Environmental Localization Mapping and Guidance for Visual Prosthesis Users

视觉假肢用户的环境定位测绘和指导

• 批准号: 10238093
• 负责人: Seth D Billings
• 金额: $ 55.1万
• 依托单位: JOHNS HOPKINS UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-30 至 2023-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10238093
• 关键词: 3-Dimensional; Address; Age related macular degeneration; Algorithms; American; Competence; Complex; Computer Vision Systems; Cues; Data; Dependence; Detection; Development; Devices; Disease; Effectiveness; Environment; Evaluation; FDA approved; Feedback; Glaucoma; Goals; Image; Implant; Late-Onset Disorder; Lead; Learning; Life; Location; Maps; Medical Device; Modality; Motion; Ocular Prosthesis; Patients; Performance; Psyche structure; Research; Retinitis Pigmentosa; Running; Semantics; Sensory; Societies; System; Systems Integration; Techniques; Technology; Testing; Time; Training; Update; Vision; Visual; Volition; aged; auditory feedback; base; behavior test; blind; cognitive load; falls; human subject; improved; innovation; legally blind; navigation aid; object recognition; portability; prosthesis wearer; prototype; research and development; retina implantation; retinal prosthesis; sensory feedback; smartphone Application; synergism; visual feedback; visual information

Project Summary About 1.3 million Americans aged 40 and older are legally blind, a majority because of diseases with onset later in life, such as glaucoma and age-related macular degeneration. Second Sight has developed the world's first FDA approved retinal implant, Argus II, intended to restore some functional vision for people suffering from retinitis pigmentosa (RP). In this era of smart devices, generic navigation technology, such as GPS mapping apps for smartphones, can provide directions to help guide a blind user from point A to point B. However, these navigational aids do little to enable blind users to form an egocentric understanding of the surroundings, are not suited to navigation indoors, and do nothing to assist in avoiding obstacles to mobility. The Argus II, on the other hand, provides blind users with a limited visual representation of their surroundings that improves users' ability to orient themselves and traverse obstacles, yet lacks features for high-level navigation and semantic interpretation of the surroundings. The proposed research aims to address these limitations of the Argus II through a synergy of state-of-the-art stimultaneous localization and mapping (SLAM) and object recognition technologies. For the past three years, JHU/APL has collaborated with Second Sight to develop similar advanced vision-based capabilities for the Argus II, including capabilities for object recognition and obstacle detection by stereo vision. This proposal is driven by the hypothesis that navigation for users of retinal prosthetics can be greatly improved by incorporating SLAM and object recognition technology conveying environmental information via a retinal prosthesis and auditory feedback. SLAM enables the visual prosthesis system to construct a map of the user's environment and locate the user within that map. The system then provides object location and navigational cues via appropriate sensory modalities enabling the user to mentally form an egocentric map of the environment. We propose to develop and test a visual prosthesis system which 1) constructs a map of unfamiliar environments and localizes the user using SLAM technology 2) automatically identifies navigationally-relevant objects and landmarks using object recognition and 3) provides sensory feedback for navigation, obstacle avoidance, and object/landmark identification.

项目摘要 约有130万40岁及40岁以上的美国人在法律上是盲人，这是由于疾病的多数 生命后期发作，例如青光眼和与年龄相关的黄斑变性。第二瞄准具 已经开发了世界上第一个FDA批准的视网膜植入物Argus II，旨在恢复一些 患有视网膜炎色素炎（RP）的人的功能视野。 在这个智能设备时代，通用导航技术，例如GPS映射应用程序 智能手机，可以提供指示，以帮助指导盲人用户从点到点。 这些导航辅助工具无济于事，使盲人用户能够对 周围环境不适合在室内航行，也无助于避免 流动性障碍。另一方面，Argus II为盲人用户提供了有限的视觉 其周围环境的代表，可以提高用户定向自己的能力和 遍历障碍，但缺乏用于高级导航和语义解释的特征 环境。拟议的研究旨在通过A来解决Argus II的这些局限 最先进的刺激定位和映射（SLAM）和物体的协同作用 识别技术。在过去的三年中，JHU/APL与第二 为Argus II开发类似基于高级视力的能力的视力，包括 立体声愿景的对象识别和障碍物检测的功能。 该建议是由以下假设驱动的：视网膜假肢用户的导航可以是 通过合并大满贯和对象识别技术传达，可以极大地改善 环境信息通过视网膜假体和听觉反馈。猛击启用 视觉假体系统构建用户环境的地图并在其中找到用户 那个地图。然后，系统通过适当的 感官方式使用户能够在心理上形成以环境的自负地图。 我们建议开发和测试一个视觉假体系统，1）构造 不熟悉的环境并使用SLAM技术定位用户2）自动 使用对象识别标识与导航相关的对象和地标，3） 导航，避免障碍物和对象/地标识别的感觉反馈。