Environmental Localization Mapping and Guidance for Visual Prosthesis Users

视觉假肢用户的环境定位测绘和指导

• 批准号: 10838685
• 负责人: Seth D Billings
• 金额: $ 9.94万
• 依托单位: JOHNS HOPKINS UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-30 至 2024-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10838685
• 关键词: Address; Administrative Supplement; Age related macular degeneration; American; Competence; Computer Vision Systems; Cues; Development; Devices; Disease; Effectiveness; Environment; Evaluation; FDA approved; Glaucoma; Goals; Implant; Lead; Learning; Legal Blindness; Life; Location; Maps; Modality; Ocular Prosthesis; Patients; Persons; Psyche structure; Research; Retinitis Pigmentosa; Semantics; Sensory; Societies; System; Systems Integration; Techniques; Technology; Testing; Time; Training; Update; Vision; Visual; Work; aged; auditory feedback; blind; falls; human subject; improved; innovation; legally blind; object recognition; parent grant; prosthesis wearer; prototype; retina implantation; retinal prosthesis; sensory feedback; smartphone application; synergism; visual information

Environmental Localization Mapping and Guidance for Visual Prosthesis Users Project Summary About 1.3 million Americans aged 40 and older are legally blind, a majority because of diseases with onset later in life, such as glaucoma and age-related macular degeneration. Second Sight developed the world’s first FDA approved retinal implant, Argus II, intended to restore some functional vision for people suffering from retinitis pigmentosa (RP). In this era of smart devices, generic navigation technology, such as GPS mapping apps for smartphones, can provide directions to help guide a blind user from point A to point B. However, these navigational aids do little to enable blind users to form an egocentric understanding of the surroundings, are not suited to navigation indoors, and do nothing to assist in avoiding obstacles to mobility. Prosthetic vision devices like the Argus II, on the other hand, provides blind users with a limited visual representation of their surroundings that improve users’ ability to orient themselves and traverse obstacles, yet lack features for high-level navigation and semantic interpretation of the surroundings. The proposed research aims to address these limitations through a synergy of state-of-the-art simultaneous localization and mapping (SLAM) and object recognition technologies. This proposal is driven by the hypothesis that navigation for users of retinal prosthetics can be greatly improved by incorporating SLAM and object recognition technology conveying environmental information via a retinal prosthesis and auditory feedback. SLAM enables the visual prosthesis system to construct a map of the user’s environment and locate the user within that map. The system then provides object location and navigational cues via appropriate sensory modalities enabling the user to mentally form an egocentric map of the environment. We have developed an advanced visual prosthesis system which 1) constructs a map of unfamiliar environments and localizes the user using SLAM technology 2) automatically identifies the structural semantics of the surroundings to determine navigationally-relevant obstacles and features and 3) provides sensory feedback for navigation, obstacle avoidance, and object/landmark identification. This project focuses on the completion of human subject testing of these capabilities.

环境定位映射和视觉假体用户的指导 项目摘要 约有130万40岁及40岁以上的美国人在法律上是盲人，这是由于疾病的多数 生命后期发作，例如青光眼和与年龄相关的黄斑变性。第二瞄准具 开发了世界上第一个FDA批准的视网膜植入物Argus II，旨在恢复一些 患有视网膜炎色素炎（RP）的人的功能视野。 在这个智能设备时代，通用导航技术，例如GPS映射应用程序 智能手机，可以提供指示，以帮助指导盲人用户从点到点。 这些导航辅助工具无济于事，使盲人用户能够对 周围环境不适合在室内航行，也无助于避免障碍 流动性。另一方面 对周围环境的视觉表现有限，可以提高用户定向自己的能力 和遍历障碍，但缺乏高级导航和语义解释的功能 周围环境。拟议的研究旨在通过协同作用来解决这些局限性 最新的同时本地化和映射（大满贯）和对象识别 技术。 该建议是由以下假设驱动的：视网膜假肢用户的导航可以是 通过合并大满贯和对象识别技术传达，可以极大地改善 环境信息通过视网膜假体和听觉反馈。猛击启用 视觉假体系统构建用户环境的地图并将用户定位 那个地图。然后，系统通过适当的感觉提供对象位置和导航提示 使用户能够在心理上形成以环境的自我地图的方式。我们有 开发了一个先进的视觉假体系统，1）构建一张不熟悉的地图 环境和使用SLAM技术本地化用户2）自动识别 周围环境的结构语义，以确定与航行相关的障碍和 功能和3）提供导航，避免障碍和的感觉反馈 对象/地标识别。该项目着重于完成人类主题测试的 这些功能。