Development of an AI Assistant for Individuals with Low Vision and Blindness

为低视力和失明人士开发人工智能助手

• 批准号: 10615838
• 负责人: David M. Rosler
• 金额: --
• 依托单位: PROVIDENCE VA MEDICAL CENTER
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-07-01 至 2025-06-30
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10615838
• 关键词: 3-Dimensional; Address; Age; Algorithms; Architecture; Artificial Intelligence; Auditory; Automobiles; Blindness; Canes; Cellular Phone; Central Scotomas; Clothing; Color; Complete Blindness; Computer Vision Systems; Computer software; Computers; Cues; Development; Device or Instrument Development; Devices; Diabetic Retinopathy; Effectiveness; Electrical Stimulation of the Brain; Electrodes; Environment; Eyeglasses; Face; Feedback; Focus Groups; Glaucoma; Goals; Hand; Head; Healthcare Systems; Home; Implant; Individual; Instruction; Intelligence; Internet; Interview; Intuition; Language; Learning; Legal Blindness; Light; Location; Machine Learning; Macular degeneration; Microcomputers; Names; National Eye Institute; Operative Surgical Procedures; Paper; Perception; Performance; Peripheral; Persons; Quality of life; Reader; Research; Residual state; Resolution; Retina; Retinitis Pigmentosa; Robotics; Self-Help Devices; Software Engineering; Specific qualifier value; System; Tactile; Technology; Terrorism; Testing; Text; Training; Unemployment; United States; Universities; Veterans; Video Games; Vision; Vision Disorders; Visual; Visual Cortex; Visual impairment; Visually Impaired Persons; blind; communication device; convolutional neural network; cost; detector; effective therapy; experience; grasp; improved; interest; legally blind; neural network; novel; object recognition; open source; optical character recognition; prototype; rehabilitation service; sensor; sight restoration; smartphone application; social; sound; speech recognition; surgical risk; system architecture; voice recognition

The National Eye Institute estimates that in the United States there are presently over 1 million legally blind people and this number will increase to about 4 million by 2050. According to the VA Office of Blind Rehabilitation Services, there are approximately 130,000 US Veterans who are legally blind and more than one million who have lost the ability to perform daily tasks because of low vision. Some of these people can be helped by relatively simple and inexpensive devices such as magnifiers and canes. However, with more severe vision loss, people are likely to experience loss of independence and reduced quality of life. Blindness has a major impact on Veterans and others, as evidenced by the fact that over 70% of working-age people with significant visual impairment are unemployed. While there are causes of blindness that can be readily corrected, in many cases there is no effective treatment. One solution is to restore vision by surgically placing electrodes in the retina or visual cortex and electrically stimulating the brain based on the light level recorded by a head-mounted camera. Systems of this type are being developed and they have great potential. However, there are downsides to the implanted systems that include the risks of surgery, high cost, and a form of vision that is severely limited compared to normal perception. There is also a wide array of devices and smartphone apps that perform specialized functions (text readers, obstacle detectors, color identifiers). The goal of the proposed research is to develop and test a new type of visual assistive device, named AEyes, for individuals with blindness and low vision. It takes advantage of recent advances in artificial intelligence including computer vision, machine learning, optical character recognition, speech recognition, and 3D sound rendering to give the user the ability to recognize, localize, and interact with objects and people in their vicinity. In other words, the AI technology that can benefit people with low vision already exists and it will be implemented in a device tailored to the needs of people with visual disorders. The system will be intuitive and easy to learn and interact with as it understands spoken instructions and it speaks to the user. AEyes has been developed based on a needs-analysis and feedback from focus groups, one-on-one interviews, and prototype tests with Veterans conducted at the VA Providence Healthcare System. Aim 1 concerns device development. The system architecture and algorithms are largely implemented and functional, but a range of refinements to the software and the integration of separate functional modules into the overall system are needed. These improvements include upgrading to a higher resolution camera, retraining the recognition neural network, and integrating the face recognition, hand tracking, and voice recognition modules into the central architecture. Aim 2 consists of device tests that will be conducted with subjects having a range of low vision conditions that include no light sense, retinitis pigmentosa, glaucoma, macular degeneration, and diabetic retinopathy. The tests to be conducted include the ability of a user to recognize and localize people, reach for and grasp objects, read text on handheld paper, computer displays, and signs, and train the system to recognize new people. Comparing performance with residual vision alone and with the addition of AEyes will establish the device’s effectiveness as a visual assistant and indicate where improvements are needed.

美国国家眼科研究所估计，目前美国有超过 100 万合法眼科患者。 盲人，到 2050 年，这一数字将增加到约 400 万。根据 VA 盲人办公室的数据 康复服务部门，大约有 130,000 名美国退伍军人在法律上是失明的，并且超过一名 数以百万计的人因视力低下而丧失了执行日常任务的能力。 借助放大镜和手杖等相对简单且廉价的设备，但情况更为严重。 视力丧失，人们可能会失去独立性并降低生活质量。 对退伍军人和其他人产生重大影响，事实证明，超过 70% 的工作年龄人口患有 严重视力障碍者失业。 虽然有些失明原因可以很容易地纠正，但在许多情况下没有有效的方法 一种解决方案是通过手术将电极放置在视网膜或视觉皮层中来恢复视力。 根据头戴式摄像头系统记录的光线水平对大脑进行电刺激。 类型正在开发中，它们具有巨大的潜力，但是，植入式也有缺点。 与传统系统相比，该系统存在手术风险、高成本以及严重受限的视力 还有许多执行专门功能的设备和智能手机应用程序。 功能（文本阅读器、障碍物检测器、颜色识别器）。 拟议研究的目标是开发和测试一种新型视觉辅助设备，名为 Aeyes，针对失明和视力低下的人士，它利用了人工技术的最新进展。 智能包括计算机视觉、机器学习、光学字符识别、语音识别等 3D 声音渲染使用户能够识别、定位周围的物体和人并与之交互 换句话说，可以造福弱视人群的人工智能技术已经存在，并且将会存在。 该系统将在专为满足视觉障碍人士的需求而定制的设备中实施。 易于学习和交互，因为它能够理解语音指令并与用户对话。 是根据需求分析和焦点小组反馈、一对一访谈而制定的 在退伍军人管理局普罗维登斯医疗系统中与退伍军人进行原型测试。 目标1涉及设备开发，主要实现系统架构和算法。 和功能，但对软件进行了一系列改进以及单独功能模块的集成 这些改进包括升级到更高分辨率的相机， 重新训练识别神经网络，整合人脸识别、手部追踪、语音 识别模块集成到中央架构中。 目标 2 包括将针对一系列低视力受试者进行的设备测试 包括无光感、色素性视网膜炎、青光眼、黄斑变性和糖尿病等疾病 进行的测试包括用户识别和定位人物、伸手取物的能力。 抓住物体，阅读手持纸、计算机显示屏和标志上的文本，并训练系统 单独使用残余视力以及添加 AEyes 来识别新人。 确定设备作为视觉助手的有效性，并指出需要改进的地方。