CDI-Type II: Collaborative Research: Perception of Scene Layout by Machines and Visually Impaired Users

CDI-Type II：协作研究：机器和视障用户对场景布局的感知

• 批准号: 1028009
• 负责人: Kostas Daniilidis
• 金额: $ 32.57万
• 依托单位: University of Pennsylvania
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-10-01 至 2014-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1028009&HistoricalAwards=false
• 关键词: CDI; Type; II; Collaborative; Research

The project investigates computational methods for object detection, spatial scene construction, and natural language spatial descriptions derived from real-time visual images to describe prototypical indoor spaces (e.g., rooms, offices, etc.). The primary application of this research is to provide blind or visually impaired users with spatial information about their surroundings that may otherwise be difficult to obtain from non-visual sensing. Such knowledge will assist in development of accurate cognitive models of the environment and will support better informed execution of spatial behaviors in everyday tasks. A second motivation for the work is to contribute to the improvement of spatial capacities for computers and robots. Computers and robots are similarly "blind" to images unless they have been provided some means to "see" and understand them. Currently, no robotic system is able to reliably perform high-level processing of spatial information on the basis of image sequences, e.g., to find an empty chair in a room, which not only means finding an empty chair in an image, but also localizing the chair in the room, and performing an action of reaching the chair. The guiding tenet of this research is that a better understanding of spatial knowledge acquisition from visual images and concepts of spatial awareness by humans can also be applied to reducing the ambiguity and uncertainty of information processing by autonomous systems. A central contribution of this work is to make the spatial information content of visual images available to the visually impaired, a rapidly growing demographic of our aging society. In an example scenario a blind person and her guide dog are walking to her doctor's office, an office which she has not previously visited. At the office she needs information for performing some essential tasks such as finding the check-in counter, available seating, or the bathroom. No existing accessible navigation systems are able to describe the spatial parameters of an environment and help detect and localize objects in that space. Our work will provide the underlying research and elements to realize such a system.

该项目研究了用于对象检测，空间场景构建和自然语言空间描述的计算方法。这项研究的主要应用是为盲人或视觉障碍用户提供有关其周围环境的空间信息，否则这些信息可能很难从非视觉感应中获得。这种知识将有助于开发环境的准确认知模型，并将支持在日常任务中更好地了解空间行为的情况。这项工作的第二个动机是为改善计算机和机器人的空间能力做出贡献。除非提供了一些“看到”并理解它们的方法，否则计算机和机器人对图像的“盲目”。当前，没有机器人系统能够根据图像序列（例如，在房间中找到一个空椅子）可靠地对空间信息进行高级处理，这不仅意味着在图像中找到空椅子，而且还意味着将椅子定位在房间中，并执行到达椅子的动作。这项研究的指导性宗旨是，从人类的空间意识和空间意识的概念中更好地理解了人类的空间意识概念，也可以应用于降低自主系统信息处理的歧义和不确定性。这项工作的核心贡献是使视觉障碍的视觉图像的空间信息内容是我们衰老社会的快速增长的人群。在一个示例场景中，一个盲人和她的导犬正走到医生的办公室，这是她以前从未访问过的办公室。在办公室，她需要信息来执行一些重要任务，例如查找登机柜台，可用座位或浴室。没有现有的可访问导航系统能够描述环境的空间参数，并有助于检测和定位该空间中的对象。我们的工作将为实现这种系统提供基本的研究和要素。