Collaborative Proposal: Object and Action Recognition in Time Sequences of Images: Computational Neuroscience and Neurophysiology

协作提案：图像时间序列中的对象和动作识别：计算神经科学和神经生理学

• 批准号: 0827483
• 负责人: Tomaso Poggio
• 金额: $ 47.5万
• 依托单位: Massachusetts Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2008
• 资助国家: 美国
• 起止时间: 2008-09-01 至 2012-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0827483&HistoricalAwards=false
• 关键词: Collaborative; Proposal; Object; Action; Recognition

Normal vision is not static: time is a key dimension of the natural world we see. The eventual understanding of biological vision requires understanding the neural mechanisms used to recognize objects and actions over time. Thus the focus of the proposed research is to study how the primate visual system recognizes objects and actions in time sequences of images. A meta-goal of this project is to exploit the synergies between computational approaches and physiological experiments to lead to a better understanding of brain function and at the same time to develop better computer vision algorithms. Object recognition in time sequences of images presents a significant challenge for recognition systems, because it requires both selectivity to shape and invariance to changes of appearance in time.. This project will extend an existing computational model of the ventral stream by adding temporal dynamics in its model neurons and the ability to process video sequences. It will also expand a working model of the dorsal stream to understand the relative roles that it and the ventral stream play in dynamic visual recognition. At the same time, recordings from single units, and multiple single units, from high level visual areas including IT and regions of the STS will be made in order to characterize the tuning of single neurons to the shape dynamics of specific image sequences. By combining modeling and physiology, this work will search for a computational explanation for how the higher areas of the visual cortex recognize objects and actions over time and how they can learn.This integrative effort, which is focused on processing of dynamic perceptual information, can have a significant and direct impact on current theories of autism, dyslexia, and effects of stroke, in addition to directly guiding modeling and engineering efforts in computer vision. The proposed research is tightly coupled to education and teaching, and resources used in the research, including databases of videos, visual stimuli, the modeling software and the experimental data will be made available to the broad scientific community. Information on the project and its progress will be available at http://cbcl.mit.edu/projects/NSF-CRCNS/index.html

正常视觉不是静态的：时间是我们所看到的自然世界的一个关键维度。对生物视觉的最终理解需要理解随着时间的推移用于识别物体和动作的神经机制。因此，本研究的重点是研究灵长类动物视觉系统如何识别图像时间序列中的物体和动作。 该项目的一个元目标是利用计算方法和生理实验之间的协同作用，以更好地理解大脑功能，同时开发更好的计算机视觉算法。图像时间序列中的对象识别对识别系统提出了重大挑战，因为它需要形状选择性和外观随时间变化的不变性。该项目将通过在腹侧流中添加时间动力学来扩展现有的计算模型。模型神经元和处理视频序列的能力。它还将扩展背侧流的工作模型，以了解背侧流和腹侧流在动态视觉识别中发挥的相对作用。同时，将进行来自包括 IT 和 STS 区域在内的高级视觉区域的单个单元和多个单个单元的记录，以便表征单个神经元对特定图像序列的形状动态的调整。 通过结合建模和生理学，这项工作将寻找一种计算解释，解释视觉皮层的较高区域如何随着时间的推移识别物体和动作以及它们如何学习。这项综合工作专注于动态感知信息的处理，可以除了直接指导计算机视觉领域的建模和工程工作之外，还对当前的自闭症、阅读障碍和中风影响的理论产生重大和直接的影响。 拟议的研究与教育和教学紧密结合，研究中使用的资源，包括视频数据库、视觉刺激、建模软件和实验数据，将向广大科学界开放。有关该项目及其进展的信息将在 http://cbcl.mit.edu/projects/NSF-CRCNS/index.html 上提供。