RI: Large: Collaborative Research: 3D Structure and Motion in Dynamic Natural Scenes

RI：大型：协作研究：动态自然场景中的 3D 结构和运动

• 批准号: 1111765
• 负责人: Bruno Olshausen
• 金额: $ 68万
• 依托单位: University of California-Berkeley
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-09-01 至 2016-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1111765&HistoricalAwards=false
• 关键词: RI; Large; Collaborative; Research; 3D

How does a vision system recover the 3-dimensional structure of the world -- such as the layout of the environment, surface shape, or object motion -- from the dynamic 2-dimensional images received by the sensors in a camera, or the retinas in our eyes? This problem is fundamental to both computer and biological vision. Computer vision has developed a variety of algorithms for estimating specific aspects of a scene such as the 3-dimensional positions of points whose correspondence over time can be established, but obtaining complete and robust scene representations for complex natural scenes and viewing conditions remains a challenge. Biological vision systems have evolved impressive capabilities that suggest they have detailed and robust representations of the 3-dimensional world, but the neural representations that subserve this are poorly understood and neurophysiological studies thus far have provided little insight into the computational process. This project will pursue an interdisciplinary approach by attempting the understand the universal principles that lie at the heart of 3-dimensional scene analysis.Specifically, the project will 1) develop a novel class of computational models that recover and represent 3-dimensional scene information, 2) collect high quality video and range data of dynamic natural scenes under a variety of controlled motion conditions, and 3) test the perceptual implications of these models in psychophysical experiments. The computational models will utilize non-linear decomposition - i.e., the ability to explain complex, time-varying images in terms of the non-linear interaction of multiple factors, such as the interaction between observer motion, the 3-dimensional scene layout, and surface patterns. Importantly, the components of these models will be adapted to the statistics of natural motion patterns that arise from observer motion through natural scenes and movement around points of fixation.The project is a collaboration between three laboratories that have played a leading role in developing theoretical models of natural image statistics, visual neural representations, and perceptual processes. The investigators seek to combine their efforts to develop new models, data sets, and characterizations of 3-dimensional natural scene structure that go beyond previous studies of natural image statistics, and that can be tested in neurophysiological and psychophysical experiments. This project has the potential to bring about fundamental advances in neuroscience, visual perception, and computer vision by developing new classes of models that robustly infer representations of the 3-dimensional natural environment. It will create a set of high quality databases that will be made available to help other investigators study these issues. It will also open up new possibilities for generating realistic stimuli that can guide novel investigations of neural representation and processing.

视觉系统如何从相机或视网膜中的传感器接收到的动态二维图像中恢复世界的 3 维结构，例如环境的布局、表面形状或物体运动在我们眼中？ 这个问题对于计算机和生物视觉都是基础问题。 计算机视觉已经开发了多种算法来估计场景的特定方面，例如可以建立随时间变化的对应点的 3 维位置，但获得复杂自然场景和观看条件的完整且稳健的场景表示仍然是一个挑战。 生物视觉系统已经发展出令人印象深刻的能力，这表明它们对 3 维世界具有详细而强大的表示，但是促进这一点的神经表示却知之甚少，并且迄今为止的神经生理学研究对计算过程几乎没有提供任何见解。 该项目将采用跨学科方法，尝试理解 3 维场景分析核心的普遍原理。具体来说，该项目将 1) 开发一类新颖的计算模型，用于恢复和表示 3 维场景信息， 2）收集各种受控运动条件下动态自然场景的高质量视频和范围数据，3）在心理物理学实验中测试这些模型的感知含义。 计算模型将利用非线性分解，即能够根据多个因素的非线性相互作用来解释复杂的、随时间变化的图像，例如观察者运动、3 维场景布局和图像之间的相互作用。表面图案。 重要的是，这些模型的组件将适应自然运动模式的统计数据，这些自然运动模式是由观察者在自然场景中的运动以及围绕固定点的运动产生的。该项目是三个实验室之间的合作，这三个实验室在开发理论模型方面发挥了主导作用自然图像统计、视觉神经表征和感知过程。 研究人员寻求共同努力开发新的模型、数据集和 3 维自然场景结构的表征，这些模型、数据集和表征超越了先前的自然图像统计研究，并且可以在神经生理学和心理物理学实验中进行测试。 该项目有潜力通过开发能够稳健地推断 3 维自然环境表示的新型模型，从而在神经科学、视觉感知和计算机视觉领域带来根本性进步。 它将创建一组高质量的数据库，以帮助其他研究人员研究这些问题。 它还将为生成现实刺激开辟新的可能性，从而指导神经表征和处理的新颖研究。