Early representation of 3D volumetric shape in visual object processing

视觉对象处理中 3D 体积形状的早期表示

基本信息

批准号：
10412966
负责人：
CHARLES E CONNOR
金额：
$ 48.43万
依托单位：
JOHNS HOPKINS UNIVERSITY
依托单位国家：
美国
项目类别：
财政年份：
2018
资助国家：
美国
起止时间：
2018-09-01 至 2024-05-31
项目状态：
已结题

来源：
https://reporter.nih.gov/project-details/10412966
关键词：
3-Dimensional 3D world Area Biological Brain Calcium Signaling Code Computer Vision Systems Cues Data Electrodes Functional Imaging Future Genetic Programming Glass Goals Human Image Implant Lead Learning Measures Medial Microelectrodes Microscopic Microscopy Modeling Monkeys Network-based Neurons Ocular Prosthesis Pathway interactions Patients Pattern Performance Population Prevalence Prosthesis Rehabilitation therapy Scientist Series Shapes Signal Transduction Stimulus Structure Surface Synapses System Testing Texture Three-dimensional analysis Time V4 neuron Vision Visual Cortex Visual Perception Visual impairment Work area V4 area striata base convolutional neural network experimental study individual response network models neurotransmission nonhuman primate novel object perception object recognition operation relating to nervous system response three dimensional structure two-photon virtual reality visual object processing

项目摘要

Project Summary The goal of this project is to test a novel theoretical framework for understanding how the ventral pathway subserves object vision. In the standard framework, a series of neural operations on 2D image data through many intermediate cortical stages, including area V4, leads to high-level perceptual representations, including representation of object identity, at the final stages of the ventral pathway. However, our preliminary microelectrode data from a fixating monkey show that many neurons in V4 represent volumetric (volume- enclosing) 3D shape, not 2D image patterns. These neurons respond to many different 2D images that convey the same 3D shape with different shape-in-depth cues, including shading, reflection, and refraction. They even respond preferentially to random dot stereograms that convey 3D volumetric shape with no 2D cues whatsoever. Moreover, our preliminary results with 2-photon functional imaging in anesthetized monkey V4 show that 3D shape signals are grouped by their similarity, and also group with isomorphic (same outline) 2D shape signals (which could contribute evidence to corresponding 3D shape inferences). We propose to capitalize on these preliminary data by demonstrating the prevalence of 3D shape tuning in area V4, analyzing the 3D shape coding strategies used by these neurons, and measuring how 2D and 3D shape signals are arranged at a microscopic level across the surface of V4. We expect these results to provide strong evidence that extraction of 3D shape fragments is a primary goal of V4 processing. This early extraction of 3D shape information, just two synapses beyond primary visual cortex, would suggest a competing framework for understanding the ventral pathway, in which the initial goal is to represent 3D physical structure, independent of the various 2D image cues used to infer it. In this framework, object recognition would be based on preceding information about 3D physical structure, which would explain why human object recognition is so robust to image changes, in a way that the best computational vision systems are not. The scientific impact of this work would be to divert vision experiments toward understanding representation of real world 3D structure (rather than 2D planar stimuli) and to encourage computational vision scientists to incorporate early 3D shape processing into the deep convolutional network models that are the current state of the art.

项目摘要该项目的目的是测试一个新的理论框架，以了解腹侧路径子服务对象视觉。在标准框架中，通过2D图像数据的一系列神经操作通过包括区域V4在内的许多中间皮质阶段，都导致高级感知表示，包括对象身份的表示，在腹侧路径的最后阶段。但是，我们的初步来自固定猴子的微电极数据表明，V4中的许多神经元表示体积（体积 - 封闭）3D形状，而不是2D图像模式。这些神经元响应传达的许多不同的2D图像具有不同形状的深度线索的相同3D形状，包括阴影，反射和折射。他们甚至优先响应传达3D体积形状的随机DOT刻度图，没有2D提示任何。此外，在麻醉猴子V4中使用2光子功能成像的初步结果证明3D形状信号由它们的相似性分组，并与同构（相同概述）2D分组形状信号（可能为相应的3D形状推断提供证据）。我们建议通过证明区域V4中3D形状调整的流行率，分析这些初步数据这些神经元使用的3D形状编码策略，并测量2D和3D形状信号在V4的表面上以微观水平排列。我们希望这些结果能提供有力的证据提取3D形状片段是V4处理的主要目标。早期提取3D形状信息，只有两个超出主要视觉皮层的突触，这将为竞争框架提供了解最初目标是表示3D物理结构，独立的腹侧途径用于推断它的各种2D图像提示中。在此框架中，对象识别将基于有关3D物理结构的前面信息，这将解释为什么人类对象识别是如此最佳的计算视觉系统不是稳健的图像变化。科学的影响这项工作将是将视觉实验转向理解现实世界3D结构的表示（而不是2D平面刺激）并鼓励计算视觉科学家合并早期的3D形状处理为当前最新状态的深卷卷网络模型。