CRCNS: Joint coding of shape and texture in the primate brian

CRCNS：灵长类动物布莱恩形状和纹理的联合编码

• 批准号: 9765318
• 负责人: Anitha Pasupathy
• 金额: $ 23.05万
• 依托单位: UNIVERSITY OF WASHINGTON
• 依托单位国家: 美国
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-01 至 2021-08-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9765318
• 关键词: Biological; Biological Sciences; Categories; Characteristics; Code; Collaborations; Computer Simulation; Computers; Cues; Data; Dimensions; Discrimination; Facial Expression; Goals; Human; Image; Individual; Japan; Japanese Population; Joints; Laboratories; Modeling; Modification; Monkeys; Neurons; Pathway interactions; Perception; Performance; Physiology; Primates; Process; Psychophysics; Scientist; Shapes; Stimulus; Structure; Subgroup; Surface; Surface Properties; Text; Texture; Time; Tokyo; Training; United States; Universities; V4 neuron; Visual; Visual Pathways; Visual Perception; Visual system structure; Washington; Weight; area V4; base; convolutional neural network; experimental study; food quality; human model; human subject; information processing; luminance; neurophysiology; novel; object perception; object recognition; response; statistics; visual information; visual processing; visual stimulus

PROJECT DESCRIPTION Collaborating Pis and Consultant United States Pl: Anitha Pasupathy, Dept. of Biological Structure, University of Washington, Seattle, USA Co-Pl: Wyeth Bair, Dept. of Biological Structure, University of Washington, Seattle, USA Japan Pl: lsamu Motoyoshi, Dept. of Life Sciences, The University of Tokyo, Japan Consultant: Hidehiko Komatsu, Tamagawa University, Japan Specific Aims Our visual system endows us with a diverse set of abilities: to recognize and manipulate objects, avoid obstacles and danger during navigation, evaluate the quality of food, read text, interpret facial expressions, etc. This relies on the neuronal processing of information about form and material texture along the ventral pathway of the primate visual system (Ungerleider & Mishkin, 1982; Felleman & Van Essen, 1991). Studies over the past several decades have produced detailed models of how visual information is processed in V1, the earliest stage along . this pathway (Hubel & Wiesel, 1959, 1968; Movshon et al., 1978a, b; Albrecht et al., 1980), but beyond V1 our understanding of visual processing and representation is limited. This is particularly true with regard to our understanding of how visual representations of form and texture jointly contribute to object perception and recognition. The broad goal of this proposal is two-fold-to develop an experimentally-driven image-computable model for how naturalistic visual stimuli are processed in area V4, an important intermediate stage along the ventral visual pathway (Aim 1) and to discover how such a representation contributes to perception (Aim 2). Past studies have shown that V4 neurons are sensitive to both the form (Desimone and Schein, 1987; Kobatake and Tanaka, 1994; Gallant et al., 1993; Pasupathy and Connor, 2001; Nandy et al., 2013) and the surface texture of visual stimuli (Arcizet et al., 2008; Goda et al., 2014; Okazawa et al., 2015). But, because expertise is narrow and experimental time limited, scientists tend to focus exclusively on the encoding of form or texture and not on their joint coding. For example, in the laboratories of the USA portion of this collaboration, we have until now focused on form processing by carrying out neurophysiological studies using 2D shapes with uniform surface properties to investigate how object boundaries are encoded (Oleskiw et al., 2014; Popovkina et al., 2016). We have modeled our data by comparing the representation of V4 neurons to that of the units in AlexNet (Pospisil et al., 2015), a prominent convolutional neural net (CNN) trained to recognize objects (Krizhevsky et al., 2012). At the same time, the Japanese contingent of this collaboration has investigated the encoding of surface texture and gloss in human perception without associated form encoding (Motoyoshi et al., 2007; Sharan et al., 2008; Motoyoshi, 2010; Motoyoshi & Matoba, 2012). Here we propose to bring our respective expertise in studying form and texture encoding to bear on the question of how naturalistic stimuli with both form and surface cues are encoded in area V4 and how these representations support human visual perception. Our specific aims are: Aim1. To build a unified image-computable model for neuronal responses to shapes and textures in area V4 V4 responses to 2D shapes with uniform luminance/chromatic characteristics can be explained by a hierarchical-Max (HMax) model for object recognition that emphasizes boundary features (Cadieu et al., 2007). Such responses can also be explained by units in artificial deep convolutional networks, in which boundary features are not explicitly emphasized (all features are learned from initially random weights). On the other hand, V4 responses to texture patches can be well explained by a higher-order image-statistics-based model (Okazawa et al., 2015). Using shape data from the Pasupathy lab and texture data from the Komatsu lab (Japanese consultant), we will ask whether responses of V4 neurons to shapes and textures can be Page 21

项目描述 与PIS合作和顾问 美国 PL：美国西雅图华盛顿大学生物结构系Anitha Pasupathy，美国西雅图 共同：美国西雅图华盛顿大学生物结构系Wayeth Bair。 日本 PL：日本东京大学生命科学系Lsamu Motoyoshi 顾问：日本塔川大学的Hidehiko Komatsu 具体目标 我们的视觉系统赋予我们各种能力：识别和操纵 物体，避免在航行过程中遇到障碍和危险，评估食物的质量，阅读文字， 解释面部表情等。这取决于有关信息的神经元处理 沿着灵长类动物视觉系统的腹侧路径（Ungerleider＆ 米什金（Mishkin），1982年； Felleman＆Van Essen，1991）。在过去的几十年中 生成了如何在V1中处理视觉信息的详细模型，这是最早的阶段 。这条途径（Hubel＆Wiesel，1959，1968; Movshon等，1978a，b; Albrecht等，1980），但 除了V1之外，我们对视觉处理和表示的理解是有限的。这是 关于我们对形式的视觉表示方式的理解尤其如此 纹理共同有助于对象感知和识别。该提议的广泛目标是 两倍用于开发一个实验驱动的图像计算模型，用于自然主义 视觉刺激是在V4区域处理的，该区域是沿腹侧视觉的重要中间阶段 途径（AIM 1），并发现这种表示形式如何促进感知（AIM 2）。 过去的研究表明，V4神经元对形式敏感（Desimone和Schein， 1987; Kobatake和Tanaka，1994年； Gallant等，1993； Pasupathy和Connor，2001年； Nandy等 Al。，2013）和视觉刺激的表面纹理（Arcizet等，2008； Goda等，2014； Okazawa等人，2015年）。但是，由于专业知识狭窄且实验时间有限，所以 科学家倾向于专注于形式或纹理的编码，而不是关节 编码。例如，在这项合作的美国实验室中，我们必须直到 现在，通过使用2D形状进行神经生理研究，专注于形式处理 具有统一的表面特性，以研究对象边界的编码方式（Oleskiw et Al。，2014年； Popovkina等，2016）。我们通过比较表示形式来建模我们的数据 Alexnet单位的V4神经元（Pospisil等，2015），这是一个突出的卷积 神经网（CNN）接受了识别对象的训练（Krizhevsky等，2012）。同时， 日本合作的特遣队研究了表面纹理的编码和 没有相关形式编码的人类感知中的光泽（Motoyoshi等，2007； Sharan等 Al。，2008； Motoyoshi，2010年； Motoyoshi＆Matoba，2012年）。在这里，我们提议带来我们的 各自研究形式和纹理编码的专业知识，以解决如何解决 具有形式和表面提示的自然主义刺激在区域V4中编码 表示支持人类的视觉感知。我们的具体目的是： AIM1。建立一个统一的图像计算模型，用于对形状的神经元素响应和 区域V4的纹理 V4对具有均匀亮度/色彩特征的2D形状的响应可以解释 通过强调边界特征的对象识别的层次 - 最大最大（HMAX）模型 （Cadieu等，2007）。这种反应也可以通过人工深处的单位来解释 卷积网络，其中未明确强调边界特征（所有功能 从最初的随机重量中学到）。另一方面，V4对纹理补丁的响应 可以通过高阶基于图像统计的模型来很好地解释（Okazawa等，2015）。 使用来自Pasupathy实验室的形状数据和Komatsu实验室的纹理数据（日语 顾问），我们将询问V4神经元对形状和纹理的反应是否可以 第21页