Fast and Flexible Conjunction Coding in Biological and Artificial Vision

生物和人工视觉中快速灵活的联合编码

• 批准号: 10611323
• 负责人: JohnMark Edward Taylor
• 金额: $ 6.91万
• 依托单位: COLUMBIA UNIV NEW YORK MORNINGSIDE
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-04-16 至 2025-04-15
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10611323
• 关键词: Anatomy; Architecture; Artificial Intelligence; Attention; Automobile Driving; Behavior; Biological; Brain; Code; Cognitive; Cognitive Science; Color; Complex; Computer Models; Computer Vision Systems; Crowding; Data Set; Development; Disparate; Doctor of Philosophy; Endowment; Engineering; Environment Design; Exhibits; Feedback; Functional Magnetic Resonance Imaging; Future; Glass; Homologous Gene; Human; Image; Industrialization; Influentials; Lateral; Learning; Link; Literature; Mentorship; Methodology; Methods; Modeling; Modernization; Natural experiment; Neurologic Deficit; Neurons; Neurosciences; Patients; Population; Prevalence; Psychophysics; Recurrence; Research; Safety; Shapes; Stimulus; Stream; Sum; Syndrome; Techniques; Technology; Testing; Toy; Training; Vision; Visual; Visual Agnosias; Visual System; Work; artificial neural network; career; cognitive neuroscience; computational neuroscience; deep learning; deep neural network; experience; experimental study; flexibility; human model; improved; neural; neuroimaging; neurophysiology; novel; object recognition; open source; operation; presynaptic; response; synergism; theories; vision science; visual search; Anatomy; Architecture; Artificial Intelligence; Attention; Automobile Driving; Behavior; Biological; Brain; Code; Cognitive; Cognitive Science; Color; Complex; Computer Models; Computer Vision Systems; Crowding; Data Set; Development; Disparate; Doctor of Philosophy; Endowment; Engineering; Environment Design; Exhibits; Feedback; Functional Magnetic Resonance Imaging; Future; Glass; Homologous Gene; Human; Image; Industrialization; Influentials; Lateral; Learning; Link; Literature; Mentorship; Methodology; Methods; Modeling; Modernization; Natural experiment; Neurologic Deficit; Neurons; Neurosciences; Patients; Population; Prevalence; Psychophysics; Recurrence; Research; Safety; Shapes; Stimulus; Stream; Sum; Syndrome; Techniques; Technology; Testing; Toy; Training; Vision; Visual; Visual Agnosias; Visual System; Work; artificial neural network; career; cognitive neuroscience; computational neuroscience; deep learning; deep neural network; experience; experimental study; flexibility; human model; improved; neural; neuroimaging; neurophysiology; novel; object recognition; open source; operation; presynaptic; response; synergism; theories; vision science; visual search

Project Summary How does the human brain encode visual feature conjunctions? Two influential, yet disparate, research traditions have proposed two different mechanisms. Research in neurophysiology and computer vision has explored static conjunction coding, where feature conjunctions are automatically extracted via a feedforward hierarchy. While efficient, this mechanism may be limited to encoding conjunctions enabled by the hierarchy's learned connectivity. By contrast, research in cognitive psychology has explored dynamic conjunction coding, which sequentially encodes task-relevant conjunctions via attentional selection. While slow, this mechanism is flexible and capable of encoding any feature conjunction. Despite evidence for both mechanisms, their interplay remains unclear. In this project, we leverage advances in deep learning and open neuroimaging datasets to understand how these two mechanisms interact in the human brain, yielding the best of both worlds: fast, but flexible conjunction coding. Through the three complementary Specific Aims, we advance our understanding of these fundamental issues in vision science, and on a practical level, develop approaches that can be used to improve computer vision, aiding the development of useful technologies like autonomous vehicles. In the course of this project, I will master modern approaches in deep learning and computational neuroscience through the mentorship of my sponsor Dr. Kriegeskorte and my co-sponsor Dr. Fusi, equipping me for a career leading a lab that bridges cognitive science, neuroscience, and artificial intelligence. Hypotheses: The human brain implements static conjunction coding via neural populations with “and-like” tuning to feature combinations that emerges via feedforward convergence of neurons tuned to single features, while dy- namic conjunction coding requires recurrent connections. Static conjunction coding can rapidly encode familiar, but not unfamiliar conjunctions, while dynamic conjunction coding can encode any conjunction, but more slowly. Aim 1: We use a massive open-source fMRI dataset to chart the prevalence of static conjunction coding through- out the human visual system, using a method we developed in preliminary analyses. Aim 2: We apply “synthetic neurophysiology” to feedforward artificial neural networks to understand how static conjunction coding emerges in a feedforward hierarchy, beginning by characterizing conjunction-tuned units iden- tified in preliminary analyses, followed by testing influential models for how this occurs in biological vision. Aim 3: We test whether feedforward artificial neural networks exhibit similar limitations on a visual search task compared to known results for human feedforward vision, followed by testing whether introducing recurrent con- nections to networks can overcome these limitations.

项目摘要 人脑如何编码视觉特征组合？两个具有影响的但不同的研究传统 提出了两种不同的机制。神经生理学和计算机视觉研究探索了静态 连词编码，其中特征组合是通过前馈层次结构自动提取的。尽管 有效的是，该机制可能仅限于层次结构所学的连接启用的编码连接。 相比之下，认知心理学的研究探索了动态连接编码，依次 通过注意选择编码与任务相关的连接。虽然很慢，但这种机制灵活而有能力 编码任何功能连接。尽管有两种机制的证据，但它们的相互作用仍不清楚。在 这个项目，我们利用深度学习和开放神经影像学数据集的进步来了解这些 两种机制在人的大脑中相互作用，在两全其美中产生了最好的：快速但灵活的组合编码。 通过三个完善的特定目标，我们提出了对这些基本问题的理解 视觉科学以及实际级别，开发了可用于改善计算机视觉的方法 开发有用的技术，例如自动驾驶汽车。在这个项目的过程中，我将掌握 通过我的赞助商博士的心态，深度学习和计算神经科学的现代方法 克里格斯科特（Kriegeskorte）和我的共同赞助者Fusi博士，为我提供了一个职业，领导着一个实验室，该实验室弥合了认知科学， 神经科学和艺术智力。 假设：人脑通过“且般的”调整通过神经种群实现静态连接编码 要特征通过调谐到单个特征的神经元的前馈收敛而出现的组合，而dy- NAMIC连接编码需要经常性连接。静态连接编码可以迅速编码熟悉， 但是不是不熟悉的连接，而动态连接编码可以编码任何连接，但更慢。 AIM 1：我们使用大量的开源fMRI数据集来绘制静态连接编码的流行率 使用我们在初步分析中开发的一种方法来消除人类视觉系统。 目标2：我们将“合成神经生理学”应用于喂养艺术神经元网络，以了解静态 连词编码在馈电层次结构中出现，首先要表征连接调节的单元iDen- 在初步分析中进行了研究，然后测试了对生物视觉中这种情况的影响模型。 AIM 3：我们测试Feedforward艺术家官方神经网络是否在视觉搜索任务上暴露了类似的限制 与人类前馈视力的已知结果相比，随后测试是否引入了复发性 网络的纽约可以克服这些局限性。