Fast and Flexible Conjunction Coding in Biological and Artificial Vision

生物和人工视觉中快速灵活的联合编码

• 批准号: 10389898
• 负责人: JohnMark Edward Taylor
• 金额: $ 6.68万
• 依托单位: COLUMBIA UNIV NEW YORK MORNINGSIDE
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-04-16 至 2025-04-15
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10389898
• 关键词: Anatomy; Architecture; Artificial Intelligence; Attention; Automobile Driving; Behavior; Biological; Brain; Code; Cognitive; Cognitive Science; Color; Complex; Computer Models; Computer Vision Systems; Crowding; Data Set; Development; Doctor of Philosophy; Engineering; Environment Design; Exhibits; Feedback; Functional Magnetic Resonance Imaging; Future; Glass; Homologous Gene; Human; Image; Industrialization; Influentials; Lateral; Link; Literature; Mentorship; Methodology; Methods; Modeling; Modernization; Names; Natural experiment; Neurologic Deficit; Neurons; Neurosciences; Patients; Population; Prevalence; Psychophysics; Recurrence; Research; Safety; Shapes; Stimulus; Stream; Sum; Syndrome; Techniques; Technology; Testing; Toy; Training; Vision; Visual; Visual Agnosias; Visual system structure; Work; artificial neural network; base; career; cognitive neuroscience; computational neuroscience; deep learning; deep neural network; experience; experimental study; flexibility; human model; improved; neuroimaging; neurophysiology; novel; object recognition; open source; operation; presynaptic; relating to nervous system; response; synergism; theories; vision science; visual search

Project Summary How does the human brain encode visual feature conjunctions? Two influential, yet disparate, research traditions have proposed two different mechanisms. Research in neurophysiology and computer vision has explored static conjunction coding, where feature conjunctions are automatically extracted via a feedforward hierarchy. While efficient, this mechanism may be limited to encoding conjunctions enabled by the hierarchy's learned connectivity. By contrast, research in cognitive psychology has explored dynamic conjunction coding, which sequentially encodes task-relevant conjunctions via attentional selection. While slow, this mechanism is flexible and capable of encoding any feature conjunction. Despite evidence for both mechanisms, their interplay remains unclear. In this project, we leverage advances in deep learning and open neuroimaging datasets to understand how these two mechanisms interact in the human brain, yielding the best of both worlds: fast, but flexible conjunction coding. Through the three complementary Specific Aims, we advance our understanding of these fundamental issues in vision science, and on a practical level, develop approaches that can be used to improve computer vision, aiding the development of useful technologies like autonomous vehicles. In the course of this project, I will master modern approaches in deep learning and computational neuroscience through the mentorship of my sponsor Dr. Kriegeskorte and my co-sponsor Dr. Fusi, equipping me for a career leading a lab that bridges cognitive science, neuroscience, and artificial intelligence. Hypotheses: The human brain implements static conjunction coding via neural populations with “and-like” tuning to feature combinations that emerges via feedforward convergence of neurons tuned to single features, while dy- namic conjunction coding requires recurrent connections. Static conjunction coding can rapidly encode familiar, but not unfamiliar conjunctions, while dynamic conjunction coding can encode any conjunction, but more slowly. Aim 1: We use a massive open-source fMRI dataset to chart the prevalence of static conjunction coding through- out the human visual system, using a method we developed in preliminary analyses. Aim 2: We apply “synthetic neurophysiology” to feedforward artificial neural networks to understand how static conjunction coding emerges in a feedforward hierarchy, beginning by characterizing conjunction-tuned units iden- tified in preliminary analyses, followed by testing influential models for how this occurs in biological vision. Aim 3: We test whether feedforward artificial neural networks exhibit similar limitations on a visual search task compared to known results for human feedforward vision, followed by testing whether introducing recurrent con- nections to networks can overcome these limitations.

项目概要 人类大脑如何编码视觉特征连接？两种有影响力但又截然不同的研究传统 提出了两种不同的机制，神经生理学和计算机视觉研究探索了静态。 连词编码，通过前馈层次结构自动提取特征连词。 为了高效，该机制可能仅限于对由层次结构学习的连接性启用的连词进行编码。 相比之下，认知心理学的研究探索了动态连词编码，它依次 通过注意力选择对与任务相关的连词进行编码，虽然速度较慢，但​​这种机制灵活且功能强大。 尽管有证据表明这两种机制的存在，但它们的相互作用仍不清楚。 在这个项目中，我们利用深度学习和开放神经影像数据集的进步来了解这些 人脑中的两种机制相互作用，产生两全其美的效果：快速而灵活的联合编码。 通过三个互补的具体目标，我们加深了对这些基本问题的理解 视觉科学，并在实践层面上，开发可用于改进计算机视觉的方法，帮助 在这个项目的过程中，我将掌握自动驾驶汽车等有用技术的开发。 在我的赞助商 Dr. 的指导下，深入学习和计算神经科学的现代方法 Kriegeskorte 和我的共同赞助者 Fusi 博士为我提供了领导认知科学实验室的职业生涯， 神经科学和人工智能。 假设：人脑通过具有“类似”调整的神经群体实现静态连接编码 通过调整为单一特征的神经元前馈收敛而出现的特征组合，同时 纳米连词编码需要循环连接，静态连词编码可以快速编码熟悉的、 但不包括陌生的连词，而动态连词编码可以对任何连词进行编码，但速度较慢。 目标 1：我们使用大量开源 fMRI 数据集来绘制静态连词编码的流行情况： 使用我们在初步分析中开发的方法来模拟人类视觉系统。 目标 2：我们应用“合成神经生理学”来前馈人工神经网络，以了解静态 联合编码出现在前馈层次结构中，首先描述联合调整单元的特征 进行初步分析，然后测试有影响力的模型来了解这种情况在生物视觉中是如何发生的。 目标 3：我们测试前馈人工神经网络在视觉搜索任务上是否表现出类似的局限性 与人类前馈视觉的已知结果进行比较，然后测试引入是否经常并发 网络连接可以克服这些限制。