Computational foundations of active visual sensing

主动视觉传感的计算基础

• 批准号: 10431247
• 负责人: Mackenzie Weygandt Mathis
• 金额: $ 478.14万
• 依托单位: BAYLOR COLLEGE OF MEDICINE
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-03-15 至 2025-02-28
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10431247
• 关键词: 3-Dimensional; Animal Behavior; Animals; Area; Augmented Reality; Behavior; Behavioral; Behavioral Model; Brain; Clutterings; Code; Complex; Cues; Data; Decision Making; Discrimination; Electrophysiology (science); Environment; Eye; Foundations; Future; Genetic; Goals; Head; Image; Intelligence; Life; Limb structure; Machine Learning; Measures; Mediating; Methods; Modeling; Movement; Mus; Neural Network Simulation; Neurosciences; Performance; Physics; Policies; Positioning Attribute; Process; Property; Psyche structure; Published Comment; Publishing; Resolution; Rewards; Risk; Role; Sensory; Stimulus; System; Task Performances; Testing; Time; Training; Uncertainty; Vision; Visual; Visual Cortex; Visual system structure; active vision; base; behavior measurement; behavioral response; brain dysfunction; cell type; cost; deep learning; high dimensionality; in silico; insight; motor control; neural circuit; neural model; novel; object recognition; predictive modeling; relating to nervous system; response; theories; tool; visual control; visual information; visual neuroscience; visual processing; visual stimulus

Abstract Vision is an active process: we move our head and eyes to explore the sensory world. This is particularly important in situations where a stationary view provides limited information, such as when looking for an object that is occluded or obscured, which is common in complex natural scenes. However, our understanding of active vision is limited due to experimental and theoretical challenges, including the difficulty of studying vision in freely moving animals and the lack of formal theoretical frameworks that integrate visual representations with actions. In this team project, we will combine expertise in visual neuroscience, behavior, machine learning, and theory, to determine the behavioral, neural, and computational underpinnings of active sensing. Our approach is based on a new theoretical framework of Bounded Rational Control (BRC), and a behavioral task in which mice perform an object recognition task in the presence of occlusion and image corruptions. To enable active sensing, stimuli in the task are rendered real-time in augmented reality based on the animal's viewpoint. In our first aim, we will develop models of active sensing based on constrained visual representations in BRC. In the second aim, we measure behavioral performance (both correct/incorrect responses and full-body movements) during the task, and in the third aim we will measure neural activity across visual cortical areas during the task. For both Aims 2 and 3, we will fit our models to the corresponding behavioral and neural data, and then perform causal tests of our models by presenting novel stimuli predicted to elicit specific responses from the model. Together, these aims will provide a foundational understanding of active vision in the mouse that will support a subsequent U19 proposal taking advantage of genetic tools to investigate the underlying local and long-range neural circuits.

抽象的 视觉是一个积极的过程：我们移动头和眼睛探索感官世界。尤其是 在固定视图提供有限信息的情况下，例如寻找对象时的重要信息 这是遮挡或遮盖的，这在复杂的自然场景中很常见。但是，我们对活跃的理解 视力由于实验性和理论挑战而受到限制，包括研究自由的视觉的困难 移动动物以及缺乏将视觉表示与动作整合的形式理论框架。 在这个团队项目中，我们将结合视觉神经科学，行为，机器学习和理论方面的专业知识， 确定主动灵敏度的行为，神经和计算基础。我们的方法是基于 在有限理性控制（BRC）的新理论框架以及小鼠执行的行为任务上 在遮挡和图像校正的情况下，对象识别任务。为了使主动传感刺激 在任务中，根据动物的观点将实时渲染在增强现实中。在我们的第一个目标中，我们将 基于BRC中受约束的视觉表示形式开发主动传感的模型。在第二个目标中，我们 在任务期间测量行为绩效（正确/不正确的响应和全身运动）， 在第三个目标中，我们将在任务期间测量各种视觉皮质区域的神经活动。对于两个目标 2和3，我们将使模型拟合到相应的行为和神经数据，然后进行因果测试 通过展示新的刺激，预计该模型会引起该模型的特定反应。在一起，这些 目标将为鼠标中的主动视力提供基本的理解，以支持随后的U19 提案利用遗传工具来研究潜在的局部和远程神经回路。

项目成果

Asymmetric distribution of color-opponent response types across mouse visual cortex supports superior color vision in the sky.

小鼠视觉皮层颜色对抗反应类型的不对称分布支持卓越的天空色觉。

• DOI: 10.1101/2023.06.01.543054
• 发表时间: 2023
• 期刊: bioRxiv : the preprint server for biology
• 作者: Franke,Katrin;Cai,Chenchen;Ponder,Kayla;Fu,Jiakun;Sokoloski,Sacha;Berens,Philipp;Tolias,AndreasS
• 通讯作者: Tolias,AndreasS