Using Population Vectors to Understand Visual Working Memory for Natural Stimuli

使用群体向量来理解自然刺激的视觉工作记忆

• 批准号: 10543101
• 负责人: STEVEN J LUCK
• 金额: $ 39.94万
• 依托单位: UNIVERSITY OF CALIFORNIA AT DAVIS
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-01-01 至 2025-12-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10543101
• 关键词: Algorithms; Architecture; Area; Attention; Behavior; Behavioral; Blinking; Brain region; Categories; Classification; Code; Color; Complex; Computer Models; Consensus; Data; Detection; Diagnosis; Disease; Electroencephalography; Event-Related Potentials; Eye Movements; Face; Frequencies; Functional Magnetic Resonance Imaging; Goals; Human; Individual Differences; Learning; Life; Literature; Location; Measures; Memory; Modeling; Nature; Neural Network Simulation; Neurons; Neurosciences; Noise; Pathway interactions; Pattern; Perception; Performance; Persons; Play; Population; Primates; Property; Proxy; Reporting; Research; Role; Sampling; Short-Term Memory; Specific qualifier value; Stimulus; Stream; Structure; Surface; System; Testing; Training; Vision; Visual; Visual Perception; Visual impairment; Vocabulary; Work; base; behavior prediction; convolutional neural network; design; experimental study; feeding; human subject; insight; machine vision; neural; neural patterning; object recognition; response; theories; vector; verbal; visual memory; visual processing; visual search

Visual working memory plays a fundamental role in visual perception and visually guided behavior, and much has been learned about the nature of this memory system by studies using arrays of artificial but easily controlled stimuli (e.g., arrays of colored disks or oriented Gabor patches). However, current quantitative models of visual working memory based on these artificial stimuli cannot be readily extended to the kinds of complex, structured scenes that humans face in daily life. The central goal of the proposed research is to develop and test a new quantitative approach to understanding the representation of complex objects and scenes in working memory, which will lead to a better understanding of real-world vision. In our model, a scene is represented in visual working memory as a noisier version of the pattern of activation that was produced during the perception of that scene. We model this by feeding the scene into a neural network model of the ventral object recognition pathway and using the resulting pattern of activation across the population of units (the population vector) as a model of the working memory representation. We then use this model to make predictions about both behavioral performance and neural activity in human subjects. For example, the change detection task involves presenting a sample scene followed after delay by a test scene and asking subjects to report whether the two scenes are the same or different. We can model this task by feeding the sample and test scenes into the model and computing the distance between the population vectors for the sample and test scenes. In our preliminary data, we find that the distance between the vectors can predict behavioral change detection performance extremely well. Moreover, we find that the vector for the sample scene can predict the pattern of neural activity during the delay between the sample and test scenes (measured via event-related potentials). Note that previous quantitative models of visual working memory cannot make any predictions at all for the natural scenes used in these preliminary studies. We propose testing and extending this model in several ways. First, we will conduct several experiments to assess the ability of the model to predict behavioral performance and neural activity across a broad range of natural stimuli. Second, we will compare the ability of population vectors from different cortical regions (as estimated from the model and from fMRI data) to predict behavioral performance and delay-period activity, providing new insights into the specific brain regions that underlie visual working memory. Third, we will determine whether our model can predict performance in visually guided tasks (e.g., visual search) that rely on visual working memory. Finally, we will assess different versions of our model that implement competing mechanisms for producing capacity limitations, and we will compare the ability of these models to account for behavioral performance for both natural scenes and classic examples of artificial stimuli. Together, these experiments will provide a new and broader understanding of visual working memory.

视觉工作记忆在视觉感知和视觉引导行为中起着基本作用，并且 通过使用人造数组但很容易的研究，对这种记忆系统的性质有了很多了解 受控的刺激（例如，彩色磁盘或定向的Gabor斑块的阵列）。但是，电流定量 基于这些人造刺激的视觉工作记忆模型不能轻易扩展到那种 人类在日常生活中面临的复杂，结构化的场景。拟议研究的核心目标是 开发和测试一种新的定量方法，以理解复杂对象的表示和 工作记忆中的场景将使人们对现实世界的视野有更好的了解。 在我们的模型中，一个场景在视觉工作记忆中表示为模式的嘈杂版本 在对该场景感知期间产生的激活。我们通过将场景喂入一个 腹侧对象识别途径的神经网络模型并使用所得激活模式 在整个单位（人口矢量）中，作为工作记忆表示的模型。我们 然后使用此模型对人类行为表现和神经活动进行预测 主题。例如，更改检测任务涉及在延迟后呈现样本场景，然后 测试场景并要求受试者报告两个场景是相同还是不同。我们可以为此建模 通过将样本和测试场景喂入模型并计算总体之间的距离来任务 样品和测试场景的向量。在我们的初步数据中，我们发现向量之间的距离 可以很好地预测行为变化检测性能。此外，我们发现 样本场景可以预测样本和测试场景之间的延迟期间的神经活动模式 （通过事件相关电位测量）。请注意，以前的视觉工作记忆模型 对于这些初步研究中使用的自然场景，根本无法做出任何预测。 我们建议以多种方式进行测试并扩展此模型。首先，我们将进行几个实验 评估模型预测广泛范围的行为表现和神经活动的能力 自然刺激。其次，我们将比较来自不同皮质区域的人群向量的能力（作为 从模型和fMRI数据估计）以预测行为性能和延迟周期活动， 提供有关视觉工作记忆构成的特定大脑区域的新见解。第三，我们会的 确定我们的模型是否可以预测依赖的视觉指导任务（例如，视觉搜索）中的性能 视觉工作记忆。最后，我们将评估实施竞争的模型的不同版本 产生容量限制的机制，我们将比较这些模型的能力 自然场景和人造刺激的经典例子的行为表现。在一起，这些 实验将为视觉工作记忆提供新的更广泛的了解。