Identifying mechanisms of multisensory memory using virtual reality and fMRI

使用虚拟现实和功能磁共振成像识别多感觉记忆的机制

• 批准号: 10676058
• 负责人: Shea Duarte
• 金额: $ 4.03万
• 依托单位: UNIVERSITY OF CALIFORNIA AT DAVIS
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-10 至 2026-12-11
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10676058
• 关键词: Address; Affect; Behavior; Behavioral; Binding; Brain; Brain region; Canis familiaris; Characteristics; Cognitive; Collection; Data; Environment; Episodic memory; Event; Familiarity; Functional Magnetic Resonance Imaging; Goals; Hearing; Hippocampus; Human; Individual; Knowledge; Literature; Location; Loudness; Medial; Memory; Modeling; Participant; Pattern; Process; Research; Retrieval; Role; Sensory; Squirrel; Stimulus; Temporal Lobe; Tennis; Testing; Time; Visual; Work; animation; court; experience; experimental study; improved; long term memory; member; memory encoding; memory recognition; memory retrieval; multimodality; multisensory; neural; neuromechanism; novel; novel strategies; object recognition; sensory stimulus; sound; spatial memory; spatiotemporal; virtual reality; virtual reality environment; visual stimulus

Project Summary Forming and retrieving memories for objects and events is fundamental to human experience. Complementary mechanisms within the medial temporal lobe (MTL) support the formation and retrieval of memories for objects, such that the perirhinal cortex supports memory for features of individual items, and the hippocampus supports recollection of the object within the context it was originally encountered. Although most research on object memory has been conducted using visual stimuli alone, a small body of research has shown that visual objects encoded along with their characteristic sound (e.g., a dog and a bark) are better remembered later on than objects encoded only visually, or with a meaningless or non-characteristic sound. Despite this evidence and the ubiquity of multimodal stimuli within natural environments, our knowledge of the extent of multisensory influences on memory and its underlying mechanisms is very limited. My recent behavioral work showed that the benefits of multisensory encoding are based on improved recollection of the context in which those objects were encoded, suggesting that multisensory processing uniquely engages mechanisms that bind these objects to surrounding information at encoding. However, it remains unclear what features of the encoding context are better retrieved, and what neural mechanisms are modulated by multisensory processing to allow for this benefit. The proposed research addresses this gap in knowledge by collecting the critical evidence to determine the facets of memory for items and surrounding events that are influenced by multisensory processing, and the brain activity patterns and regions that are involved in retrieving such information. We hypothesize that multisensory processing at encoding improves object memory by increasing the likelihood that the object and context will be bound into an episodic memory to support later recognition of that object, rather than by enhancing memory for the individual object itself. Further, we predict that patterns of activation within the hippocampus during retrieval of multimodal objects will also carry information about the environment in which these items were encoded. To rigorously test the type of memory that multisensory processing impacts, there is a need to assess memory for unimodal and multimodal objects that are embedded within rich, naturalistic spatiotemporal context. To this end, we have developed an immersive, naturalistic encoding task within virtual reality (VR) environments, which contain controlled but animated visual and audiovisual objects. This novel approach will allow us to assess whether multisensory processing influences memory for individual objects alone or if these objects are more readily bound to their surroundings to support memory for events within context (Aim 1). Further, analyses of fMRI data will be used to determine which regions of the MTL and cortex are specifically involved in memories of multisensory objects seen in context (Aim 2). The overarching goal of this project is to advance our knowledge of how episodic memories are formed during naturalistic experiences and this work will contribute to our ability to explain and predict real-world behaviors.

项目概要 形成和检索对物体和事件的记忆是人类经验的基础。补充 内侧颞叶（MTL）内的机制支持物体记忆的形成和检索， 这样，鼻周皮层支持对单个项目特征的记忆，海马体支持 在最初遇到的上下文中回忆对象。尽管大多数研究都是针对对象 记忆仅通过视觉刺激进行，一小部分研究表明视觉对象 与其特征声音（例如，狗和吠声）一起编码，以后会比记住更好 仅以视觉方式编码的对象，或者使用无意义或无特征的声音进行编码的对象。尽管有这些证据和 自然环境中普遍存在的多模式刺激，我们对多感官程度的了解 对记忆及其潜在机制的影响非常有限。我最近的行为研究表明 多感官编码的好处是基于对这些物体所处环境的更好的记忆 被编码，表明多感官处理独特地涉及结合这些物体的机制 编码时周围的信息。然而，目前尚不清楚编码上下文的特征是什么 更好地检索，以及通过多感官处理调节哪些神经机制来实现这一点 益处。拟议的研究通过收集关键证据来解决这一知识差距 确定受多感官影响的物品和周围事件的记忆方面 处理，以及参与检索此类信息的大脑活动模式和区域。我们 假设编码时的多感官处理通过增加可能性来改善对象记忆 对象和上下文将被绑定到情景记忆中以支持以后对该对象的识别， 而不是通过增强对单个对象本身的记忆。此外，我们预测激活模式 在检索多模态物体时，海马体内也会携带有关环境的信息 这些项目被编码在其中。严格测试多感官处理的记忆类型 影响，需要评估嵌入丰富的单模态和多模态物体的记忆， 自然主义时空背景。为此，我们开发了一种沉浸式、自然主义的编码任务 在虚拟现实 (VR) 环境中，其中包含受控但动画的视觉和视听对象。 这种新颖的方法将使我们能够评估多感官处理是否影响个体的记忆 单独的物体，或者这些物体更容易与其周围环境结合以支持事件记忆 在上下文中（目标 1）。此外，功能磁共振成像数据的分析将用于确定 MTL 的哪些区域和 皮层特别参与对上下文中看到的多感官物体的记忆（目标 2）。首要的 该项目的目标是增进我们对自然主义时期情景记忆如何形成的了解 经验和这项工作将有助于我们解释和预测现实世界行为的能力。