Neuronal mechanisms of model-based learning

基于模型的学习的神经机制

• 批准号: 10722261
• 负责人: Joni D Wallis
• 金额: $ 57.79万
• 依托单位: UNIVERSITY OF CALIFORNIA BERKELEY
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-08 至 2028-07-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10722261
• 关键词: Affect; Algorithms; Anxiety; Area; Behavioral; Brain; Cells; Code; Cognitive; Communication; Complex; Compulsive Behavior; Computer Models; Decision Making; Devices; Disease; Electric Stimulation; Endowment; Environment; Functional disorder; Future; Goals; Grant; Graph; Hippocampus; Impairment; Learning; Location; Major Depressive Disorder; Maps; Mental Depression; Modeling; Monkeys; Neurons; Pathway interactions; Patients; Pharmaceutical Preparations; Play; Post-Traumatic Stress Disorders; Predictive Value; Prevalence; Probability; Process; Property; Psychiatry; Research; Rewards; Role; Schizophrenia; Severities; Specific qualifier value; Symptoms; Testing; Therapeutic; Therapeutic Intervention; Theta Rhythm; Update; awake; environmental change; falls; flexibility; frontal lobe; knowledge graph; learning algorithm; maladaptive behavior; microstimulation; neural; neural circuit; neuromechanism; neurophysiology; neuropsychiatric disorder; novel; response; spatiotemporal

The field of computational psychiatry seeks to understand the symptoms and causes of neuropsychiatric diseases as dysfunctional learning processes. The learning algorithms used by the brain fall along a continuum between two extremes. At one end of the continuum is model-free learning, an automatic process that relies on trial-and-error, storing the values of past actions, and inflexibly repeating those actions that led to higher values. On the other end is model-based learning, which generates predictions via a computationally expensive, deliberative process that models the environment, which endows flexibility to respond to environmental changes. Dysfunction of these algorithms can produce maladaptive behaviors. For example, compulsive behavior is argued to arise from disruption of model-based learning, which biases patients towards more inflexible model-free learning mechanisms. Although a great deal of progress has been made in understanding the neural mechanisms underlying model-free learning, we have limited understanding of how the brain uses models to generate reward predictions. The grant aims to test the hypothesis that interactions between hippocampus (HPC) and orbitofrontal cortex (OFC) implement model-based learning. Specifically, we predict that HPC is responsible for constructing a cognitive map that instantiates a neural representation of behavioral tasks, and OFC is responsible for using the cognitive map to generate reward predictions that can be used to generate flexible decision-making. The current grant will test key predictions of this hypothesis. Our first aim uses a novel task that temporally separates the presentation of information about states and values. We will use high-channel count recordings from HPC and OFC and closed-loop microstimulation to examine how the putative HPC state representation affects the coding of value in OFC. In addition, we will examine whether this interaction occurs through the synchronization of the theta rhythm between the two areas. In the second aim, we will examine how a more complex map involving multiple distinct states might be used to enable rapid readjustments to reward changes. Dysfunction of pathways between HPC and frontal cortex are implicated in several neuropsychiatric disorders, including schizophrenia, major depression, and post-traumatic stress disorder. Medication-based treatments have failed to show significant reduction in the prevalence or severity of these disorders. An alternative approach is to use electrical stimulation, but to date this has also yielded mixed results. Our goal is to develop more sophisticated devices that will interact with neural circuits in a more principled way to treat neuropsychiatric disorders, such as using neural activity to detect symptoms and microstimulation to intervene. An impediment to this approach is that the neural coding in many of these circuits remains poorly understood. The aim of the current grant is to understand the neuronal properties of HPC and OFC to help lay the groundwork for future potential therapeutic approaches based on closed-loop microstimulation.

计算精神病学领域试图了解神经精神病的症状和原因 疾病作为功能失调的学习过程。大脑使用的学习算法沿连续 在两个极端之间。连续体的一端是无模型学习，这是一个依赖的自动过程 反复试验，存储过去的行动的价值，并僵化地重复那些导致更高的行动 值。另一方面是基于模型的学习，该学习通过计算生成预测 建模环境的昂贵，审议的过程，它赋予了响应的灵活性 环境变化。这些算法的功能障碍会产生适应不良的行为。例如， 强迫行为是由基于模型的学习的中断引起的，这使患者偏向于 更僵化的无模型学习机制。尽管在 了解无模型学习的神经机制，我们对如何了解 大脑使用模型来产生奖励预测。 该赠款旨在检验海马（HPC）和眶额皮质之间相互作用的假设 （OFC）实施基于模型的学习。具体来说，我们预测HPC负责构建 实例化行为任务的神经表示的认知图，OFC负责使用 认知图生成可用于产生灵活决策的奖励预测。这 当前赠款将检验该假设的关键预测。我们的第一个目标使用了一项新任务， 分开有关状态和价值观的信息的介绍。我们将使用高通道计数录音 从HPC和OFC以及闭环微刺激来检查假定的HPC状态表示 影响OFC中价值的编码。此外，我们将检查这种相互作用是否通过 这两个区域之间theta节奏的同步。在第二个目标中，我们将研究更多 涉及多个不同状态的复杂地图可用于使快速调整以奖励变化。 HPC和额叶皮质之间的途径功能障碍与几种神经精神疾病有关， 包括精神分裂症，重度抑郁症和创伤后应激障碍。基于药物治疗的治疗 这些疾病的患病率或严重程度未显示出明显降低。另一种 方法是使用电刺激，但迄今为止，这也产生了混合的结果。我们的目标是发展 更复杂的设备将以更有原则的方式与神经回路互动 神经精神疾病，例如使用神经活动来检测症状和微刺激进行干预。 对这种方法的障碍是，许多这些电路中的神经编码仍然知之甚少。 当前赠款的目的是了解HPC和OFC的神经元特性，以帮助建立 基于闭环微刺激的未来潜在治疗方法的基础。