Neuronal mechanisms of model-based learning

基于模型的学习的神经机制

• 批准号: 10722261
• 负责人: Joni D Wallis
• 金额: $ 57.79万
• 依托单位: UNIVERSITY OF CALIFORNIA BERKELEY
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-08 至 2028-07-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10722261
• 关键词: Affect; Algorithms; Anxiety; Area; Behavioral; Brain; Cells; Code; Cognitive; Communication; Complex; Compulsive Behavior; Computer Models; Decision Making; Devices; Disease; Electric Stimulation; Endowment; Environment; Functional disorder; Future; Goals; Grant; Graph; Hippocampus; Impairment; Learning; Location; Major Depressive Disorder; Maps; Mental Depression; Modeling; Monkeys; Neurons; Pathway interactions; Patients; Pharmaceutical Preparations; Play; Post-Traumatic Stress Disorders; Predictive Value; Prevalence; Probability; Process; Property; Psychiatry; Research; Rewards; Role; Schizophrenia; Severities; Specific qualifier value; Symptoms; Testing; Therapeutic; Therapeutic Intervention; Theta Rhythm; Update; awake; environmental change; falls; flexibility; frontal lobe; knowledge graph; learning algorithm; maladaptive behavior; microstimulation; neural; neural circuit; neuromechanism; neurophysiology; neuropsychiatric disorder; novel; response; spatiotemporal

The field of computational psychiatry seeks to understand the symptoms and causes of neuropsychiatric diseases as dysfunctional learning processes. The learning algorithms used by the brain fall along a continuum between two extremes. At one end of the continuum is model-free learning, an automatic process that relies on trial-and-error, storing the values of past actions, and inflexibly repeating those actions that led to higher values. On the other end is model-based learning, which generates predictions via a computationally expensive, deliberative process that models the environment, which endows flexibility to respond to environmental changes. Dysfunction of these algorithms can produce maladaptive behaviors. For example, compulsive behavior is argued to arise from disruption of model-based learning, which biases patients towards more inflexible model-free learning mechanisms. Although a great deal of progress has been made in understanding the neural mechanisms underlying model-free learning, we have limited understanding of how the brain uses models to generate reward predictions. The grant aims to test the hypothesis that interactions between hippocampus (HPC) and orbitofrontal cortex (OFC) implement model-based learning. Specifically, we predict that HPC is responsible for constructing a cognitive map that instantiates a neural representation of behavioral tasks, and OFC is responsible for using the cognitive map to generate reward predictions that can be used to generate flexible decision-making. The current grant will test key predictions of this hypothesis. Our first aim uses a novel task that temporally separates the presentation of information about states and values. We will use high-channel count recordings from HPC and OFC and closed-loop microstimulation to examine how the putative HPC state representation affects the coding of value in OFC. In addition, we will examine whether this interaction occurs through the synchronization of the theta rhythm between the two areas. In the second aim, we will examine how a more complex map involving multiple distinct states might be used to enable rapid readjustments to reward changes. Dysfunction of pathways between HPC and frontal cortex are implicated in several neuropsychiatric disorders, including schizophrenia, major depression, and post-traumatic stress disorder. Medication-based treatments have failed to show significant reduction in the prevalence or severity of these disorders. An alternative approach is to use electrical stimulation, but to date this has also yielded mixed results. Our goal is to develop more sophisticated devices that will interact with neural circuits in a more principled way to treat neuropsychiatric disorders, such as using neural activity to detect symptoms and microstimulation to intervene. An impediment to this approach is that the neural coding in many of these circuits remains poorly understood. The aim of the current grant is to understand the neuronal properties of HPC and OFC to help lay the groundwork for future potential therapeutic approaches based on closed-loop microstimulation.

计算精神病学领域旨在了解神经精神病学的症状和原因 疾病是功能失调的学习过程。大脑使用的学习算法是一个连续体 两个极端之间。连续体的一端是无模型学习，这是一种依赖于 反复试验，存储过去行动的价值，并不灵活地重复那些导致更高的行动 价值观。另一方面是基于模型的学习，它通过计算生成预测 昂贵的、深思熟虑的环境建模过程，赋予了响应的灵活性 环境变化。这些算法的功能障碍可能会产生适应不良的行为。例如， 强迫行为被认为是由于基于模型的学习的破坏而引起的，这使患者偏向于 更不灵活的无模型学习机制。尽管已经取得了很大进展 尽管我们了解无模型学习背后的神经机制，但我们对其如何进行了解还很有限 大脑使用模型来生成奖励预测。 该拨款旨在检验海马体 (HPC) 和眶额皮质之间相互作用的假设 （OFC）实施基于模型的学习。具体来说，我们预测 HPC 负责构建 认知图实例化行为任务的神经表征，OFC 负责使用 认知图生成奖励预测，可用于生成灵活的决策。这 目前的拨款将测试这一假设的关键预测。我们的第一个目标是使用一项新颖的任务 将有关状态和值的信息的表示分开。我们将使用高通道数录音 从 HPC 和 OFC 以及闭环微刺激来检查假定的 HPC 状态表示如何 影响 OFC 中的值编码。此外，我们将检查这种相互作用是否通过 两个区域之间的 theta 节律同步。在第二个目标中，我们将研究如何更加 涉及多个不同状态的复杂地图可用于实现快速调整以奖励变化。 HPC 和额叶皮层之间的通路功能障碍与多种神经精神疾病有关， 包括精神分裂症、重度抑郁症和创伤后应激障碍。以药物为基础的治疗 未能显示出这些疾病的患病率或严重程度显着降低。另一种选择 方法是使用电刺激，但迄今为止，这也产生了不同的结果。我们的目标是发展 更复杂的设备将以更有原则的方式与神经回路相互作用来治疗 神经精神疾病，例如利用神经活动检测症状和微刺激进行干预。 这种方法的一个障碍是许多这些电路中的神经编码仍然知之甚少。 当前拨款的目的是了解 HPC 和 OFC 的神经元特性，以帮助奠定 为未来基于闭环微刺激的潜在治疗方法奠定了基础。