Frontostriatal Rhythms Underlying Reinforcement Learning.

强化学习背后的额纹状体节律。

基本信息

批准号：
10401263
负责人：
Joni D Wallis
金额：
$ 37.97万
依托单位：
UNIVERSITY OF CALIFORNIA BERKELEY
依托单位国家：
美国
项目类别：
财政年份：
2018
资助国家：
美国
起止时间：
2018-07-23 至 2024-04-30
项目状态：
已结题

来源：
https://reporter.nih.gov/project-details/10401263
关键词：
Adopted Animals Area Artificial Arm Brain Brain region Chronic Clinical Clinical Trials Code Communication Computers Corpus striatum structure Decision Making Deep Brain Stimulation Disease Environment Frequencies Functional disorder Future Human Impairment Intervention Learning Measurement Measures Methods Modeling Monkeys Mood Disorders Motor Movement Movement Disorders Nature Neurons Neurosciences Obsessive-Compulsive Disorder Outcome Pattern Pharmacologic Substance Phase Predictive Value Primates Process Psychological reinforcement Research Rodent Rodent Model Role Stimulus Structure Symptoms Testing Theta Rhythm Time Training Translating Work addiction awake base brain machine interface caudate nucleus classical conditioning cognitive process computer framework electrical microstimulation frontal lobe learned behavior microstimulation neurophysiology neuropsychiatric disorder novel therapeutic intervention prevent relating to nervous system response spatiotemporal success time use

项目摘要

Project summary Many neuropsychiatric disorders, including obsessive-compulsive disorder, mood disorders and addiction, involve compromised evaluative and decision-making processes, and maladaptive learned associations. A computational framework that has proven useful in reconciling these different clinical symptoms is reinforcement learning (RL), which dictates how to optimally interact with the environment to maximize potential benefits and mitigate negative consequences. Work over the past several decades has revealed that frontostriatal brain circuits are key mechanistic components of RL. However, the precise nature of the interaction between the frontal cortex and the striatum, and how this communication is achieved, remains unclear. The current proposal focuses on orbitofrontal cortex (OFC) and the caudate nucleus (CN). OFC assigns values to stimuli in our environment, which enables us to make optimal decisions. However, it contains little information about potential motor responses. Our hypothesis is that OFC transfers value information to CN, where it in can be used to select the choice response that will lead to the highest value outcome. We hypothesize that this communication occurs via a phase reset of the local field potential in the theta band at the time of the choice. To test this hypothesis, we will simultaneously record both single neurons and local field potentials from OFC and CN in awake, behaving animals trained to perform an RL task. We will particularly focus on the spatiotemporal dynamics in LFPs between regions, and their relationship to local neuronal computations. We will test the causal role of OFC theta in enabling frontostriatal communication by applying frequency specific microstimulation to OFC while simultaneously recording neural activity in CN. Finally, we will determine whether we can manipulate RL processes using `closed-loop' control, in which we use neural measurements of OFC theta to control the application of microstimulation to CN. Taken together, the results of this proposal will provide convergent correlative and causal evidence for the role of OFC and CN in RL, as well as determine the mechanism by which the two areas communicate. In addition, it will lay the groundwork for future BMI approaches focused on frontolimbic interventions to manipulate maladaptive associations.

项目摘要许多神经精神疾病，包括强迫症，情绪障碍和成瘾，涉及折衷的评估和决策过程，以及适应不良的学术关联。一个被证明可用于调和这些不同临床症状有用的计算框架是强化学习（RL），它决定了如何最佳与环境互动以最大化潜在的好处并减轻负面后果。在过去的几十年中，工作表明额叶脑电路是RL的关键机械组件。但是，额叶皮层与纹状体之间的相互作用，以及如何实现这种交流，仍然存在不清楚。当前的建议集中于眶额皮层（OFC）和尾状核（CN）。 OFC分配值在我们的环境中刺激，这使我们能够做出最佳决定。但是，它很少有关潜在电机响应的信息。我们的假设是OFC将价值信息传递给CN，它可以使用它来选择会导致最高价值结果的选择响应。我们假设这种通信是通过在theta频段中局部场电位的相位重置在选择的时间。为了检验这一假设，我们将同时记录OFC的单个神经元和局部场电位和CN中的CN，经过训练的动物，可以执行RL任务。我们将特别关注区域之间LFP的时空动力学及其与局部神经元计算的关系。我们将通过应用频率特定微刺激为OFC，同时记录CN中的神经活动。最后，我们将确定我们是否可以使用“闭环”控制来操纵RL过程，其中我们使用的神经测量值 OFC theta控制微刺激在CN中的应用。综上所述，该提议的结果将为OFC和CN在RL中的作用提供收敛的相关和因果证据，并确定这两个领域交流的机制。此外，它将为未来的BMI奠定基础侧侧干预措施的方法以操纵不良适应性关联。

项目成果

期刊论文数量（5）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Distributional reinforcement learning in prefrontal cortex.

前额皮质的分布式强化学习。

DOI：
10.1038/s41593-023-01535-w
发表时间：
2024
期刊：
Nature neuroscience
影响因子：
25
作者：
Muller,TimothyH;Butler,JamesL;Veselic,Sebastijan;Miranda,Bruno;Wallis,JoniD;Dayan,Peter;Behrens,TimothyEJ;Kurth-Nelson,Zeb;Kennerley,StevenW
通讯作者：
Kennerley,StevenW

Hybrid dedicated and distributed coding in PMd/M1 provides separation and interaction of bilateral arm signals.