The role of distributional reinforcement learning in human neurons during impulsive choices

分布式强化学习在人类神经元冲动选择过程中的作用

基本信息

批准号：
10335061
负责人：
Elliot H Smith
金额：
$ 50.98万
依托单位：
UNIVERSITY OF UTAH
依托单位国家：
美国
项目类别：
财政年份：
2022
资助国家：
美国
起止时间：
2022-02-03 至 2026-12-31
项目状态：
未结题

来源：
https://reporter.nih.gov/project-details/10335061
关键词：
Address Affect Amygdaloid structure Animals Anterior Area Artificial Intelligence Basic Science Behavior Behavior Disorders Behavioral Brain Brain Stem Categories Choice Behavior Code Color Complex Corpus striatum structure Data Decision Making Development Dopamine Dopamine Receptor Epilepsy Exhibits Feedback Future Goals Hippocampus (Brain)Human Impulse Control Disorders Impulsive Behavior Impulsivity Intractable Epilepsy Knowledge Learning Link Mathematics Measures Medical Mental Health Mental disorders Modeling Monitor Mus Neurons Neurosciences Organism Outcome Patients Performance Population Probability Psychiatric therapeutic procedure Psychological reinforcement Research Reversal Learning Rewards Risk Role Signal Transduction Substance Use Disorder Temporal Lobe Testing Time Translating Update Work analog base cingulate cortex dopaminergic neuron expectation experience experimental study human subject improved learning outcome neuromechanism neuropsychiatric disorder neuropsychiatry novel optimism relating to nervous system response

项目摘要

ABSTRACT Recent developments in artificial intelligence and neuroscience have revealed neural codes for reinforcement that represent predictions of a range of possible future reward outcomes, rather than a singular expected value. This distributional reinforcement learning has enabled improved performance of artificial agents and has straightforward implications for numerous neuropsychiatric disorders, particularly impulse control and substance use disorders. This proposal aims to leverage our experience recording neuronal activity from the brains of human neurosurgical patients in order to translate these recordings in a novel research direction: to understand the mechanisms of human choice behavior. We will determine where distributional codes exist in the human prefrontal and mesial temporal cortices, and how those codes are expressed dynamically in time as humans make impulsive choices during the Balloon Analog Risk Task (BART) and a probabilistic reversal learning task. The results of these experiments will have both important basic scientific implications and will begin to address how distributional reinforcement learning in the human brain contributes to impulsive choices. In order to begin translating this new area of knowledge to understand the underpinnings of human decisions, we will first establish the presence of distributional reinforcement learning in four brain areas that comprise a human decision-making circuit: Orbitofrontal Cortex, Anterior Cingulate Cortex, Amygdala, and Hippocampus. Specific Aim 1 will test the three essential predictions of distributional RL: whether populations of neurons in each of these brain areas exhibit 1) asymmetric scaling of reward prediction errors, 2) diverse reversal points, and 3) that prediction error asymmetries and reversal points correlate across neurons. Specific Aim 2 seeks to decode BART reward prediction distributions from neurons in the aforementioned brain areas and determine how changes in BART reward distributions correlate with the propensity to make impulsive choices. Specific Aim 3 will test how diversity in optimism and pessimism in each neuron recorded from the aforementioned brain areas correlates with valuation or devaluation across trials. The completion of these aims will constitute important basic research findings in discovering distributional RL in the human prefrontal and mesial temporal cortices. By uncovering neural population codes that underlie potentially impulsive choices in human decision-making circuits, these experiments also address fundamental neural mechanisms underlying impulsive choices. This issue is central to addressing important problems for contemporary mental health including substance use disorder and a many other neuropsychiatric disorders. These findings will have readily translatable implications for improving targeted electrical therapies for psychiatric disorders.

抽象的人工智能和神经科学的最新发展揭示了强化的神经代码代表对一系列可能的未来奖励结果的预测，而不是单一的期望值。这种分布式强化学习提高了人工智能体的性能，并且对许多神经精神疾病的直接影响，特别是冲动控制和物质使用障碍。该提案旨在利用我们记录大脑神经元活动的经验人类神经外科患者，以便将这些录音转化为一个新的研究方向：理解人类选择行为的机制。我们将确定人类中分布代码的存在位置前额叶和内侧颞叶皮质，以及这些代码如何随着人类的时间动态表达在气球模拟风险任务 (BART) 和概率逆转学习任务中做出冲动选择。这些实验的结果将具有重要的基本科学意义，并将开始解决人脑中的分布式强化学习如何促进冲动选择。为了开始转化这一新的知识领域来理解人类决策的基础，我们将首先在四个大脑区域中确定分布式强化学习的存在，这些区域包括人类决策回路：眶额皮层、前扣带皮层、杏仁核和海马体。具体目标 1 将测试分布 RL 的三个基本预测：神经元群体是否每个大脑区域都表现出 1) 奖励预测误差的不对称缩放，2) 不同的逆转点， 3）预测误差不对称性和反转点在神经元之间相关。具体目标 2 旨在解码上述大脑区域神经元的 BART 奖励预测分布并确定 BART 奖励分配的变化如何与做出冲动选择的倾向相关。具体的目标 3 将测试从上述大脑记录的每个神经元的乐观和悲观多样性如何区域与试验中的估值或贬值相关。这些目标的完成将构成发现分布式强化学习的重要基础研究成果。人类前额叶皮质和内侧颞叶皮质。通过揭示背后的神经群体密码人类决策回路中潜在的冲动选择，这些实验还解决了基本问题冲动选择背后的神经机制。这个问题对于解决重要问题至关重要当代心理健康，包括物质使用障碍和许多其他神经精神疾病。这些发现对于改善精神科靶向电疗法具有易于转化的意义失调。