Does prefrontal dopamine modulate error signals to optimally adjust learning?

前额叶多巴胺是否会调节错误信号以最佳地调整学习？

基本信息

批准号：
8784640
负责人：
Matthew Nassar
金额：
$ 5.33万
依托单位：
BROWN UNIVERSITY
依托单位国家：
美国
项目类别：
财政年份：
2014
资助国家：
美国
起止时间：
2014-09-01 至 2017-08-31
项目状态：
已结题

来源：
https://reporter.nih.gov/project-details/8784640
关键词：
Accounting Address Affect Animals Anterior Area Attention deficit hyperactivity disorder Automobile Driving Basal Ganglia Behavior Behavioral Biological Biology Brain Brain region Computer Simulation Corpus striatum structure Data Disease Dopamine Dorsal Electroencephalography Elements Employee Strikes Environment Event Failure Feedback Future Genotype Glean Goals Human Lead Learning Learning Module Maintenance Measurement Measures Medial Mediating Mental disorders Modeling N-Methylaspartate Neurons Outcome Pattern Play Prefrontal Cortex Probability Process Protocols documentation Psychological reinforcement Recurrence Rewards Role Schizophrenia Side Signal Transduction Simulate Speed Statistical Models Symptoms System Testing Training Uncertainty Update Ventral Striatum Work abstracting base behavioral pharmacology biophysical model cingulate cortex dopaminergic neuron expectation experience human subject improved insight learned behavior network models neural circuit public health relevance research study response tolcapone tool

项目摘要

DESCRIPTION (provided by applicant): Humans and animals learn to effectively select actions based on past experience. One particular form of reinforcement learning that involves learning from errors in predicting rewards has provided parsimonious explanations for a broad range of learning phenomena. Such models have also provided some insights into the biological machinery involved in this process. Dopamine neurons projecting to the striatum are thought to encode a "reward prediction error" that is used to train neurons in striatum to reflect the value o a particular action in a particular state. While traditional reinforcement learning models are both simple and effective, they fail to capture at least one striking aspect of human learning behavior: that people learn more from some errors than from others. In particular, people tend to be more influenced by errors so salient as to suggest a context change or ones that occur during a moment of uncertainty. This behavior is well described by abstract statistical models of optimal inference, but the mechanisms by which it could be implemented in the brain remain unknown. Here I examine a potential mechanism by which this rational adjustment of learning might be implemented in the brain: anterior cingulate cortex (ACC), an area of the brain important for behavioral updating, might represent the current context and relay this information to neurons in the striatum encoding action values. By representing a new context after a salient error, ACC may drive the activation of a new set of striatal neurons, thereby discarding the irrelevant information gleaned in the previous context and speeding learning. While such a system allows for rational adjustments in learning, it would require very fine tuned control over the maintenance and discarding of context representations in ACC. One potential mechanism by which this fine tuning might be achieved depends on tonic (persisting) dopamine levels in ACC. Higher tonic dopamine levels are thought to improve network stability which, in ACC, might lead to stable context representations and a rate of learning that is optimized for stable environments. The goal of this proposal is to provide me with training in computational modeling, human EEG measurements, and behavioral pharmacology. This training allows me to test the hypothesis that dopaminergic neuromodulatory systems and networks in ACC serve complementary roles in adjusting influence of outcomes on future actions through two specific Aims. The first Aim will examine whether feedback locked EEG responses emanating from ACC reflect rational adjustments of learning, predict behavioral updating, and are consistent with changes to a context representation. The second Aim will examine whether pharmacologically increasing cortical dopamine levels slows learning and mitigates feedback locked EEG responses.

描述（由申请人提供）：人类和动物学会根据过去的经验有效地选择动作。强化学习的一种特殊形式涉及从错误中学习在预测奖励时为广泛的学习现象提供了简约的解释。这样的模型还提供了一些有关此过程中涉及的生物机械的见解。人们认为投射到纹状体的多巴胺神经元被认为编码了一种“奖励预测误差”，用于训练纹状体中的神经元以反映特定状态下的特定作用的值。而传统的强化学习模型都是简单有效地，他们无法捕捉到人类学习行为的至少一个引人注目的方面：人们从某些错误中学到的比从其他错误中学到的更多。特别是，人们倾向于受到如此明显的错误影响，以暗示上下文变化或在不确定性时刻发生的情况。最佳推断的抽象统计模型很好地描述了这种行为，但是在大脑中可以实现的机制仍然未知。在这里，我研究了一种潜在的机制，通过该机制可以在大脑中实施这种合理的学习：前扣带回皮层（ACC），大脑的大脑区域对于行为更新很重要，可以代表当前的环境，并将此信息传递给纹状体编码动作值中的神经元。通过表示明显错误后的新上下文，ACC可能会驱动一组新的纹状体神经元的激活，从而丢弃在上下文中收集的无关信息并加速学习。尽管这样的系统允许在学习中进行合理的调整，但它需要对ACC中上下文表示的维护和丢弃的微调控制。可以实现这种微调的一种潜在机制取决于ACC中多巴胺水平。人们认为，较高的补品多巴胺水平可以提高网络稳定性，在ACC中，这可能会导致稳定的上下文表示和对稳定环境进行优化的学习速率。该建议的目的是为我提供计算建模，人脑电图测量和行为药理学的培训。这项培训使我能够检验以下假设：ACC中的多巴胺能神经调节系统和网络在调整结果对未来对未来行动的影响通过两个特定目标中起互补作用。第一个目标将检查反馈锁定的脑电图响应是否反映了学习的合理调整，预测行为更新，并且与上下文表示的更改一致。第二个目的将检查药理学上增加的皮质多巴胺水平是否会减慢学习速度，并减轻反馈锁定的脑电图反应。