CRCNS Research Proposal: Cortico-amygdalar substrates of adaptive learning

CRCNS 研究提案：适应性学习的皮质杏仁核基质

• 批准号: 9982289
• 负责人: Alicia Izquierdo
• 金额: $ 33.06万
• 依托单位: DARTMOUTH COLLEGE
• 依托单位国家: 美国
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-15 至 2023-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9982289
• 关键词: Activities of Daily Living; Amygdaloid structure; Area; Behavioral; Brain; Brain region; Calcium; Computer Models; Data; Dissection; Dorsal; Environment; Feedback; Food; Functional Magnetic Resonance Imaging; Grain; Human; Image; Learning; Metaplasia; Modeling; Monitor; Monkeys; Neurons; Organism; Outcome; Pathway interactions; Property; Rattus; Research Proposals; Resolution; Reversal Learning; Rewards; Rodent; Role; Schedule; Signal Transduction; Speed; Stimulus; Study models; Synapses; System; Techniques; Testing; Time; Uncertainty; Update; adaptive learning; cell type; experimental study; learning strategy; network models; neuromechanism; predictive modeling; preference; response; statistics; visual stimulus; word learning

PROJECT DESCRIPTION 1. BACKGROUND AND SIGNIFICANCE Learning from feedback in the real w'orld is limited by constant fluctuations in reward outcomes associated with choosing certain options or actions. Some of these fluctuations are caused by fundamental changes in the reward values of those options/actions that necessitate dramatic adjustments to the current learning strategies, like in epiphany learning or one-shot learning [Chen & Krajbich, 2017; Lee et al. 2015]. Other changes represent inherent stochasticity in an otherwise stable environment and should be tolerated and ignored to maintain stable choice preferences. In other words, learning in dynamic environments is bounded by a tradeoff between being adaptable (i.e. respond quickly to changes in the environment) and being precise (i.e. update slowly after each feedback to be more accurate), which we refer to as the adaptability-precision tradeoff [Farashahi et al., 2017; Khorsand & Soltani, 2017]. Therefore, distinguishing meaningful changes in the environment from natural fluctuations can greatly enhance adaptive learning, indicating that adaptive learning depends on interactions between multiple brain areas. To date, most computational models of learning under uncertainty are very high-level and/or descriptive [Behrens et al., 2007; Costa et al., 2015; ligaya, 2016; Jang et al., 2015; Nassar et al., 201 O; Payzan-LeNestour & Bossaerts, 2011] and therefore, do not provide specific testable predictions. On the other hand, neural mechanisms of uncertainty monitoring for adaptive learning have been predominantly investigated in humans, and in a few cases monkeys, both of which are limited in terms of circuit-level manipulations. However, interactions between brain areas unfold on short timescales and can be specific to certain cell types. These properties have severely limited the ability of functional MRI [Logothetis, 2003] or MEG [Dale et al., 2000; Mostert et al., 2015] to reveal the microcircuit mechanisms within brain regions and fine-grained contributions between brain regions. To overcome these limitations and reveal neural mechanisms underlying adaptive learning under uncertainty, we propose a combination of detailed computational modeling, imaging of stable neuronal ensembles, and precise system-level manipulation of interactions between multiple brain areas in rodents. The latter is possible in part due to powerful circuit- dissection techniques in rodents that allow manipulations of genetically-tractable cell types and thus, specific projections between brain regions. Combined with decoding of neuronal activity in cortex and guided by mechanistic computational modeling, this approach enables us to investigate both microcircuit and system-level mechanisms of adaptive learning under uncertainty. We have recently proposed a mechanistic model for adaptive learning under uncertainty [Farashahi et al., 2017]. This model, which we refer to as reward-dependent metaplasticity (ROMP) model, provides a synaptic mechanism for how learning can be self-adjusted to reward statistics in the environment. The model predicts as more time spent in a given environment with a certain reward schedule, the organisms should become less sensitive to feedback that does not support what is learned. This and other predictions of the model were confirmed using a large set of behavioral data in monkeys during a probabilistic reversal learning task [Farashahi et al., 2017]. Although the proposed metaplasticity mechanism enables the model to become more robust against random fluctuations, it also causes the model to not respond quickly to actual changes in the environment. This limitation can be partially mitigated by allowing synapses to become unstable in response to changes in the environment [ligaya, 2016]. Interestingly, in our model, the changes in the activity of neurons that encode reward values can be used by another system to compute volatility in the environment. This signal can be used subsequently to increase the speed of learning when volatility is high, that is, when there is a higher chance of real changes in the environment. We hypothesize that such interactions between value-encoding and uncertainty-monitoring systems can enhance adaptability required in dynamic environments. In addition to this modeling study, we recently have shown that both basolateral amygdala (BLA) and orbitofrontal cortex (OFC) have complementary roles in adaptive value learning under uncertainty in rodents [Stolyarova & Izquierdo, 2017]. In this experiment, rats learned the variance in delays for food rewards associated with different visual stimuli upon selecting between them. We found that OFC is necessary to accurately learn such stimulus-outcome association (in terms of 1 21

项目描述 1。背景和意义 在真实世界中从反馈中学习受到奖励结果不断波动的限制 与选择某些选项或动作相关。这些波动中的一些是由 这些选项/行动的奖励价值的根本变化，这些选择/行动需要引人注目 调整当前的学习策略，例如顿悟学习或一声学习[Chen＆ 克拉比奇，2017年； Lee等。 2015]。其他变化代表否则的固有随机性 稳定的环境，应被容忍和忽略以保持稳定的选择偏好。在 换句话说，在动态环境中的学习是由适应能力的权衡取决于 （即对环境变化的变化迅速响应）和精确（即每次慢慢更新 反馈更准确），我们称之为适应性的权衡[farashahi et eToff Al。，2017年； Khorsand＆Soltani，2017年]。因此，区分有意义的变化 自然波动的环境可以极大地增强自适应学习，表明适应性 学习取决于多个大脑区域之间的相互作用。 迄今为止，不确定性下的大多数学习计算模型都是非常高级和/或 描述性[Behrens等，2007; Costa等人，2015年； Ligaya，2016年； Jang等人，2015年；纳萨尔等人， 201 o; Payzan-Lenestour＆Bossaerts，2011年]，因此不提供特定的可测试 预测。另一方面，自适应学习的不确定性监测的神经机制 在人类和少数情况下，主要研究了猴子 在电路级操作方面有限。但是，大脑区域之间的相互作用展开 时间尺度短，可以特定于某些单元格。这些特性严重限制了 功能MRI的能力[Logothetis，2003]或MEG [Dale等，2000; Mostert等，2015]揭示 大脑区域内的微电路机制和大脑之间的细粒贡献 地区。克服这些局限性并揭示自适应的神经机制 在不确定性下学习，我们提出了详细的计算建模的组合， 稳定的神经元合奏的成像，以及对相互作用的精确系统级操纵 在啮齿动物的多个大脑区域之间。后者可能部分是由于强大的电路 - 啮齿动物中的解剖技术允许操纵可摘要的细胞类型，因此 大脑区域之间的特定预测。与皮质中神经元活性的解码结合 在机械计算建模的指导下，这种方法使我们能够研究这两者 在不确定性下自适应学习的微电路和系统级机制。 我们最近提出了一个在不确定性下自适应学习的机械模型 [Farashahi等，2017]。这个模型，我们称之为奖励依赖性的塑性（ROMP） 模型，提供了一种突触机制，用于如何自我调整学习以奖励统计数据 环境。该模型预测，在给定环境中花费的时间更多 奖励时间表，生物应该对不支持的反馈敏感 学到了什么。使用大量的模型和其他预测 在概率逆转学习任务中，猴子中的行为数据[Farashahi等，2017]。 尽管提出的跨塑性机制使模型能够变得更加强大 随机波动，这也导致该模型不对实际变化的实际变化迅速响应 环境。通过允许突触变得不稳定，可以部分缓解这种限制 对环境变化的反应[Ligaya，2016年]。有趣的是，在我们的模型中，变化 另一个系统可以使用编码奖励值的神经元的活动来计算 环境的波动。该信号随后可用于提高学习速度 当波动率很高时，也就是说，当环境发生真正变化的可能性更高时。我们 假设价值编码和不确定性监控系统之间的这种相互作用可以 在动态环境中提高适应性。 除了这项建模研究外，我们最近还表明，这两个基底外侧杏仁核 （BLA）和眶额皮层（OFC）在自适应价值学习中具有互补的作用 啮齿动物的不确定性[Stolyarova＆izquierdo，2017年]。在此实验中，大鼠了解了 选择与不同的视觉刺激相关的食物奖励延迟。我们 发现OFC对于准确学习这种刺激结果关联是必要的（就 1 21