CRCNS Research Proposal: Cortico-amygdalar substrates of adaptive learning

CRCNS 研究提案：适应性学习的皮质杏仁核基质

• 批准号: 9982289
• 负责人: Alicia Izquierdo
• 金额: $ 33.06万
• 依托单位: DARTMOUTH COLLEGE
• 依托单位国家: 美国
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-09-15 至 2023-07-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9982289
• 关键词: Activities of Daily Living; Amygdaloid structure; Area; Behavioral; Brain; Brain region; Calcium; Computer Models; Data; Dissection; Dorsal; Environment; Feedback; Food; Functional Magnetic Resonance Imaging; Grain; Human; Image; Learning; Metaplasia; Modeling; Monitor; Monkeys; Neurons; Organism; Outcome; Pathway interactions; Property; Rattus; Research Proposals; Resolution; Reversal Learning; Rewards; Rodent; Role; Schedule; Signal Transduction; Speed; Stimulus; Study models; Synapses; System; Techniques; Testing; Time; Uncertainty; Update; adaptive learning; cell type; experimental study; learning strategy; network models; neuromechanism; predictive modeling; preference; response; statistics; visual stimulus; word learning

PROJECT DESCRIPTION 1. BACKGROUND AND SIGNIFICANCE Learning from feedback in the real w'orld is limited by constant fluctuations in reward outcomes associated with choosing certain options or actions. Some of these fluctuations are caused by fundamental changes in the reward values of those options/actions that necessitate dramatic adjustments to the current learning strategies, like in epiphany learning or one-shot learning [Chen & Krajbich, 2017; Lee et al. 2015]. Other changes represent inherent stochasticity in an otherwise stable environment and should be tolerated and ignored to maintain stable choice preferences. In other words, learning in dynamic environments is bounded by a tradeoff between being adaptable (i.e. respond quickly to changes in the environment) and being precise (i.e. update slowly after each feedback to be more accurate), which we refer to as the adaptability-precision tradeoff [Farashahi et al., 2017; Khorsand & Soltani, 2017]. Therefore, distinguishing meaningful changes in the environment from natural fluctuations can greatly enhance adaptive learning, indicating that adaptive learning depends on interactions between multiple brain areas. To date, most computational models of learning under uncertainty are very high-level and/or descriptive [Behrens et al., 2007; Costa et al., 2015; ligaya, 2016; Jang et al., 2015; Nassar et al., 201 O; Payzan-LeNestour & Bossaerts, 2011] and therefore, do not provide specific testable predictions. On the other hand, neural mechanisms of uncertainty monitoring for adaptive learning have been predominantly investigated in humans, and in a few cases monkeys, both of which are limited in terms of circuit-level manipulations. However, interactions between brain areas unfold on short timescales and can be specific to certain cell types. These properties have severely limited the ability of functional MRI [Logothetis, 2003] or MEG [Dale et al., 2000; Mostert et al., 2015] to reveal the microcircuit mechanisms within brain regions and fine-grained contributions between brain regions. To overcome these limitations and reveal neural mechanisms underlying adaptive learning under uncertainty, we propose a combination of detailed computational modeling, imaging of stable neuronal ensembles, and precise system-level manipulation of interactions between multiple brain areas in rodents. The latter is possible in part due to powerful circuit- dissection techniques in rodents that allow manipulations of genetically-tractable cell types and thus, specific projections between brain regions. Combined with decoding of neuronal activity in cortex and guided by mechanistic computational modeling, this approach enables us to investigate both microcircuit and system-level mechanisms of adaptive learning under uncertainty. We have recently proposed a mechanistic model for adaptive learning under uncertainty [Farashahi et al., 2017]. This model, which we refer to as reward-dependent metaplasticity (ROMP) model, provides a synaptic mechanism for how learning can be self-adjusted to reward statistics in the environment. The model predicts as more time spent in a given environment with a certain reward schedule, the organisms should become less sensitive to feedback that does not support what is learned. This and other predictions of the model were confirmed using a large set of behavioral data in monkeys during a probabilistic reversal learning task [Farashahi et al., 2017]. Although the proposed metaplasticity mechanism enables the model to become more robust against random fluctuations, it also causes the model to not respond quickly to actual changes in the environment. This limitation can be partially mitigated by allowing synapses to become unstable in response to changes in the environment [ligaya, 2016]. Interestingly, in our model, the changes in the activity of neurons that encode reward values can be used by another system to compute volatility in the environment. This signal can be used subsequently to increase the speed of learning when volatility is high, that is, when there is a higher chance of real changes in the environment. We hypothesize that such interactions between value-encoding and uncertainty-monitoring systems can enhance adaptability required in dynamic environments. In addition to this modeling study, we recently have shown that both basolateral amygdala (BLA) and orbitofrontal cortex (OFC) have complementary roles in adaptive value learning under uncertainty in rodents [Stolyarova & Izquierdo, 2017]. In this experiment, rats learned the variance in delays for food rewards associated with different visual stimuli upon selecting between them. We found that OFC is necessary to accurately learn such stimulus-outcome association (in terms of 1 21

项目描述 一、背景及意义 从现实世界的反馈中学习受到奖励结果不断波动的限制 与选择某些选项或操作相关。其中一些波动是由 这些选项/行动的奖励价值发生根本性变化，需要采取戏剧性的行动 调整当前的学习策略，例如顿悟学习或一次性学习[Chen & 克拉伊比奇，2017；李等人。 2015]。其他变化代表了其他方面的固有随机性 稳定的环境，应该被容忍和忽视，以保持稳定的选择偏好。在 换句话说，动态环境中的学习受到适应性之间的权衡的限制 （即快速响应环境的变化）和精确（即每次更新后缓慢更新） 反馈更准确），我们将其称为适应性-精度权衡 [Farashahi et 等，2017；霍尔桑德和索尔塔尼，2017]。因此，区分有意义的变化 来自自然波动的环境可以极大地增强适应性学习，这表明适应性学习 学习取决于多个大脑区域之间的相互作用。 迄今为止，大多数不确定性下的学习计算模型都是非常高级的和/或 描述性的 [Behrens et al., 2007;科斯塔等人，2015；利加亚，2016；张等人，2015；纳萨尔等人， 201 奥； Payzan-LeNestour & Bossaerts, 2011]，因此，不提供具体的可测试 预测。另一方面，自适应学习的不确定性监控神经机制 主要在人类身上进行了研究，也有少数情况下在猴子身上进行了研究，这两种动物都 电路级操作方面受到限制。然而，大脑区域之间的相互作用是在 时间尺度短，并且可以针对某些细胞类型。这些特性严重限制了 功能性 MRI [Logothetis, 2003] 或 MEG [Dale et al., 2000； Mostert et al., 2015] 揭示 大脑区域内的微电路机制以及大脑之间的细粒度贡献 地区。为了克服这些限制并揭示适应性的神经机制 在不确定性下学习，我们提出了详细计算模型的组合， 稳定神经元整体的成像以及相互作用的精确系统级操作 啮齿动物的多个大脑区域之间。后者之所以成为可能，部分原因在于强大的电路 啮齿动物的解剖技术允许操纵遗传易处理的细胞类型，因此， 大脑区域之间的特定投影。结合皮质神经元活动的解码 在机械计算模型的指导下，这种方法使我们能够研究 不确定性下自适应学习的微电路和系统级机制。 我们最近提出了一种不确定性下自适应学习的机制模型 [Farashahi 等人，2017]。这个模型，我们称之为奖赏依赖性化塑性（ROMP） 模型，提供了一种突触机制，用于如何自我调整学习以奖励统计数据 环境。该模型预测在特定环境下花费的时间会增加 奖励计划，生物体应该对不支持的反馈变得不太敏感 学到了什么。该模型的这一预测和其他预测已通过大量数据得到证实 概率逆转学习任务期间猴子的行为数据 [Farashahi et al., 2017]。 尽管所提出的形塑性机制使模型能够变得更加鲁棒 随机波动，也会导致模型不能快速响应实际变化 环境。这种限制可以通过允许突触变得不稳定来部分缓解。 对环境变化的响应[ligaya，2016]。有趣的是，在我们的模型中，变化 编码奖励值的神经元的活动可以被另一个系统用来计算 环境的波动。随后可以使用该信号来提高学习速度 当波动性较高时，即环境发生实际变化的可能性较高时。我们 假设价值编码和不确定性监控系统之间的这种相互作用可以 增强动态环境中所需的适应性。 除了这项建模研究之外，我们最近还发现基底外侧杏仁核 （BLA）和眶额皮层（OFC）在适应性价值学习中具有互补作用 啮齿动物的不确定性 [Stolyarova & Izquierdo, 2017]。在这个实验中，老鼠了解到了 在选择不同的视觉刺激时，食物奖励的延迟与它们相关。我们 发现 OFC 对于准确学习这种刺激-结果关联是必要的（就 1 21