Ensemble neural dynamics in the medial prefrontal cortex underlying cognitive flexibility and reinforcement learning

内侧前额叶皮层的整体神经动力学是认知灵活性和强化学习的基础

• 批准号: 9450063
• 负责人: Surya Ganguli
• 金额: $ 23.76万
• 依托单位: STANFORD UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-26 至 2019-08-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9450063
• 关键词: Address; Algorithms; Animal Behavior; Animals; Architecture; Area; Attention Deficit Disorder; Behavior; Behavioral; Belief; Biological Neural Networks; Brain; Cells; Code; Cognition; Cognitive; Computational algorithm; Data; Data Analyses; Data Set; Decision Making; Dimensions; Disease; Electrophysiology (science); Elements; Employee Strikes; Feedback; Future; Goals; Health; Image; Impairment; Individual; Joints; Learning; Link; Medial; Mental Depression; Methods; Microscope; Modeling; Monitor; Mus; Neural Network Simulation; Neurons; Optics; Outcome; Pattern; Persons; Play; Population; Prefrontal Cortex; Probability; Psychological reinforcement; Pyramidal Cells; Recording of previous events; Recurrence; Research; Rewards; Rodent; Role; Schizophrenia; Signal Transduction; Slice; Specific qualifier value; Structure; Supervision; Technology; Testing; Training; Update; Viral; Work; arm; base; cell cortex; cellular imaging; computer framework; data modeling; expectation; experimental study; flexibility; high dimensionality; microendoscope; mouse model; network models; neural circuit; neuromechanism; novel; relating to nervous system; theories; way finding

Abstract The prefrontal cortex is thought to play a crucial role in cognitive flexibility, in part by updating a person's expectations about the external world and the likely consequences of candidate actions based on the feedback gained from past actions. Deficits in this form of cognition occur in multiple psychiatric conditions in which prefrontal cortex is implicated. Despite much research, the mechanisms by which prefrontal neural circuits contribute to flexible decision-making and switches in cognitive strategy remain unclear. We will examine these issues using reinforcement learning theory, which specifies the optimal strategies for selecting future actions given a subject's past history of actions taken and rewards received. We will first gather the largest set of multi- neuronal recordings ever taken in prefrontal cortex, and then use reinforcement learning theory to analyze the data and deduce the circuit mechanisms by which the prefrontal cortex stores and updates its internal beliefs about the external world and the likely results of future actions. Past studies in behaving animals have found evidence for individual prefrontal cells that, on average, encode information related to cognitive strategy and action selection, but with limited data it has not been possible to identify how prefrontal circuits maintain and update this information over the course of multiple decisions, actions and outcomes. To collect sufficient data and create better models of prefrontal circuits, we will use a miniature microscope enabling us to monitor large neural ensembles in active mice. Our goals are to: (1) Develop and validate an experimental paradigm for imaging the concurrent dynamics of hundreds of prefrontal cells in mice flexibly switching between two different strategies of spatial navigation. Our pilot data show mice can perform the task well, that prefrontal activity is crucial for strategy-switching, and that prefrontal cortex contains cells whose dynamics appear to signal estimates of the optimal strategy. We will verify mice can follow bona fide navigation strategies and not just memorize spatial paths that yield reward. We will also confirm the prefrontal cells stay healthy and have normal activity patterns throughout the multi-day experiment. (2) Use reinforcement learning theory to analyze our large datasets and create neural circuit models of how prefrontal cortex stores and updates its beliefs to guide future actions. Using the theory we will first create observer-actor models of mouse behavior. We will then apply supervised and unsupervised methods of data analysis to assess whether prefrontal neural ensembles encode task-related, abstract variables such as belief and value. Using our observer-actor models and analyses of neural dynamics, we will train recurrent neural network models to solve the strategy-switching task. The resulting circuit models of reinforcement learning will then yield testable predictions about how the mice and prefrontal cells should behave when we modify the task. Overall, our study will address key unanswered questions about prefrontal function and seeks to attain a mechanistic understanding of how prefrontal circuits contribute to flexible decision-making.

抽象的 前额叶皮层被认为在认知灵活性方面发挥着至关重要的作用，部分是通过更新一个人的 对外部世界的期望以及基于反馈的候选人行动可能产生的后果 从过去的行动中获得的。这种形式的认知缺陷发生在多种精神疾病中，其中 前额皮质也受到牵连。尽管进行了大量研究，但前额神经回路的机制 有助于灵活决策和认知策略转换仍不清楚。我们将检查这些 使用强化学习理论的问题，该理论指定了选择未来行动的最佳策略 给定受试者过去采取的行动和收到的奖励的历史。我们首先会收集最大的一组多 前额皮质中的神经元记录，然后使用强化学习理论来分析 数据并推断出前额叶皮层存储和更新其内部信念的电路机制 关于外部世界以及未来行动的可能结果。 过去对行为动物的研究发现了单个前额叶细胞的证据，平均而言， 对与认知策略和行动选择相关的信息进行编码，但由于数据有限，尚未实现 可以确定前额叶回路如何在多个过程中维护和更新此信息 决定、行动和结果。为了收集足够的数据并创建更好的前额叶回路模型，我们 将使用微型显微镜，使我们能够监测活跃小鼠的大型神经群。我们的目标是： (1) 开发并验证一个实验范例，用于对数百个并发动态进行成像 小鼠的前额细胞在两种不同的空间导航策略之间灵活切换。我们的试点数据 表明小鼠可以很好地执行任务，前额叶活动对于策略转换至关重要，并且前额叶 皮层包含的细胞的动态似乎可以发出对最佳策略的估计的信号。我们将验证小鼠 可以遵循真正的导航策略，而不仅仅是记住产生奖励的空间路径。我们也会 确认前额叶细胞在多天的实验中保持健康并具有正常的活动模式。 (2) 使用强化学习理论来分析我们的大型数据集并创建神经回路模型 前额皮质存储并更新其信念以指导未来的行动。使用我们首先创建的理论 小鼠行为的观察者-行动者模型。然后我们将应用有监督和无监督的数据方法 分析评估前额叶神经元是否编码与任务相关的抽象变量，例如信念 和价值。使用我们的观察者-行动者模型和神经动力学分析，我们将训练循环神经网络 网络模型来解决策略切换任务。由此产生的强化学习电路模型将 然后得出关于当我们修改任务时小鼠和前额叶细胞应该如何表现的可测试预测。 总体而言，我们的研究将解决有关前额叶功能的关键未解答问题，并力求实现 对前额叶回路如何有助于灵活决策的机械理解。