Understanding the role of quantitative internal signals in behavioral flexibility

了解定量内部信号在行为灵活性中的作用

基本信息

批准号：
10208212
负责人：
GABY MAIMON
金额：
$ 144.6万
依托单位：
ROCKEFELLER UNIVERSITY
依托单位国家：
美国
项目类别：
财政年份：
2021
资助国家：
美国
起止时间：
2021-04-15 至 2024-03-31
项目状态：
已结题

来源：
https://reporter.nih.gov/project-details/10208212
关键词：
Adaptive Behaviors Anatomy Animal Behavior Animal Model Animals Behavior Behavioral Biological Models Brain Cognition Decision Making Deterioration Drosophila genus Drosophila melanogaster Electrophysiology (science)Environment Face Future Genetic Glean Grant Hand Human Impairment Individual Insecta Laboratories Learning Lobe Mammals Measures Methods Modeling Movement Mushroom Bodies Nature Neurologic Neurons Neuropil Neurosciences Obsessive-Compulsive Disorder Output Pattern Physiological Preparation Ramp Reflex action Response to stimulus physiology Rewards Role Signal Transduction Social Interaction Societies Structure System Testing Therapeutic Thinking Time Update Wasps Work autism spectrum disorder base cellular imaging classical conditioning cognitive neuroscience conditioning dopaminergic neuron egg expectation experience experimental study flexibility fly insight millisecond nervous system disorder neural circuit neurophysiology neurotransmission nonhuman primate novel therapeutic intervention relating to nervous system sensory input sensory stimulus visual learning visual stimulus

项目摘要

Project Summary / Abstract This grant focuses on how very recent experiences––over the past few seconds to minutes––allow brains to update expectations about the world and then use these expectations to guide behavior. The ability to flexibly adjust one's course of action in this manner is a hallmark of adaptive human behavior. At the neural level, relevant cellular-activity correlates have been described in non-human primates and other vertebrate model systems. For example, ramping neural activity has been observed in the few hundred milliseconds, or seconds, leading up to behavioral decisions and the rate of rise of these ramps tracks the gradual accumulation of information relevant for the decision being made. Ramping activity is thus a correlate of an increasing expectation that an important decision needs to be made and the moment at which the ramp reaches a threshold level typically signifies when a final decision is taken. Another salient correlate of internal expectations are reward-prediction error signals: bursts of dopamine-neuron activity when an animal receives an unexpected reward or a reward is surprisingly omitted. Reward-prediction error signals seem well poised to adjust animal and human behavior based on learned expectations. A clearer picture of how quantitative internal signals––like ramping and reward-prediction error activity––contribute to behavioral flexibility would be an important step forward for cognitive neuroscience. Here, we propose to develop two new behavioral tasks in tethered Drosophila, where we can perform simultaneous neurophysiology. Our first aim is to use one of these tasks to test the hypothesis that ramping neural signals are fundamental in forming behavioral decisions over tens-of-seconds to minutes timescales in ethologically relevant contexts, rather than just on much shorter timescales and in laboratory defined tasks (as has been shown to date). Such a discovery would argue that expectations built over minutes in real-world conditions are ultimately fed into slowly ramping neuronal signals so as to guide natural decision-making. Our second aim is to discover reward-prediction error signals in fruit flies actively performing a trial-by-trial conditioning task and to define a circuit mechanism through which such signals allow brains to form quantitatively precise expectations––updated on a trial-by-trial basis––on the likelihood of rewards arriving or not arriving in the near future. Such discoveries in a genetically tractable model will inform our thinking on how our brains generate expectations that allow for flexible, adaptive behaviors, ultimately informing new therapeutic approaches to neurological conditions in which flexibility is impaired, such as obsessive-compulsive disorder.

项目概要/摘要这笔资助的重点是最近的经历——过去几秒钟到几分钟——如何让大脑更新对世界的期望，然后利用这些期望来指导行为的能力。以这种方式调整自己的行动方针是人类适应性行为的标志。相关的细胞活动相关性已在非人类灵长类动物和其他脊椎动物模型中得到描述例如，在几百毫秒或几秒内观察到神经活动的增加。导致行为决策，并且这些斜坡的上升率跟踪了逐渐积累的因此，与正在做出的决策相关的信息与增加的活动相关。期望需要做出重要决定以及坡道达到某个时刻阈值水平通常表示何时做出最终决定。期望是奖励预测错误信号：当动物接受奖励时，多巴胺神经元活动爆发意外的奖励或奖励被意外地遗漏了——奖励预测错误信号似乎已经做好了准备。根据习得的期望调整动物和人类的行为更清楚地了解如何定量。内部信号——如斜坡和奖励预测错误活动——有助于行为灵活性认知神经科学向前迈出的重要一步，我们建议在其中开发两项新的行为任务。拴系果蝇，我们可以在那里进行同步神经生理学研究，我们的首要目标是使用其中一个。测试以下假设的任务：斜坡神经信号对于形成行为决策至关重要在道德相关的背景下，从几十秒到几分钟的时间尺度，而不仅仅是更短的时间尺度时间尺度和实验室定义的任务（正如迄今为止所显示的那样）。在现实条件下几分钟内建立的期望最终会被反馈到缓慢上升的神经信号中我们的第二个目标是发现水果中的奖励预测误差信号。积极地执行逐个试验的调节任务，并定义一个电路机制，通过该机制信号使大脑能够形成数量上精确的预期——在逐次试验的基础上进行更新—— 奖励在不久的将来到达或不到达的可能性。将告诉我们关于我们的大脑如何产生允许灵活、适应性行为的期望的想法，最终为灵活性受损的神经系统疾病提供新的治疗方法，例如作为强迫症。

项目成果

期刊论文数量（2）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

An internal expectation guides Drosophila egg-laying decisions.

内部期望指导果蝇的产卵决定。

DOI：
发表时间：
2022-10-28
期刊：
影响因子：
13.6
作者：
Vijayan, Vikram;Wang, Zikun;Chandra, Vikram;Chakravorty, Arun;Li, Rufei;Sarbanes, Stephanie L;Akhlaghpour, Hessameddin;Maimon, Gaby
通讯作者：
Maimon, Gaby

Reinforcement Learning for Cost-Aware Markov Decision Processes

成本感知马尔可夫决策过程的强化学习