Collaborative Research: CIF: Small: Mathematical and Algorithmic Foundations of Multi-Task Learning

协作研究：CIF：小型：多任务学习的数学和算法基础

• 批准号: 2343600
• 负责人: Justin Romberg
• 金额: $ 30万
• 依托单位: Georgia Tech Research Corporation
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-04-01 至 2027-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2343600&HistoricalAwards=false
• 关键词: Collaborative; Research; CIF; Small; Mathematical

Reinforcement learning has emerged as one of the predominant frameworks for real-time decision making and control. It has been the driving force behind several recent high-profile successes of artificial intelligence, enjoying success in areas as diverse as robotic control, wireless communications, and protein structure prediction. While reinforcement learning provides a powerful and flexible framework for learning, data efficiency is a fundamental challenge: this framework is known to require significant computational resources and vast amount of data. This challenge limits the applicability of reinforcement learning and keeps it from being applied in problems where training data and computational power are limited, including important applications such as wildfire monitoring and the search-and-rescue of lost people using unmanned aerial vehicles. This project addresses this challenge by developing new mathematical foundations of multi-task reinforcement learning and novel learning algorithms that require less data in the aggregate when multiple tasks are jointly learned. The project integrates the research findings with rigorous educational and outreach activities, course development, and student training. This project focuses on answering two fundamental questions: (1) Under what conditions does it take less data and computation to learn multiple tasks jointly than it would to learn each task individually? and (2) Can reinforcement learning algorithms learn something meaningful with only a limited amount of data and computation? Our approach to answering these questions draws on techniques from online learning, compressed sensing, and stochastic modeling. In particular, this project covers both offline settings, where the similarity structure between tasks is learned from a given data set, and online settings, where this learned structure is used to efficiently adapt to a new task “on the fly”. The project also addresses the fundamental problem of catastrophic forgetting in multi-task learning, where the learned policy loses the ability to perform a previous task after training for a new task. Over the course of this project, the proposed research activities will be evaluated systematically through a series of simulations of multi-robot navigation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

强化学习已成为实时决策和控制的主要框架之一，它是人工智能最近取得的几项引人注目的成功的驱动力，在机器人控制、无线通信和人工智能等领域取得了成功。蛋白质结构预测。虽然强化学习提供了强大且灵活的学习框架，但数据效率是一个基本挑战：众所周知，该框架需要大量的计算资源和大量数据，这一挑战限制了强化学习的适用性并使其无法发挥作用。应用于训练数据和计算的问题电力有限，包括野火监测和使用无人机搜救失踪人员等重要应用，该项目通过开发多任务强化学习的新数学基础和需要较少数据的新颖学习算法来解决这一挑战。该项目将研究结果与严格的教育和推广活动、课程开发和学生培训相结合。该项目重点回答两个基本问题：（1）在什么条件下需要更少的数据。和计算来学习多个任务联合学习而不是单独学习每个任务？（2）强化学习算法是否可以仅使用有限的数据和计算来学习有意义的东西？我们回答这些问题的方法借鉴了在线学习、压缩感知和随机建模的技术。特别是，该项目涵盖离线设置和在线设置，其中任务之间的相似性结构是从给定的数据集中学习的，而在线设置则使用这种学习的结构来有效地适应“即时”的新任务。灾难性遗忘的根本问题在多任务学习中，学习的策略在接受新任务训练后失去了执行先前任务的能力。在该项目的过程中，将通过一系列多机器人导航模拟来系统地评估所提出的研究活动。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。