Collaborative Research: CIF: Small: Mathematical and Algorithmic Foundations of Multi-Task Learning

协作研究：CIF：小型：多任务学习的数学和算法基础

• 批准号: 2343600
• 负责人: Justin Romberg
• 金额: $ 30万
• 依托单位: Georgia Tech Research Corporation
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-04-01 至 2027-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2343600&HistoricalAwards=false
• 关键词: Collaborative; Research; CIF; Small; Mathematical

Reinforcement learning has emerged as one of the predominant frameworks for real-time decision making and control. It has been the driving force behind several recent high-profile successes of artificial intelligence, enjoying success in areas as diverse as robotic control, wireless communications, and protein structure prediction. While reinforcement learning provides a powerful and flexible framework for learning, data efficiency is a fundamental challenge: this framework is known to require significant computational resources and vast amount of data. This challenge limits the applicability of reinforcement learning and keeps it from being applied in problems where training data and computational power are limited, including important applications such as wildfire monitoring and the search-and-rescue of lost people using unmanned aerial vehicles. This project addresses this challenge by developing new mathematical foundations of multi-task reinforcement learning and novel learning algorithms that require less data in the aggregate when multiple tasks are jointly learned. The project integrates the research findings with rigorous educational and outreach activities, course development, and student training. This project focuses on answering two fundamental questions: (1) Under what conditions does it take less data and computation to learn multiple tasks jointly than it would to learn each task individually? and (2) Can reinforcement learning algorithms learn something meaningful with only a limited amount of data and computation? Our approach to answering these questions draws on techniques from online learning, compressed sensing, and stochastic modeling. In particular, this project covers both offline settings, where the similarity structure between tasks is learned from a given data set, and online settings, where this learned structure is used to efficiently adapt to a new task “on the fly”. The project also addresses the fundamental problem of catastrophic forgetting in multi-task learning, where the learned policy loses the ability to perform a previous task after training for a new task. Over the course of this project, the proposed research activities will be evaluated systematically through a series of simulations of multi-robot navigation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

强化学习已成为实时决策和控制的主要框架之一。这一直是最近几项人工智能成功成功的推动力，在机器人控制，无线通信和蛋白质结构预测的潜水区中取得了成功。尽管增强学习为学习提供了一个强大而灵活的框架，但数据效率是一个基本挑战：已知该框架需要大量的计算资源和大量数据。这项挑战限制了强化学习的适用性，并防止将其应用于训练数据和计算能力有限的问题，包括诸如野火监控等重要应用以及使用无人驾驶飞机的失落人员的搜索搜索。该项目通过开发多任务增强学习和新颖学习算法的新数学基础来解决这一挑战，这些学习算法需要在共同学习多个任务时需要较少的总数据。该项目将研究结果与严格的教育和外展活动，课程发展和学生培训相结合。该项目重点是回答两个基本问题：（1）在什么条件下，共同学习多个任务的数据和计算要比单独学习每个任务所需的数据和计算要少？ （2）仅通过有限的数据和计算，增强学习算法可以学习有意义的东西吗？我们回答这些问题的方法取决于在线学习，压缩敏感性和随机建模中的技术。特别是，该项目涵盖了两个离线设置，其中任务之间的相似性结构是从给定的数据集和在线设置中学到的，在该设置中，该学习的结构用于“即时”“飞行”。还解决了多任务学习中灾难性遗忘的基本问题，在培训新任务后，学会的政策失去了执行先前任务的能力。在该项目的整个过程中，拟议的研究活动将通过一系列多机器人导航进行系统评估。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子和更广泛的影响审查标准来通过评估来评估。