CAREER: Continual Learning with Evolving Memory, Soft Supervision, and Cross-Domain Knowledge - Foundational Theory and Advanced Algorithms

职业：利用进化记忆、软监督和跨领域知识进行持续学习——基础理论和高级算法

• 批准号: 2338506
• 负责人: Jie Ding
• 金额: $ 54.44万
• 依托单位: University of Minnesota-Twin Cities
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-09-01 至 2029-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2338506&HistoricalAwards=false
• 关键词: CAREER; Continual; Learning; Evolving; Memory

As the landscape of data and learning environments expands rapidly, the utility of artificial intelligence hinges on its ability to scale and adapt. In response, continual learning is emerging as a promising paradigm to meet this demand. Distinct from other learning paradigms, continual learning emphasizes the ability to maintain performance on previously learned tasks while seamlessly integrating new information. It focuses on rapid adaptation to new environments by appropriately recalling past knowledge and actively seeking side information to accelerate learning and improve accuracy. However, current methods in continual learning are predominantly empirical and lack a clear theoretical foundation, limiting their wider application and hindering further progress. This project aims to bridge this gap by developing a principled framework for continual learning, consisting of novel theoretical insights and practical algorithmic designs. The research outcomes will substantially advance our understanding of continual learning, enrich its algorithmic framework, and provide scalable algorithm packages. The outcomes will be integrated into machine learning courses at all levels, benefiting students across disciplines such as electrical engineering, statistics, and computer science. The project will actively involve underrepresented students in STEM, synergizing research and education at undergraduate and graduate levels, and developing introductory materials for K-12 students through AI apprenticeship programs.The overarching goal of this project is to pioneer a methodological framework of continual learning for autonomous machine learners navigating dynamic data environments. The framework addresses three essential facets of continual learning. First, it will establish mathematical foundations of continual learning and scalable online algorithms to recollect an evolving memory for fast adaptation and capability expansion. Second, it will develop methods to amplify learning efficiency from informative, even less reliable, pseudo labels. Lastly, it will develop approaches for expanding the predictive power with task-specific assistance from peer learners who possess cross-modal data. The developed foundations will lead to novel algorithms to empower intelligent engineering systems to learn more efficiently and effectively in an ever-changing world. The project will also offer deep insights into pivotal machine learning questions, such as understanding the fundamental trade-offs between adaptivity and forgetting in dynamic learning environments, gauging the maximal influences of minor yet jointly dependent data perturbations on modeling results, and quantifying a model's room for improvement when the underlying law of probability remains elusive. Furthermore, these developments are anticipated to have broader impacts, enabling resource-constrained machine learners to continually broaden their learning capabilities in complex practical applications, such as intelligent transportation systems and cellular performance management.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着数据和学习环境的迅速扩展，人工智能的效用取决于其扩展和适应的能力。作为回应，持续学习正在成为满足这一需求的有前景的范式。与其他学习范式不同，持续学习强调在无缝集成新信息的同时保持先前学习任务表现的能力。它注重通过适当回忆过去的知识并积极寻找辅助信息来加速学习并提高准确性，从而快速适应新环境。然而，目前持续学习的方法主要是经验性的，缺乏明确的理论基础，限制了其更广泛的应用并阻碍了进一步的进展。该项目旨在通过开发持续学习的原则框架来弥补这一差距，其中包括新颖的理论见解和实用的算法设计。研究成果将极大地增进我们对持续学习的理解，丰富其算法框架，并提供可扩展的算法包。研究成果将被整合到各级机器学习课程中，使电气工程、统计学和计算机科学等跨学科的学生受益。 该项目将积自主机器学习者在动态数据环境中导航。该框架解决了持续学习的三个基本方面。首先，它将建立持续学习和可扩展在线算法的数学基础，以重新收集不断变化的记忆，以实现快速适应和能力扩展。其次，它将开发方法来提高从信息丰富、甚至不太可靠的伪标签中学习的效率。最后，它将开发通过拥有跨模式数据的同伴学习者针对特定任务的帮助来扩展预测能力的方法。所开发的基础将带来新颖的算法，使智能工程系统能够在不断变化的世界中更高效地学习。该项目还将深入洞察关键的机器学习问题，例如了解动态学习环境中适应性和遗忘之间的基本权衡，衡量微小但共同依赖的数据扰动对建模结果的最大影响，以及量化模型的空间当基本的概率定律仍然难以捉摸时，需要进行改进。此外，这些发展预计将产生更广泛的影响，使资源有限的机器学习者能够不断扩大其在复杂的实际应用中的学习能力，例如智能交通系统和蜂窝性能管理。该奖项反映了 NSF 的法定使命，并被认为是值得的通过使用基金会的智力优势和更广泛的影响审查标准进行评估来提供支持。