Collaborative Research: MLWiNS: Dino-RL: A Domain Knowledge Enriched Reinforcement Learning Framework for Wireless Network Optimization

合作研究：MLWiNS：Dino-RL：用于无线网络优化的领域知识丰富的强化学习框架

• 批准号: 2003131
• 负责人: Jing Yang
• 金额: $ 18.16万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-06-01 至 2024-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2003131&HistoricalAwards=false
• 关键词: Collaborative; Research; MLWiNS; Dino; RL

Reinforcement learning (RL) methods have met with renewed interest in recent years for adaptively configuring wireless networks. Despite the promising early results and the conceptual match, many existing approaches do not develop and tailor the RL methods to fit the unique characteristics of wireless networking. The goal of this project is to develop a novel domain knowledge enriched RL framework, or Dino-RL, to address this problem. The Dino-RL framework aims to seamlessly integrate the physical-law based modeling and an abstract episodic memory into the RL process, and has the potential to revamp the operation and management of future wireless networks. Developing this novel technology would also help maintain the nation's continued leadership in wireless technologies and its pipeline of highly qualified engineers. The project pursues synergistic activities for the successful design and implementation of Dino-RL, followed by a comprehensive, real-world data driven evaluation. Episodic RL is first studied with the objective to incorporate domain knowledge into building an efficient episodic memory. In addition, a hierarchical hidden variable model is built to enable meta-reinforcement learning for knowledge transfer and efficient exploration. Lastly, the conflict between enhancing the physical-law based modeling and reinforcement learning is balanced via novel sample-efficient model selection algorithms.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

近年来，增强学习方法（RL）方法已引起了新的兴趣，以适应无线网络。尽管有希望的早期结果和概念上的匹配，但许多现有方法并未开发和量身定制RL方法以适合无线网络的独特特征。该项目的目的是开发一种新颖的领域知识丰富的RL框架或Dino-RL来解决此问题。 Dino-RL框架旨在将基于物理法的建模和抽象的情节内存无缝整合到RL过程中，并有可能改进未来无线网络的操作和管理。开发这种新颖的技术还将有助于维持美国在无线技术方面的持续领导力及其高素质工程师的管道。该项目为成功设计和实施Dino-RL提供协同活动，然后进行全面的现实数据驱动的评估。首先研究了情节RL的目标，其目标是将域知识纳入建立有效的情节记忆中。此外，构建了一个层次隐藏的变量模型，以实现用于知识转移和有效探索的荟萃方面学习。最后，通过新颖的样本效率模型选择算法增强基于物理法的建模和增强学习之间的冲突是平衡的。该奖项反映了NSF的法定任务，并认为通过基金会的知识分子的优点和更广泛的影响，认为值得通过评估来支持。

项目成果

Safe Exploration Incurs Nearly No Additional Sample Complexity for Reward-free RL

• DOI: 10.48550/arxiv.2206.14057
• 发表时间: 2022-06
• 期刊: ArXiv
• 作者: Ruiquan Huang;J. Yang;Yingbin Liang

Provably Efficient Offline Reinforcement Learning with Perturbed Data Sources

• DOI: 10.48550/arxiv.2306.08364
• 发表时间: 2023-06
• 期刊: ArXiv
• 作者: Chengshuai Shi;Wei Xiong;Cong Shen;Jing Yang

On Federated Learning with Energy Harvesting Clients

• DOI: 10.1109/icassp43922.2022.9746608
• 发表时间: 2022-02
• 期刊: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
• 作者: Cong Shen;Jing Yang;Jie Xu

Federated Linear Contextual Bandits

• 发表时间: 2021-10
• 作者: Ruiquan Huang;Weiqiang Wu;Jing Yang;Cong Shen

Heterogeneous Multi-player Multi-armed Bandits: Closing the Gap and Generalization

• 发表时间: 2021-10
• 作者: Chengshuai Shi;Wei Xiong;Cong Shen;Jing Yang