Collaborative Research: CNS CORE: Small: RUI: Hierarchical Deep Reinforcement Learning for Routing in Mobile Wireless Networks

合作研究：CNS CORE：小型：RUI：移动无线网络中路由的分层深度强化学习

• 批准号: 2154190
• 负责人: Victoria Manfredi
• 金额: $ 32.63万
• 依托单位: Wesleyan University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-04-15 至 2025-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2154190&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; CORE; Small

The use of multi-hop routing in mobile wireless networks is becoming more prevalent, just as these networks are becoming more dense, dynamic, and heterogeneous. Designing a universal multi-hop routing strategy for mobile wireless networks is challenging, however, due to the need to seamlessly adapt routing behavior to spatially diverse and temporally changing network conditions. An alternative to using hand-crafted routing strategies is to use Reinforcement Learning (RL) to learn adaptive multi-hop routing strategies automatically. RL focuses on the design of intelligent agents: an RL agent interacts with its environment to learn a policy, i.e., which actions to take in different environmental states. By using function approximation like deep neural networks (DNNs) as in deep reinforcement learning (DeepRL) to approximate the policy, the RL agent can learn to generalize from its training experience to unseen network conditions and scale the learned routing strategy to larger networks. The PIs will continue their current practice of involving under-represented groups in research, and will use the project research to promote teaching and training through postdoctoral mentoring, course development, and outreach activities.The goal of this project is to use DeepRL to develop a universal multi-hop routing strategy for mobile wireless networks that is scalable, generalizable, and adaptive. Specifically, this project will build a novel routing framework that uses hierarchical DeepRL to design an option hierarchy, comprised of multiple layers of routing decisions working together to achieve the overall goals of the network. To enable the same routing strategy to be used at different devices and in unseen network scenarios, the framework will use relational features combined with novel neural network models to handle mobility and perform feature estimation. To further enhance generalizability, the framework will use continual learning to ensure that the routing behaviors learned for more recently seen network scenarios do not dominate the learned routing policy. The developed routing strategies will be thoroughly evaluated using both simulation and experimental testbeds. Through the use of hierarchical DeepRL, this project will provide a significant step forward in developing RL-based routing strategies, and will facilitate development of adaptive strategies for a wide range of mobile wireless networks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在移动无线网络中多跳路由的使用变得越来越普遍，就像这些网络变得越来越密集，动态和异质性一样。但是，为移动无线网络设计通用的多跳路由策略是具有挑战性的，但是由于需要无缝将路由行为调整到空间多样化和时间上变化的网络条件上。使用手工制作的路由策略的替代方法是使用增强学习（RL）自动学习自适应多跳路由策略。 RL专注于智能代理的设计：RL代理与其环境互动以学习政策，即采取不同环境状态的行动。通过使用函数近似（例如深神经网络（DNN）），例如在深度强化学习（DEEPRL）中近似策略，RL代理可以学会从其培训经验中概括以使网络条件概括，并将学习的路由策略扩展到较大的网络。 PI将继续他们目前的实践，即参与研究中代表性不足的群体，并将利用项目研究通过博士后指导，课程开发和外展活动来促进教学和培训。该项目的目的是使用DEEPRL来制定一种可扩展，可扩展，通用和适应性的移动无线网络的通用多跳路由策略。 具体而言，该项目将建立一个新颖的路由框架，该框架使用层级deepRL来设计选项层次结构，该层次结构由多层路由决策组成，共同实现网络的整体目标。为了使在不同设备和看不见的网络方案中使用相同的路由策略，该框架将使用关系功能与新型神经网络模型相结合来处理移动性并执行功能估计。为了进一步提高概括性，该框架将使用持续学习来确保最近看到的网络方案所学的路由行为不会主导学习的路由策略。将使用仿真和实验测试床对开发的路由策略进行彻底评估。通过使用层次deepRL，该项目将在制定基于RL的路由策略方面迈出重要一步，并促进为广泛的移动无线网络开发自适应策略。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子和更广泛的影响来评估CRITERIA CRITERIA的评估。

项目成果

Hierarchical Prototype Networks for Continual Graph Representation Learning

• DOI: 10.1109/tpami.2022.3186909
• 发表时间: 2021-11
• 期刊: IEEE Transactions on Pattern Analysis and Machine Intelligence
• 影响因子: 23.6
• 作者: Xikun Zhang;Dongjin Song;D. Tao

Sparsified Subgraph Memory for Continual Graph Representation Learning

• DOI: 10.1109/icdm54844.2022.00177
• 发表时间: 2022-11
• 期刊: 2022 IEEE International Conference on Data Mining (ICDM)
• 作者: Xikun Zhang;Dongjin Song;D. Tao

Learning an adaptive forwarding strategy for mobile wireless networks: resource usage vs. latency

• DOI: 10.1007/s10994-024-06601-3
• 发表时间: 2024-08-07
• 期刊: MACHINE LEARNING
• 影响因子: 7.5
• 作者: Manfredi，Victoria;Wolfe，Alicia P.;Wang，Bing
• 通讯作者: Wang，Bing