CNS Core: Medium: Resource Constrained Reinforcement Learning for Computing Systems

CNS 核心：中：计算系统的资源受限强化学习

• 批准号: 1955997
• 负责人: Christina Yu
• 金额: $ 120万
• 依托单位: Cornell University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-07-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1955997&HistoricalAwards=false
• 关键词: CNS; Core; Medium; Resource; Constrained

Reinforcement Learning has surfaced as a promising algorithmic paradigm for online decision making and control. Recent efforts demonstrate impressive results, but with relatively unlimited computational resources, data, and training. However, many real-world decision-making and control problems require real-time decisions but are severely constrained in terms of throughput, latency, memory, and power. Existing algorithms are employed in software on general purpose processors, and do not translate well to real-timeresource-constrained computation in hardware. This project will develop new algorithmic paradigms as well as hardware acceleration primitives that work in sync to enable real-time reinforcement learning at scale in embedded applications. From the algorithmic perspective, this work will study efficiency in computational resource tradeoffs, and design reinforcement learning algorithms that efficiently adapt to given hardware constraints and hardware primitives. From the hardware perspective, this work will design new hardware primitives that support the developed reinforcement learning algorithms for real-time applications in a storage- and energy-efficient manner.Our reinforcement learning algorithms, analyses, and experiments will shed new insights about how to make optimal trade-offs between performance and different system constraints such as memory, power, and latency. The research will be guided by systems applications including switch scheduling for network routers, resource management for data centers, network congestion control, memory management for computers, and adaptive sensing for wireless networks and the Internet of things. The outcomes of this project have the potential to transform how data centers, communication networks, and wireless systems are managed. Applications beyond computer and network systems include operational challenges in large-scale systems, such as inventory management and resource allocation, or low latency control problems that arise in brain-machine interfaces. The project also includes an extensive outreach plan that involves women and underrepresented minorities in computing research in reinforcement learning from an algorithmic and architecture-level perspective.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

强化学习已成为用于在线决策和控制的有希望的算法范式。最近的努力表明了令人印象深刻的结果，但具有相对无限的计算资源，数据和培训。但是，许多现实世界的决策和控制问题需要实时决策，但在吞吐量，延迟，记忆和权力方面受到严格限制。现有的算法用于通用处理器的软件中，并且不能很好地转化为硬件中的真实托管限制的计算。该项目将开发新的算法范式以及同步工作的硬件加速度基础，以在嵌入式应用程序中进行大规模实时增强学习。从算法的角度来看，这项工作将研究计算资源折衷的效率，并设计强化学习算法，这些算法有效地适应了给定的硬件约束和硬件原始词。 From the hardware perspective, this work will design new hardware primitives that support the developed reinforcement learning algorithms for real-time applications in a storage- and energy-efficient manner.Our reinforcement learning algorithms, analyses, and experiments will shed new insights about how to make optimal trade-offs between performance and different system constraints such as memory, power, and latency.该研究将以系统应用程序为指导，包括网络路由器的开关计划，数据中心的资源管理，网络拥塞控制，计算机的内存管理以及用于无线网络和物联网的自适应感测。该项目的结果有可能改变数据中心，通信网络和无线系统的管理方式。计算机和网络系统以外的应用程序包括大规模系统中的操作挑战，例如库存管理和资源分配，或在脑机界面中出现的低潜伏期控制问题。该项目还包括一项广泛的外展计划，该计划涉及妇女和代表性不足的少数群体在计算从算法和建筑级别的研究中学习研究方面的研究。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的智力和更广泛影响的评估来进行评估的支持，这是值得的。

项目成果

Sequential Fair Allocation: Achieving the Optimal Envy-Efficiency Trade-off Curve

• DOI: 10.1287/opre.2022.2397
• 发表时间: 2023-11-23
• 期刊: OPERATIONS RESEARCH
• 影响因子: 2.7
• 作者: Sinclair，Sean R.;Jain，Gauri;Yu，Christina Lee
• 通讯作者: Yu，Christina Lee

Nonparametric Matrix Estimation with One-Sided Covariates

• DOI: 10.1109/isit50566.2022.9834608
• 发表时间: 2021-10
• 期刊: 2022 IEEE International Symposium on Information Theory (ISIT)
• 作者: C. Yu

Nonasymptotic Analysis of Monte Carlo Tree Search

蒙特卡罗树搜索的非渐近分析

• DOI: 10.1287/opre.2021.2239
• 发表时间: 2022
• 期刊: Operations Research
• 影响因子: 2.7
• 作者: Shah, Devavrat;Xie, Qiaomin;Xu, Zhi
• 通讯作者: Xu, Zhi

CSI-Based Multi-Antenna and Multi-Point Indoor Positioning Using Probability Fusion

• DOI: 10.1109/twc.2021.3109789
• 发表时间: 2020-09
• 期刊: IEEE Transactions on Wireless Communications
• 影响因子: 10.4
• 作者: Emre Gonultacs;E. Lei;Jack Langerman;Howard Huang;Christoph Studer

Bias and Extrapolation in Markovian Linear Stochastic Approximation with Constant Stepsizes

• DOI: 10.1145/3578338.3593526
• 发表时间: 2022-10
• 期刊: Abstract Proceedings of the 2023 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems
• 作者: D. Huo;Yudong Chen;Qiaomin Xie