CNS Core: Small: Offline Inference for Ultra-Efficient Memory Management

CNS 核心：小型：离线推理以实现超高效内存管理

• 批准号: 2128653
• 负责人: Harry Xu
• 金额: $ 50万
• 依托单位: University of California-Los Angeles
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2128653&HistoricalAwards=false
• 关键词: CNS; Core; Small; Offline; Inference

Machine learning (ML) has made its way into systems of various kinds, helping them make informed decisions at critical points. A typical approach to ML-for-systems is to perform inference online by querying a model with runtime data. Online inference incurs non-trivial overheads, imposing a tight restriction on model size and complexity. In fact, systems that involve ML in their decision making often use very simple models (e.g., linear models) with inferior accuracy. This project develops a transformative approach to ML-for-systems - instead of doing online inference, this project advocates to train models that can predict runtime properties directly from program source code. As such, inference can be done offline and their results can be encoded and efficiently looked up during execution. Given that inference no longer contributes to run time, the proposed approach removes the above-discussed restrictions, enabling systems to employ state-of-the-art model architectures. This project further applies offline inference to memory management tasks that are critical to cloud applications. Modern society relies on services provided by large-scale systems. Improving the throughput and efficiency of such systems improves the service-level efficiency and scalability that human can experience in their lives. Replacing complicated and heuristics-driven decision making in today’s memory management systems with learning has a potential to dramatically reduce the cost of allocation and deallocation, which is a significant component in an application’s execution. Traditionally, inference is performed online, restricting what models to use and how high the accuracy can reach. This project develops techniques that make ML a more appealing approach by removing these restrictions. The techniques proposed span runtime system and ML. This interdisciplinary nature produces research that has impact in both areas. The project also makes efforts in education and diversity by incorporating research into courses and recruiting researchers from underrepresented groups.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习 (ML) 已进入各种类型的系统，帮助他们在关键点做出明智的决策。 机器学习的典型方法是通过使用运行时数据查询模型来执行在线推理。 - 琐碎的开销，对模型大小和复杂性施加严格限制。事实上，在决策中涉及机器学习的系统通常使用非常简单的模型（例如线性模型），但精度较低。 ML-for-systems - 该项目主张训练可以直接从程序源代码预测运行时属性的模型，而不是进行在线推理，因此可以离线进行推理，并且可以在执行期间对其结果进行编码和有效查找。鉴于推理不再影响运行时间，所提出的方法消除了上述限制，使系统能够采用最先进的模型架构，该项目进一步将离线推理应用于对云应用程序至关重要的内存管理任务。 .现代社会依赖提高此类系统的吞吐量和效率可以提高人们在生活中体验到的服务级别效率和可扩展性，用学习取代当今内存管理系统中复杂的、启发式的决策制定具有重要意义。显着降低分配和释放成本的潜力，这是应用程序执行中的一个重要组成部分。传统上，推理是在线执行的，限制了使用的模型以及可以达到的准确度。通过消除这些限制，机器学习成为一种更具吸引力的方法，这种跨学科性质产生了对这两个领域都有影响的研究，该项目还通过将研究纳入课程并招募代表性不足的研究人员来努力实现教育和多样性。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。