Equality Saturation for Deep Learning Compilers

深度学习编译器的等式饱和

• 批准号: 2873105
• 金额: --
• 依托单位: University of Cambridge
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2873105
• 关键词: Equality; Saturation; Deep; Learning; Compilers

Exponential growth in the computational demands of state-of-the-art Deep Learning (DL) models poses both societal andenvironmental concerns as research is increasingly monopolized by corporate labs which can afford multi-million dollartraining processes which in turn produce tens of thousands of kilograms of CO2. Improving the efficiency of DL istherefore paramount to democratizing research and protecting the planet. Equality Saturation, a novel compileroptimization technique, is one promising approach, reducing DL runtime by up to 70% and finding such improvements300x faster than traditional methods in some scenarios. However, its widespread application to Deep Learning Compilersis limited by its complexity, scalability and the difficulty in adapting it to the heterogeneous DL frameworks availabletoday. By integrating Equality Saturation into the MLIR compiler, Equality Saturation could be widely applied acrossframeworks without requiring tens of thousands of lines of handwritten optimization code, thereby increasing itsaccessibility to developers and greatly improving the efficiency of modern DL systems.

最先进的深度学习 (DL) 模型的计算需求呈指数级增长，引发了社会和环境问题，因为研究越来越被企业实验室垄断，而企业实验室可以承担数百万美元的训练过程，而训练过程又会产生数万公斤的重量二氧化碳。因此，提高深度学习的效率对于研究民主化和保护地球至关重要。 Equality Saturation 是一种新颖的编译器优化技术，是一种很有前途的方法，可将 DL 运行时间减少多达 70%，并且在某些情况下发现这种改进比传统方法快 300 倍。然而，它在深度学习编译器中的广泛应用受到其复杂性、可扩展性以及难以适应当今可用的异构深度学习框架的限制。通过将等式饱和度集成到 MLIR 编译器中，等式饱和度可以跨框架广泛应用，而不需要数万行手写优化代码，从而提高了开发人员的可访问性，并大大提高了现代深度学习系统的效率。