Hardware-aware Network Architecture Search under ML Training workloads

ML 训练工作负载下的硬件感知网络架构搜索

• 批准号: 2904511
• 金额: --
• 依托单位: University of Cambridge
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2024
• 资助国家: 英国
• 起止时间: 2024 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2904511
• 关键词: Hardware; aware; Network; Architecture; Search

Achieving the optimal delay/energy/accuracy requires the co-optimization of Neural Network (NN) workload together with the system hardware it is deployed in. This is particularly important on mobile systems running on batteries, where to deploy more complex workloads, retraining-adaptation might be limited due to energy or computation constraints.Neural Architecture Search (NAS) solutions enable the exploration of the NN graph characteristics in an efficient way, adapting the design specifics of the neural architecture to the machine learning task at hand. The aim of this PhD tackles the design of an open-source, hardware aware NAS methodology to optimize not only inference but also on-device training. Furthermore, hardware considerations will be expanded to lower-level processor specifications (for instance, NVM parameters, DVFS design) alongside system-level characteristics.Key Research QuestionsThe objective of the research is to investigate the open problems in the domain of hardware-aware network architecture search (NAS). In particular, it will focus on a proof of concept system model that will be aware of established opportunities for efficiency. Examples of these include different frequency/voltage operation points (DVFS) and hybrid memories (off-chip/on-chip, non-volatile/volatile) with multiple power operation modes. A key challenge will be integrating this hardware-defined search-space with that of search-space of the neural architecture to perform joint-optimization of both neural architecture and hardware design spaces in a tractable manner. The design of this NAS method will be a useful tool, but just as important will be the results this tool is able to produce - specifically novel neural architectures and novel hardware designs that especially when run jointly produce high efficiency systems. It is expected that by analysing the workload-aware optimized systems, design and micro-architectural innovations will result in the exceptional power, performance and area metrics for such heterogeneous hardware designs.Work expected of the ICASE studentDuring the Ph.D, the student will research and address the computing and memory requirements of a system model that includes the NAS scheme and optimises an on-device training workload.While addressing the key research questions, the student will work in a top-class environment, with colleagues from the CaMLSys group at the University of Cambridge. There will be a minimum of a 3 month placement with the industrial partner IMEC where the student will work in a diverse environment with scientists devoted to world leading R&D and innovation in nanoelectronics and digital technologies.

实现最佳的延迟/能量/准确性需要进行神经网络（NN）工作负载以及部署的系统硬件的合作化。这对于在电池上运行的移动系统尤为重要，在该电池上运行更复杂的工作量，重新适应可能会受到能量或计算范围的限制。神经体系结构的细节到手头的机器学习任务。该博士的目的是应对开源，硬件意识到的NAS方法的设计，不仅可以优化推理，而且还要优化设备培训。此外，硬件注意事项将扩展到较低级别的处理器规范（例如，NVM参数，DVFS设计）以及系统级特征。特别是，它将集中于概念系统模型的证明，该模型将意识到既定的效率机会。这些示例包括具有多个功率操作模式的不同频率/电压操作点（DVF）和混合记忆（片外/片上，非挥发性/挥发性）。一个关键的挑战将是将这种硬件定义的搜索空间与神经体系结构的搜索空间集成在一起，以以可操作的方式对神经体系结构和硬件设计空间进行联合优化。这种NAS方法的设计将是一个有用的工具，但是重要的是该工具能够产生的结果 - 特别是新型的神经体系结构和新型硬件设计，尤其是当运行共同生成高效率系统时。可以预期，通过分析工作负载的优化系统，设计和微构造创新将导致这种异质硬件设计的特殊功能，绩效和面积指标。学院期望的工作期望的工作是学生居住博士学位的工作，该学生将研究和解决NAS培训的系统模型，并解决一项启动工作的计算和记忆要求，并将工作培训培训。顶级环境，剑桥大学的CAMLSYS集团的同事。工业合作伙伴IMEC至少要有3个月的安置，在该公司将在多元化的环境中工作，科学家致力于世界领先的研发以及纳米电子和数字技术的创新。