Real-Time Federated Learning at the Wireless Edge via Algorithm-Hardware Co-Design

通过算法-硬件协同设计在无线边缘进行实时联合学习

基本信息

批准号：
EP/X019160/1
负责人：
JIA HU
金额：
$ 25.67万
依托单位：
UNIVERSITY OF EXETER
依托单位国家：
英国
项目类别：
Research Grant
财政年份：
2023
资助国家：
英国
起止时间：
2023 至无数据
项目状态：
未结题

来源：
https://gtr.ukri.org/projects?ref=EP%2FX019160%2F1
关键词：
Real Time Federated Learning Wireless

项目摘要

The past years have witnessed a rapidly growing number of wirelessly-connected devices such as smartphones and Internet-of-Things (IoT) equipment, which generate ever-increasing amounts of data driving key Artificial Intelligence (AI) applications. However, users are increasingly unwilling to allow their private data (such as media, location, or sensor data) to be uploaded to a central location (e.g., cloud datacentre) for training Machine Learning (ML) models, and data-protection laws such as the Data Protection Act 2018 are growing more restrictive towards data usage. Federated Learning (FL) is a game-changing technology conceived to address the growing data privacy concern by moving training from the datacentre to user devices at the network edge, allowing sensitive data to remain on the devices where it was generated. FL has enormous potential for real-world, privacy-sensitive applications such as autonomous driving, diagnostic healthcare, and predictive maintenance.The operating environment for FL at the edge is extremely challenging for a variety of reasons: 1) the data owned by FL clients is highly heterogeneous (in regard to data distribution, quality, and quantity) and dynamic (data distributions change over time); 2) the hardware devices have diverse computing and communication capabilities with stringent resource constraints (e.g., battery power); and 3) FL clients work under unreliable wireless edge network conditions. Hence, despite FL's huge promise, there are considerable barriers to its wider real-world adoption for mission-critical AI applications that need real-time, on-demand responses, caused by several grand challenges: Challenge 1) lack of FL algorithms delivering consistent performance for dynamic client data, diverse client hardware, and unreliable wireless connections simultaneously; Challenge 2) lack of rigorous theoretical analyses of real-time, real-world FL algorithms; Challenge 3) lack of optimised, energy-efficient, versatile hardware acceleration for real-time FL.To address these important challenges, this project will create revolutionary algorithm-hardware co-design approaches to make FL a real-time process with unparalleled speed, performance, and energy-efficiency at the wireless edge, capable of meeting the stringent requirements of mission-critical applications. This research will pioneer a set of original methods and innovative technologies including: 1) disruptive lightweight hardware-aware FL algorithms that significantly reduce communication, computing, and energy costs while achieving fast model updates; 2) rigorous mathematical analyses of the proposed algorithms to prove their convergence rates and offer theoretical insights into how they perform under various edge network conditions; 3) an automatic hardware-software co-optimisation framework integrating specialised training-acceleration and power-reduction methods to realise optimised, energy-efficient hardware acceleration; and 4) a unique prototype system that will integrate the designed FL hardware accelerator and real-time FL algorithms and be evaluated in a realistic wireless edge networking testbed.This project has the potential to transform FL from a lengthy and disjointed process to a continuous, real-time procedure with concurrent model training and deployment. The proposed research will contribute to the UK's digital transformation and green economy by creating ground-breaking technologies for creating innovative AI-empowered products with significantly improved performance and energy-efficiency while complying with strict data-privacy protection.

过去几年，智能手机和物联网 (IoT) 设备等无线连接设备的数量迅速增长，这些设备产生的数据量不断增加，驱动关键的人工智能 (AI) 应用。然而，用户越来越不愿意允许他们的私人数据（例如媒体、位置或传感器数据）上传到中央位置（例如云数据中心）以训练机器学习（ML）模型，以及数据保护法，例如随着《2018 年数据保护法》对数据使用的限制越来越严格。联合学习 (FL) 是一项改变游戏规则的技术，旨在通过将训练从数据中心转移到网络边缘的用户设备来解决日益增长的数据隐私问题，从而允许敏感数据保留在生成数据的设备上。 FL 在现实世界中对隐私敏感的应用程序（例如自动驾驶、诊断医疗保健和预测性维护）具有巨大潜力。由于多种原因，边缘 FL 的操作环境极具挑战性：1）FL 客户端拥有的数据高度异构（在数据分布、质量和数量方面）和动态（数据分布随时间变化）； 2）硬件设备具有多样化的计算和通信能力，并且具有严格的资源限制（例如电池电量）； 3) FL 客户端在不可靠的无线边缘网络条件下工作。因此，尽管 FL 有着巨大的前景，但由于以下几个重大挑战，其在现实世界中更广泛地采用需要实时、按需响应的任务关键型 AI 应用程序仍存在相当大的障碍：挑战 1）缺乏提供一致的 FL 算法同时处理动态客户端数据、不同客户端硬件和不可靠的无线连接的性能；挑战2）缺乏对实时、现实世界FL算法的严格理论分析；挑战3）缺乏针对实时FL的优化、节能、多功能硬件加速。为了解决这些重要挑战，该项目将创建革命性的算法-硬件协同设计方法，使FL成为具有无与伦比速度的实时过程，无线边缘的性能和能效，能够满足关键任务应用的严格要求。这项研究将开创一系列原创方法和创新技术，包括：1）颠覆性的轻量级硬件感知FL算法，可显着降低通信、计算和能源成本，同时实现快速模型更新； 2）对所提出的算法进行严格的数学分析，以证明其收敛速度，并为它们在各种边缘网络条件下的表现提供理论见解； 3）自动软硬件协同优化框架，集成专门的训练加速和节能方法，实现优化、节能的硬件加速； 4) 一个独特的原型系统，将集成设计的 FL 硬件加速器和实时 FL 算法，并在现实的无线边缘网络测试台中进行评估。该项目有潜力将 FL 从一个漫长且脱节的过程转变为一个连续的、具有并发模型训练和部署的实时过程。拟议的研究将通过创造突破性技术来创建创新的人工智能产品，显着提高性能和能源效率，同时遵守严格的数据隐私保护，从而为英国的数字化转型和绿色经济做出贡献。