Collaborative Research: SHF: Medium: Heterogeneous Architecture for Collaborative Machine Learning

协作研究：SHF：媒介：协作机器学习的异构架构

• 批准号: 2106754
• 负责人: Xiaoyong Yuan
• 金额: $ 40万
• 依托单位: Michigan Technological University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-07-15 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2106754&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Medium; Heterogeneous

The recent breakthrough of on-device machine learning with specialized artificial-intelligence hardware brings machine intelligence closer to individual devices. To leverage the power of the crowd, collaborative machine learning makes it possible to build up machine-learning models based on datasets that are distributed across multiple devices while preventing data leakage. However, most existing efforts are focused on homogeneous devices; given the widespread yet heterogeneous participants in practice, it is urgently important but challenging to manage immense heterogeneity. The research team develops heterogeneous architectures for collaborative machine learning to achieve three objectives under heterogeneity: efficiency, adaptivity, and privacy. The proposed heterogeneous architecture for collaborative machine learning is bringing tangible benefits for a wide range of disciplines that employ artificial intelligence technologies, such as healthcare, precision medicine, cyber physical systems, and education. The research findings of this project are intended to be integrated with the existing courses and K-12 programs. Furthermore, the research team is actively engaged in activities that encourage students from underrepresented groups to participate in computer science and engineering research.This project provides the theoretical underpinning and empirical evidence for an efficient, adaptive and privacy-preserving design under heterogeneity, which fills a critical void - the existing collaborative machine-learning approach fails to manage the immense heterogeneity in practice. This project centers on three aspects: (1) design of specialized neural architectures for heterogeneous hardware platforms to cope with the limited efficiency of collaborative training due to heterogeneity; (2) design of an efficient and adaptive knowledge-transfer framework to bridge heterogeneous participants based on their underlying proximity benefits; (3) privacy strategies for heterogeneous collaboration by identifying new vulnerabilities and developing privacy-preserving mechanisms. A general-purpose testbed is built to rigorously validate the proposed research and expand the impact of this project. It is expected that this project opens a new research paradigm to unleash the utmost potential of heterogeneous and collaborative machine intelligence.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

最近，专门的人工智能硬件在设备上机器学习方面取得了突破，使机器智能更接近个人设备。为了利用人群的力量，协作机器学习可以基于分布在多个设备上的数据集构建机器学习模型，同时防止数据泄漏。然而，大多数现有的努力都集中在同质设备上。鉴于实践中参与者广泛而异质，管理巨大的异质性既紧迫又具有挑战性。研究团队开发了用于协作机器学习的异构架构，以实现异构下的三个目标：效率、适应性和隐私。所提出的协作机器学习异构架构正在为医疗保健、精准医学、网络物理系统和教育等采用人工智能技术的广泛学科带来切实的好处。该项目的研究成果旨在与现有课程和 K-12 项目相结合。此外，研究团队还积极开展鼓励弱势群体学生参与计算机科学和工程研究的活动。该项目为异质性下的高效、自适应和隐私保护设计提供了理论基础和经验证据，填补了关键空白——现有的协作机器学习方法无法管理实践中的巨大异质性。该项目主要围绕三个方面进行：（1）针对异构硬件平台设计专门的神经架构，以应对异构性导致的协作训练效率有限； (2) 设计一个高效且适应性强的知识转移框架，根据不同参与者的潜在邻近优势来桥接异质参与者； （3）通过识别新的漏洞和开发隐私保护机制来实现异构协作的隐私策略。建立通用测试平台是为了严格验证拟议的研究并扩大该项目的影响。预计该项目将开启一个新的研究范式，以释放异构和协作机器智能的最大潜力。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

FedZKT: Zero-Shot Knowledge Transfer towards Resource-Constrained Federated Learning with Heterogeneous On-Device Models

• DOI: 10.1109/icdcs54860.2022.00094
• 发表时间: 2021-09
• 期刊: 2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS)
• 作者: Lan Zhang;Dapeng Wu;Xiaoyong Yuan

ES Attack: Model Stealing Against Deep Neural Networks Without Data Hurdles

• DOI: 10.1109/tetci.2022.3147508
• 发表时间: 2020-09
• 期刊: IEEE Transactions on Emerging Topics in Computational Intelligence
• 影响因子: 5.3
• 作者: Xiaoyong Yuan;Lei Ding;Lan Zhang;Xiaolin Li;D. Wu

Membership Inference Attacks and Defenses in Neural Network Pruning

• 发表时间: 2022-02
• 作者: Xiaoyong Yuan;Lan Zhang