Federated Learning Infrastructure for Collaborative Machine Learning on Heterogeneous Environments

用于异构环境下协作机器学习的联邦学习基础设施

• 批准号: 22KJ2289
• 负责人: Thonglek Kundjanasith
• 金额: $ 1.09万
• 依托单位: Nara Institute of Science and Technology
• 依托单位国家: 日本
• 项目类别: Grant-in-Aid for JSPS Fellows
• 财政年份: 2023
• 资助国家: 日本
• 起止时间: 2023-03-08 至 2024-03-31
• 项目状态: 已结题
• 来源: https://kaken.nii.ac.jp/en/grant/KAKENHI-PROJECT-22KJ2289/
• 关键词: Collaborative Develop; Distributed Computing; Edge Machine Learning; Federated Learning; Privacy Preservation; Resource Heterogeneity

I proposed an infrastructure to allow individuals to collaboratively develop machine learning models on their environments, which are usually heterogeneous.The proposed infrastructure allows researchers to work together and potentially build better models than big companies can. The proposed infrastructure applied federated learning to train the models while preserving data privacy. I proposed three components in the proposed infrastructure to support training a model on diverse storage, computing, and network resources efficiently. First, I proposed a component to reduce the model size to fit the storage capacity of the heterogeneous environment. Second, I proposed a component to aggregate the models trained on heterogeneous computing resources. Third, I proposed a component to sparsify the model for exchanging the models between a server and clients. The proposed infrastructure was evaluated using state-of-the-art neural network models to detect COVID-19 cases from chest X-ray images. COVID-19 detection is one of the most popular machine learning applications for privacy-sensitive data. As a result, the ensemble model with heterogeneous structures on six different hardware environments from the proposed infrastructure produces accuracy higher than a trained single COVID-NET by 5.39%.

我提出了一个基础架构，以允许个人在其环境上共同开发机器学习模型，这些基础架构通常是异质的。拟议的基础架构使研究人员可以共同努力，并有可能比大公司能够建立更好的模型。拟议的基础设施应用联合学习来培训模型，同时保留数据隐私。我在拟议的基础架构中提出了三个组件，以有效地支持培训有关多种存储，计算和网络资源的模型。首先，我提出了一个组件来减少模型大小，以适合异质环境的存储容量。其次，我提出了一个组件来汇总在异质计算资源中训练的模型。第三，我提出了一个组件，以稀疏该模型以交换服务器和客户端之间的模型。使用最先进的神经网络模型评估了提出的基础架构，以检测胸部X射线图像的COVID-19病例。 COVID-19检测是对隐私敏感数据的最受欢迎的机器学习应用程序之一。结果，来自提议的基础设施的六个不同硬件环境上具有异质结构的集合模型可产生的准确性高于训练有素的单一covid-net 5.39％。

项目成果

Automated Quantization and Retraining for Neural Network Models Without Labeled Data

无标记数据的神经网络模型的自动量化和再训练

• DOI: 10.1109/access.2022.3190627
• 发表时间: 2022
• 期刊: IEEE Access
• 影响因子: 3.9
• 作者: Kundjanasith Thonglek;Keichi Takahashi;Kohei Ichikawa;Chawanat Nakasan;Hidemoto Nakada;Ryousei Takano;Pattara Leelaprute;Hajimu Iida
• 通讯作者: Hajimu Iida

Kasetsart University(タイ)

农业大学（泰国）

Sparse Communication for Federated Learning

• DOI: 10.1109/icfec54809.2022.00008
• 发表时间: 2022-05
• 期刊: 2022 IEEE 6th International Conference on Fog and Edge Computing (ICFEC)
• 作者: Kundjanasith Thonglek;Keichi Takahashi;Koheix Ichikawa;Chawanat Nakasan;P. Leelaprute;Hajimu Iida