III: Small: Multiple Device Collaborative Learning in Real Heterogeneous and Dynamic Environments

III：小：真实异构动态环境中的多设备协作学习

• 批准号: 2311990
• 负责人: Eric Xing
• 金额: $ 59.94万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2311990&HistoricalAwards=false
• 关键词: III; Small; Multiple; Device; Collaborative

In collaborative learning, different devices such as smartphones or organizations like banks or hospitals learn together, using their own (sometimes private) data to build a shared model. This project tackles the challenge of scaling up this kind of learning for large, constantly changing, and diverse datasets. It proposes a move away from traditional computing systems, towards more flexible systems that can handle changing data types, decentralized computing, and different strategies between learning devices. Currently, problems arise from the diverse nature and volume of real-world data, like user photos, surveillance videos, or medical information. Present collaborative learning schemes often try to average out the learning updates from all devices, ignoring the individual characteristics of each device and their unique computing abilities. Moreover, these systems struggle with "free riding", where some participants benefit from the improved learning model without contributing any data. This difficulty calls for creating fitting incentives to encourage equal participation. The goal of this project is to improve collaborative learning by solving these issues, resulting in more efficient and fair learning systems that cater to individual devices' unique characteristics. This advancement goes beyond improving machine learning as it encourages data sharing, participation, and inclusivity, bringing about broader societal benefits. Current collaborative learning frameworks often achieve consensus by averaging model updates from participating agents, an approach that may disregard the unique attributes and diverse hardware capabilities of the agents involved. Such an oversight could lead to a mismatch between the model architecture and the hardware capabilities of specific devices, particularly edge devices with limited memory or computational power, thereby impeding efficient model training or underutilizing available resources. Additionally, the extant collaborative learning frameworks tend to overlook crucial factors such as algorithm trustworthiness and mechanism design. These challenges highlight the urgent need for a reimagined approach to collaborative learning. This project focuses on the development of a collaborative learning framework specifically designed for dynamic and diverse environments, with particular emphasis on standard hardware computing platforms, such as those comprising edge devices. The project's objectives are threefold: First, it seeks to innovate model-parallel collaborative learning by designing unique model architectures and efficient algorithms, underpinned by theoretical explorations employing a structured variational inference approach. Second, the project aims to facilitate the creation of practical, efficient training and communication learning algorithms for on-device usage. This aim will be achieved by introducing new algorithmic components and an authentic on-device testing platform. Lastly, the project intends to evaluate the system's trustworthiness and mechanism design in a decentralized setting. It will design incentives that promote data sharing and algorithm adoption, thereby maximizing benefits at both the community and individual levels. The project's targeted applications encompass disaster forecasting, AI-assisted clinical diagnosis, and treatment, and decentralized strategic decision-making.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在协作学习中，使用自己的（有时是私人）数据来构建共享模型，例如银行或医院等不同的设备（例如银行或医院）一起学习。该项目应对大型，不断变化和多样化的数据集扩展这种学习的挑战。它提议从传统的计算系统转向更灵活的系统，这些系统可以处理不断变化的数据类型，分散计算以及学习设备之间的不同策略。当前，问题是由现实数据的多样性和数量引起的，例如用户照片，监视视频或医疗信息。当前的协作学习方案通常试图平均从所有设备中平均学习更新，而忽略了每个设备的个人特征及其独特的计算能力。此外，这些系统在“自由骑行”中遇到了困难，其中一些参与者从改进的学习模型中受益，而无需贡献任何数据。这个困难要求创造合适的激励措施来鼓励平等参与。该项目的目的是通过解决这些问题来改善协作学习，从而产生更高效，公平的学习系统，以迎合各个设备的独特特征。这种进步不仅仅是改善机器学习，因为它鼓励了数据共享，参与和包容性，从而带来了更广泛的社会利益。当前的协作学习框架通常通过平均参与代理的模型更新来达成共识，这种方法可能会忽略所涉及的代理的独特属性和多样化的硬件功能。这样的监督可能会导致模型体系结构与特定设备的硬件功能之间的不匹配，尤其是具有有限内存或计算能力的边缘设备，从而阻碍有效的模型培训或不利于可用资源。此外，现存的协作学习框架倾向于忽视关键因素，例如算法可信度和机制设计。这些挑战强调了迫切需要重新想象合作学习的方法。该项目的重点是专门为动态和多样化环境设计的协作学习框架的开发，特别着重于标准硬件计算平台，例如包括边缘设备的设备。该项目的目标是三倍：首先，它试图通过设计独特的模型体系结构和有效算法来创新模型并行协作学习，这是由理论探索采用结构化变分推断方法的基础的。其次，该项目旨在促进创建实用，高效的培训和交流学习算法以实现设备使用。通过引入新的算法组件和一个真实的内部设备测试平台来实现此目标。最后，该项目打算在分散的环境中评估系统的可信度和机制设计。它将设计激励措施，以促进数据共享和算法采用，从而最大程度地提高社区和个人层面的收益。该项目的针对性应用程序包括灾难预测，AI辅助临床诊断和治疗以及权力下放的战略决策。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛影响的审查标准通过评估来获得支持的。