CSR: Small: Collaborative Research: Decentralized Real-Time Machine Learning Systems on Near-User Edge Devices

CSR：小型：协作研究：近用户边缘设备上的分散式实时机器学习系统

• 批准号: 2104416
• 负责人: Michael Ryoo
• 金额: $ 25万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2022-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2104416&HistoricalAwards=false
• 关键词: CSR; Small; Collaborative; Research; Decentralized

The ever-increasing number of Internet of Things (IoT) devices generate large quantities of raw data that need to be processed and analyzed in real time. Since conducting computationally expensive tasks, such as computer vision and natural language processing, is often a challenge for IoT devices, most of their computations are currently offloaded to cloud servers. However, this offloading leads to an increased risk for privacy as well as a dependency on network connectivity. To solve this challenge, the project utilizes the distributed computing power of already connected IoT devices to perform high computing power applications in real time.The project is composed of three tasks. First is the development of distributed machine learning (ML) systems for multiple IoT devices. The project will involve studying how to communicate between nodes with reliable connections and how to dynamically change the job of each node at run-time with little overhead. Second is the development of optimal task assignment and scheduling algorithms. Here, a machine learning approach will be used to generate a recognition model architecture optimal for each distributed system configuration. Third is the development of low-resolution deep neural network (DNN) systems to utilize low-power computing nodes. The development of these DNN systems will involve identifying multiple low-resolution filters that are optimal for varying configurations.The proposed technical work will advance the state of the art in implementation of parallel and decentralized DNN systems, thereby benefiting all scientific fields of endeavor that rely on computing. The decentralized DNN system will offer new opportunities in power constrained mobile platforms for applications including surveillance and automotive. The research results will lead to new materials/courses for computer architecture and systems. The proposed infrastructure will also be used to guide undergraduate students' research activities. The software infrastructure will be maintained as an open source project, which can be found at https://github.com/parallel-ml. It will be updated periodically as new outcomes become available. The results will be published in conferences, journals and technical reports.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

物联网 (IoT) 设备数量不断增加，产生大量需要实时处理和分析的原始数据。 由于执行计算机视觉和自然语言处理等计算量大的任务通常对物联网设备来说是一个挑战，因此它们的大部分计算目前都被卸载到云服务器上。然而，这种卸载会导致隐私风险增加以及对网络连接的依赖。为了解决这一挑战，该项目利用已连接的物联网设备的分布式计算能力来实时执行高计算能力应用。该项目由三个任务组成。首先是为多个物联网设备开发分布式机器学习（ML）系统。 该项目将涉及研究如何在具有可靠连接的节点之间进行通信，以及如何在运行时以很少的开销动态改变每个节点的工作。其次是开发最优任务分配和调度算法。在这里，将使用机器学习方法来生成最适合每个分布式系统配置的识别模型架构。第三是开发低分辨率深度神经网络（DNN）系统以利用低功耗计算节点。这些 DNN 系统的开发将涉及识别最适合不同配置的多个低分辨率滤波器。所提出的技术工作将推进并行和分散 DNN 系统实施的最先进技术，从而使所有依赖于这些系统的科学领域受益关于计算。分散式 DNN 系统将为监控和汽车等应用的功率受限移动平台提供新的机会。研究成果将带来计算机体系结构和系统的新材料/课程。拟议的基础设施还将用于指导本科生的研究活动。软件基础设施将作为开源项目进行维护，可以在 https://github.com/parallel-ml 上找到。 随着新结果的出现，它将定期更新。结果将在会议、期刊和技术报告中发表。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?

自监督学习真的能改善像素强化学习吗？

• 发表时间: 2022-11
• 期刊: 2022 Conference on Neural Information Processing Systems (NeurIPS
• 作者: Li, Xiang;Shang, Jinghuan;Das, Srijan;Ryoo, Michael S.
• 通讯作者: Ryoo, Michael S.

AViD Dataset: Anonymized Videos from Diverse Countries

AViD 数据集：来自不同国家的匿名视频

• 发表时间: 2020-12
• 期刊: Advances in neural information processing systems
• 作者: Piergiovanni, AJ;Ryoo, Michael S
• 通讯作者: Ryoo, Michael S

Neural Neural Textures Make Sim2Real Consistent

神经网络纹理使 Sim2Real 保持一致

• 发表时间: 2022-12
• 期刊: Conference on Robot Learning (CoRL
• 作者: Burgert, Ryan;Shang, Jinghuan;Li, Xiang;Ryoo, Michael S.
• 通讯作者: Ryoo, Michael S.

Coarse-Fine Networks for Temporal Activity Detection in Videos

用于视频中时间活动检测的粗细网络

• 发表时间: 2021-06
• 期刊: IEEE Conference on Computer Vision and Pattern Recognition
• 作者: Kumara, Kahatapitiya;Ryoo, Michael S
• 通讯作者: Ryoo, Michael S

Learning Viewpoint-Agnostic Visual Representations by Recovering Tokens in 3D Space

通过恢复 3D 空间中的标记来学习与视点无关的视觉表示

• 发表时间: 2022-11
• 期刊: 2022 Conference on Neural Information Processing Systems (NeurIPS
• 作者: Shang, Jinghuan;Das, Srijan;Ryoo, Michael S.
• 通讯作者: Ryoo, Michael S.