CAREER: Scalable and Adaptive Edge Stream Processing

职业：可扩展和自适应边缘流处理

• 批准号: 2205677
• 负责人: Liting Hu
• 金额: $ 48.87万
• 依托单位: Virginia Polytechnic Institute and State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2023-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2205677&HistoricalAwards=false
• 关键词: CAREER; Scalable; Adaptive; Edge; Stream

Internet-of-Things (IoT) applications such as self-driving cars, augmented reality, interactive gaming, and event monitoring have a tremendous potential to improve our lives. These applications generate a large influx of sensor data at massive scales. Under many time-critical scenarios, these massive data streams must be processed in a very short time to derive actionable intelligence. This CAREER project aims to support time-critical IoT applications by applying the stream processing paradigm to the Edge computing architecture. The success of this research will benefit many time-critical IoT applications in the areas such as factory automation, the tactile internet, autonomous vehicles, and process automation. It will also substantially improve the performance profiles of a variety of data processing systems, such as wide-area data analytics systems, mobile data access systems, event tracking systems, and streaming databases. As an integral part of its research program, this CAREER project involves K-12, undergraduate and graduate level education in partnership with the local Public School system.Specifically, this CAREER project will build a scalable and adaptive Edge stream processing engine, which enables fast stream processing of a large number of concurrent IoT queries in the dynamic, heterogeneous Edge environment. This work includes three primary research directions. First, a new dynamic dataflow graph abstraction will be implemented, which automatically chains, parallelizes and replicates stream operators to adapt to the Edge dynamics and handle failures in a scalable way. Second, a new customizable data shuffling service abstraction will be implemented, which customizes the data shuffling path (e.g., ring shuffle, hierarchical tree shuffle, butterfly wrap shuffle) at runtime for the given network topology and workload. Third, a fully decentralized architecture with many distributed schedulers will be implemented, in which each scheduler operates autonomously to process IoT queries. All three parts of the project will be prototyped and implemented on real-world stream processing systems and validated by performing real-world experiments.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

自动驾驶汽车、增强现实、互动游戏和事件监控等物联网 (IoT) 应用具有改善我们生活的巨大潜力。这些应用程序会产生大量的大规模传感器数据。在许多时间紧迫的场景下，必须在很短的时间内处理这些海量数据流才能获得可操作的情报。该职业项目旨在通过将流处理范例应用于边缘计算架构来支持时间关键的物联网应用程序。这项研究的成功将使工厂自动化、触觉互联网、自动驾驶汽车和过程自动化等领域的许多时间关键的物联网应用受益。它还将大大提高各种数据处理系统的性能，例如广域数据分析系统、移动数据访问系统、事件跟踪系统和流数据库。作为其研究计划的一个组成部分，该职业项目涉及与当地公立学校系统合作的 K-12、本科和研究生水平的教育。具体来说，该职业项目将构建一个可扩展和自适应的边缘流处理引擎，该引擎能够实现快速在动态、异构边缘环境中对大量并发 IoT 查询进行流处理。这项工作包括三个主要研究方向。首先，将实现一个新的动态数据流图抽象，它自动链接、并行化和复制流运算符，以适应边缘动态并以可扩展的方式处理故障。其次，将实现新的可定制数据混洗服务抽象，该抽象在运行时针对给定的网络拓扑和工作负载定制数据混洗路径（例如，环混洗、分层树混洗、蝴蝶包装混洗）。第三，将实施具有许多分布式调度程序的完全去中心化架构，其中每个调度程序自主运行以处理物联网查询。该项目的所有三个部分都将在现实世界的流处理系统上进行原型设计和实施，并通过进行现实世界的实验进行验证。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和更广泛的影响进行评估，被认为值得支持审查标准。

项目成果

SR3: Customizable Recovery for Stateful Stream Processing Systems

• DOI: 10.1145/3423211.3425681
• 发表时间: 2020-12
• 期刊: Proceedings of the 21st International Middleware Conference
• 作者: Hailu Xu;Pinchao Liu;Susana Cruz-Diaz;D. D. Silva-D.;Liting Hu

Towards low-latency I/O services for mixed workloads using ultra-low latency SSDs

• DOI: 10.1145/3524059.3532378
• 发表时间: 2022-06
• 期刊: Proceedings of the 36th ACM International Conference on Supercomputing
• 作者: Mingzhe Liu;Haikun Liu;Chencheng Ye;Xiaofei Liao;Hai Jin;Yu Zhang;Ran Zheng;Liting Hu

XUnified: A Framework for Guiding Optimal Use of GPU Unified Memory

XUnified：指导 GPU 统一内存优化使用的框架

• DOI: 10.1109/access.2022.3196008
• 发表时间: 2022
• 期刊: IEEE Access
• 影响因子: 3.9
• 作者: Xu, Hailu;Lin, Pei-Hung;Emani, Murali;Hu, Liting;Liao, Chunhua
• 通讯作者: Liao, Chunhua

DART: A Scalable and Adaptive Edge Stream Processing Engine

DART：可扩展的自适应边缘流处理引擎

• 发表时间: 2021
• 期刊: 2021 USENIX Annual Technical Conference (USENIX ATC 21
• 作者: Liu, Pinchao;Silva, Dilma Da;Hu, Liting.
• 通讯作者: Hu, Liting.