XPS: DSD: Adaptive Stream-Processing Compilers for a Messy World

XPS：DSD：适用于混乱世界的自适应流处理编译器

• 批准号: 1337242
• 负责人: Ryan Newton
• 金额: $ 74.56万
• 依托单位: Indiana University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-09-15 至 2017-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1337242&HistoricalAwards=false
• 关键词: XPS; DSD; Adaptive; Stream; Processing

Stream processing is an increasingly important application domain; a significant portion of the data-deluge beleaguering society takes the form of real-time data, ranging from scientific data to tweets. Fortunately, in the last decade, a variety of stream-processing languages have sprung up, including WaveScope, StreamIt, Feldspar, and others. While these languages enable stream-processing programs (which take the form of flow-graphs of stream operators) to execute automatically and efficiently on multicores and small clusters of machines, they optimize assuming an unchanging streaming workload and cannot handle dynamic conditions found in many realistic streaming situations, such as inside modern networks.This proposal pursues a more adaptive approach: fast, incremental compilation and recompilation of subgraphs of a stream-processing flow-graph to support dynamic placement and optimization policies while retaining high performance. The goal is to allow streaming applications to start instantly and in parallel, or restart if the program changes, while adapting to predictable features of the environment over time including streams of constant rate. Existing just-in-time (JIT) compilers for languages such as JavaScript are a mature technology, but a new body of techniques are needed to apply the more radical optimizations of stream compilers in a dynamic context. This project aims to develop these techniques and evaluate them in a specific application domain: high-speed in-network processing.

流处理是一个日益重要的应用领域；困扰社会的数据洪流很大一部分采用实时数据的形式，从科学数据到推文。 幸运的是，在过去十年中，各种流处理语言如雨后春笋般涌现，包括 WaveScope、StreamIt、Feldspar 等。 虽然这些语言使流处理程序（采用流运算符流程图的形式）能够在多核和小型机器集群上自动高效地执行，但它们在假设流工作负载不变的情况下进行优化，并且无法处理许多现实中发现的动态条件。该提案追求一种更具适应性的方法：流处理流图的子图的快速增量编译和重新编译，以支持动态放置和优化策略，同时保持高性能。 目标是允许流应用程序立即并行启动，或者在程序更改时重新启动，同时适应环境随时间的可预测特征，包括恒定速率的流。 JavaScript 等语言的现有即时 (JIT) 编译器是一项成熟的技术，但需要一套新的技术来在动态上下文中应用流编译器的更彻底的优化。 该项目旨在开发这些技术并在特定应用领域（高速网络内处理）中对其进行评估。