Approximate Distributed Stream Tracking: Enabling the Next Generation of Data-Streaming Applications

近似分布式流跟踪：支持下一代数据流应用程序

• 批准号: 0414852
• 负责人: Shanmugavelayu Muthukrishnan
• 金额: $ 27万
• 依托单位: Rutgers University New Brunswick
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2005
• 资助国家: 美国
• 起止时间: 2005-09-01 至 2009-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0414852&HistoricalAwards=false
• 关键词: Approximate; Distributed; Stream; Tracking; Enabling

This research develops models, algorithmic methods and software solutions for tracking of massive data streams for monitoring applications such as IP (Internet Protocol) network traffic analysis. Such monitoring applications are inherently distributed, relying on correlating multiple streams, and therefore present challenges in terms of severe communication constraints. In addition, such massive streams are also faced with the traditional storage and per-item processing time constraints even in the centralized or the single stream cases. Under such severe constraints, monitoring is necessarily approximate. This research project develops principled methods for performing essential monitoring tasks on distributed streams under the accumulation of all such constraints. In particular, new methods are developed that trade off accuracy of analysis for meeting communication, space and time constraints. Distributed monitoring of massive data streams arises in many communication systems, primarily in security applications. The resulting models and solutions address such applications and yield better understanding of how to perform detailed data analyses within existing resource constraints. This research is carried out in collaboration with industry researchers (Minos Garofalakis and Rajeev Rastogi of Lucent) who bring extensive knowledge of stream data mining and provide data sets for testing the new algorithms for approximate distributed stream tracking. In addition, the industrial participation in this project increases the impact of this project via technology transfer. Studying the algorithmic, database and networking aspects of the problem jointly will lead to significant new insights and training. Solutions and resulting software programs will be made freely available via the project's Web site (http://www.cs.rutgers.edu/~muthu/adst.html).

这项研究开发了模型、算法方法和软件解决方案，用于跟踪海量数据流，以监控 IP（互联网协议）网络流量分析等应用。此类监控应用本质上是分布式的，依赖于关联多个流，因此在严格的通信限制方​​面提出了挑战。此外，即使在集中式或单流情况下，如此海量的流也面临着传统存储和每项处理时间的限制。在如此严格的限制下，监测必然是近似的。该研究项目开发了在所有此类约束的累积下对分布式流执行基本监控任务的原则方法。特别是，开发了新的方法，以牺牲分析的准确性来满足通信、空间和时间的限制。海量数据流的分布式监控出现在许多通信系统中，主要是在安全应用中。由此产生的模型和解决方案解决了此类应用程序的问题，并更好地理解如何在现有资源限制下执行详细的数据分析。这项研究是与行业研究人员（朗讯的 Minos Garofalakis 和 Rajeev Rastogi）合作进行的，他们带来了流数据挖掘的丰富知识，并提供了用于测试近似分布式流跟踪新算法的数据集。此外，工业界参与该项目通过技术转让增加了该项目的影响力。共同研究问题的算法、数据库和网络方面将带来重要的新见解和培训。解决方案和最终的软件程序将通过该项目的网站（http://www.cs.rutgers.edu/~muthu/adst.html）免费提供。