BIGDATA: Small: DCM: JetStream: A Flexible Distributed System for Online and In-Place Data Analysis

BIGDATA：小型：DCM：JetStream：用于在线和就地数据分析的灵活分布式系统

• 批准号: 1250990
• 负责人: Michael Freedman
• 金额: $ 70万
• 依托单位: Princeton University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-09-15 至 2018-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1250990&HistoricalAwards=false
• 关键词: BIGDATA; Small; DCM; JetStream; Flexible

Many research and commercial endeavors are experiencing dramatic transformations through the use of Big Data, wherein large data repositories are collected and analyzed to reveal trends, correlation, and information that may not be apparent in smaller samples. Current approaches assume centralizing the repository, which may be a poor fit in environments where the data generation rate exceeds the network capabilities. In this project, the PIs investigate system architectures for both real-time and historical analysis of geographically distributed data, combined with research in adaptively reducing data volumes to optimize bandwidth capabilities. This combination allows better use of the computation and storage associated with smarter end devices, including, but not limited to, distributed sensors, smart meters, and even full servers, without requiring network upgrades. Given the historical trends of the growth of computation and storage versus the capacity limits of wide-area networks, this research enables more data collection and analysis to be performed at a lower overall system cost. Further, the ability to dynamically adapt data precision and fidelity to available network bandwidth allows systems to gracefully and automatically improve performance in the presence of higher-capacity networks. The research enables the collection and analysis of data that is currently left unanalyzed because of network constraints. Such data can include finer-granularity usage data, which could indicate actionable steps to reducing household energy consumption, or it could include a greater olume of debugging and monitoring data, which could better predict system failures or provide greater insight than with current methods.

许多研究和商业努力通过使用大数据经历了戏剧性的转换，其中收集和分析了大型数据存储库，以揭示在较小的样本中可能并不明显的趋势，相关性和信息。当前的方法假定集中存储库，这可能是在数据生成速率超过网络功能的环境中的拟合度。在该项目中，PIS调查了用于地理分布数据的实​​时和历史分析的系统体系结构，结合了自适应减少数据量的研究，以优化带宽功能。这种组合允许更好地利用与更智能的端设备相关的计算和存储，包括但不限于分布式传感器，智能电表甚至完整的服务器，而无需网络升级。鉴于计算和存储的历史趋势与广域网络的容量限制的历史趋势，这项研究可以以较低的总体系统成本进行更多的数据收集和分析。 此外，动态调整数据精度和保真度的能力能够使系统在存在较高容量网络的情况下优雅，自动提高性能。该研究可以收集和分析由于网络限制，目前未分析的数据。此类数据可以包括较细的粒度用法数据，这可能表明可行的步骤可以减少家庭能源消耗，或者可以包括更大的调试和监视数据，这可以更好地预测系统故障或提供比当前方法更大的洞察力。