BIGDATA: Small: DA: A Random Projection Approach

大数据：小：DA：随机投影方法

• 批准号: 1419210
• 负责人: Ping Li
• 金额: $ 44.74万
• 依托单位: Rutgers University New Brunswick
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-12-01 至 2019-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1419210&HistoricalAwards=false
• 关键词: BIGDATA; Small; DA; Random; Projection

With the advent of Internet, numerous applications in the context of network traffic, search, and databases are faced with very large, inherently high-dimensional, or naturally streaming datasets. To effectively tackle these extremely large-scale practical problems (e.g., building statistical models from massive data, real-time network traffic monitoring and anomaly detection), methods based on statistics and probability have become increasingly popular. This proposal aims at developing theoretical, well-grounded statistical methods for massive data based on random projections, including data stream algorithms, quantized projection algorithms, and sparse projection algorithms.Massive data are often generated as high-rate streams. Network traffic is a typical example. Effective measurements (and updates) of network traffic in real-time using small storage space are crucial for detecting anomaly events, for example the DDoS (Distributed Denial of Service) attacks. For many applications such as databases and machine learning, appropriate quantization of random projections will substantially improve the accuracies (in terms of variance per bit) and provide efficient indexing and dimension reductions to facilitate efficient search and learning. The proposed research will tackle a series of mathematically challenging problems in the development of random projections. A wide range of statistical learning and numerical linear algebra algorithms will be re-engineered to take advantage of the state-the-art projection methods.These days, many industries such as search are in urgent demand for statistical algorithms which can effectively handle massive data. It is expected that algorithms to be developed in this proposal will be integrated with parallel platforms, to solve truly large-scale real-world problems. Research results will be disseminated to practitioners through publications, conference presentations, industry visits and collaborations, tutorials, and open-source distributions. Many of the proposed research problems involve statistical analysis and may continue to help attract statisticians/mathematicians to work on area of big data. The proposed research activities will engage both undergraduate and graduate students in statistics and engineering, through innovative curriculum and research training.

随着Internet的出现，网络流量，搜索和数据库中的众多应用程序面临着非常大的，固有的高维或自然流数据集。为了有效解决这些极大的实用问题（例如，从大量数据，实时网络流量监测和异常检测中构建统计模型），基于统计和概率的方法变得越来越流行。该建议旨在基于随机投影，包括数据流算法，量化投影算法和稀疏投影算法，开发理论，良好的统计方法，用于大规模数据，通常以高速率流的形式生成。网络流量是一个典型的示例。实时使用小存储空间实时测量网络流量（和更新）对于检测异常事件，例如DDOS（分布式拒绝服务）攻击至关重要。对于许多应用程序（例如数据库和机器学习），对随机预测的适当量化将显着提高准确性（根据每位方差），并提供有效的索引和尺寸降低，以促进有效的搜索和学习。拟议的研究将在随机预测的发展中解决一系列数学上具有挑战性的问题。将重新设计各种统计学习和数值线性代数算法，以利用现状的投影方法。这些天，许多行业（例如搜索）迫切需要对可以有效处理大量数据的统计算法的需求。预计该提案将与并行平台集成在一起，以解决真正的大规模现实世界中的问题。研究结果将通过出版物，会议演示，行业访问和协作，教程和开源分布传播给从业人员。许多提出的研究问题涉及统计分析，并可能继续帮助吸引统计学家/数学家在大数据领域工作。拟议的研究活动将通过创新的课程和研究培训来吸引本科生和研究生统计学和工程学。