BIGDATA: Collaborative Research: F: Big Data, It's Not So Big: Exploiting Low-Dimensional Geometry for Learning and Inference

BIGDATA：协作研究：F：大数据，它并不是那么大：利用低维几何进行学习和推理

• 批准号: 1546132
• 负责人: Shayn Mukherjee
• 金额: $ 32.22万
• 依托单位: Duke University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2015
• 资助国家: 美国
• 起止时间: 2015-12-01 至 2018-11-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1546132&HistoricalAwards=false
• 关键词: BIGDATA; Collaborative; Research; Big; Data

This research will leverage ideas from algebraic and differential geometry to address core problems in modern high-dimensional and massive data science. The project will develop statistical methods and numerical tools, grounded in solid mathematical, statistical, and computational foundations, to extract low dimensional geometry from massive data with applications in clustering, data summarization, prediction, dimension reduction, and visualization. The solutions developed as part of this project can result in fundamental advances in practical applications across fields as diverse as biology, medicine, social sciences, communication networks, and engineering. In addition to internal validation via statistical and mathematical theory and simulation studies, the methods developed in the project will involve external validation via interdisciplinary applications. These applications include: (1) inference of population structure from genomic data; (2) document analysis via topic models; and (3) inference of subsets of putative gene networks relevant to drug resistance in melanoma.The research is motivated by the central premise that, even though the amount of data may be massive, a compact model can represent these data. Specifically, high-dimensional and/or massive data can be reasonably approximated by a mixture of subspaces, for which sparse representations exist. A mixture of subspaces of potentially different dimensions is a flexible, rich representation of data with nice mathematical properties that can scale to large data. There are several fundamental challenges in modeling mixtures of subspaces that will be addressed in this research: 1) the subspaces will be of different dimensions, 2) both the subspace parameters and the mixing parameters need to be inferred, 3) efficient algorithms for inference are required for both high-dimensional and massive data. The central foundational impediment in all of these challenges is that the model is a stratified space (a union of manifolds), and therefore has singularities. The key insight in this research is that there exist embeddings and representations of the model space that mitigate these singularities. These ideas are implemented as concrete Bayesian, frequentist, and numerical algorithms and models to address the real world examples listed above.

这项研究将利用代数和差异几何形状的思想来解决现代高维数据科学的核心问题。 该项目将开发以固体数学，统计和计算基础为基础的统计方法和数值工具，以从大量数据中提取低维几何形状，并在聚类，数据汇总，预测，尺寸降低和可视化中应用。 作为该项目的一部分开发的解决方案可能会导致在生物学，医学，社会科学，通信网络和工程等多样化领域的实际应用中的基本进步。 除了通过统计和数学理论和仿真研究进行内部验证外，项目中开发的方法还将涉及通过跨学科应用程序进行外部验证。 这些应用包括：（1）从基因组数据中推断人口结构； （2）通过主题模型进行文档分析； （3）推断与黑色素瘤中与耐药性相关的推定基因网络子集的推断。该研究的动机是由中心前提的，即，即使数据量可能大量，紧凑的模型也可以代表这些数据。 具体而言，高维和/或大量数据可以通过子空间的混合物合理地近似，为此存在稀疏表示。 潜在不同维度的子空间的混合物是具有具有良好数学属性的数据的灵活，丰富的数据，可以扩展到大数据。 在本研究中将解决的子空间的混合物建模时面临一些基本挑战：1）子空间将具有不同的维度，2）需要推断子空间参数和混合参数，需要推断出有效的推理算法。 所有这些挑战中的中心基础障碍是该模型是一个分层空间（一种流形的结合），因此具有奇异性。 这项研究的关键见解是，模型空间的嵌入和表示可以减轻这些奇异性。 这些想法被实现为具体的贝叶斯，频繁主义和数值算法和模型，以解决上面列出的现实示例。