EFFICIENT SPECTRAL APPROACHES FOR FINDING UNDERLYING STRUCTURES IN BIG DATA

用于查找大数据底层结构的高效谱方法

• 批准号: 9278252
• 负责人: Yuval Kluger
• 金额: $ 41万
• 依托单位: YALE UNIVERSITY
• 依托单位国家: 美国
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-05-24 至 2019-04-30
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9278252
• 关键词: Algorithmic Analysis; Algorithms; Architecture; Big Data; Binding; Bioinformatics; Biological; Categories; Cells; ChIP-seq; Clinical Research; Computer software; Confusion; Data; Data Analyses; Data Analytics; Data Discovery; Data Set; Databases; Detection; Development; Dimensions; Employee Strikes; Event; Excision; Future; Genomics; Goals; Knowledge Discovery; Language; Lead; Massive Parallel Sequencing; Mathematics; Memory; Methods; Microprocessor; Nucleotides; Numeric Rating Scale; Pattern; Performance; Randomized; Research; Running; Science; Signal Transduction; Stream; Structure; Techniques; Time; Validation; Variant; base; big biomedical data; computerized data processing; computerized tools; design; experimental study; falls; genome wide association study; genomic data; health care delivery; high dimensionality; improved; insertion/deletion mutation; insight; interest; learning strategy; novel; novel strategies; population stratification; prototype; public health relevance; repository; tool; user-friendly; whole genome

 DESCRIPTION (provided by applicant)l Recently we developed several spectral approaches for analyzing very large genomics datasets or complete databases that fall into the category of big data (BD). The first approach is designed to perform SVD or PCA based on randomization that can dramatically accelerate the computation of their eigenvectors and eigenvalues relative to the standard Lanczos algorithm implemented in all common software packages. Computing PCA and the SVD more efficiently could revolutionize the innumerable biomedical applications based on PCA and the SVD, e.g. population stratification in very large GWAS. These algorithms produce higher accuracy than classical (deterministic) methods, enable the processing of data streams that are too large to store, and parallelize easily to be used in multicore microprocessors. Our second novel approach is an unsupervised spectral learning method. It provides new mathematical insights of striking conceptual simplicity for ranking multiple competing algorithms without access to validation data and for optimally combining this ensemble of algorithms to obtain improved predictions in the absence of ground truth. Constructing a tool that provides end users an option to optimally rank or combine algorithms for analysis of genomics data is a practical and efficient solution to remove the confusion among end-users or bioinformaticians who are faced with the need to decide which tool to choose for their study, as a large number of biological results inferred by the different tools are often in disagreement. The choice of the best performing algorithm or pipeline is essential as it can often lead to substantial improvement in quality of the readout from massively parallel sequencing experiments. Moreover, combining these tools typically results in performance superior to the best performing algorithm. Our goal is to establish a team whose focus is to provide and disseminate full-blown implementations of spectral BD tools and methods that have broad applicability across the spectrum of biomedical sciences, clinical research, and healthcare delivery. Specifically we will develop scalable PCA and SVD for Genomics and biomedical applications, further advance our spectral method for ranking the performance of competing pipelines and combine them to achieve better predictions without access to validation data. Moreover, we will develop scalable dimensional reduction techniques for organizing BD from biomedical applications.

 描述（由申请人提供）最近，我们开发了几种光谱方法，用于分析属于大数据（BD）类别的非常大的基因组数据集或完整数据库。第一种方法旨在执行基于随机化的 SVD 或 PCA，可以显着地执行 SVD 或 PCA。相对于所有常见软件包中实现的标准 Lanczos 算法，加速其特征向量和特征值的计算，更有效地计算 PCA 和 SVD 可以彻底改变无数的问题。基于 PCA 和 SVD 的生物医学应用，例如非常大的 GWAS 中的群体分层，这些算法比传统（确定性）方法具有更高的精度，可以处理太大而无法存储的数据流，并且可以轻松并行化以在多核中使用。我们的第二个新颖方法是一种无监督的光谱学习方法，它提供了概念简单的新数学见解，可以在无需访问验证数据的情况下对多个竞争算法进行排名，并最佳地组合该算法集合以获得改进的预测。构建一个工具，为最终用户提供最佳排名或组合基因组数据分析算法的选项，是消除最终用户或需要做出决定的生物信息学家之间的困惑的实用且有效的解决方案。选择哪种工具进行研究，因为不同工具推断出的大量生物学结果往往不一致。选择性能最佳的算法或流程至关重要，因为它通常可以显着提高读数的质量。大规模并行测序实验。此外，结合这些工具通常会产生优于最佳性能算法的性能，我们的目标是建立一个团队，其重点是提供和传播在生物医学科学领域具有广泛适用性的光谱 BD 工具和方法的成熟实现。具体来说，我们将为基因组学和生物医学应用开发可扩展的 PCA 和 SVD，进一步推进我们的光谱方法对竞争管道的性能进行排名，并将它们结合起来以实现更好的预测，而无需验证。此外，我们将开发可扩展的降维技术，用于组织生物医学应用中的 BD。