CAREER: Distilling information structure from big and dirty data: Efficient learning of clusters and graphs in modern datasets

职业：从大数据和脏数据中提取信息结构：现代数据集中集群和图的高效学习

• 批准号: 1252412
• 负责人: Aarti Singh
• 金额: $ 50万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-03-01 至 2018-02-28
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1252412&HistoricalAwards=false
• 关键词: CAREER; Distilling; information; structure; big

This CAREER project aims to advance the state-of-the-art in theory and methods for extracting clusters and graphs from big and dirty datasets arising in modern application domains. Clusters and graphs provide a meaningful representation of the structure of information contained in data, e.g. in neuroscience and health care domains, clustering patients with similar phenotypes and genotypes helps identify target groups for drug design, clustering fiber tracks generated by high-resolution Digital Surface Imaging (DSI) scans of brains help identify significant neural pathways, and graph structures can reflect connectivity between brain regions. The results of this work will significantly enhance the ability to exploit such modern datasets through new methods for learning clusters and graphs from data that is large-scale, high-dimensional, under-sampled, corrupted, and often only available in a compressed or streaming representation. Specifically, this project will develop computationally efficient and principled methods for learning clusters and graphs that can (i) perform unsupervised feature selection to discard irrelevant features in high dimensions, (ii) leverage feedback based on intelligent adaptive queries that focus resources on most informative variables and features, (iii) use compressive measurement design that adapts to the information structure for measurement and computation efficiency, and (iv) be able to handle noisy streaming data. The algorithms will be accompanied with performance guarantees in the form of a precise characterization of the mis-clustering rate and graph recovery error. Additionally, the project will investigate the tradeoffs between number of measurements, computational complexity and robustness in these problems. The methods and theory developed will be evaluated through simulations as well as their applicability to real datasets in neuroscience and healthcare domain, in collaboration with practitioners from these fields. The results of this research could potentially transform many application domains that involve grouping similar variables and learning complex interactions between them, based on big and dirty datasets. In particular, the neuroscience and healthcare applications are likely have very direct and significant implications for society. Accurately mapping neural pathways will help diagnose and treat brain pathologies at an early stage, and help understand brain functioning. Clustering patients and discovering disease spreading pathways based on few measurements of relevant genetic features or indicators could help prevent and cure diseases, and also minimize healthcare costs. The research activities will be tightly integrated with education efforts that aim to develop a diverse workforce that is better equipped with cross-disciplinary tools to address the challenges of modern datasets. The education plan includes development of two inter-disciplinary courses, and enhancement of the joint Statistics & Machine Learning PhD program at Carnegie Mellon University (CMU). Outreach activities include promoting undergraduate research, broadening participation of women and underrepresented groups in STEM fields through OurCS (Opportunities for Undergraduate Research in Computer Science), Andrew?s Leap (a summer enrichment program for area high school and middle school students) and CS4HS program aimed at High School and K-8 teachers at Carnegie Mellon University. The results of this project (including publications, data sets, and software) will be disseminated online at http://www.cs.cmu.edu/~aarti/research_projects/.

该职业项目旨在推进理论的最新方法，以及从现代应用程序域中出现的大而肮脏数据集中提取群集和图形的方法。簇和图提供了数据中包含的信息结构的有意义表示，例如在神经科学和医疗保健领域中，具有相似表型和基因型的聚类患者有助于识别药物设计的目标群，聚集纤维轨道由高分辨率数字表面成像（DSI）扫描产生的大脑扫描有助于识别重要的神经途径，并且图形结构可以反映大脑区域之间的连接性。这项工作的结果将大大增强通过学习群集和图形的新方法来利用此类现代数据集的能力，这些方法是从大规模，高维，不采样，损坏，损坏且通常仅在压缩或流式传输表示中可用的数据的能力。具体而言，该项目将开发用于学习簇和图表的计算高效和原则性方法，可以（i）执行无监督的功能选择，以丢弃高维度的无关特征，（ii）利用基于智能的自适应查询来利用反馈，以智能的适应性查询，这些查询将资源侧重于大多数信息的效率和计算效率，以适应大多数信息的效率，（III），（iii）适用于适应性的效果，（III），（III）适应了信息（III），（III）效率（iii），（iii）适应性（iii），（iii）的信息（iii），（iii）的信息（iii）效率（iii）的信息（iii）效率（iii）效率（iii）。处理嘈杂的流数据。该算法将伴随性能保证，以精确表征误差率和图形恢复误差的形式。此外，该项目将调查这些问题中测量数量，计算复杂性和鲁棒性之间的权衡。开发的方法和理论将通过模拟以及它们适用于神经科学和医疗保健领域的真实数据集的适用性，并与这些领域的从业人员合作。这项研究的结果可能会改变许多应用程序域，这些应用程序域涉及基于大和肮脏的数据集将相似变量分组和学习复杂的相互作用。特别是，神经科学和医疗保健应用可能对社会具有非常直接和重要的影响。准确地绘制神经通路将有助于早期诊断和治疗大脑病理，并有助于了解大脑功能。基于几乎相关的遗传特征或指标的测量，将患者聚类并发现疾病扩散途径可以帮助预防和治愈疾病，并最大程度地减少医疗保健费用。研究活动将与教育工作紧密整合，旨在开发多样化的劳动力，该劳动力更好地配备了跨学科工具来应对现代数据集的挑战。教育计划包括开发两项跨学科课程，以及在卡内基·梅隆大学（CMU）的联合统计和机器学习博士计划的增强。外展活动包括促进本科研究，通过ORCS（在计算机科学领域的本科研究的机会），Andrew'S Leap（针对地区高中和中学生的夏季丰富计划）以及CS4HS计划的夏季丰富计划和Carnegie Mellon Mellon Mellon University的K-8老师。该项目的结果（包括出版物，数据集和软件）将在http://www.cs.cmu.edu/~aarti/~aarti/research_projects/上在线传播。