III: Large: Collaborative Research: Analysis Engineering for Robust End-to-End Data Science

III：大型：协作研究：稳健的端到端数据科学的分析工程

• 批准号: 1856641
• 负责人: Brad Myers
• 金额: $ 71.25万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1856641&HistoricalAwards=false
• 关键词: III; Large; Collaborative; Research; Analysis

From poor statistical practices leading to retractions of scientific "discoveries" to low-level spreadsheet errors subverting high-stakes analyses, failures of data analysis can have catastrophic consequences. The rapid growth of data science practice in the last decade has led to large collaborative efforts to develop new data processing, machine learning, and analytics tools that put more advanced data analysis into the hands of a wider audience of practitioners, from students to scientists to designers. The most dominant tool for data science is code, where cutting-edge algorithms can be applied from an existing libraries. However, as this democratization of data science has lowered the barrier to using advanced methods, safely using these tools under sound statistical practice remains as difficult as ever. To facilitate more robust data science, this project investigates models and tools for analysis engineering by data scientists who write programs. The focus is on the complete end-to-end process of data analysis performed with code: the iterative, and often exploratory, steps that analysts go through to turn data into This project will contribute insights and characterizations of analytic work, novel methods for capturing and analyzing data science activities, and develop new programming tools and visualization methods for authoring and validating analyses. If successful, this project will augment people's ability to conduct and assess data analyses, promoting more robust results and reducing the gap between novice and expert analysts. The findings and tools from the project will be incorporated into educational efforts, including classroom teaching and tutorials and available as open source software integrated into popular analytical environments (e.g., Jupyter).Data analysis is a central activity to scientific research, yet is too often conducted in an undisciplined fashion. This project treats the entire analytic process as our central phenomenon of study. The project will employ mixed methods to study and characterize common analysis practices and pitfalls, including direct observations of data analysts, large-scale analysis of computational notebooks, and instrumentation of analytic programming environments like JupyterLab. The project will contribute new methods for specifying and safeguarding analyses, including domain-specific languages and program synthesis methods to guide users to preferred next steps. It will also explore "multiverse" workflows to manage and assess a diversity of analysis decisions. Analogues of debugging and testing tools will be developed to flag problems and perform error analysis, while the capture and visualization of analytic provenance to aid reproducibility, verification, and collaborative review. The work will be evaluated through controlled studies, classroom use, and open-source deployment for wide-scale field use.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

从导致科学“发现”的缩回到颠覆高风险分析的低水平电子表格错误中，数据分析的失败可能会带来灾难性的后果。在过去十年中，数据科学实践的快速发展导致了开发新的数据处理，机器学习和分析工具的巨大协作努力，这些工具将更高级的数据分析置于从学生到科学家再到设计师的更多受众群体的手中。数据科学的最主要工具是代码，可以从现有库中应用最先进的算法。但是，由于数据科学的民主化降低了使用先进方法的障碍，因此在合理的统计实践下安全地使用这些工具仍然一如既往地困难。为了促进更强大的数据科学，该项目研究了编写程序的数据科学家分析工程的模型和工具。重点是使用代码执行的数据分析的完整端到端过程：分析师将数据转化为该项目的迭代，通常是探索性的步骤，将有助于分析工作的见解和特征，新的方法捕获和分析数据科学活动的方法，以及开发新的编程工具和可视化工具和可视化方法，用于创作和验证分析。如果成功，该项目将增强人们进行和评估数据分析的能力，促进更强大的结果并减少新手和专家分析师之间的差距。该项目的发现和工具将被纳入教育工作，包括课堂教学和教程，并作为集成到流行的分析环境中的开源软件（例如，Jupyter）。DATA分析是科学研究的核心活动，但通常以一种无关的方式进行。该项目将整个分析过程视为我们的研究中心现象。该项目将采用混合方法来研究和表征常见的分析实践和陷阱，包括直接观察数据分析师，对计算笔记本的大规模分析以及诸如Jupyterlab之类的分析编程环境的仪器。该项目将为指定和保护分析的新方法提供新的方法，包括特定于领域的语言和程序合成方法，以指导用户获得首选的下一步。它还将探索“多元宇宙”工作流程，以管理和评估各种分析决策。调试和测试工具的类似物将被开发以提示问题并执行错误分析，而分析起源的捕获和可视化以帮助可重复性，验证和协作审查。这项工作将通过对照研究，课堂使用和开源部署进行评估，以供大规模现场使用。该奖项反映了NSF的法定任务，并使用基金会的知识分子优点和更广泛的影响评估标准，认为值得通过评估来获得支持。