SHF:Small:Language Support for Ad Hoc Data Processing

SHF:Small:对即席数据处理的语言支持

• 批准号: 1016937
• 负责人: David Walker
• 金额: $ 50万
• 依托单位: Princeton University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-08-15 至 2014-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1016937&HistoricalAwards=false
• 关键词: SHF; Small; Language; Support; Ad

In every business, engineering endeavour and scientific discipline, workers are digitizing their knowledge with the hope of using computational methods to categorize, query, filter, search, diagnose, and visualize their data. While this effort is leading to remarkable industrial and scientific advances, it is also generating enormous amounts of ad hoc data (i.e., that data for which standard data processing tools such as query engines, statistical packages, graphing tools, or other software is not readily available). Ad hoc data poses tremendous challenges to its users because it is often highly varied, poorly documented, filled with errors, and continuously evolving --- yet ad hoc data also contains much valuable information. The goal of this research is to develop general-purpose software tools and techniques capable of managing ad hoc data efficiently. This research has the potential for a broad impact on society by dramatically improving the productivity of industrial data analysts, computer systems administrators and academics who must deal with ad hoc data on a day-to-day basis.The central technical challenge of the research involves designing, implementing and evaluating a new domain-specific programming language that facilitates the management of ad hoc data sets. This new programming language will allow data analysts to specify the structure of ad hoc data files, how those files are arranged in a file system and what meta-data is associated with them. Once a specification is complete, it will be possible to use it as documentation for the data set or for generating data-processing tools. The research will also involve developing new methods for enabling users to generate specifications quickly and accurately, without actually having to write down all of the details by hand. Finally, the research will develop new algorithms for implementing the generated data-processing tools efficiently.

在每个业务，工程努力和科学纪律中，工人都在数字化知识，希望使用计算方法对其数据进行分类，查询，过滤，搜索，诊断和可视化数据。尽管这项努力导致了显着的工业和科学进步，但它还产生了大量的临时数据（即，不随时可用的标准数据处理工具，例如查询引擎，统计包，绘制工具或其他软件的数据）。临时数据对其用户构成了巨大的挑战，因为它通常多样化，文献不佳，充满错误并不断发展，但临时数据也包含很多有价值的信息。 这项研究的目的是开发能够有效管理临时数据的通用软件工具和技术。 这项研究通过极大地提高工业数据分析师，计算机系统管理人员和学者的生产率来对社会产生广泛的影响，他们必须每天处理临时数据。该研究的核心技术挑战涉及设计，实施和评估新的领域特异性编程语言，以促进Ad Hoc数据集管理的管理。 这种新的编程语言将允许数据分析师指定临时数据文件的结构，这些文件在文件系统中的排列方式以及与它们相关的元数据。 规范完成后，就可以将其用作数据集或生成数据处理工具的文档。 该研究还将涉及开发新方法，以使用户能够快速，准确地生成规格，而无需实际手工写下所有细节。 最后，该研究将开发新算法，以有效地实施生成的数据处理工具。