CAREER: Declarative Uncertainty

职业：声明的不确定性

• 批准号: 1750460
• 负责人: Oliver Kennedy
• 金额: $ 54.23万
• 依托单位: SUNY at Buffalo
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-03-01 至 2024-02-29
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1750460&HistoricalAwards=false
• 关键词: CAREER; Declarative; Uncertainty

Data is messy. Fortunately, with minimal human intervention, good data cleaning heuristics produce mostly reliable, usually actionable information from big, messy data. For instance, analysts might automate their curation workflows by using classifiers to predict missing attribute values, or by using an entity-resolver to find and merge duplicate records. Unfortunately, heuristics are also dangerous, as the result of heuristic curation is often taken as fact. Serious mistakes like people being denied a loan due to someone else's bad credit, 12-year olds being identified as terrorists, or billion dollar investment errors, often result when low-confidence, or uncertain heuristic inferences are treated as truth. Many principled tools like probabilistic databases already exist for automatically tracking potential errors in unreliable data, but these tools are not easy to use. As a result, analysts more often resort to simply documenting potential errors and hoping that anyone using the data will realize the implications. This proposal will enable data management systems that can query and organize uncertain data, without being hard to use. The specific aim of this proposal is to decouple the process of asking questions about uncertain data from mechanical concerns like why the data is uncertain, how the user wants to view uncertainty in query results, or which algorithms should be used. To enable this sort of "declarative uncertainty management," the project team will build on a system called Mimir that virtualizes uncertainty by augmenting data curation workflows (e.g., ETL pipelines) with a form of provenance capture through which heuristics can register alternative outputs (e.g., a schema matcher may register multiple potential matches). This provenance can then be used to synthesize a wide range of different physical and visual representations of uncertainty in data and in query results. To enable declarative uncertainty management, this proposal will address specific problems that fall into two general categories: (1) selecting and efficiently constructing qualitative summaries of uncertainty in query results, and (2) enhancing database query compilers and optimizers to support practical, efficient query processing over uncertain data. For further information see the project web page: http://mimirdb.infoThis award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据很乱。 幸运的是，在最少的人为干预的情况下，良好的数据清理启发式方法可以从大而混乱的数据中产生大部分可靠且通常可操作的信息。 例如，分析师可以通过使用分类器来预测缺失的属性值，或使用实体解析器来查找和合并重复记录来自动化其管理工作流程。 不幸的是，启发式方法也很危险，因为启发式管理的结果通常被视为事实。 当人们将低信心或不确定的启发式推论视为事实时，通常会导致严重错误，例如人们因他人的不良信用而被拒绝贷款、12岁的孩子被认定为恐怖分子或数十亿美元的投资错误。 许多原则性工具（例如概率数据库）已经存在，用于自动跟踪不可靠数据中的潜在错误，但这些工具并不易于使用。 因此，分析师通常只是简单地记录潜在的错误，并希望使用这些数据的任何人都能意识到其中的含义。 该提案将使数据管理系统能够查询和组织不确定的数据，并且不会难以使用。该提案的具体目标是将询问不确定数据的问题的过程与机械问题解耦，例如数据为何不确定、用户希望如何查看查询结果中的不确定性，或者应该使用哪些算法。 为了实现这种“声明式不确定性管理”，项目团队将构建一个名为 Mimir 的系统，该系统通过采用来源捕获形式增强数据管理工作流程（例如 ETL 管道）来虚拟化不确定性，启发式方法可以通过这种形式注册替代输出（例如，模式匹配器可以注册多个潜在的匹配）。 然后，可以使用该来源来合成数据和查询结果中不确定性的各种不同的物理和视觉表示。 为了实现声明性不确定性管理，该提案将解决分为两大类的具体问题：（1）选择和有效构建查询结果中不确定性的定性摘要，以及（2）增强数据库查询编译器和优化器以支持实用、高效的查询对不确定数据的处理。欲了解更多信息，请参阅项目网页：http://mimirdb.info 该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Runtime provenance refinement for notebooks

笔记本的运行时出处细化

• DOI: 10.1145/3530800.3534535
• 发表时间: 2022
• 期刊: Proceedings of the 14th International Workshop on the Theory and Practice of Provenance
• 作者: Deo, Nachiket;Glavic, Boris;Kennedy, Oliver
• 通讯作者: Kennedy, Oliver

Loki: Streamlining Integration and Enrichment

Loki：简化集成和丰富

• 发表时间: 2020
• 期刊: Human in the Loop Data Analytics
• 作者: Spoth, William;Kumari, Poonam;Kennedy, Oliver;Nargesian, Fatemeh
• 通讯作者: Nargesian, Fatemeh

Uncertainty Annotated Databases - A Lightweight Approach for Approximating Certain Answers

不确定性注释数据库 - 近似某些答案的轻量级方法

• DOI: 10.1145/3299869.3319887
• 发表时间: 2019
• 期刊: SIGMOD
• 作者: Feng, Su;Huber, Aaron;Glavic, Boris;Kennedy, Oliver
• 通讯作者: Kennedy, Oliver

DataSense: Display Agnostic Data Documentation

DataSense：显示不可知的数据文档

• 发表时间: 2021
• 期刊: Conference on Innovative Data Systems Research
• 作者: Kumari, Poonam;Brachmann, Michael;Kennedy, Oliver;Feng, Su;Glavic, Boris
• 通讯作者: Glavic, Boris

Query Log Compression for Workload Analytics

用于工作负载分析的查询日志压缩

• DOI: 10.14778/3291264.3291265
• 发表时间: 2018
• 期刊: Proceedings of the VLDB Endowment
• 影响因子: 2.5
• 作者: Xie, Ting;Chandola, Varun;Kennedy, Oliver
• 通讯作者: Kennedy, Oliver