III: Small: Collaborative Research: Algorithms, systems, and theories for exploiting data dependencies in crowdsourcing

III：小型：协作研究：在众包中利用数据依赖性的算法、系统和理论

基本信息

批准号：
2007941
负责人：
Qi Li
金额：
$ 25万
依托单位：
Iowa State University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2020
资助国家：
美国
起止时间：
2020-10-01 至 2024-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2007941&HistoricalAwards=false
关键词：
III Small Collaborative Research Algorithms

项目摘要

Data are abundantly available to encode knowledge in many domains, such as biomedical research, online commerce, open government, education, and public health. Machine learning is a powerful tool to discover novel knowledge from data and to help individuals and organizations make informed decisions. However, machine learning needs to be bootstrapped by human-annotated knowledge, which can be expensive to obtain and also contain human errors. The team of researchers discovers and exploits the dependencies in the data, via novel methodologies to significantly reduce the cost and noises when providing critical knowledge for machine learning. The research outputs, including algorithms, systems, and theories, are sufficiently generic to benefit many domains where machine learning is applicable. By conducting the fundamental research, the team will train undergraduates and graduates for the STEM workforce in the nation.The researchers will collaborate to develop algorithms, systems, and theories for reducing costs and noises when annotating dependent data, termed as “structured annotations”, to provide supervision knowledge for machine learning. While the dependencies can make data annotations costly and error-prone, the researchers view the dependencies as a useful inductive bias for selective and accurate annotations. In particular, the research team proposes a human-in-the-loop system to aid the construction of proper probabilistic graphical models to encode the dependencies. The project team combines contextual and multi-armed bandits with scalable graph inference algorithms to reduce labeling costs. Based on the graphical bandits, the team addresses the budget allocation when querying labels of the same data point repetitively for robustness. With noisy human annotations, the team formulates optimization problems and algorithms to jointly infer the annotator competences and the ground truth labels of the data. From the theoretical perspective, the project will advance the active learning in crowdsourcing settings with more realistic noise distributions and will analyze the regrets in structured annotations. The project will result in datasets, algorithms, and a testbed system that benefit not only the core machine learning research community but also many domains that use machine learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据可用于编码许多领域的知识，例如生物医学研究、在线商务、开放政府、教育和公共卫生。机器学习是从数据中发现新知识并帮助个人和组织做出明智决策的强大工具。然而，机器学习需要由人工注释的知识来引导，这些知识的获取成本可能很高，而且还包含人为错误。研究人员团队通过新颖的方法发现和利用数据中的依赖性，以显着降低成本和噪音。为机器提供关键知识研究成果（包括算法、系统和理论）足够通用，可以使机器学习适用的许多领域受益。通过进行基础研究，该团队将为全国的 STEM 劳动力培训本科生和研究生。研究人员将合作开发算法、系统和理论，以减少注释依赖数据（称为“结构化注释”）时的成本和噪音，为机器学习提供监督知识。虽然依赖关系可能会使数据注释成本高昂且容易出错。查看依赖关系作为选择性和准确注释的有用归纳偏差，研究团队提出了一种人机循环系统，以帮助构建适当的概率图形模型来编码依赖关系。使用可扩展的图形推理算法来降低标记成本基于图形 bandits，该团队在重复查询同一数据点的标签时解决了预算问题，以确保鲁棒性。制定优化问题和算法来共同推断注释者能力和数据的真实标签，该项目将以更真实的噪声分布推进众包环境中的主动学习，并分析结构化注释项目中的遗憾。将产生数据集、算法和测试平台系统，不仅有利于核心机器学习研究社区，而且有利于许多使用机器学习的领域。该奖项是 NSF 的法定使命，并通过使用基金会的智力评估进行评估，被认为值得支持优点和更广泛的影响审查标准。

项目成果

期刊论文数量（9）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

SagDRE: Sequence-Aware Graph-Based Document-Level Relation Extraction with Adaptive Margin Loss

DOI：
10.1145/3534678.3539304
发表时间：
2022-08
期刊：
Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
影响因子：
0
作者：
Ying Wei;Qi Li
通讯作者：
Ying Wei;Qi Li

Relation-Aware Network with Attention-Based Loss for Few-Shot Knowledge Graph Completion

DOI：
10.1007/978-3-031-33380-4_8
发表时间：
2023-06
期刊：
ArXiv
影响因子：
0
作者：
Qiao Qiao-Qiao;Yuepei Li;Kang Zhou;Qi Li
通讯作者：
Qiao Qiao-Qiao;Yuepei Li;Kang Zhou;Qi Li

CPTAM: Constituency Parse Tree Aggregation Method

CPTAM：选区解析树聚合方法

DOI：
10.1137/1.9781611977172.71
发表时间：
2022
期刊：
Proceedings of the SIAM International Conference on Data Mining
影响因子：
0
作者：
Kulkarni, Adithya;Sabetpour, Nasim;Markin, Alexey;Eulenstein, Oliver;Li, Qi
通讯作者：
Li, Qi

Data Poisoning Attacks and Defenses to Crowdsourcing Systems

DOI：
10.1145/3442381.3450066
发表时间：
2021-02
期刊：
Proceedings of the Web Conference 2021
影响因子：
0
作者：
Minghong Fang;Minghao Sun;Qi Li;N. Gong;Jinhua Tian;Jia Liu
通讯作者：
Minghong Fang;Minghao Sun;Qi Li;N. Gong;Jinhua Tian;Jia Liu

Open-Domain Aspect-Opinion Co-Mining with Double-Layer Span Extraction

DOI：
10.1145/3534678.3539386
发表时间：
2022-08
期刊：
Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
影响因子：
0
作者：
Mohna Chakraborty;Adithya Kulkarni;Qi Li
通讯作者：
Mohna Chakraborty;Adithya Kulkarni;Qi Li