CHS:Small:Utilizing synergy between human and computer information processing for complex visual information organization and use

CHS:Small：利用人与计算机信息处理之间的协同作用来组织和使用复杂的视觉信息

基本信息

批准号：
1814450
负责人：
Qi Yu
金额：
$ 49.74万
依托单位：
Rochester Institute of Tech
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2018
资助国家：
美国
起止时间：
2018-07-15 至 2024-06-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1814450&HistoricalAwards=false
关键词：
CHS Small Utilizing synergy between

项目摘要

CHS: Small: Utilizing synergy between human and computer information processing for complex visual information organization and useRecent years have brought important advances in the use of computational approaches to automatically extract the meaning of an image (aka, image semantics). But, understanding these images when they relate to specialized areas such as medicine is significantly more challenging because it depends on human expertise. This project brings together human and computer capabilities to discover image semantics by (1) encoding human image inspection and analysis behaviors that represent domain expertise, and (2) algorithmically fusing human expertise with image data. The outcomes will help to provide truly meaningful interpretations of complex images in areas such as medicine, science, and security intelligence. This interdisciplinary project will provide extensive research opportunities for undergraduate and graduate students and for broadening participation in computing. The team will leverage the college?s successful program for Women in Computing and PhD Program that has a strong track record in recruiting students from underrepresented and culturally-diverse groups. The research will contribute novel computational models to capture the complex and unique features of human language and vision related to performing image understanding tasks, and an innovative probabilistic framework to fuse human knowledge data with image features. Interpretable knowledge patterns will be extracted to inform high-level abstractions of human expertise and establish cross-modality relationships. The hierarchical probabilistic framework will promote a systematic fusion of multimodal knowledge data with image content. By fusing data from multiple, complimentary modalities, the framework is robust to sparseness, noise, and ambiguity in human knowledge data while being flexible when one or more data modalities become unavailable. Through nonparametric modeling, the framework can account for the novel semantics resulting from human expertise, hence closely represent the knowledge-based processing in human image understanding.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

CHS：小：用于复杂的视觉信息组织和用户订婚年之间的人与计算机信息处理之间的协同作用已在使用计算方法自动提取图像含义（aka，图像语义）方面带来了重要的进步。但是，当这些图像与专业领域（例如医学）相关时，理解它们是更具挑战性的，因为它取决于人类的专业知识。该项目汇集了人类和计算机功能，通过（1）编码代表领域专业知识的人类图像检查和分析行为，以及（2）算法将人类专业知识与图像数据融合在一起。结果将有助于在医学，科学和安全情报等领域提供对复杂图像的真正有意义的解释。这个跨学科项目将为本科生和研究生提供广泛的研究机会，并扩大参与计算的参与。该团队将利用该学院在计算机和博士学位课程中为女性成功计划，该计划在招募代表性不足和文化多样性团体的学生方面具有良好的记录。该研究将促进新颖的计算模型，以捕获与执行图像理解任务有关的人类语言和视觉的复杂和独特特征，以及一个创新的概率框架，将人类知识数据与图像特征融合在一起。将提取可解释的知识模式，以告知人类专业知识的高级抽象并建立跨模式关系。分层概率框架将促进多模式知识数据与图像内容的系统融合。通过融合来自多种，免费的方式的数据，该框架对人类知识数据的稀疏，噪音和歧义是可靠的，而当一个或多个数据模式变得不可用时，则具有灵活性。通过非参数建模，该框架可以解释人类专业知识所产生的新语义，因此非常代表了人类形象理解中基于知识的处理。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子和更广泛影响的审查标准通过评估来进行评估的。

项目成果

期刊论文数量（26）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Integrating Bayesian and Discriminative Sparse Kernel Machines for Multi-class Active Learning

DOI：
发表时间：
2019
期刊：
影响因子：
0
作者：
Weishi Shi;Qi Yu
通讯作者：
Weishi Shi;Qi Yu

Knowledge Acquisition for Human-In-The-Loop Image Captioning

DOI：
发表时间：
2023
期刊：
影响因子：
0
作者：
Ervine Zheng;Qi Yu;Rui Li;Pengcheng Shi;Anne R. Haake
通讯作者：
Ervine Zheng;Qi Yu;Rui Li;Pengcheng Shi;Anne R. Haake

An Efficient Many-Class Active Learning Framework for Knowledge-Rich Domains

知识丰富领域的高效多类主动学习框架

DOI：
10.1109/icdm.2018.00164
发表时间：
2018
期刊：
2018 IEEE International Conference on Data Mining (ICDM
影响因子：
0
作者：
Shi, Weishi;Yu, Qi
通讯作者：
Yu, Qi

Integrating Multi-level Tag Recommendation with External Knowledge Bases for Automatic Question Answering

多级标签推荐与外部知识库融合实现自动问答

DOI：
10.1145/3319528
发表时间：
2019
期刊：
ACM Transactions on Internet Technology
影响因子：
5.3
作者：
Lima, Eduardo;Shi, Weishi;Liu, Xumin;Yu, Qi
通讯作者：
Yu, Qi

Balancing Bias and Variance for Active Weakly Supervised Learning

DOI：
10.1145/3534678.3539264
发表时间：
2022-06
期刊：
Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
影响因子：
0
作者：
Hitesh Sapkota;Qi Yu
通讯作者：
Hitesh Sapkota;Qi Yu