Research Initiation Award: Integrating Image and Text Information for Biomedical Literature-Based Cross and Multimodal Retrieval

研究启动奖：基于图像和文本信息的生物医学文献交叉和多模态检索整合

• 批准号: 1601044
• 负责人: Md Rahman
• 金额: $ 29.78万
• 依托单位: Morgan State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-06-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1601044&HistoricalAwards=false
• 关键词: Research; Initiation; Award; Integrating; Image

The Historically Black Colleges and Universities-Undergraduate Program (HBCU-UP) Research Initiation Awards (RIAs) provide support to STEM junior faculty at HBCUs who are starting to build a research program, as well as for mid-career faculty who may have returned to the faculty ranks after holding an administrative post or who needs to redirect and rebuild a research program. Faculty members may pursue research at their home institution, at an NSF-funded Center, at a research intensive institution or at a national laboratory. The RIA projects are expected to help further the faculty member's research capability and effectiveness, to improve research and teaching at his or her home institution, and to involve undergraduate students in research experiences. With support from the National Science Foundation, Morgan State University will conduct research in information retrieval using search strategies based on techniques from image processing as well as natural language processing. This would enable public access to both visual information and take away messages from journal articles. This project will provide valuable research experience and mentorship for several minority undergraduate students at Morgan State University. In addition, the project will help Morgan State University build its research capacity and enhance the educational and research experiences of their undergraduate students. Within the larger goal of expanding queries for information retrieval, the project will 1) use a crowdsourcing based approach to perform large scale manual annotation of visual regions of interest (ROIs) by pairing automatically detected ROIs to concepts occurring in a brief caption, 2) use a feature learning approach to extract discriminative features from ROIs and automatically map the ROIs to concepts in an existing textual ontology, such as RadLex, 3) aided by a visual ontology, consider the semantic relations between the visual words when assessing the distance between images described with the bag-of-visual-words feature representation scheme, 4) in addition to cross modal search by mapping image regions to concepts in ontology, perform multimodal search by fusing weighted text and image features generated by a multi-response linear regression (MLR)-based meta-learner in a classification-driven task-specific manner, and 5) evaluate the retrieval techniques using benchmark and realistic datasets by participating in the yearly ImageCLEF retrieval evaluation campaign. The labeled set of biomedical images with annotated regions of interest will be made available to the research community.

历史上黑人学院和大学本科项目 (HBCU-UP) 研究启动奖 (RIA) 为 HBCU 中开始建立研究项目的 STEM 初级教师以及可能已返回美国的职业中期教师提供支持。教师按照担任行政职务或需要重新调整和重建研究项目的人员进行排名。教职员工可以在其所在机构、国家科学基金会资助的中心、研究密集型机构或国家实验室进行研究。 RIA 项目预计将有助于提高教师的研究能力和效率，改善其所在机构的研究和教学，并让本科生参与研究经验。在国家科学基金会的支持下，摩根州立大学将使用基于图像处理和自然语言处理技术的搜索策略进行信息检索研究。 这将使公众能够获取视觉信息并从期刊文章中获取信息。 该项目将为摩根州立大学的几名少数族裔本科生提供宝贵的研究经验和指导。 此外，该项目还将帮助摩根州立大学建设研究能力，并增强本科生的教育和研究经验。在扩展信息检索查询的更大目标中，该项目将 1) 使用基于众包的方法，通过将自动检测到的 ROI 与简短标题中出现的概念配对，对视觉感兴趣区域 (ROI) 进行大规模手动注释，2)使用特征学习方法从 ROI 中提取判别性特征，并自动将 ROI 映射到现有文本本体中的概念，例如 RadLex，3）在视觉本体的辅助下，考虑视觉词之间的语义关系评估用视觉词袋特征表示方案描述的图像之间的距离，4）除了通过将图像区域映射到本体中的概念来进行跨模态搜索之外，还通过融合由多个生成器生成的加权文本和图像特征来执行多模态搜索-以分类驱动的特定任务方式响应基于线性回归（MLR）的元学习器，5）通过参与年度 ImageCLEF 检索评估活动，使用基准和实际数据集评估检索技术。 带有注释的感兴趣区域的标记的生物医学图像集将提供给研究界。