CAREER: Weakly-Supervised Visual Scene Understanding: Combining Images and Videos, and Going Beyond Semantic Tags
职业:弱监督视觉场景理解:结合图像和视频,超越语义标签
基本信息
- 批准号:2150012
- 负责人:
- 金额:$ 50.05万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Continuing Grant
- 财政年份:2021
- 资助国家:美国
- 起止时间:2021-09-01 至 2024-03-31
- 项目状态:已结题
- 来源:
- 关键词:
项目摘要
The internet provides an endless supply of images and videos, replete with weakly-annotated meta-data such as text tags, GPS coordinates, timestamps, or social media sentiments. This huge resource of visual data provides an opportunity to create scalable and powerful recognition algorithms that do not depend on expensive human annotations. The research component of this project develops novel visual scene understanding algorithms that can effectively learn from such weakly-annotated visual data. The main novelty is to combine both images and videos together. The developed algorithms could have broad impact in numerous fields including AI, security, and agricultural sciences. In addition to scientific impact, the project performs complementary educational and outreach activities. Specifically, it provides mentorship to high school, undergraduate, and graduate students, teaches new undergraduate and graduate computer vision courses that have been lacking at UC Davis, and organizes an international workshop on weakly-supervised visual scene understanding.This project develops novel algorithms to advance weakly-supervised visual scene understanding in two complementary ways: (1) learning jointly with both images and videos to take advantage of their complementarity, and (2) learning from weak supervisory signals that go beyond standard semantic tags such as timestamps, captions, and relative comparisons. Specifically, it investigates novel approaches to advance tasks like fully-automatic video object segmentation, weakly-supervised object detection, unsupervised learning of object categories, and mining of localized patterns in the image/video data that are correlated with the weak supervisory signal. Throughout, the project explores ways to understand and mitigate noise in the weak labels and to overcome the domain differences between images and videos.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
互联网提供了无穷无尽的图像和视频,其中充满了弱注释的元数据,例如文本标签、GPS 坐标、时间戳或社交媒体情绪。这种巨大的视觉数据资源提供了创建可扩展且强大的识别算法的机会,这些算法不依赖于昂贵的人工注释。该项目的研究部分开发了新颖的视觉场景理解算法,可以有效地从此类弱注释的视觉数据中学习。主要的新颖之处是将图像和视频结合在一起。开发的算法可能会对人工智能、安全和农业科学等众多领域产生广泛影响。除了科学影响之外,该项目还开展补充性的教育和推广活动。具体来说,它为高中生、本科生和研究生提供指导,教授加州大学戴维斯分校缺乏的新本科生和研究生计算机视觉课程,并组织弱监督视觉场景理解国际研讨会。该项目开发新颖的算法以两种互补的方式推进弱监督视觉场景理解:(1)与图像和视频联合学习以利用它们的互补性,以及(2)从超越标准语义标签(例如时间戳、标题、和相对比较。具体来说,它研究了先进任务的新方法,例如全自动视频对象分割、弱监督对象检测、对象类别的无监督学习以及挖掘图像/视频数据中与弱监督信号相关的局部模式。自始至终,该项目都在探索如何理解和减轻弱标签中的噪音,并克服图像和视频之间的领域差异。该奖项反映了 NSF 的法定使命,并通过使用基金会的智力优点和更广泛的影响审查进行评估,被认为值得支持标准。
项目成果
期刊论文数量(13)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
The Two Dimensions of Worst-case Training and Their Integrated Effect for Out-of-domain Generalization
- DOI:10.1109/cvpr52688.2022.00941
- 发表时间:2022-04
- 期刊:
- 影响因子:0
- 作者:Zeyi Huang;Haohan Wang;Dong Huang;Yong Jae Lee;Eric P. Xing
- 通讯作者:Zeyi Huang;Haohan Wang;Dong Huang;Yong Jae Lee;Eric P. Xing
Generating Furry Cars: Disentangling Object Shape and Appearance across Multiple Domains
生成毛茸茸的汽车:跨多个领域理清对象形状和外观
- DOI:
- 发表时间:2021
- 期刊:
- 影响因子:0
- 作者:Ojha, Utkarsh;Singh, Krishna Kumar;Lee, Yong Jae
- 通讯作者:Lee, Yong Jae
Collaging Class-specific GANs for Semantic Image Synthesis
- DOI:10.1109/iccv48922.2021.01415
- 发表时间:2021-10
- 期刊:
- 影响因子:0
- 作者:Yuheng Li;Yijun Li;Jingwan Lu;Eli Shechtman;Yong Jae Lee;Krishna Kumar Singh
- 通讯作者:Yuheng Li;Yijun Li;Jingwan Lu;Eli Shechtman;Yong Jae Lee;Krishna Kumar Singh
Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features
- DOI:
- 发表时间:2021-11
- 期刊:
- 影响因子:0
- 作者:Haohan Wang;Zeyi Huang;Hanlin Zhang;Eric P. Xing
- 通讯作者:Haohan Wang;Zeyi Huang;Hanlin Zhang;Eric P. Xing
Contrastive Learning for Diverse Disentangled Foreground Generation
- DOI:10.1007/978-3-031-19787-1_19
- 发表时间:2022-11
- 期刊:
- 影响因子:0
- 作者:Yuheng Li;Yijun Li;Jingwan Lu;Eli Shechtman;Yong Jae Lee;Krishna Kumar Singh
- 通讯作者:Yuheng Li;Yijun Li;Jingwan Lu;Eli Shechtman;Yong Jae Lee;Krishna Kumar Singh
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Yong Jae Lee其他文献
Polytetrafluoroethylene as a spacer graft for the correction of lower eyelid retraction.
聚四氟乙烯作为间隔移植物,用于矫正下眼睑退缩。
- DOI:
10.3341/kjo.2005.19.4.247 - 发表时间:
2005 - 期刊:
- 影响因子:0
- 作者:
Yong Jae Lee;S. Khwarg - 通讯作者:
S. Khwarg
Analyzing the time frame for the transition from leisure-cyclist to commuter-cyclist
分析从休闲骑行者到通勤骑行者转变的时间框架
- DOI:
- 发表时间:
2011 - 期刊:
- 影响因子:0
- 作者:
Hyochul Park;Yong Jae Lee;H. Shin;Keemin Sohn - 通讯作者:
Keemin Sohn
Ray-based Color Image Segmentation
基于光线的彩色图像分割
- DOI:
10.1109/crv.2008.33 - 发表时间:
2008 - 期刊:
- 影响因子:0
- 作者:
Changhai Xu;Yong Jae Lee;B. Kuipers - 通讯作者:
B. Kuipers
Evaluation of fatigue crack propagation in surface modification layer by a load-controlled plate bending fatigue tester
利用负载控制板弯曲疲劳试验机评估表面改性层疲劳裂纹扩展
- DOI:
10.1299/transjsme.2014smm0022 - 发表时间:
2014 - 期刊:
- 影响因子:0
- 作者:
Naoki Kuramoto;Kyung Ho Chang;Kenichi Fujii;Yong Jae Lee;K. Sugihara;髙桑 脩,左奈田 一将,祖山 均 - 通讯作者:
髙桑 脩,左奈田 一将,祖山 均
Who Moved My Cheese? Automatic Annotation of Rodent Behaviors with Convolutional Neural Networks
谁动了我的奶酪?
- DOI:
- 发表时间:
2017 - 期刊:
- 影响因子:0
- 作者:
Zhongzheng Ren;Adriana Noronha Annie;Vogel Ciernia;Yong Jae Lee - 通讯作者:
Yong Jae Lee
Yong Jae Lee的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Yong Jae Lee', 18)}}的其他基金
RI:Small:Collaborative Research: Understanding Human-Object Interactions from First-person and Third-person Videos
RI:Small:协作研究:从第一人称和第三人称视频中理解人与物体的交互
- 批准号:
2204808 - 财政年份:2021
- 资助金额:
$ 50.05万 - 项目类别:
Standard Grant
CAREER: Weakly-Supervised Visual Scene Understanding: Combining Images and Videos, and Going Beyond Semantic Tags
职业:弱监督视觉场景理解:结合图像和视频,超越语义标签
- 批准号:
1751206 - 财政年份:2018
- 资助金额:
$ 50.05万 - 项目类别:
Continuing Grant
RI:Small:Collaborative Research: Understanding Human-Object Interactions from First-person and Third-person Videos
RI:Small:协作研究:从第一人称和第三人称视频中理解人与物体的交互
- 批准号:
1812850 - 财政年份:2018
- 资助金额:
$ 50.05万 - 项目类别:
Standard Grant
EAGER: Leveraging Synthetic Data for Visual Reasoning and Representation Learning with Minimal Human Supervision
EAGER:在最少的人类监督下利用合成数据进行视觉推理和表示学习
- 批准号:
1748387 - 财政年份:2017
- 资助金额:
$ 50.05万 - 项目类别:
Standard Grant
相似国自然基金
语义扩散驱动的弱监督语义分割
- 批准号:62301613
- 批准年份:2023
- 资助金额:30 万元
- 项目类别:青年科学基金项目
集成在线学习,弱监督深度学习和动态交互的三维医学图像分割平台
- 批准号:62301326
- 批准年份:2023
- 资助金额:30 万元
- 项目类别:青年科学基金项目
基于弱监督信息的精细化时域动作定位方法研究
- 批准号:62306294
- 批准年份:2023
- 资助金额:30 万元
- 项目类别:青年科学基金项目
基于多粒度弱监督学习的APT攻击智能检测方法
- 批准号:62372410
- 批准年份:2023
- 资助金额:51 万元
- 项目类别:面上项目
面向图像目标检测的新型弱监督学习方法研究
- 批准号:62371157
- 批准年份:2023
- 资助金额:50 万元
- 项目类别:面上项目
相似海外基金
深層学習は関連タスクを学べるか:関連タスク学習能力の獲得とバイオ画像への応用
深度学习能否学习相关任务:相关任务学习能力的获取及其在生物图像中的应用
- 批准号:
22KJ2396 - 财政年份:2023
- 资助金额:
$ 50.05万 - 项目类别:
Grant-in-Aid for JSPS Fellows
Algebraic Structures in Weakly Supervised Disentangled Representation Learning
弱监督解缠表示学习中的代数结构
- 批准号:
22KJ0880 - 财政年份:2023
- 资助金额:
$ 50.05万 - 项目类别:
Grant-in-Aid for JSPS Fellows
Foundations of Unsupervised and Weakly Supervised Learning
无监督和弱监督学习的基础
- 批准号:
RGPIN-2019-06018 - 财政年份:2022
- 资助金额:
$ 50.05万 - 项目类别:
Discovery Grants Program - Individual
Deep Weakly-Supervised Neural Networks for Cross-Domain Video Recognition and Localization
用于跨域视频识别和定位的深度弱监督神经网络
- 批准号:
DGDND-2022-05397 - 财政年份:2022
- 资助金额:
$ 50.05万 - 项目类别:
DND/NSERC Discovery Grant Supplement
Development of deepometry for the supervised and weakly supervised learning of imaging data
用于成像数据监督和弱监督学习的深度测量的发展
- 批准号:
2748735 - 财政年份:2022
- 资助金额:
$ 50.05万 - 项目类别:
Studentship