RI:Small: Improve Visual Tracking by Large Scale Learning, Diagnosis, and Evaluation

RI:Small：通过大规模学习、诊断和评估改进视觉跟踪

• 批准号: 2006665
• 负责人: Haibin Ling
• 金额: $ 49.83万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2006665&HistoricalAwards=false
• 关键词: RI; Small; Improve; Visual; Tracking

Video understanding and analysis has a wide range of applications. As a cornerstone in video understanding, visual tracking provides online motion information of objects of interests, such as walking pedestrians in autonomous driving, moving cells in bioengineering study, and deforming guidewire in medical intervention, to name a few. Despite recent advances in deep learning-based visual tracking systems, however, a significant gap remains between state-of-the-art algorithms and real-world applications. A conjecture is that the advantage of deep learning is not fully explored, especially due to the lack of large-scale quality tracking datasets. Currently, the largest published fully annotated tracking dataset contains less than 2,000 videos, which are hardly sufficient for effectively learning a robust tracking model. This project confronts the issue by directly and explicitly working on large-scale learning of tracking algorithms, and aims to improving tracker systems from various aspects including accuracy, efficiency, robustness, as well as generalization capability. The produced datasets, benchmark, diagnosis toolkit, tracking algorithms and temporal modeling techniques, will be made publicly available and expected to generate significant contributions to the computer vision and related fields.The overall goal of this research is to push the frontier of visual object tracking though large-scale learning. The project divides the research activities into three thrusts. Firstly, large-scale quality tracking datasets will be constructed with full annotation. Based on such datasets, an online benchmark platform will be derived and a tracking diagnosis toolkit be developed for studying challenge factors in visual tracking. These results will provide the data basis, test beds, and analytic tools for facilitating research in visual tracking. Secondly, efforts will be devoted to improving the robustness of deep trackers against various challenge factors, by either optimizing tracker architectures or integrating predictions of these factors. Thirdly, effective deep temporal models will be developed in two ways: one implicitly encodes temporal information in joint spatial-temporal CNN structures, while the other develops attention-guided dual memory models.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

视频理解和分析具有广泛的应用。作为视频理解的基石，视觉跟踪提供了感兴趣的对象的在线运动信息，例如在自动驾驶中行走的行人，在生物工程研究中移动细胞以及在医疗干预中变形指南，仅举几例。尽管在基于深度学习的视觉跟踪系统方面取得了最新进展，但是在最新算法和现实世界应用之间仍然存在很大的差距。一个猜想是，深度学习的优势没有得到充分探索，尤其是由于缺乏大规模质量跟踪数据集。目前，最大的完全注释的跟踪数据集包含不到2,000个视频，这几乎不足以有效学习强大的跟踪模型。该项目通过直接和明确地研究跟踪算法的大规模学习来面临问题，并旨在从各个方面改善跟踪器系统，包括准确性，效率，鲁棒性以及概括能力。生产的数据集，基准，诊断工具包，跟踪算法和时间建模技术将被公开可用，并期望对计算机视觉和相关领域产生重大贡献。这项研究的总体目标是推动视觉对象的前沿跟踪，虽然学习大型学习。该项目将研究活动分为三个推力。首先，将通过完整注释构建大型质量跟踪数据集。基于此类数据集，将得出一个在线基准平台，并开发一个跟踪诊断工具包，以研究视觉跟踪中的挑战因素。这些结果将提供数据基础，测试床和分析工具，以促进视觉跟踪中的研究。其次，通过优化跟踪器体系结构或整合对这些因素的预测，将努力致力于提高深层跟踪器对各种挑战因素的鲁棒性。第三，有效的深度时间模型将以两种方式开发：一种隐含地编码联合时空CNN结构中的时间信息，而另一个则开发了引导引导的双重记忆模型。该奖项反映了NSF的法定任务，并被认为是通过基金会的智力功能和广泛影响的评估来评估CRETERIA的评估。

项目成果

GTCaR: Graph Transformer for Camera Re-localization

• DOI: 10.1007/978-3-031-20080-9_14
• 发表时间: 2022
• 作者: Xinyi Li;Haibin Ling

TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Tracking

• DOI: 10.1109/wacv56688.2023.00485
• 发表时间: 2023-01-01
• 期刊: 2023 IEEE/CVF WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV)
• 作者: Chu, Peng;Wang, Jiang;Liu, Zicheng
• 通讯作者: Liu, Zicheng

Transparent Object Tracking with Enhanced Fusion Module

• DOI: 10.1109/iros55552.2023.10341597
• 发表时间: 2023-09
• 期刊: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
• 作者: Kalyan Garigapati;Erik Blasch;Jie Wei;Haibin Ling

TracKlinic: Diagnosis of Challenge Factors in Visual Tracking

• DOI: 10.1109/wacv48630.2021.00101
• 发表时间: 2019-11
• 期刊: 2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
• 作者: Heng Fan;Fan Yang;Peng Chu;Lin Yuan;Haibin Ling

Self-Supervised Bulk Motion Artifact Removal in Optical Coherence Tomography Angiography

• DOI: 10.1109/cvpr52688.2022.01996
• 发表时间: 2022-02
• 期刊: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 作者: Jiaxiang Ren;K. Park;Yingtian Pan;H. Ling