RI:Small: Improve Visual Tracking by Large Scale Learning, Diagnosis, and Evaluation

RI:Small：通过大规模学习、诊断和评估改进视觉跟踪

• 批准号: 2006665
• 负责人: Haibin Ling
• 金额: $ 49.83万
• 依托单位: SUNY at Stony Brook
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2006665&HistoricalAwards=false
• 关键词: RI; Small; Improve; Visual; Tracking

Video understanding and analysis has a wide range of applications. As a cornerstone in video understanding, visual tracking provides online motion information of objects of interests, such as walking pedestrians in autonomous driving, moving cells in bioengineering study, and deforming guidewire in medical intervention, to name a few. Despite recent advances in deep learning-based visual tracking systems, however, a significant gap remains between state-of-the-art algorithms and real-world applications. A conjecture is that the advantage of deep learning is not fully explored, especially due to the lack of large-scale quality tracking datasets. Currently, the largest published fully annotated tracking dataset contains less than 2,000 videos, which are hardly sufficient for effectively learning a robust tracking model. This project confronts the issue by directly and explicitly working on large-scale learning of tracking algorithms, and aims to improving tracker systems from various aspects including accuracy, efficiency, robustness, as well as generalization capability. The produced datasets, benchmark, diagnosis toolkit, tracking algorithms and temporal modeling techniques, will be made publicly available and expected to generate significant contributions to the computer vision and related fields.The overall goal of this research is to push the frontier of visual object tracking though large-scale learning. The project divides the research activities into three thrusts. Firstly, large-scale quality tracking datasets will be constructed with full annotation. Based on such datasets, an online benchmark platform will be derived and a tracking diagnosis toolkit be developed for studying challenge factors in visual tracking. These results will provide the data basis, test beds, and analytic tools for facilitating research in visual tracking. Secondly, efforts will be devoted to improving the robustness of deep trackers against various challenge factors, by either optimizing tracker architectures or integrating predictions of these factors. Thirdly, effective deep temporal models will be developed in two ways: one implicitly encodes temporal information in joint spatial-temporal CNN structures, while the other develops attention-guided dual memory models.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

视频理解和分析有着广泛的应用。作为视频理解的基石，视觉跟踪提供感兴趣对象的在线运动信息，例如自动驾驶中的行走行人、生物工程研究中的移动细胞以及医疗干预中的变形导丝等等。然而，尽管基于深度学习的视觉跟踪系统最近取得了进展，但最先进的算法和实际应用之间仍然存在巨大差距。一个猜想是，深度学习的优势没有得到充分发挥，特别是由于缺乏大规模的质量跟踪数据集。目前，最大的已发布的完全注释跟踪数据集包含不到 2000 个视频，这不足以有效地学习鲁棒的跟踪模型。该项目通过直接、明确地致力于跟踪算法的大规模学习来面对这个问题，旨在从准确性、效率、鲁棒性和泛化能力等各个方面改进跟踪系统。所产生的数据集、基准测试、诊断工具包、跟踪算法和时间建模技术将公开提供，预计将为计算机视觉和相关领域做出重大贡献。这项研究的总体目标是推动视觉对象跟踪的前沿虽然大规模学习。该项目将研究活动分为三个重点。首先，将构建具有完整注释的大规模质量跟踪数据集。基于这些数据集，将衍生出一个在线基准平台，并开发一个跟踪诊断工具包，用于研究视觉跟踪中的挑战因素。这些结果将为促进视觉跟踪研究提供数据基础、测试平台和分析工具。其次，将致力于通过优化跟踪器架构或整合这些因素的预测来提高深度跟踪器针对各种挑战因素的鲁棒性。第三，有效的深度时间模型将通过两种方式开发：一种在联合时空CNN结构中隐式编码时间信息，另一种开发注意力引导的双记忆模型。该奖项反映了NSF的法定使命，被认为值得支持通过使用基金会的智力优点和更广泛的影响审查标准进行评估。

项目成果

ARCHIE++ : A Cloud-enabled Framework for Conducting AR System Testing in the Wild

ARCHIE：用于在野外进行 AR 系统测试的云支持框架

• DOI: 10.1109/tvcg.2022.3141029
• 发表时间: 2022-01
• 期刊: IEEE Transactions on Visualization and Computer Graphics
• 影响因子: 5.2
• 作者: Lehman, Sarah;Elezovikj, Semir;Ling, Haibin;Tan, Chiu
• 通讯作者: Tan, Chiu

Self-Supervised Bulk Motion Artifact Removal in Optical Coherence Tomography Angiography

光学相干断层扫描血管造影中自监督体运动伪影去除

• 发表时间: 2022-01
• 期刊: IEEE Conference on Computer Vision and Pattern Recognition
• 作者: Ren, J.;Park, K.;Pan, Y.;Ling, H.
• 通讯作者: Ling, H.

MART: Motion-Aware Recurrent Neural Network for Robust Visual Tracking

MART：用于鲁棒视觉跟踪的运动感知循环神经网络

• DOI: 10.1109/wacv48630.2021.00061
• 发表时间: 2021-01
• 期刊: Winter Conference on Applications of Computer Vision
• 作者: Fan, Heng;Ling, Haibin
• 通讯作者: Ling, Haibin

Modeling Deep Learning Based Optical Attacks to Mail Privacy

对基于深度学习的邮件隐私光攻击进行建模

• 发表时间: 2021-01
• 期刊: Proc. of AAAI Conference on Artificial Intelligence (AAAI
• 作者: Huang, B;Lian, R;Samaras, D;Ling, H
• 通讯作者: Ling, H

One-Shot Neural Ensemble Architecture Search by Diversity Guided Search Space Shrinking

通过多样性引导搜索空间缩小进行一次性神经集成架构搜索

• 发表时间: 2021-01
• 期刊: Proc. of IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR
• 作者: Chen, M.;Fu, J.;Ling, H.
• 通讯作者: Ling, H.