CRII: RI: Learning with Low-Quality Visual Data: Handling Both Passive and Active Degradations

CRII：RI：使用低质量视觉数据学习：处理被动和主动退化

• 批准号: 2053269
• 负责人: Zhangyang Wang
• 金额: $ 7.73万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-08-01 至 2021-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2053269&HistoricalAwards=false
• 关键词: CRII; RI; Learning; Low; Quality

This project is focused on effectively and robustly exploiting low-quality (LQ) visual data for computer vision tasks. While most current computer vision systems are designed for high-quality visual data, collected from "clear" environments where subjects are well observable without significant attenuation or alteration, a dependable vision system must reckon with the entire spectrum of degradations from unconstrained environments. With various degradations arising from the visual data acquisition and processing pipeline, the ubiquitous LQ visual data can dramatically deteriorate the model performance in practice. The project outcome can broadly benefit a variety of real-world applications, such as video surveillance, autonomous/assisted driving, robotics and medical image analysis, where LQ visual data has constituted major performance and reliability bottlenecks. This research categorizes common degradations into the two types: "passive degradations" that are caused by uncontrollable environment factors (such as bad weather and low light); and "active degradations" that are intentionally introduced in a controllable way to meet certain budget requirements (such as lossy compression). The project will mainly addresses two important technical questions: i) how to overcome passive degradations and achieve more robust high-level task performance on LQ video data, using end-to-end deep learning models; and ii) how to properly introduce and control active degradations to generate the desired form of LQ data, that both satisfies certain budget requirements and maintains the target task utility, using deep adversarial learning models. The resulting new techniques are to be verified on application examples such as video recognition, video annotation, video compression, and de-identified video data sharing for recognition purpose.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目的重点是有效，健壮地利用低质量（LQ）视觉数据来进行计算机视觉任务。尽管当前的大多数计算机视觉系统都是为高质量的视觉数据而设计的，这些数据是从“清晰”环境中收集的，在没有大幅度衰减或更改的情况下，受试者可以观察到，但可靠的视觉系统必须估计来自无约束环境的整个降解范围。随着视觉数据采集和处理管道引起的各种降解，无处不在的LQ视觉数据可以在实践中极大地恶化模型性能。该项目结果可以广泛受益于各种现实世界应用，例如视频监视，自动驾驶/辅助驾驶，机器人技术和医学图像分析，其中LQ视觉数据构成了主要的性能和可靠性瓶颈。这项研究将常见的降解归类为两种类型：“被动降解”，这些降解是由无法控制的环境因素（例如恶劣的天气和低光）引起的；以及以可控的方式有意引入以满足某些预算要求（例如有损压缩）的“主动降解”。该项目将主要解决两个重要的技术问题：i）如何使用端到端的深度学习模型克服被动降解并在LQ视频数据上实现更强大的高级任务性能； ii）如何使用深厚的对抗性学习模型正确介绍和控制主动降解以生成所需的LQ数据形式，这些LQ数据都满足某些预算要求并保持目标任务实用程序。由此产生的新技术将在申请示例中进行验证，例如视频识别，视频注释，视频压缩和识别视频数据共享，以识别目的。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛影响的审查标准来通过评估来获得支持的。

项目成果

Privacy-Preserving Deep Action Recognition: An Adversarial Learning Framework and A New Dataset

• DOI: 10.1109/tpami.2020.3026709
• 发表时间: 2020-09
• 期刊: IEEE Transactions on Pattern Analysis and Machine Intelligence
• 影响因子: 23.6
• 作者: Zhenyu Wu;Haotao Wang;Zhaowen Wang;Hailin Jin;Zhangyang Wang