Illuminating the Path of Video Visualization

照亮视频可视化之路

• 批准号: EP/G006555/1
• 负责人: Min Chen
• 金额: $ 107.43万
• 依托单位: Swansea University
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2009
• 资助国家: 英国
• 起止时间: 2009 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FG006555%2F1
• 关键词: Illuminating; Path; Video; Visualization; Illuminating; Path; Video; Visualization

The notion of video visualization was coined by the PI and his postgraduate student in their 2003 IEEE VIS paper. It is a technology drawing the concepts and methodologies from volume and flow visualization, image and video processing, and vision science. It extracts meaningful information from a video data set and conveys the extracted information to users in appropriate visual representations. It is not intended to provide fully automatic solutions to the traditional problems in video processing, but involves human in the loop of intelligent reasoning while reducing the burden of viewing videos. In the subsequent work in collaboration with Stuttgart, the PI and CI introduced the concept of visual signatures in video visualization, and reported a major user study conducted at Swansea involving some 92 subjects [IEEE TVCG 2006]. This work offered an important scientific insight as to how human observers may learn to recognize visual signatures of events depicted in an abstract visual representation of a video.[Tsotsos01] stated that a bounded visual search (e.g., looking for all moving pixel clusters with 20-60 pixels) can be achieved in linear time, whist an unbounded visual search (e.g., looking for something abnormal in a video) is NP-complete. For most practical problems in video processing and computer vision, we rarely have perfectly bounded visual search. We often search simultaneously for entities (e.g., objects, motions or events) in different classes. The models that are used to guide a search are usually incomplete and may lead to uncertainty or errors in detection, segmentation andclassification. The dynamic and unpredictable nature of the input videos instigates mechanisms for heuristic reasoning and iterative decision optimization, which further depart from linear or polynomial performance.In contrast, the human eye-brain system is undeniable more powerful than any current vision system in performing visual searches, especially unbounded visual searches. Even we suppose that the human eye-brain system is a Turing machine, its 100 billion neurons and 100-500 trillion synaptic connections between neurons will unlikely to be matched by computers in the near future. Hence this raises the possibility that using video visualization to aid unbound visual search may provide a more scalable means for dealing with large volumes of video datasets.Video visualization can be deployed in many application areas, such as scientific experimentation and computation, security industry and media and entertainment industry. However, in traditional visualization (e.g., medical visualization), the users are normally familiar with the 3D objects (e.g., bones or organs) depicted in a visual representation. In contrast, human observers are not familiar with the 3D objects depicted in a visual representation of a video because one spatial dimension of these objects shows the temporal dimension. The problem is further complicated by the fact that, in most videos, each 2D frame is the projective view of a 3D scene. Hence, a visual representation of a video on a computer display is, in effect, a 2D projective view of a 4D spatiotemporal domain. In order to for us to see 'time' without using 'time', we need to address a range of challenges in science, technology, visual perception and applications. This project is intended to continue the UK's leadership in tackling these challenges by building on the existing expertise and excellence in video visualization.

视频可视化的概念是由PI和他的研究生在2003年IEEE VIS论文中创造的。这是一项从数量和流量可视化，图像和视频处理以及视觉科学中汲取概念和方法的技术。它从视频数据集中提取有意义的信息，并将提取的信息传达给用户中适当的视觉表示形式。它并不是要为视频处理中传统问题提供全自动解决方案，而是涉及人类参与智能推理的循环，同时减少观看视频的负担。在随后与Stuttgart合作的工作中，PI和CI在视频可视化中介绍了视觉签名的概念，并报告了在斯旺西进行的一项主要用户研究，其中涉及92个主题[IEEE TVCG 2006]。这项工作提供了一个重要的科学见解，即人类观察者如何学会识别视频抽象的视觉表示中描述的事件的视觉特征。[TSOTSOS01]指出，有界的视觉搜索（例如，寻找所有具有20-60个像素的移动像素群集）可以在线性的视频中实现（E.G. e.g.），以实现一致的视频。 NP完整。对于视频处理和计算机视觉中的大多数实际问题，我们很少有完美的视觉搜索。我们经常在不同类别中同时搜索实体（例如对象，动作或事件）。用于引导搜索的模型通常不完整，可能导致检测，分割和分类的不确定性或错误。输入视频的动态和不可预测的性质促进了启发式推理和迭代决策优化的机制，这进一步脱离了线性或多项式性能。相反，人类眼睛脑系统在执行视觉搜索时，尤其是无符合的视觉搜索时，人眼脑系统不可否认地比当前的视觉系统更强大。即使我们认为人眼脑系统是一台图灵机，其1000亿个神经元和神经元之间的100-500万亿个突触连接不太可能在不久的将来与计算机匹配。因此，这增加了这样的可能性，即使用视频可视化来帮助未结合的视觉搜索可能会提供更可扩展的方法来处理大量视频数据集。视频可视化可以在许多应用领域中部署，例如科学实验和计算，安全行业以及媒体和媒体和娱乐行业。但是，在传统的可视化（例如医学可视化）中，用户通常熟悉视觉表示中描述的3D对象（例如骨骼或器官）。相比之下，人类观察者对视频的视觉表示中描述的3D对象并不熟悉，因为这些对象的空间维度显示了时间维度。在大多数视频中，每个2D帧都是3D场景的投影视图，这一事实使问题更加复杂。因此，在计算机显示器上的视频的视觉表示实际上是4D时空域的2D投影视图。为了让我们看到“时间”而不使用“时间”，我们需要应对科学，技术，视觉感知和应用方面的一系列挑战。该项目旨在通过建立视频可视化的现有专业知识和卓越的卓越知识来继续英国的领导地位，以应对这些挑战。