基于旋转不变Gabor特征的视频动态几何与纹理提取技术研究

Content structure plays an important role in the understanding and processing of videos. Current video characterization and analysis systems rely on image representations based on low-level visual primitives such as color, texture, and motion. While practical and computationally efficient, it is still difficult challenging to bridge the semantic gap between the low-level nature of the primitives and the high-level semantics. Therefore it is interesting to investigate alternative representations and content descriptors based on the texture and geometric structure. We introduce statistical models for two important components of video content, texture and geometric structure, and demonstrate the usefulness of the model with practical applications. First we construct the Gabor feature space, which is important for video pixel similarity computations. We formalize this by using rotation-invariant Gabor filter banks and applying optimization in texture feature space. This Gabor feature space can be further applied to video analysis, we present a simple and efficient video tracking method based on the feature space using rotation-aware Gabor flow optimization. Our approach extends optical flow computation for constructing the Gabor flow to represent the pixel similarity to preserve temporal coherence when applied to video sequences. In particular, our video processing approach is designed in a spatiotemporal manner to keep temporal coherence in resultant video samples. While the highly parallel implementation on GPU is facilitated to achieve realtime performance of tracking the texture and geometric structure along Gabor flow among the video frames, which are also useful for controlling the color refining. We also plan to allow the user to specify the video content using the sketch-based interface on the keyframe images. For each of these techniques, we provide simple closed-form solutions in energy optimization that yield high-quality results. We also suggest ways in which these properties can be used as a basis for intuitive content-based video analysis and processing. While building good models for video is very important, the ultimate measure of success of a given model is the efficiency of the solutions for practical and objective tasks that may be derived from it. And we are sure that this work has a broad application prospects in the industrial, entertainment，military, scientific research，as well as in our daily life．

视频动态几何结构和纹理提取是数字媒体的关键技术之一,广泛应用于影视制作与动画设计。前期研究发现Gabor滤波器通过方向配准，可以实现对于旋转纹理的有效检测。本项目基于Gabor小波与旋转不变特性，探索建立良好纹理结构识别特性的视频域特征空间，并以此Gabor特征对视频中的动态内容进行追踪和分析，从而提高视频编辑与再渲染的质量和处理效率；通过草图设计界面改善用户交互的体验，研究并实现视频特征相关的纹理分析、基于旋转不变的Gabor流的视频几何与纹理信息配准、以及后期增强现实应用与视频渲染的算法系统。本研究将有效提高视频场景的自然纹理的特征分析与提取的准确性、实现基于纹理特征的视频内容的编辑技术、以及保持时间连续性的视频几何与纹理渲染系统。本课题的研究工作旨在通过自然纹理的准确特征选择与提取、通过视频特征空间的构建与高效处理,为数字媒体发展提供新的研究靶点和应用方向。