NRI: Small: Collaborative Research: Active Sensing for Robotic Cameramen

NRI：小型：协作研究：机器人摄影师的主动传感

• 批准号: 1317947
• 负责人: Kostas Daniilidis
• 金额: $ 43.37万
• 依托单位: University of Pennsylvania
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-09-15 至 2015-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1317947&HistoricalAwards=false
• 关键词: NRI; Small; Collaborative; Research; Active

With advances in camera technologies, and as cloud storage, network bandwidth and protocols become available, visual media are becoming ubiquitous. Video recording became de facto universal means of instruction for a wide range of applications such as physical exercise, technology, assembly, or cooking. This project addresses the scientific and technological challenges of video shooting in terms of coverage and optimal views planning while leaving high level aspects including creativity to the video editing and post-production stages. Camera placement and novel view selection challenges are modeled as optimization problems that minimize the uncertainty in the location of actors and objects, maximize coverage and effective appearance resolution, and optimize object detection for the sake of semantic annotation of the scene. New probabilistic models capture long range correlations when the trajectories of actors are only partially observable. Quality of potential novel views is modeled in terms of resolution that is optimized by maximizing the coverage of a 3D orientation histogram while an active view selection process for object detection minimizes a dynamic programming objective function capturing the loss due to classification error as well as the resources spent for each view.The project advances active sensing and perception and provides the technology for further automation on video capturing. Such technology has broader impact on the production of education videos for online courses as well as in telepresence applications. Research results are integrated into robotics and digital media programs addressing K-12 students.

随着相机技术的进步，随着云存储，网络带宽和协议的可用，视觉媒体变得无处不在。视频录制成为事实上的通用教学手段，用于广泛的应用，例如体育锻炼，技术，组装或烹饪。该项目解决了视频拍摄在覆盖范围和最佳视图计划方面面临的科学和技术挑战，同时留下了高水平的方面，包括对视频编辑和后期制作阶段的创造力。摄像头放置和新颖的视图选择挑战被建模为优化问题，可最大程度地减少演员和物体位置的不确定性，最大​​化覆盖范围和有效的外观分辨率，并优化对象检测场景的语义注释。当参与者的轨迹仅部分可观察到时，新的概率模型捕获了远程相关性。 潜在的新型视图的质量是根据分辨率建模的，该分辨率通过最大化3D方向直方图的覆盖范围而进行了优化，而主动视图选择过程以最小化对象检测，可以最大程度地捕获动态的编程目标函数，从而捕获了由于分类错误以及为每种视图所花费的资源而导致的损失。这样的技术对在线课程以及远程申请应用程序的教育视频的制作产生更大的影响。研究结果纳入了针对K-12学生的机器人技术和数字媒体计划。