Creating and Evaluating New Media Interfaces for Expression

创建和评估新的表达媒体界面

• 批准号: RGPIN-2020-07054
• 负责人: Fels, Sidney
• 金额: $ 3.5万
• 依托单位: University of British Columbia
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=718445
• 关键词: Creating; Evaluating; New; Media; Interfaces

I propose to push the boundaries of two forms of interactive visual media to increase the capacity for humans to learn and work together. First, new forms of interaction with a novel 3D display will be investigated. Second, advanced interactive, collaborative video (IV) with three critical new affordances to elevate video to be a first-class' learning medium will be created. First, research using our 3D fish-tank virtual reality (FTVR) spherical display (3DD) focuses on two main types of interactions: 1) co-located and remote multi-person interaction requiring real-time 3D capture, and 2) human-3D agent interaction. To support co-located four-person interaction, a curved polarizing lens will be created to match the curvature of the spherical display surface coupled to active glasses to enable four different monoscopic perspective corrected views to be projected on the display but seen by different users. These displays provide virtual contents anchored to real-world coordinates allowing everyone to share a common reference frame. Thus, deictic interactions, like pointing and saying “That thing there.” are supported where everyone can see what is being pointed at. For remote participants, a real-time 3D capture system using depth cameras supporting eye-contact will be created. Perceptual effects as well as task performance on 3D tasks will be studied and compared to similar configurations using flat screens. Second, viewer and room interaction with an intelligent virtual agent (IVA) inside the spherical display will be studied. As with real-participants, voice and gesture interaction will investigated. Likewise, interaction semantics for collaborative pointing, selecting, translating, and rotating content will be explored in remote and co-located scenarios. This research will create advanced 3D spherical displays for groups of people and agents to work and play across space and time. Second, from past work, we discovered three video affordances critical for education are missing: mutability, bidirectionality and interoperability. For mutability, interfaces for collaborative video making while each state change and annotation is managed will be created. For interoperability, interfaces supporting video structure and semantic capture and copy for effective integration with students' information ecology, including notes will be created. For bidirectionality, rich referencing interfaces enabling complex methods of communicating with video elements will be investigated. These interactions will be studied in educational scenarios such as: note taking, presenting, and social learning. Ultimately, the goal of this research provides the pen and paper' for video so people can learn to read/watch and write/compose' video as easily as they can through text. This research directly involves 33 highly qualified personnel working in world-class international, interdisciplinary teams to become leaders in advanced media interaction.

我建议推动两种形式的交互式视觉媒体的界限，以增加人类学习和共同努力的能力。首先，将研究与新型3D显示的新形式。其次，将创建高级交互式协作视频（IV），并将创建三个关键的新负担能力，以将视频提升为一流的学习媒体。 首先，使用我们的3D Fish-Tank虚拟现实（FTVR）球形显示（3DD）的研究重点介绍了两种主要互动类型：1）需要实时3D捕获的共同和远程的多人互动，以及2）Human-3D代理互动。为了支持共同定居的四人交互，将创建一个曲线偏振镜头，以匹配球形显示表面的曲率，该镜头耦合到活动眼镜，以使四个不同的单镜透视透视图校正后的视图在显示屏上投影，但由不同的用户看到。这些显示提供了固定在现实世界坐标的虚拟内容，使每个人都可以共享一个共同的参考框架。那是一种互动，例如指着并说“那东西在那里”。在每个人都可以看到所指向的内容的地方得到支持。对于远程参与者，将创建一个使用深度摄像头的实时3D捕获系统。将研究3D任务的感知效果以及任务性能，并将使用平面屏幕进行比较。其次，将研究与球形显示内部的智能虚拟代理（IVA）的观众和房间互动。与实际参与者一样，语音和手势相互作用也将研究。同样，将在远程和共同定位的方案中探索用于协作指向，选择，翻译和旋转内容的交互语义。这项研究将为一群人和代理人创建高级的3D球形显示，以跨越时空工作和玩耍。 其次，从过去的工作中，我们发现三个视频缺失了至关重要的教育：可熔性，双向性和互操作性。为了进行变异性，将创建每个状态更改和注释时进行协作视频制作的接口。为了互操作性，将创建支持视频结构和语义捕获的接口，并复制与学生信息生态学有效整合，包括笔记。对于双向，将研究丰富的引用界面，从而可以研究与视频元素进行通信的复杂方法。这些互动将在教育场景中进行研究，例如：注释，表现和社会学习。最终，这项研究的目标为视频提供了笔和纸，以便人们可以尽可能轻松地学习阅读和观看/写作/撰写“视频”。 这项研究直接涉及33名在世界一流的国际跨学科团队中工作的高素质人员，以成为高级媒体互动的领导者。