Creating and Evaluating New Media Interfaces for Expression

创建和评估新的表达媒体界面

• 批准号: RGPIN-2020-07054
• 负责人: Fels, Sidney
• 金额: $ 3.5万
• 依托单位: University of British Columbia
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2021
• 资助国家: 加拿大
• 起止时间: 2021-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=741391
• 关键词: Creating; Evaluating; New; Media; Interfaces

I propose to push the boundaries of two forms of interactive visual media to increase the capacity for humans to learn and work together. First, new forms of interaction with a novel 3D display will be investigated. Second, advanced interactive, collaborative video (IV) with three critical new affordances to elevate video to be a `first-class' learning medium will be created. First, research using our 3D fish-tank virtual reality (FTVR) spherical display (3DD) focuses on two main types of interactions: 1) co-located and remote multi-person interaction requiring real-time 3D capture, and 2) human-3D agent interaction. To support co-located four-person interaction, a curved polarizing lens will be created to match the curvature of the spherical display surface coupled to active glasses to enable four different monoscopic perspective corrected views to be projected on the display but seen by different users. These displays provide virtual contents anchored to real-world coordinates allowing everyone to share a common reference frame. Thus, deictic interactions, like pointing and saying "That thing there." are supported where everyone can see what is being pointed at. For remote participants, a real-time 3D capture system using depth cameras supporting eye-contact will be created. Perceptual effects as well as task performance on 3D tasks will be studied and compared to similar configurations using flat screens. Second, viewer and room interaction with an intelligent virtual agent (IVA) inside the spherical display will be studied. As with real-participants, voice and gesture interaction will investigated. Likewise, interaction semantics for collaborative pointing, selecting, translating, and rotating content will be explored in remote and co-located scenarios. This research will create advanced 3D spherical displays for groups of people and agents to work and play across space and time. Second, from past work, we discovered three video affordances critical for education are missing: mutability, bidirectionality and interoperability. For mutability, interfaces for collaborative video making while each state change and annotation is managed will be created. For interoperability, interfaces supporting video structure and semantic capture and copy for effective integration with students' information ecology, including notes will be created. For bidirectionality, rich referencing interfaces enabling complex methods of communicating with video elements will be investigated. These interactions will be studied in educational scenarios such as: note taking, presenting, and social learning. Ultimately, the goal of this research provides the `pen and paper' for video so people can learn to `read/watch and write/compose' video as easily as they can through text. This research directly involves 33 highly qualified personnel working in world-class international, interdisciplinary teams to become leaders in advanced media interaction.

我建议突破两种形式的交互式视觉媒体的界限，以提高人类学习和协作的能力，首先，将研究新型 3D 显示器的新交互形式，其次是先进的交互式协作视频 (IV)。首先，使用我们的 3D 鱼缸虚拟现实 (FTVR) 球形显示器 (3DD) 进行的研究将重点关注两种主要的交互类型：1 ）需要实时 3D 捕捉的同地远程多人交互，以及 2) 人与 3D 代理交互 为了支持同地四人交互，将创建一个弯曲偏光透镜来匹配球形显示器的曲率。表面耦合到主动眼镜，使四种不同的单视场透视校正视图能够投影在显示器上，但由不同的用户看到。这些显示器提供锚定到现实世界坐标的虚拟内容，允许每个人共享一个共同的参考系。支持指向并说“那儿的东西”这样的互动，对于远程参与者来说，还将创建一个使用支持眼神接触的深度摄像头的实时 3D 捕捉系统。首先，我们将研究 3D 任务的任务表现，并与使用平面屏幕的类似配置进行比较；其次，将研究观看者和房间与球形显示器内的智能虚拟代理 (IVA) 的交互，以及与真实参与者的语音和手势交互。将被调查。同样，将在远程和同地场景中探索协作指向、选择、翻译和旋转内容的交互语义。这项研究将为人们和代理群体创建先进的 3D 球形显示器，以便他们跨空间和时间工作和娱乐。从过去的工作中，我们发现缺少对教育至关重要的三个视频功能：可变性、双向性和互操作性。对于可变性，将创建用于协作视频制作的接口，同时管理每个状态更改和注释。对于互操作性，将创建支持的接口。视频结构以及语义捕获和复制，以便与学生的信息生态有效集成，包括创建双向性，将研究支持与视频元素通信的复杂方法的丰富参考接口。 ：记笔记、演示和社交学习的最终目标是为视频提供“笔和纸”，以便人们可以像通过文本一样轻松地学习“阅读/观看和书写/撰写”视频。研究直接涉及高素质人员的工作世界一流的国际跨学科团队，成为先进媒体互动领域的领导者。