Collaborative Research: HCC: Small: RUI: Drawing from Life in Extended Reality: Advancing and Teaching Cross-Reality User Interfaces for Observational 3D Sketching

合作研究：HCC：小型：RUI：从扩展现实中的生活中汲取灵感：推进和教授用于观察 3D 草图绘制的跨现实用户界面

• 批准号: 2326998
• 负责人: Daniel Keefe
• 金额: $ 40万
• 依托单位: University of Minnesota-Twin Cities
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-15 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2326998&HistoricalAwards=false
• 关键词: Collaborative; Research; HCC; Small; RUI

This project advances the science of how people work effectively and creatively within future computing environments that span the real world, virtual reality (VR), and mixed digital+physical spaces in between. Since drawing is fundamental to creative work across many disciplines (e.g., product design, architecture, engineering, art), the investigators focus on the emerging medium of 3D drawing, a form of computer-aided drawing where sweeping movements of a 3D-tracked pen are used to create virtual 3D drawings that appear to float in the air. In the hands of artists, architects, product designers, engineers, and others, virtual reality 3D drawing technologies have already led to exciting new design processes and products. However, despite some early successes and the great potential, 3D drawing, like so many other early applications of VR, remains limited when compared to the physical, real-world counterpart of traditional drawing. Take, for example, the way artists and product designers are taught to draw; learning to draw in the real world starts with learning to see, i.e., making careful observations of a subject. For advanced drawers, real-world drawing from observation is even a way to study a subject, a mode of inquiry. Yet, creative work in virtual environments takes the opposite approach, users have powerful tools and an endless 3D canvas, but no means of connecting these to their notes, sketches, models, experiments, and other real-world artifacts. This project presents a new vision for 3D drawing that makes it applicable to impactful work with real-world subjects (e.g., scientists working in the laboratory or the field, engineers conducting a design review, artists working for social change). The key approach is to re-center 3D drawing, and other creative tasks typically performed in virtual environments, on the foundational skill of careful observation. Specifically, the project develops new Augmented- and Mixed-Reality (AR and MR) tools for making observations from multiple perspectives and using those observations to create 3D drawings. The success of these foundational tools is assessed in lab-based user studies. The integrated educational program creates college courses and educational materials to train students in these new cross-reality modes of working and build capacity among potential 3D drawing instructors for teaching this new style of observational 3D drawing in AR. Undergraduate students in both computer science and art/design disciplines are trained as key members of the interdisciplinary research team.Prior research on 3D drawing has made 3D pens more controllable and led to many exciting software tools, but none of them help people to work across realities to draw what they see. This project creates novel user interface techniques and viewpoint entropy algorithms for making observations (e.g., judging relative proportion, angles, and negative space) by identifying, visualizing, and navigating the most useful viewpoints for observations of a real-world subject. It also develops spatial user interfaces for translating observations into hand-drawn 3D forms with hybrid 2D+3D sketching interfaces and computer graphics sketch-based 3D modeling algorithms. The deep focus on real-world observation makes the proposed techniques ideal examples of emerging research on cross-reality user interfaces that support seamless transitions between reality, virtuality, and mixed spaces in between. The project includes a tightly integrated educational plan, and the novel systems developed will be deployed in classrooms at multiple institutions. The success and potential of the new technologies and educational modules developed will be assessed through a combination of lab- and classroom-based user studies. This research effort will open the door to using 3D drawing as a mode of inquiry to leverage drawing's expressive power for representing the world around us.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该项目推进了人们如何在跨越现实世界、虚拟现实 (VR) 以及两者之间的混合数字+物理空间的未来计算环境中有效和创造性地工作的科学。由于绘图是许多学科（例如产品设计、建筑、工程、艺术）创意工作的基础，因此研究人员将重点放在 3D 绘图这一新兴媒体上，这是一种计算机辅助绘图形式，其中 3D 跟踪笔的扫动运动用于创建看起来漂浮在空中的虚拟 3D 绘图。在艺术家、建筑师、产品设计师、工程师等人的手中，虚拟现实 3D 绘图技术已经带来了令人兴奋的新设计流程和产品。然而，尽管 3D 绘图取得了一些早期的成功并具有巨大的潜力，但与许多其他早期的 VR 应用一样，与传统绘图的物理、现实世界对应物相比，3D 绘图仍然受到限制。以艺术家和产品设计师的绘画方式为例；学习在现实世界中绘画始于学习观察，即仔细观察某个主题。对于高级画家来说，通过观察来绘制现实世界甚至是一种研究主题的方式，一种探究的模式。然而，虚拟环境中的创造性工作则​​采用相反的方法，用户拥有强大的工具和无尽的 3D 画布，但无法将这些与他们的笔记、草图、模型、实验和其他现实世界的工件连接起来。该项目提出了 3D 绘图的新愿景，使其适用于现实世界主题的有影响力的工作（例如，在实验室或现场工作的科学家、进行设计审查的工程师、致力于社会变革的艺术家）。关键方法是将 3D 绘图和其他通常在虚拟环境中执行的创意任务重新集中在仔细观察的基本技能上。具体来说，该项目开发了新的增强现实和混合现实（AR 和 MR）工具，用于从多个角度进行观察，并使用这些观察结果来创建 3D 绘图。这些基础工具的成功通过基于实验室的用户研究进行评估。该综合教育计划创建了大学课程和教材，以培训学生这些新的跨现实工作模式，并培养潜在 3D 绘图教师的能力，以教授这种新型 AR 观察性 3D 绘图。计算机科学和艺术/设计学科的本科生都被培训为跨学科研究团队的关键成员。先前对 3D 绘图的研究使 3D 笔更加可控，并催生了许多令人兴奋的软件工具，但它们都无法帮助人们跨学科工作现实来画出他们所看到的。该项目创建了新颖的用户界面技术和视点熵算法，通过识别、可视化和导航对现实世界主题的观察最有用的视点来进行观察（例如，判断相对比例、角度和负空间）。它还开发空间用户界面，通过混合 2D+3D 草图界面和基于计算机图形草图的 3D 建模算法，将观察结果转化为手绘 3D 形式。对现实世界观察的深度关注使得所提出的技术成为跨现实用户界面新兴研究的理想示例，支持现实、虚拟和混合空间之间的无缝过渡。该项目包括一个紧密集成的教育计划，所开发的新颖系统将部署在多个机构的教室中。所开发的新技术和教育模块的成功和潜力将通过实验室和课堂用户研究的结合来评估。这项研究工作将为使用 3D 绘图作为一种探究模式打开大门，以利用绘图的表达能力来代表我们周围的世界。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的评估进行评估，被认为值得支持。影响审查标准。