HCC: Small: 3DStylus: User-Guided Shape Manipulation using Neural Priors

HCC：小型：3DStylus：使用神经先验进行用户引导的形状操作

基本信息

批准号：
2304481
负责人：
Rana Hanocka
金额：
$ 59.99万
依托单位：
University of Chicago
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2023
资助国家：
美国
起止时间：
2023-04-01 至 2026-03-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2304481&HistoricalAwards=false
关键词：
HCC Small 3DStylus User Guided

项目摘要

Digital 3D models are used in a wide variety of applications, including manufacturing, engineering, medicine, and entertainment. The rapid growth in demand for 3D models, however, has outpaced our ability to construct them. Employing current modeling tools to perform even the most basic of 3D operations, such as selecting a region on a shape, requires years of extensive training and experience. This project will develop new and accessible tools and techniques for creating and manipulating 3D objects, eliminating existing technical barriers of entry and thereby democratizing 3D content creation. Project outcomes will have broad impact by making 3D modeling more accessible, empowering both expert and lay users to create and transform objects for applications across a wide range of industries and professions. The project is centered around the development of 3DStylus, a suite of foundational tools that will enable users to edit and modify existing 3D shapes and create new ones from scratch, using simple text as input. 3D Stylus will encompass three central components: (i) 3D Editor, which will enhance existing 3D models by incorporating text-specified textures, materials, and localized modifications that preserve the underlying shape; (ii) 3D Morpher, which will transform existing 3D models into new text-specified geometry, while preserving the original texture details; and (iii) 3D Creator, which will allow users to intuitively create novel 3D geometries, and make desired edits and manipulations, using only a text description. The research will leverage deep learning techniques which have been used so successfully in 2D (as evidenced by the revolutionary impact of DALL-E), but which have generally been out-of-reach in 3D given the relatively small amount of available high-quality 3D data. The novel approaches envisioned will instead use the abundantly available 2D datasets as a signal for editing 3D objects. Thus, the transformational potential of deep learning can be harnessed to achieve advanced 3D modeling capabilities, without the need for large 3D datasets.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数字3D模型用于多种应用，包括制造，工程，医学和娱乐。但是，对3D模型的需求的快速增长超过了我们构建它们的能力。使用当前的建模工具，即使在形状上选择一个区域，也需要多年的培训和经验，甚至需要执行最基本的3D操作。该项目将开发新的且可访问的工具和技术，以创建和操纵3D对象，消除现有的技术障碍，从而使创建3D内容的创建民主化。项目成果将通过使3D建模更容易访问，赋予专家和外行用户，从而为各种行业和专业的应用程序创建和转换对象，从而产生广泛的影响。该项目围绕着3Dstylus的开发，这是一套基础工具，它将使用户使用简单的文本作为输入来编辑和修改现有的3D形状并从头开始创建新形状。 3D手写笔将涵盖三个中心组件：（i）3D编辑器，该编辑器将通过合并文本指定的纹理，材料和局部修改来增强现有的3D模型，从而保留基本形状；（ii）3D Morpher，它将将现有的3D模型转换为新的文本指定的几何形状，同时保留原始纹理细节；（iii）3D创建者，它将允许用户直观地创建新颖的3D几何形状，并仅使用文本描述进行所需的编辑和操作。这项研究将利用在2D中成功使用的深度学习技术（如DALL-E的革命性影响所证明的），但鉴于相对较少的可用高质量的3D数据，通常在3D方面已经有远见。小说方法所设想的方法将使用大量可用的2D数据集作为编辑3D对象的信号。因此，可以利用深度学习的变革潜力来实现先进的3D建模功能，而无需大型3D数据集。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子的智力优点和更广泛影响的评估来审查标准的评估。