RI: Small: Integrating physics, data, and art-based insights for controllable generative models

RI：小型：集成物理、数据和基于艺术的见解以实现可控生成模型

• 批准号: 2323086
• 负责人: Pavan Turaga
• 金额: $ 59.55万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2323086&HistoricalAwards=false
• 关键词: RI; Small; Integrating; physics; data

Generative models refer to a large class of machine learning techniques that can generate user-specified media – including images, video, 3D environments, and text – from inputs such as text prompts, sketches, or other user provide images. New generative models are rapidly being developed and are seen as increasingly important in many different applications such as in chatbots and automation. Current generative models are characterized by extremely large models trained on web-scale data, but on closer inspection are found to be unreliable in critically important contexts. This project focuses on generative models for visual media, where current generative models will be advanced by leveraging prior knowledge about how visual features can be described by physical and statistical laws. The sources of knowledge that will be leveraged include physics-based knowledge, insights from traditional content creation techniques, and advances in modeling latent-spaces using novel geometric methods. The anticipated benefits include more robust models, smaller scale models, and more interpretable and modular models. This research systematically investigating the basics of generative-adversarial networks. The first task considers the role of the input probability distribution from which samples are drawn, generalizing to non-parametric distributions tuned to reduce distribution mismatch under sample mixing. The second task involves architectural novelty in terms of detail layering, where synthesis is broken into a series of simpler architectures. The third task focuses on developing reduced parameter discriminator models, using orthogonality-type constraints as a proxy for physical variables like lighting, texture, and deformation. The fourth task focuses on developing shape-aware architectures, using learnable polynomial basis functions to represent shape more directly. Applications for these methods include augmenting training-sets to create trustworthy machine learning models in contexts such as manufacturing and health, where it is difficult to gather large training sets. Curricular innovations include creating access to these approaches for non-STEM students, in a class titled Machine Learning for Media Arts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

生成模型是指一大类机器学习技术，可以根据文本提示、草图或其他用户提供的图像等输入生成用户指定的媒体（包括图像、视频、3D 环境和文本）。正在开发中，并且在聊天机器人和自动化等许多不同的应用中被认为越来越重要。当前的生成模型的特点是在网络规模数据上训练的非常大的模型，但仔细检查发现在极其重要的环境中是不可靠的。项目重点视觉媒体的生成模型，其中当前的生成模型将通过利用有关如何通过物理和统计定律描述视觉特征的先验知识来改进。将利用的知识来源包括基于物理的知识、传统内容创建的见解。技术以及使用新颖的几何方法建模潜在空间的进展。预期的好处包括更稳健的模型、更小的规模模型以及更可解释和模块化的模型。第一项任务考虑了生成对抗网络的基础知识。输入概率的作用第二个任务涉及细节分层方面的架构新颖性，其中合成被分解为一系列更简单的架构。开发简化参数鉴别器模型，使用正交类型约束作为物理变量（如光照、纹理和变形）的代理，第四项任务重点是开发形状感知架构，使用可学习的多项式基函数更直接地表示形状。这些方法包括增加训练集，以在制造和健康等环境中创建值得信赖的机器学习模型，在这些环境中很难收集大型训练集。课程创新包括为非 STEM 学生创建访问这些方法的机会，课程名称为“STEM”。媒体艺术机器学习。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。