Protecting Against Malicious Use of Image Diffusion Models

防止图像扩散模型的恶意使用

• 批准号: 2737559
• 金额: --
• 依托单位: University of Warwick
• 依托单位国家: 英国
• 项目类别: Studentship
• 财政年份: 2022
• 资助国家: 英国
• 起止时间: 2022 至 无数据
• 项目状态: 未结题
• 来源: https://gtr.ukri.org/projects?ref=studentship-2737559
• 关键词: Protecting; Against; Malicious; Use; Image

The rapid advancement in the capabilities of generative AI has been accompanied by an increase in scope for their misuse. In particular, text-to-image diffusion models such as DALL-E 2 have reacheda point at which high quality images can be generated with ease by a broad spectrum of users, not all of whom may have good intent. The aim of this PhD project is to investigate how to protect againstmalicious use of this technology. The goal of a text-to-image diffusion model is to generate realistic images based on a text prompt. These models work by using a forward process which sequentially adds noise to an input image and then learns a mapping to reverse this process. Then, starting with noise, we can use the trained model to denoise the image conditioned on the input text and end up with a output image that appears to come from the same distribution as the input images. Instead of retraining or changing how these models work, we can modify parameters in existing models to change their behaviour to be more desirable. One example use case for this methodology is editing implicit biases found in these models. Implicit biases in the training data for text-to-image diffusion models, can lead to perpetuating social and cultural biases when generating images. As an example, asking a diffusion model for an image of a cow will, with high probability, return an image of a cow in a field even although the environment was never specified. Gender biases are also present in these models which is noticeable when generating pictures of people in certain professions. Since retraining the model to avoid these biases is expensive and time consuming, methods such as TIME look to edit the weights of the model after training in order to reduce the likelihood of a chosen bias occurring. There is still a large amount of analysis to be done into the methodology. Some examples include: How is model performance impacted after editing facts? How can we reduce a broader range of biases in one application of the TIME method? Image manipulation using image diffusion models is another emerging issue. The availability and ease at which these tools can be used allow any image to be edited freely, with little in the way of safeguards to prevent malicious intent. In-painting is the process of taking an existing image and using the model to only generate specific areas of the image. One example of how this can be misused is by editing the background of an image of a person to make it appear as if they were somewhere else. Methodology has been proposed to create protections for images against this process, by adding specific perturbations to the image that cause diffusion models to struggle to generate what is prompted. This project would include an analysis into these techniques and other potential prevention strategies.

生成AI能力的快速发展伴随着滥用的范围。特别是，诸如DALL-E 2之类的文本到图像扩散模型已经到达了该点，在该点可以通过广泛的用户轻松地生成高质量的图像，并不是所有的用户都有良好的意图。该博士项目的目的是调查如何防止对该技术的极度使用。文本对图像扩散模型的目标是基于文本提示生成逼真的图像。这些模型通过使用向前过程，该过程顺序将噪声添加到输入图像，然后学习映射以扭转此过程。然后，从噪声开始，我们可以使用训练有素的模型来定义输入文本条件的图像，并最终得到一个似乎来自与输入图像相同的分布的输出图像。我们可以修改现有模型中的参数，以更改其行为以更理想。此方法的一个示例用例是编辑这些模型中发现的隐式偏见。文本到图像扩散模型的培训数据中的隐式偏见，在产生图像时会导致社会和文化偏见的延续。例如，向牛的图像提出扩散模型，即使从未指定环境，也有很高的概率将牛的图像返回田间的图像。这些模型中也存在性别偏见，在某些职业中生成人的图片时，这是显而易见的。由于对模型避免这些偏见的模型是昂贵且耗时的，因此诸如时间之类的方法希望在训练后编辑模型的权重，以减少所选择偏见的可能性。该方法仍然需要进行大量分析。一些示例包括：编辑事实后如何影响模型性能？在时间方法的一种应用中，我们如何减少更广泛的偏见？使用图像扩散模型的图像操纵是另一个新兴问题。可以使用这些工具的可用性和易用性允许自由编辑任何图像，几乎没有保护措施以防止恶意意图。镶嵌是拍摄现有图像并使用模型仅生成图像的特定区域的过程。通过编辑一个人的图像的背景使其看起来好像在其他地方，可以滥用这种情况的一个例子。已经提出了方法，以通过向图像添加特定的扰动来为图像创建保护措施，从而导致扩散模型难以生成所提示的内容。该项目将包括对这些技术和其他潜在预防策略的分析。