CopyrightMeter: Revisiting Copyright Protection in Text-to-image Models

---

📝 Introduction

Welcome to the official repository of the paper: CopyrightMeter: Revisiting Copyright Protection in Text-to-image Models

What is CopyrightMeter?

• A unified evaluation framework for copyright protection in text-to-image diffusion models.
• Core Capabilities:
  • Evaluates 17 state-of-the-art protection methods and 16 representative attacks.
  • Uses utility metrics to assess fidelity (quality retention), efficacy (protection effectiveness), and resilience (robustness against attacks) of protections under various attacks.
• Purpose:
  • Provides guidance for developing robust protection methods.
  • Establishes a standard benchmark for future research in text-to-image copyright protection.

🌟 Overview

The code is organized as follows:

• ./protections/ contains code used for protections (Obfuscation Processing, Model Sanitization, and Digital Watermarking);
• ./attacks/ includes code used for attacks (Noise Purification, Concept Recovery, and Watermark Removal);
• ./evaluation/ provides metrics for evaluating fidelity, efficacy, and resilience of copyright protection methods.

The table below provides a summary of the categories and their definitions within the CopyrightMeter framework:

Category	Definition
🔒 Protection: Obfuscation Processing	Add adversarial perturbations on images to avoid image mimicry.
⚔️ Attack: Noise Purification	Purify the protected images to nullify adversarial perturbations.
🔒 Protection: Model Sanitization	Prevent DM from generating images containing specific concepts.
⚔️ Attack: Concept Recovery	Retrieve the eliminated concept to recover the content generation.
🔒 Protection: Digital Watermarking	Embed DM-based watermark into image generation.
⚔️ Attack: Watermark Removal	Tamper with images to remove watermark.

🔍 Detailed Methods

CopyrightMeter integrates the code for copyright protection and attack methods in text-to-image diffusion models from the following projects. We extend our heartfelt gratitude to the authors whose groundbreaking work and shared code have greatly enriched the field of copyright protection in text-to-image diffusion models.

🔒 Protection Schemes

The protection schemes include:

• Obfuscation Processing: ./protections/obfuscation_processing
• Model Sanitization: ./protections/model_sanitization
• Digital Watermarking: ./protections/digital_watermarking

Obfuscation Processing

• AdvDM: C. Liang, et al., "Adversarial example does good: Preventing painting imitation from diffusion models via adversarial examples," in ICML, 2023. [Paper] [Code]

• Mist: C. Liang and X. Wu, "Mist: Towards improved adversarial examples for diffusion models," arXiv preprint arXiv:2305.12683, 2023. [Paper] [Code] [Website]

• Glaze: S. Shan, et al., "Glaze: Protecting artists from style mimicry by text-to-image models," in USENIX Security, 2023. [Paper] [Code] [Website]

• PhotoGuard: H. Salman, et al., "Raising the cost of malicious AI-powered image editing," in ICML, 2023. [Paper] [Code]

• Anti-DreamBooth: T. Van Le, et al., "Anti-dreambooth: Protecting users from personalized text-to-image synthesis," in ICCV, 2023. [Paper] [Code] [Website]

Model Sanitization

• Forget-Me-Not: G. Zhang, et al., "Forget-me-not: Learning to forget in text-to-image diffusion models," in CVPR, 2024. [Paper] [Code]

• Erasing Concepts: R. Gandikota, et al., "Erasing concepts from diffusion models," in ICCV, 2023. [Paper] [Code] [Website]

• Ablating Concepts: N. Kumari, et al., "Ablating concepts in text-to-image diffusion models," in ICCV, 2023. [Paper] [Code] [Website]

• Unified Concept Editing: R. Gandikota, et al., "Unified concept editing in diffusion models," in WACV, 2024. [Paper] [Code] [Website]

• Negative Prompt: AUTOMATIC1111, "Negative prompt," 2022. [Code] [Website]

• Safe Latent Diffusion: P. Schramowski, et al., "Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models," in CVPR, 2023. [Paper] [Code]

Digital Watermarking

• DiffusionShield: Y. Cui, et al., "DiffusionShield: A watermark for copyright protection against generative diffusion models," arXiv preprint arXiv:2306.04642, 2023. [Paper] [Code]

• DIAGNOSIS: Z. Wang, et al., "DIAGNOSIS: Detecting unauthorized data usages in text-to-image diffusion models," in ICLR, 2024. [Paper] [Code]

• Stable Signature: P. Fernandez, et al., "The stable signature: Rooting watermarks in latent diffusion models," in ICCV, 2023. [Paper] [Code] [Website]

• Tree-Rings Watermarks: Y. Wen, et al., "Tree-rings watermarks: Invisible fingerprints for diffusion images," in NeurIPS, 2023. [Paper] [Code]

• ZoDiac: L. Zhang, et al., "Attack-resilient image watermarking using stable diffusion," in NeurIPS, 2024. [Paper] [Code]

• Gaussian Shading: Z. Yang, et al., "Gaussian shading: Provable performance-lossless image watermarking for diffusion models," in CVPR, 2024. [Paper] [Code1] [Code2]

⚔️ Attack Schemes

The attack schemes include:

• Noise Purification: ./attacks/noise_purification
• Concept Recoveryn: ./attacks/concept_recovery
• Watermark Removal: ./attacks/watermark_removal

Noise Purification

• JPEG Compression: G. K. Wallace, "The JPEG Still Picture Compression Standard," in Communications of the ACM, 1991. [Paper]

• Quantization: P. Heckbert, "Color Image Quantization for Frame Buffer Display," in ACM SIGGRAPH Computer Graphics, 1982. [Paper]

• Total Variation Minimization: A. Chambolle, "An Algorithm for Total Variation Minimization and Applications," in Journal of Mathematical Imaging and Vision, 2004. [Paper]

• IMPRESS: B. Cao, et al., "IMPRESS: Evaluating the Resilience of Imperceptible Perturbations Against Unauthorized Data Usage in Diffusion-Based Generative AI," in NeurlPS, 2024. [Paper] [Code]

• DiffPure: W. Nie, et al., "Diffusion Models for Adversarial Purification," in ICML, 2022. [Paper] [Code] [Website]

Concept Recovery

• LoRA: E. J. Hu, et al., "LoRA: Low-Rank Adaptation of Large Language Models," in ICLR, 2022. [Paper] [Code]

• DreamBooth: N. Ruiz, et al., "DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation," in CVPR, 2023, pp. 22500–22510. [Paper] [Code] [Website]

• Textual Inversion: R. Gal, et al., "An Image is Worth One Word: Personalizing Text-to-Image Generation Using Textual Inversion," in ICLR, 2023. [Paper] [Code] [Website]

• Concept Inversion: M. Pham, et al., "Circumventing Concept Erasure Methods for Text-to-Image Generative Models," in ICLR, 2023. [Paper] [Code] [Website]

• Ring-A-Bell: Y.-L. Tsai, et al., "Ring-A-Bell! How Reliable Are Concept Removal Methods for Diffusion Models?" in ICLR, 2024. [Paper] [Code]

Watermark Removal

Watermark attacks are extended based on the ZoDiac framework, with code available at wmattacker.py and details in the paper Attack-Resilient Image Watermarking Using Stable Diffusion.

• Bright

• Rotation

• Crop

• Blur

• Learned Image Compression: Z. Cheng, et al., "Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules," in CVPR, 2020, pp. 7939–7948. [Paper] [Code]

• DiffPure: W. Nie, et al., "Diffusion Models for Adversarial Purification," in ICML, 2022. [Paper] [Code] [Website]

📚 How to Cite

Thanks for your interest in our work. If you find our work useful for your research, please cite the following paper:

@article{xu2024copyrightmeter,
      title={CopyrightMeter: Revisiting Copyright Protection in Text-to-image Models}, 
      author={Naen Xu and Changjiang Li and Tianyu Du and Minxi Li and Wenjie Luo and Jiacheng Liang and Yuyuan Li and Xuhong Zhang and Meng Han and Jianwei Yin and Ting Wang},
      journal={arXiv preprint arXiv:2411.13144},
      year={2024}
}

Acknowledgement

We thank the ZJU TRAIL Lab and the Stony Brook University ALPS Lab for their invaluable assistance in extending CopyrightMeter! 🎓✨