Invisible and Robust Protection Method for Content Generated by Deep Neural Networks

Academic Background

In recent years, with the revolutionary development and widespread application of deep learning models in engineering applications, phenomenon-level applications such as ChatGPT and DALL⋅E 2 have emerged, profoundly impacting people’s daily lives. At the same time, individuals can use open-source deep learning technology to create various content, such as image style transfer and image cartoonization, known as AI-generated content (AIGC). Against this backdrop, the copyright protection of commercial applications based on AIGC, such as Meitu, Prisma, and Adobe Lightroom, has become urgent and inevitable. However, because many AIGC-related technologies are open-source, skilled individuals can also create similar applications. Therefore, when copyright disputes arise, it becomes a significant challenge for commercial companies and their products.

Currently, some common copyright protection methods include but are not limited to copyright registration, copyright declaration, and encrypted copyright protection. These methods are effective for physical products (such as books and movies) in real life, but not for products generated by deep neural networks (DNN). Traditional copyright protection technologies such as barcodes, QR codes, and watermarks (visible and invisible) have the drawback of introducing visible traces on images, affecting their integrity and thus reducing user experience. In addition, the cost and efficiency of traditional copyright protection technologies may not be suitable for the high-traffic characteristics of DNN applications.

Paper Source

This paper is authored by Donghua Wang from the College of Computer Science and Technology, Zhejiang University, and Wen Yao, Tingsong Jiang, Weien Zhou, and Xiaoqian Chen from the Defense Innovation Institute, Intelligent Game and Decision Laboratory, Chinese Military Academy of Sciences, and Lang Lin from Zhejiang Economic Information Center. The paper was published on May 14, 2024, in the journal Neural Networks.

Research Process

This paper proposes a brand-new invisible and robust copyright protection method for DNN-generated content, named the Invisible Robust Copyright Protection Method. The research process includes the following aspects:

a) Research Workflow

1. Design of Copyright Encoder and Copyright Decoder:

   • Copyright Encoder (CP Encoder): It integrates the copyright image with the input image to generate an invisible perturbation containing the copyright information and overlays it on the input image to create the encoded image.
   • Copyright Decoder (CP Decoder): It extracts the copyright image from the encoded image.

2. Introduction of Robustness Module:

   • A robustness module is designed to enhance the decoding ability of the copyright decoder in dealing with various distortions on social media platforms.
   • Differential JPEG compression, color dithering, Gaussian blur, and text watermark are used, and a Spatial Transformer Network (STN) is deployed to improve the decoder’s anti-interference capability.

3. Design of Loss Functions:

   • Loss functions are designed in both feature space and color space to ensure the quality of the encoded image and the decoded image.

b) Main Research Outcomes

The research validates the method’s effectiveness through extensive objective and subjective experiments, and through practical testing by posting encoded images on social media (such as Weibo and Twitter) and downloading them for verification.

1. Workflow Result Data:

   • Integrate the copyright image with the input image to produce an invisible perturbation, overlay it on the input image to create the encoded image, and design a copyright decoder to extract the copyright image from the encoded image.
   • Introduce a robustness module, using differential JPEG compression, color dithering, Gaussian blur, and text watermarking, and deploy STN to improve the decoder’s anti-interference capability.
   • Ensure the quality of encoded and decoded images through loss functions.

2. Conclusion and Value:

   • The proposed copyright protection method achieves lossless protection in high-quality image scenarios and is validated for its effectiveness through extensive experiments.
   • This method can be used as a plug-in, protecting generated products without modifying existing deployed systems, effectively achieving tracking and evidence collection.

3. Innovation and Highlights:

   • The proposed study combines invisible perturbation with relevant copyright information, embedding the copyright information into images in a way undetectable by human observers.
   • The robustness module designed in the study significantly improves the anti-interference capability of decoded images under complex social media environment disturbances.
   • The design of loss functions covers feature space and color space, ensuring image quality.

c) Key Stages

The experimental evaluation of the robust protection method includes three parts: concealment evaluation, robustness evaluation, and physical testing evaluation.

1. Concealment Evaluation: Through quantitative and qualitative indicators and data hiding analysis, the method’s concealment effect on different datasets is verified.
2. Robustness Evaluation: Consider the decoding ability under potential corrosion conditions, conduct subjective tests to study human eye sensitivity to embedded traces, and carry out physical tests to verify the method’s feasibility.
3. Physical Testing: Post encoded images on Weibo and Twitter and download them, use the copyright decoder to decode, and observe the actual effect.

Significance and Value of the Research

The robust protection method proposed in this paper not only performs excellently in protecting copyright information but also has broad practical application prospects. It can make up for the shortcomings of existing methods in high-traffic DNN applications, providing a solid protection means for the commercialization of AIGC products. At the same time, through meticulous design and comprehensive evaluation of the method’s implementation, its efficiency and usability in practice are ensured, holding the potential to become a core component of the DNN-generated content protection ecosystem.

Other Important Content

The research also reveals potential factors and issues that need to be considered in future applications, such as image resolution and content diversity, as well as the need for a unified multimedia copyright information protection framework in practical applications. This provides valuable directions for further research.