Image Synthesis under Limited Data: A Survey and Taxonomy
Image Synthesis Under Limited Data: A Survey
Research Background and Problem Statement
In recent years, deep generative models have achieved unprecedented progress in intelligent creation tasks, especially in areas such as image and video generation, and audio synthesis. However, the success of these models relies heavily on large amounts of training data and computational resources. When the training data is limited, generative models are prone to overfitting and memorization issues, which significantly degrade the quality and diversity of generated samples. This limitation poses challenges for many practical application scenarios, such as medical image generation, industrial defect detection, and artwork creation.
To address these issues, researchers have been dedicated to developing new models capable of generating high-quality and diverse images under limited data conditions. Despite numerous studies attempting to solve this problem, there is still a lack of systematic reviews that clearly define the following points:
1. The definition, challenges, and taxonomy of image synthesis under limited data;
2. An in-depth analysis of the advantages, disadvantages, and limitations of existing literature;
3. Exploration of potential application directions and future research trends in this field.
To fill this gap, Mengping Yang and Zhe Wang authored a survey paper titled “Image Synthesis Under Limited Data: A Survey and Taxonomy,” aiming to provide a comprehensive introduction for beginners and valuable reference resources for researchers in related fields.
Source of the Paper and Author Information
This survey was co-authored by Mengping Yang and Zhe Wang, both from the Department of Computer Science and Engineering at East China University of Science and Technology, as well as the Key Laboratory of Smart Manufacturing in Energy Chemical Processes under the Ministry of Education. The paper was published in the prestigious journal International Journal of Computer Vision (IJCV) and officially went online in January 2025 (DOI: 10.1007/s11263-025-02357-y). IJCV is one of the top journals in the field of computer vision, focusing on publishing high-quality research results, thus making this paper highly valuable academically.
Main Content and Discussion
1. Main Task Classification of Image Synthesis Under Limited Data
The authors first proposed a systematic classification framework, dividing image synthesis tasks under limited data conditions into four categories:
1. Data-Efficient Generative Models: Directly learn distributions from limited data and generate new samples;
2. Few-Shot Generative Adaptation: Transfer knowledge from pre-trained large-scale generative models to target domains;
3. Few-Shot Image Generation: Generate new samples based on a few input conditional images;
4. One-Shot Image Synthesis: Generate diverse samples using only one reference image.
Each category corresponds to different technical challenges and solutions. For example, in data-efficient generative models, the main issue is how to avoid overfitting and memorization; whereas in few-shot generative adaptation, it is necessary to handle distribution differences between source and target domains.
2. Technical Approaches for Data-Efficient Generative Models
Method Overview
For data-efficient generative models, the authors summarized four major technical approaches:
- Augmentation-Based Approaches: Expand the training set through data augmentation, such as Adaptive Discriminator Augmentation (ADA), Pseudo Augmentation (APA), etc.;
- Regularization-Based Approaches: Introduce additional constraints to stabilize the training process, such as Consistency Regularization (CR), Balanced Consistency Regularization (BCR), etc.;
- Architecture Variants: Design lightweight network structures or optimize the parameter complexity of existing models, such as FastGAN and Re-GAN;
- Off-the-Shelf Model Based Approaches: Use pre-trained models to extract feature space information, such as ProjectedGAN and StyleGAN-XL.
Experimental Results and Comparisons
The authors evaluated the performance of the above methods on multiple benchmark datasets, including FFHQ (face dataset), AFHQ (animal face dataset), and some low-shot datasets (e.g., Animal-Faces-Cat). Experiments showed that combining augmentation methods with regularization methods yields the best results. For instance, in the FFHQ dataset, FakeCLR+ADA achieved FID scores of 9.9 and 7.25 on 2K and 5K samples respectively, outperforming other methods.
3. Core Strategies for Few-Shot Generative Adaptation
Method Overview
The goal of few-shot generative adaptation is to transfer knowledge from pre-trained generative models to target domains. The authors categorized it into four strategies:
1. Fine-Tuning Based Approaches: Adjust part of the parameters of pre-trained models, such as TransferGAN and EWC;
2. Extra Branches Based Approaches: Add auxiliary networks to mine target domain features, such as MineGAN and Dorm;
3. Regularization Based Approaches: Retain source domain knowledge through regularization terms, such as CDC and DCL;
4. Kernel Modulation Based Approaches: Dynamically adjust network weights to adapt to target domains, such as Adam and OKM.
Experimental Results and Comparisons
In the task of transferring from FFHQ to Babies, Sunglasses, and Sketches datasets, kernel modulation methods performed particularly well. For example, OKM achieved an FID score of 37.57 on the Babies dataset, significantly better than traditional fine-tuning methods (e.g., TransferGAN’s 104.79). This indicates that kernel modulation methods can more effectively transfer knowledge while avoiding overfitting.
4. Few-Shot Image Generation and One-Shot Image Synthesis
Few-Shot Image Generation
Few-shot image generation requires models to generate diverse samples based on a few input conditional images. Common methods include optimization-based, transformation-based, and fusion-based approaches. Experimental results show that transformation-based methods achieve a good balance between generation quality and diversity.
One-Shot Image Synthesis
The main challenge of one-shot image synthesis is capturing the internal distribution of a single reference image. To this end, researchers have proposed multi-stage training and patch-level training strategies. For example, SinGAN proposed by Shaham et al. can achieve high-quality one-shot synthesis through hierarchical generation.
5. Application Scenarios and Open Problems
Application Scenarios
Image synthesis technology under limited data has demonstrated significant value in multiple fields:
- Medical Imaging: Generate scarce disease images to assist in diagnosis;
- Industrial Inspection: Generate defect images for training detection models;
- Artistic Creation: Generate personalized artworks or restore historical paintings.
Open Problems
Despite certain progress, many unresolved issues remain:
1. How to further improve the data efficiency of models?
2. How to maintain generation quality while reducing computational costs?
3. How to design more powerful regularization methods to alleviate overfitting problems?
Research Significance and Value
Through a comprehensive review of the field of image synthesis under limited data, this paper provides readers with a clear problem definition, classification system, and detailed analysis of the latest research findings. Its main contributions include:
1. Proposing a unified task classification framework to facilitate understanding of the relationships between different research directions;
2. Conducting a comprehensive comparison of existing methods, revealing the advantages and limitations of various methods;
3. Exploring potential application directions and future research trends, providing guidance for subsequent research.
Additionally, the authors maintain a timely updated online resource repository (Awesome-Few-Shot-Generation) to continuously track the latest developments in this field. This survey not only provides valuable reference materials for academia but also lays a theoretical foundation for practical applications in industry.