EfficientDeRain+: A High-Efficiency Image Deraining Method Enhanced by RainMix Augmentation

Background

Rain significantly affects the quality of images and videos captured by computer vision systems, with raindrops and streaks impairing clarity and degrading performance in tasks like pedestrian detection, object tracking, and semantic segmentation. To enable all-weather vision systems, effective deraining has become a critical need. However, existing methods often rely on heuristic rain model assumptions, requiring complex optimization or iterative refinement. These methods tend to be computationally expensive, unsuitable for real-time applications, and limited in addressing diverse and complex real-world rain patterns.

To address these challenges, this paper introduces EfficientDeRain+, a high-efficiency deraining method that models deraining as a predictive filtering problem. The method incorporates several innovations, including uncertainty-aware cascaded predictive filtering (UC-PFILT), weight-sharing multi-scale dilated filtering (WS-MS-DFILT), and a novel data augmentation technique called RainMix. These advancements significantly enhance the efficiency and quality of image deraining.

Source of the Paper

This research is a collaborative effort by researchers from Singapore’s Agency for Science, Technology, and Research (A*STAR), Kyushu University (Japan), New York University (USA), the University of Alberta (Canada), Tianjin University (China), and Shenzhen University. It is published in the International Journal of Computer Vision.

Research Methods and Innovations

Research Process

The study was conducted through the following steps:

1. Baseline Approach with Spatially-Variant Predictive Filtering:

   • Deraining is modeled as a predictive filtering task, where the clean pixel value is reconstructed using a weighted sum of neighboring pixels, with weights predicted by a deep neural network. The convolution operation accelerates the filtering process for improved efficiency.

2. Uncertainty-Aware Cascaded Predictive Filtering (UC-PFILT):

   • To address residual rain traces, UC-PFILT generates an uncertainty map after the first-stage filtering, which guides further refinement. Experiments demonstrate its effectiveness in removing residual rain patterns.

3. Weight-Sharing Multi-Scale Dilated Filtering (WS-MS-DFILT):

   • This module handles rain streaks of varying scales without introducing additional parameters. It achieves improved deraining quality while maintaining efficiency.

4. RainMix Data Augmentation:

   • Recognizing the limited diversity of rain patterns in existing datasets, RainMix separately augments the rain layer and background, synthesizing new rainy images to significantly improve model generalization.

Experimental Design

Experiments were conducted on multiple public datasets, including synthetic datasets (Rain100H, Rain1400, Rain800, and RainCityscapes) and real-world datasets (SPA and Raindrop). Additionally, video deraining performance was validated on the NTURain dataset.

Data Analysis and Results

• Single-Image Deraining:

  • On the Rain100H dataset, EfficientDeRain+ achieved a PSNR of 34.57 and SSIM of 0.9513, outperforming the state-of-the-art RCDNet while being 74 times faster.
  • On Rain1400 and Rain800, the method demonstrated similarly high deraining quality, achieving the highest PSNR and SSIM among all baselines.

• Video Deraining:

  • On the NTURain dataset, EfficientDeRain+ achieved the highest SSIM (0.9713) and competitive PSNR (36.98), demonstrating its potential for real-time applications with a per-frame processing time of ~7ms.

Contribution of Modules

Ablation studies revealed that: - UC-PFILT significantly reduces residual rain patterns. - WS-MS-DFILT effectively handles rain streaks of different scales. - RainMix greatly enhances the model’s generalization to diverse rain patterns, making it a key innovation.

Research Significance

Scientific Contribution

• Introduced a novel and efficient framework for deraining, providing a new perspective for low-level vision tasks.
• Innovatively leveraged predictive filtering, multi-scale dilated filtering, and advanced data augmentation to enrich deraining methodologies.

Application Value

• Demonstrated superior deraining quality and efficiency across diverse scenarios, making it suitable for real-time applications like autonomous driving and video surveillance.

Highlights

• Innovation: Eliminates reliance on complex rain models, leveraging deep network predictions for high-efficiency deraining.
• Performance: Combines high-quality restoration with fast processing speeds, leading the field in efficiency.
• Robustness: RainMix improves the model’s generalization to real-world rain patterns, addressing practical challenges effectively.

Conclusion

EfficientDeRain+ presents a practical, high-efficiency solution for image and video deraining, integrating several groundbreaking techniques with excellent performance in both quality and speed. These innovations are not only relevant for deraining but also extendable to other low-level vision tasks. Future research could explore broader weather conditions and more complex scenes to further validate and expand its applicability.