Stacked Deconvolutional Network for Semantic Segmentation

Introduction

Semantic segmentation is a critical task in the field of computer vision, aiming to classify each pixel in an image and predict its category. However, existing Fully Convolutional Networks (FCNs) have limitations in handling spatial resolution, often leading to problems such as blurry object boundaries and missing small objects. To address these issues, this paper proposes a Stacked Deconvolutional Network (SDN) to improve the effectiveness of semantic segmentation.

Research Background

Driven by Deep Convolutional Neural Networks (DCNNs), significant progress has been made in semantic segmentation. DCNNs, through powerful learning capabilities, can acquire high-level semantic features for tasks like image classification, object detection, and keypoint prediction. However, in semantic segmentation tasks, the downsampling operations in the classification network architecture lead to reduced spatial resolution of the feature maps, resulting in segmented outputs with unclear object boundaries and small artifact regions.

To mitigate these adverse effects, various methods have been proposed. For instance, atrous convolutions have been used to expand the receptive field of the convolution kernels, enhancing the ability to capture contextual information. Additionally, upsampling paths or deconvolution operations have been employed to restore the spatial resolution of feature maps. Nonetheless, simply stacking multiple convolutional layers increases network depth, making gradient propagation challenging during training. Therefore, this paper introduces a novel network architecture—Stacked Deconvolutional Network (SDN), which stacks multiple shallow deconvolutional network units (SDN units), incorporating intra-unit and inter-unit connections to achieve more efficient network training and optimization.

Authors and Source

The primary authors of this paper include Jun Fu, Jing Liu, Yuhang Wang, Jin Zhou, Changyong Wang, and Hanqing Lu, from the Institute of Automation of the Chinese Academy of Sciences, the Academy of Military Medical Sciences, and other institutions. The paper is published in IEEE Transactions on Image Processing and encompasses innovative research results in the field of semantic segmentation. However, due to some author changes, the paper did not see final publication, which is regrettable.

Core Work of the Research

The proposed Stacked Deconvolutional Network (SDN) stacks multiple shallow deconvolutional network units and combines intra-unit and inter-unit connections to enhance the network’s ability to capture contextual information and feature fusion. The specific workflow is as follows:

Research Process

a) Research Process: - Design multiple shallow deconvolutional network units (SDN units). - Stack multiple SDN units. - Introduce intra-unit and inter-unit connections to facilitate information flow and gradient propagation. - Add hierarchical supervision signals to continually optimize the network while improving spatial resolution.

Each SDN unit comprises two main parts: an encoder and a decoder. The encoder is responsible for downsampling to expand the receptive field and capture multi-scale features, while the decoder gradually restores spatial resolution through deconvolution operations. The pretrained weights of DenseNet-161 are utilized to enhance initial parameter performance.

Main Results

b) Main Results: Experimental results on datasets like PASCAL VOC 2012, Camvid, Gatech, and COCO Stuff show that the proposed SDN model achieves new optimal values in segmentation accuracy (Intersection-over-Union, IoU). For instance, on the PASCAL VOC 2012 dataset, the SDN model achieved an IoU score of 86.6% without CRF post-processing.

Conclusion and Value

c) Conclusion: The proposed Stacked Deconvolutional Network significantly improves semantic segmentation tasks by stacking shallow deconvolutional networks and employing hierarchical supervision. Its outstanding performance on various datasets demonstrates the effectiveness of the method in capturing contextual information and restoring precise boundaries.

d) Research Highlights: - A novel stacked deconvolutional network (SDN) architecture is proposed to capture multi-scale contextual information by stacking multiple shallow deconvolutional network units. - Intra-unit and inter-unit connections are introduced, enhancing the flow of information and gradients and improving feature reuse. - The introduction of hierarchical supervision signals further improves the efficiency of network training and segmentation accuracy.

Other Valuable Information

e) Other Valuable Information: The study also involves optimizing network training efficiency through dense intra-unit connections, inter-unit connections, and hierarchical supervision signals, allowing very deep networks to be effectively trained. Additionally, detailed experimental comparisons analyze different supervision signal generation methods and the network’s adaptability across various datasets.

Summary

The paper proposes a Stacked Deconvolutional Network (SDN) to effectively address issues like reduced spatial resolution and blurry boundaries encountered in Fully Convolutional Networks during semantic segmentation. By introducing intra-unit and inter-unit connections and hierarchical supervision signals, the SDN not only achieves state-of-the-art segmentation accuracy across multiple datasets but also provides a new perspective on deep learning network design, offering significant insights for research and application in semantic segmentation.