Self-supervised Production Anomaly Detection and Progress Prediction Based on High-streaming Videos

Background Introduction

In modern manufacturing, real-time production monitoring, progress prediction, and anomaly detection are crucial for ensuring production quality and efficiency. However, traditional vision-based anomaly detection methods struggle with background noise during production processes and often overlook the heterogeneity of production stages. Many manufacturing environments, such as aircraft production, involve human-robot collaboration or high-precision manual assembly tasks, which are often difficult to monitor using embedded digital sensors, while real-time operation videos are readily available. Although vision-based production monitoring has been widely used in applications such as product surface inspection, existing algorithms still face challenges in distinguishing normal background variations from production-related anomalies.

To address these issues, Yifan Li and colleagues from the Department of Industrial Engineering at Tsinghua University proposed an integrated approach that combines progress prediction with anomaly detection, employing the Autoencoder Process Probability Embedding (APPE) method. This method maps the distribution of normal production images to a progress-related Gaussian Mixture Model (GMM), focusing on identifying production-related features while minimizing background interference through the proposed Spatial Activation Map (SAM).

Source of the Paper

This research was conducted by Yifan Li, Zhi-Hai Zhang, Jiaqi Xu, Xiaowei Yue, and Li Zheng from the Department of Industrial Engineering at Tsinghua University and is set to be published in IEEE Transactions on Automation Science and Engineering in 2025.

Research Process and Results

Research Process

1. Problem Definition and Background Analysis
   The paper begins with a detailed analysis of the shortcomings of existing vision-based production monitoring methods in distinguishing production-related features from background noise, particularly the complexity of background changes in dynamic production environments. The authors point out that traditional anomaly detection methods often rely on predefined rules or class labels, making it difficult to handle unforeseen anomalies during the production process.

2. Proposal of the APPE Method
   To overcome these issues, the authors proposed the APPE method. The core of this method involves mapping image features to the latent space of a Gaussian Mixture Model, guided by the SAM mechanism to focus the model on production-related features. Specifically:

   • Encoder: Maps images to low-dimensional latent variables with a GMM distribution.
     
   • Spatial Activation Map (SAM): Generated via a Multi-Layer Perceptron (MLP) to refine encoder features.
     
   • Decoder: Reconstructs images and calculates the Spatial Activated Reconstruction Error (SARE) weighted by SAM.
     
   • Loss Function: Combines Negative Log-Likelihood Loss (NLL), Triplet Loss, and SARE to train the APPE model.

3. Experimental Design and Datasets
   The study evaluated the method using two real-world production datasets: water valve production and commercial aircraft spoiler production. The water valve production dataset includes 11 assembly videos, extracting 29,529 training images and 31 assembly videos, with 700 anomaly points and 5,790 normal points annotated. The commercial aircraft spoiler production dataset was compiled from three weeks of factory surveillance footage, extracting approximately 46,000 images, divided into training and testing sets.

4. Baseline Comparison and Ablation Studies
   The authors compared APPE with three well-known anomaly detection algorithms (AE, VAE, and DAGMM) and conducted ablation studies to validate the key roles of SAM and NLL in the model. The results showed that APPE outperformed all baseline models in anomaly detection tasks, particularly on the water valve production dataset, where APPE achieved superior AUC (Area Under the Curve) and EER (Equal Error Rate) metrics.

5. Integration of Progress Prediction and Anomaly Detection
   The study also proposed integrating anomaly detection with progress prediction by analyzing posterior probabilities to optimize the accuracy and error rates of progress prediction. Experiments demonstrated that the integrated method significantly improved progress prediction accuracy and Mean Absolute Percentage Error (MAPE) on both datasets.

Main Results and Conclusions

1. Improved Anomaly Detection Performance
   On the water valve production dataset, APPE achieved an AUC of 90.79% and an EER of 17.49%, significantly outperforming other baseline models. On the commercial aircraft spoiler production dataset, APPE achieved an AUC of 80.45% and an EER of 27.24%, also demonstrating strong performance.

2. Effectiveness of SAM
   SAM significantly enhanced the model’s anomaly detection capability by focusing on production-related pixels. Ablation studies showed that removing SAM drastically reduced the model’s performance, proving its critical role in APPE.

3. Optimization of Progress Prediction
   After integrating anomaly detection with progress prediction, the progress prediction accuracy on the water valve production dataset improved from 94.60% to 95.37%, and the MAPE decreased from 5.03% to 4.97%. On the commercial aircraft spoiler production dataset, the progress prediction accuracy improved from 92.25% to 92.36%, and the MAPE decreased from 4.97% to 4.95%.

Research Significance and Highlights

1. Innovation of the APPE Method
   The APPE method introduces a novel production monitoring framework by combining Gaussian Mixture Models and Spatial Activation Maps, significantly improving the accuracy and efficiency of anomaly detection and progress prediction.

2. Self-supervised Learning Mechanism
   The proposed self-supervised learning mechanism enables the model to effectively predict unforeseen anomalies by learning features from normal production processes, even in the absence of anomaly data.

3. Practical Application Value
   This research not only holds scientific value but also provides a practical solution for real-time monitoring in production environments, especially in complex human-robot collaboration and high-precision manual assembly tasks, with broad application prospects.

Summary

The research by Yifan Li and colleagues successfully addresses the challenges of anomaly detection and progress prediction in production monitoring through the proposed APPE method. This method not only enhances anomaly detection accuracy but also optimizes progress prediction performance through integration. The study provides a powerful tool for the manufacturing industry and is expected to play a significant role in high-precision manufacturing and complex production environments.