A Robust Multi-Scale Feature Extraction Framework with Dual Memory Module for Multivariate Time Series Anomaly Detection

Information Science Artificial Intelligence Automation
anomaly detection multivariate time series robust feature extraction multi-scale global-local memory module

With the rapid development of deep learning technology, the importance of data mining and artificial intelligence training techniques in practical applications has become increasingly prominent. Especially in the field of multivariate time series anomaly detection, existing methods, though excellent, still face significant issues when dealing with noisy or contaminated data. Based on this, this paper proposes a multi-scale feature extraction framework with dual memory modules to address these challenging problems.

Research Background

Multivariate Time Series (MTS) data typically involves real-time operating states from multiple sensors in Internet of Things (IoT) applications. Effectively analyzing these data can reveal hidden information and provide warnings of anomalies to ensure the safe operation of systems. However, traditional anomaly detection methods, such as Local Outlier Factor (LOF), One-Class Support Vector Machine (OCSVM), and Isolation Forest (IF), fail to capture the complex structure and nonlinear relationships of time series data well. Furthermore, reconstruction-based methods have typical issues reconstructing anomalous data, making it difficult to distinguish between normal and anomalous data. Therefore, new efficient methods urgently need to be developed.

Framework proposed in this study

Paper Source and Authors

This paper is co-authored by Bing Xue, Xin Gao, Baofeng Li, Feng Zhai, Jiansheng Lu, Jiahao Yu, Shiyuan Fu, and Chun Xiao. These authors are from the School of Artificial Intelligence at Beijing University of Posts and Telecommunications, China Electric Power Research Institute Co., Ltd., the School of Electrical and Information Engineering at Tianjin University, and the State Grid Shanxi Marketing Service Center. The article was published in the journal “Neural Networks” on May 20, 2024.

Research Process and Methods

Workflow

The multi-scale feature extraction framework proposed in this paper includes several steps:

  1. Sequential Adjacent Windows as Input: To extract both local and long-term dependency information, continuous adjacent windows are designed as input.
  2. Dual Memory Enhanced Encoder: A dual memory enhanced encoder is proposed to extract global typical patterns and local common features. This ensures the reconstruction capability for normal data while suppressing the generalization capability for anomalous data.
  3. Multi-Scale Fusion Module: By fusing latent variables of different semantic information and time dependencies, these reconstructed latent variables are used to reconstruct samples for anomaly detection.

Experimental Methods and Data

This paper uses five public datasets from different domains to verify the effectiveness of the proposed method through experiments. They are:

  • MSL (Mars Science Laboratory)
  • SMAP (Soil Moisture Active Passive Satellite)
  • SMD (Server Machine Dataset)
  • PSM (Pooled Server Metrics)
  • SWAT (Secure Water Treatment)

Main Results

Experimental results show that the proposed method significantly outperforms 16 existing benchmark methods across five different datasets. Specifically:

  1. MSL Dataset: AUC-ROC value reaches 0.6523, FC1 value is 0.5581, and AUC-F1PA%K value is 0.3731.
  2. SMAP Dataset: AUC-ROC value is 0.5073, FC1 value is 0.2372, and AUC-F1PA%K value is 0.2782.
  3. SWAT Dataset: AUC-ROC value is 0.8452, FC1 value is 0.5960, and AUC-F1PA%K value is 0.7964.
  4. PSM Dataset: AUC-ROC value is 0.7581, FC1 value is 0.6350, and AUC-F1PA%K value is 0.6201.
  5. SMD Dataset: AUC-ROC value reaches 0.7293, FC1 value is 0.5125, and AUC-F1PA%K value is 0.3216.

Research Conclusions

This study shows that by designing continuous adjacent windows as input and using dual memory enhanced encoders and multi-scale fusion modules, the accuracy and stability of anomaly detection in multivariate time series tasks can be significantly improved. The designed feature extraction module can effectively handle noise and anomalous data present in real-world data, making the reconstructed normal data more accurate and thus better distinguishing anomalies.

Research Highlights

  1. Multi-Scale Feature Extraction: The framework integrates multi-scale features of different semantic information, making the extracted features more comprehensive and robust.
  2. Dual Memory Enhanced Encoder: By integrating global and local typical features, the model improves the reconstruction capability for normal samples while effectively suppressing anomalous data.
  3. Rich Comparative Experiments: Extensive experiments on multiple public datasets demonstrate the applicability and superiority of the proposed method in different domains and data.

Additional Valuable Information

This paper also designs a new data preprocessing module that can adaptively learn the mean and variance of data to better handle data characteristics varying over time. Additionally, a flexible sliding window technique is adopted to more accurately extract both short-term and long-term dependencies of time series. The proposed multi-scale feature extraction framework offers a new and effective solution for the task of multivariate time series anomaly detection, holding significant value for scientific research as well as providing robust technical support for practical applications in the industry.