Multi-Feature Attention Convolutional Neural Network for Motor Imagery Decoding

Engineering Medical Laboratory Science Electrical Engineering Information Science Artificial Intelligence Automation Neurology Bioengineering Life Sciences Medicine
multi-feature attention convolutional neural networks motor imagery brain-computer interface lightweight model

Brain-Computer Interface (BCI) is a communication method that connects the nervous system to the external environment. Motor Imagery (MI) is the cornerstone of BCI research, referring to the internal rehearsal before physical execution. Non-invasive techniques such as Electroencephalography (EEG) can record neural activities with high temporal resolution due to their cost-effectiveness and convenience. When subjects imagine moving specific body parts, energy changes (ERD/ERS) occur in specific brain regions, which can be recorded by EEG and used to identify motor intentions. MI-based BCI systems have achieved significant progress, capable of controlling exoskeletons and cursors, especially when combined with virtual reality technology, showing notable potential for stroke rehabilitation.

Currently, the high performance of MI decoding methods is crucial for the success of these systems. However, compared to other BCI paradigms relying on external stimuli, such as Event-Related Potential (ERP) and Steady-State Visual Evoked Potential (SSVEP), improving the classification performance of spontaneous MI faces huge challenges due to lower signal-to-noise ratios and inter-subject variability.

Research Source

This paper is authored by Yiyang Qin, Banghua Yang, Sixiong Ke, Peng Liu, Fenqi Rong, and Xinxing Xia, among others, from the School of Mechanical and Electrical Engineering and Automation at Shanghai University and other research institutions. It is published in the 32nd volume of “IEEE Transactions on Neural Systems and Rehabilitation Engineering” in 2024. This work is supported by several projects, including the China National Key R&D Program.

Research Process

1. Dataset and Preprocessing

The study used two datasets: the BCI Competition IV-2a dataset from 2008 and the BCI Robot Competition MI dataset from the 2019 World Robot Conference. Preprocessing for the entire dataset was conducted using the MNE library, including band-pass filtering from 0.5-40Hz to remove artifacts such as electrooculogram (EOG) and electromyogram (EMG) interference.

2. M-FANet Architecture Design

This paper proposes a lightweight Multi-Feature Attention Neural Network (M-FANet). The model includes multiple convolutional layers for feature extraction and designs three different attention modules for frequency-domain, local spatial, and feature map calibration: - Frequency Band Attention Module: Uses Chebyshev Type II filters to extract multi-band EEG data and incorporates point convolution to merge multi-band information. - Local Spatial Attention Module: Extracts EEG data’s local spatial features using small convolution kernels, focusing on brain regions related to MI. - Feature Map Attention Module: Computes feature maps through global average pooling layers and fully connected layers and uses the Squeeze-and-Excitation Block (SEBlock) to automatically calibrate and prioritize feature maps.

3. R-Drop Training Method

A training method named Regularized Dropout (R-Drop) was introduced, which adds a regularization term to reduce output differences between sub-models, enhancing the model’s generalization ability.

Research Results

1. Performance Evaluation

On the BCI Competition IV-2a dataset, M-FANet achieved a four-class classification accuracy of 79.28%, Kappa: 0.7259; and on the WBCIC-MI dataset, it achieved a three-class classification accuracy of 77.86%, Kappa: 0.6650. Experiments show that M-FANet outperforms the latest MI decoding methods.

2. Ablation Experiments

A series of ablation experiments were conducted to validate the contributions of M-FANet’s multi-feature attention modules in frequency band attention, local spatial attention, and feature map attention. The removal of each module resulted in performance decline, indicating their importance in the model.

Conclusion and Significance

The proposed lightweight Multi-Feature Attention Neural Network M-FANet significantly improves classification performance in MI tasks through effective selection of frequency band features, local spatial features, and attention mechanisms. The R-Drop training method reduces model overfitting by constraining output differences between sub-models. M-FANet balances performance and memory requirements, demonstrating its potential in MI-based BCI research and applications.

Highlights and Value

  • Innovative Design: Efficient data processing and feature extraction using multi-feature attention modules.
  • Outstanding Performance: Verified through extensive experiments, M-FANet’s classification performance surpasses existing top MI decoding methods.
  • Lightweight Architecture: Ensures high accuracy while consuming fewer resources.
  • Wide Application Prospects: Particularly suitable for portable and embedded devices with limited resources, showing great application potential.

Future Work

Future work will include applying M-FANet to other BCI paradigms such as ERP and exploring non-fixed bandwidth segmentation methods to distinguish valuable information. Additionally, plans to explore transfer learning applications to further improve the model’s performance across different datasets are underway.