Wavelet-Based Temporal-Spectral-Attention Correlation Coefficient for Motor Imagery EEG Classification

Brain-Computer Interface (BCI) technology has rapidly developed in recent years and is considered a cutting-edge technology that allows external devices to be controlled directly by the brain without the need for peripheral nerves and muscles. Particularly in the application of Motor Imagery Electroencephalography (MI-EEG), BCI technology has shown great potential. By analyzing MI-EEG signals, it can help improve the quality of life for patients with physical disabilities or neuromuscular degeneration. However, due to individual variations and factors such as brain activity stability and low Signal-to-Noise Ratio (SNR), extracting effective features from complex EEG signals to improve the accuracy of MI-EEG classification systems remains a significant challenge.

In MI-EEG classification, feature extraction and representation are critical to determining classification performance. Commonly used feature extraction methods, such as Common Spatial Pattern (CSP), Sub-band CSP (SBCSP), and Filter Bank CSP (FBCSP), although effective, generally focus on the energy characteristics of EEG and fail to extract highly discriminative features from the raw signals. More importantly, these methods lack consideration of deep information of EEG signals in the spatial, temporal, and frequency domains, thus limiting their decoding performance.

To address these issues, the researchers in this paper proposed a Wavelet-based Temporal-Spectral-Attention Correlation Coefficient (WTS-CC) algorithm. This algorithm significantly improves MI-EEG classification accuracy by considering features and their weighting in the spatial, channel, temporal, and spectral domains of the EEG signal simultaneously.

Research Origin

This research was conducted by Wei-Yen Hsu (IEEE Senior Member) and Ya-Wen Cheng, and the paper was published in IEEE Transactions on Neural Systems and Rehabilitation Engineering, Vol. 31, 2023. The research data and methods in the report were funded by the Ministry of Science and Technology of Taiwan (MOST110-2221-E-194-027-MY3 and MOST111-2410-H-194-038-MY3).

Research Methods and Workflow

Research Workflow

The study mainly includes the following modules: 1. Initial Temporal Feature Extraction module (ITFE) 2. Deep EEG-Channel-Attention module (DEC) 3. Wavelet-based Temporal-Spectral-Attention module (WTS) 4. Discrimination module

Initial Temporal Feature Extraction Module (ITFE)

In MI-EEG classification, extracting more feature information from EEG signals is crucial for improving classification accuracy. The ITFE module extracts preliminary features directly from raw EEG signals through convolution operations of different kernel sizes. To maximize feature extraction performance, the researchers designed three convolution kernels of different sizes (1×3, 1×5, 1×11). Through temporal convolution operations, richer feature information is extracted, allowing the model to capture features at different temporal scales.

Deep EEG-Channel-Attention Module (DEC)

While extracting richer features aids in the classification of MI tasks, these feature information often mixes a lot of unrelated or redundant information. The DEC module, based on the Squeeze-and-Excitation (SE) mechanism, automatically adjusts the importance weights of each EEG channel, thereby enhancing important channels and suppressing less important ones. Through this process, the DEC module significantly improves feature extraction quality, helping to enhance more discriminative features.

Wavelet-based Temporal-Spectral-Attention Module (WTS)

EEG signals have temporal sequence characteristics, and their frequency components change over time. The WTS module introduces wavelet transformation and independent sample t statistic to capture more significant discriminative features through feature weighting between time-spectral images. In this process, the EEG signal is first transformed into a time-spectral representation through Continuous Wavelet Transform (CWT), and then the significance of feature differences between different MI tasks is evaluated using the independent sample t statistic. The WTS module effectively enhances time-spectral features that are significantly different between different MI tasks.

Discrimination Module

The study used the correlation coefficient as the basis for MI-EEG discrimination. By evaluating the correlation between the test sample and the average time-spectral feature maps (TTSFMs) of different MI tasks in the training set, the discrimination module can achieve efficient and accurate MI task classification. Compared with traditional deep learning models, the correlation coefficient method is simpler and less prone to overfitting.

Research Results

Dataset Description

The study used three public BCI competition datasets to verify the effectiveness of the proposed method: 1. BCI Competition IV dataset 2a, containing EEG signals of 9 subjects, covering four types of MI tasks (left hand, right hand, both feet, and tongue). 2. BCI Competition IV dataset 2b, containing EEG signals of 9 subjects, covering two types of MI tasks (left hand and right hand). 3. 2020 International BCI Competition dataset Track#1, containing EEG signals of 20 subjects, covering two types of MI tasks (left hand and right hand).

Evaluation Metrics

The study used ten-fold cross-validation to test each dataset and used four indicators: classification accuracy (Accuracy), Cohen’s kappa coefficient (Kappa), F1 Score, and Area Under the Curve (AUC) to evaluate the method’s performance. The experimental results show that WTS-CC surpasses the current state-of-the-art methods in all evaluation indicators across the three datasets.

Comparison with Existing Methods

Compared with twelve other latest EEG classification methods (including Shallow ConvNet, Deep ConvNet, CP-MixedNet, TS-SEFFNet, etc.), WTS-CC achieved the highest average classification accuracy (81.45%), Kappa coefficient (0.752), F1 Score, and AUC on the BCI Competition IV dataset 2a, showing significant superiority.

On the BCI Competition IV dataset 2b, WTS-CC also demonstrated the highest average classification accuracy, further proving its efficiency and practicality in MI-EEG discrimination.

Additionally, WTS-CC achieved an average classification accuracy of 83.31% on the 2020 International BCI Competition dataset Track#1, performing well even in few-sample learning scenarios.

Component Performance Evaluation and Ablation Experiments

Further studies on the impact of DEC module and WTS module on the overall system performance revealed that the DEC module can effectively improve classification performance through the attention mechanism, while the WTS module significantly enhances classification accuracy through time-spectral feature weighting. Ablation experiment results showed that the absence of the DEC module would result in a drop in average classification accuracy to 55.49%, indicating the importance of the module.

Conclusion

Through the WTS-CC method proposed in this study, the research team achieved significant improvements in MI-EEG classification performance on a variety of public datasets, solving many problems present in traditional methods. WTS-CC has great potential in improving MI-EEG classification accuracy because it combines features and their weighting in the spatial, channel, temporal, and spectral domains, effectively extracting and enhancing discriminative features.

In the future, the research team plans to improve and extend the WTS-CC model for adaptive discrimination. Additionally, applying Transfer Learning technology to this method may further enhance its applicability and generalization in real BCI systems.