Research on Depression Prediction Method Based on Conformal Prediction

Background Introduction

Depression is a common mental disorder characterized by persistent sadness, debilitation, and loss of interest in activities. It not only increases the risk of suicide but also imposes a significant psychological burden on patients and their families. Currently, the diagnosis of depression mainly relies on mental health reports such as the Beck Depression Inventory (BDI-II), Hamilton Depression Rating Scale (HRSD), and Patient Health Questionnaire (PHQ-8). However, these diagnostic methods depend on the subjective experience of clinicians and the cognitive abilities of patients, making them time-consuming and inefficient.

In recent years, with the rapid development of deep learning technology, depression prediction methods based on deep learning have shown great potential. However, these deep models are often deployed as “black box” models, lacking trustworthiness and failing to provide confidence in predictions. For high-risk clinical applications like depression prediction, uncertainty quantification is crucial in the decision-making process. To this end, this study proposes a depression prediction method based on Conformal Prediction (CP) (Conformal Depression Prediction, CDP), aiming to provide valid confidence intervals with theoretical coverage guarantees for model predictions.

Paper Source

This paper was co-authored by Yonghong Li, Shan Qu, and Xiuzhuang Zhou. Yonghong Li and Xiuzhuang Zhou are from the School of Artificial Intelligence at Beijing University of Posts and Telecommunications, while Shan Qu is from the Department of Psychiatry at Peking University People’s Hospital. The paper has been published in the IEEE Transactions on Affective Computing journal, with a publication date of 2025, and the DOI is 10.1109/TAFFC.2025.3542023.

Research Process

1. Research Objectives and Method Overview

The main objective of this study is to develop an uncertainty quantification method for facial depression prediction. Specifically, the authors propose two methods: Conformal Depression Prediction (CDP) and its improved version, CDP-acc. CDP provides confidence intervals with marginal coverage guarantees for model predictions through Conformal Prediction, while CDP-acc further optimizes the confidence intervals via approximate conditional coverage, making them tighter and adaptive to specific inputs.

2. Datasets and Model Selection

The study uses two commonly used facial depression datasets: AVEC 2013 and AVEC 2014. These datasets contain facial videos, with each frame labeled with BDI-II scores. The study selected the classic C3D and SlowFast networks as baseline models for facial depression prediction.

3. Uncertainty Quantification Methods

a) CDP Method

The core idea of the CDP method is to use Conformal Prediction to provide confidence intervals for model predictions. The specific steps are as follows: 1. Calibration Set Construction: Divide the dataset into training, calibration, and test sets. 2. Model Training: Train a deep neural network model using the training set. 3. Confidence Interval Calculation: Calculate prediction biases on the calibration set as Conformal Scores and compute confidence intervals according to user-defined confidence levels.

b) CDP-acc Method

The CDP-acc method further optimizes the confidence intervals through approximate conditional coverage. The specific steps are as follows: 1. Prediction Distribution Estimation: Divide the predictions on the calibration set into several subintervals and estimate the conditional distribution within each subinterval. 2. Confidence Interval Construction: Construct nested sequences based on the estimated conditional distribution and calculate the shortest width confidence interval.

4. Experiments and Evaluation

The study evaluates the performance of uncertainty quantification methods through the following metrics: - Prediction Interval Coverage Probability (PICP): Measures the probability that the confidence interval covers the true value. - Mean Prediction Interval Width (MPIW): Measures the average width of the confidence interval. - Size-Stratified Coverage (SSC): Evaluates the conditional coverage of different methods across varying levels of depression severity.

Main Results

1. Prediction Errors

The study compared prediction errors under different training loss functions and network architectures. The results show that Quantile Regression (QR) exhibits smaller prediction errors in most cases, especially on the AVEC 2013 dataset.

2. Uncertainty Quantification Performance

The CDP and CDP-acc methods performed well on PICP and MPIW metrics. CDP-acc significantly reduced the width of the confidence intervals while maintaining coverage. Particularly on the AVEC 2014 dataset, CDP-acc achieved a PICP of 93.72% with an MPIW of only 20.17.

3. Conditional Coverage Evaluation

The CDP-acc method performed best in the conditional coverage evaluation, with SSC values close to 1-α, indicating that this method can better adapt to predictions across different levels of depression severity.

Research Conclusion

The CDP and CDP-acc methods proposed in this study provide an effective uncertainty quantification framework for facial depression prediction. CDP provides confidence intervals with marginal coverage guarantees for model predictions through Conformal Prediction, while CDP-acc further optimizes the confidence intervals via approximate conditional coverage, making them tighter and adaptive to specific inputs. Experimental results show that CDP-acc significantly reduces the width of the confidence intervals while maintaining coverage, providing more precise uncertainty quantification for depression prediction.

Research Highlights

1. Innovative Method: This study is the first to apply Conformal Prediction to depression prediction, proposing an uncertainty quantification method that does not require retraining the model.
2. Theoretical Guarantee: The CDP method provides confidence intervals with theoretical coverage guarantees, ensuring the reliability of predictions.
3. Practical Application Value: The CDP and CDP-acc methods offer more reliable decision support for depression prediction, which is particularly significant in high-risk clinical applications.

Other Valuable Information

The study also conducted parameter analysis to explore the impact of the number of subintervals on the performance of CDP-acc. The results show that when the number of subintervals is 14, CDP-acc achieves the best balance between PICP and MPIW. Additionally, the study compared the impact of different empirical distribution estimation methods on the performance of CDP-acc, finding that the histogram estimation method performs best in most cases.