MRQA: Addressing the Widespread Problem of MRI Protocol Non-Compliance

Background

In recent years, large-scale neuroimaging datasets have played a crucial role in studying brain-behavior relationships, such as the Alzheimer’s Disease Neuroimaging Initiative (ADNI), Human Connectome Project (HCP), and Adolescent Brain Cognitive Development (ABCD) study. These datasets are typically collected from multiple sites and different scanner models. However, a significant issue exists in cross-site or cross-device data collection, which is the inconsistency of imaging parameters. Inconsistent imaging parameters can severely affect data quality, reduce signal-to-noise ratio (SNR), and statistical power, potentially invalidating research findings.

Traditionally, ensuring MRI scan protocol consistency is a cumbersome and manual task. This is primarily due to the complexity of DICOM (Digital Imaging and Communications in Medicine) and the lack of resources dedicated to addressing this issue. Moreover, due to the frequent impromptu adjustments of parameter values at different sites, protocol non-compliance issues are often overlooked. Thus, consistent imaging protocols are particularly important when aggregating data across multiple sites.

Research and Author Information

This paper is written by Harsh Sinha and Pradeep Reddy Raamana, from the Intelligent Systems Program, Department of Biomedical Informatics, and Department of Radiology at the University of Pittsburgh. It was published in the journal Neuroinformatics, with the acceptance date of May 4, 2024.

Research Objectives and Methodology

Given the high time and effort costs of existing manual verification methods, this study proposes an open-source tool, MRQA (Magnetic Resonance Quality Assurance), aimed at automatically assessing the protocol compliance of MRI datasets. MRQA can handle datasets in both DICOM and BIDS formats, focusing on identifying protocol non-compliance issues. It has been tested on over 20 publicly available neuroimaging datasets, including the large ABCD study, revealing widespread protocol non-compliance issues. These non-compliances include deviations in repetition time (TR), echo time (TE), flip angle (FA), and phase encoding direction (PED).

Workflow

1. Data Parsing: MRQA first parses the input dataset, creating a data structure that contains all imaging parameters.
2. Compliance Check: It then summarizes and evaluates different imaging modalities (such as anatomical, functional, and diffusion MRI), generating protocol compliance reports.

Analysis Steps

a. Horizontal Audit: Checks the consistency of parameter configurations with the reference protocol for all subjects under a single imaging modality. b. Vertical Audit: Ensures the consistency of acquisition parameters across multiple imaging modalities within a single imaging session.

MRQA differentiates itself from other methods by conducting checks directly on the scanner and enabling automated scripting for regular monitoring (e.g., daily or weekly), promptly notifying researchers of any non-compliance issues.

Results

Testing on datasets such as ABCD and OpenNeuro revealed significant non-compliance issues with scanners from different manufacturers. Particularly, scanners from GE and Philips showed higher rates of non-compliance compared to Siemens scanners. Specific findings include:

• ABCD Dataset: In T1-weighted images, Philips scanners had a non-compliance rate of 64.43%, while GE scanners had a rate of 2.0%.
• OpenNeuro Dataset: Multiple datasets showed missing key parameters, such as PED, magnetic field strength, and echo train length.

These findings indicate that protocol non-compliance is a prevalent issue in multi-site collections, requiring compliance checks both before and after image acquisition to ensure data integrity and reliability.

Discussion and Significance

Compliance issues not only affect the predictive performance of computational models but also impact the broad application of clinical trials and the reproducibility of scientific research findings. Introducing an automatic protocol compliance check tool like MRQA can significantly reduce potential errors and omissions in manual verification, enhancing the quality of datasets and the effectiveness of statistical analyses.

This study emphasizes the importance of timely detection and correction of non-compliance issues during data collection. Although MRQA still has some limitations, such as the inability to parse parameters from proprietary headers of GE and Philips, it provides an important tool for achieving protocol consistency in MRI datasets and has broad application prospects.

Conclusion

The MRQA tool presented in this paper offers an effective solution to the protocol non-compliance issue in MRI datasets. It can automatically generate compliance reports, helping researchers promptly identify and correct non-compliance issues, thereby ensuring data integrity and the reliability of statistical analyses. With the continuous expansion of neuroimaging datasets, the practicality and importance of this tool will become increasingly significant.