(584) External validation and comparison of three prediction scores for severe maternal morbidity

A number of tools and scores have been proposed to assess individual risk of severe maternal morbidity. We sought to examine and compare the validity of three scoring systems.


Study Design: This was a retrospective cohort study of all individuals in the Consortium on Safe Labor dataset, which was conducted from 2002 to 2008. Individuals were excluded if they had missing information on risk factors. Severe maternal morbidity was defined based on the Centers for Disease Control and Prevention excluding blood transfusion. Risk scores were calculated for each participant using the California Maternal Quality Care Collaborative (CMQCC) scoring system, the Assessment of Perinatal Excellence (APEX) scoring system and the obstetric comorbidity index (OB-CMI). We calculated the probability of severe maternal morbidity according to the risk scores. The discriminative performance of the prediction score was examined by the receiver operating characteristic area under the curves (ROCAUCs) and their 95% confidence intervals (95% CI). ROCAUC for each score were compared using the DeLong test. Calibration plots were developed for each score to examine the goodness-of-fit.

Results: Of 117, 332 individuals, 811 (0.7%) had severe maternal morbidity. The CMQCC scoring system had a significantly higher ROCAUC compared to the APEX and OB-CMI scoring systems (0.78 [95%CI 0.77-0.80] vs. 0.74 [95%CI 0.72-0.76] vs. 0.67 [95%CI 0.64-0.69]; P < 0.001). Calibration plots showed excellent concordance between the predicted and actual SMM for the APEX scoring system and OB-CMI (both Hosmer–Lemeshow test P-values = 1.00, suggesting goodness-of-fit). However, CMQCC scoring system underestimated the probability of severe maternal morbidity if the predicted probability was more than 25%.  

Conclusion: Both CMQCC and APEX scoring systems had good discrimination to predict severe maternal morbidity. APEX scoring systems and OB-CMI had goodness-of-fit.