(986) Prospective validation of a Vital-sign augmented Peripartum Risk Stratification Model to predict severe maternal morbidity

To build and validate an equitable Vital-sign augmented Peripartum Risk Stratification Model (VPRSM) in the electronic medical record (EMR) to identify in real-time patients at highest risk for severe maternal morbidity (SMM) during delivery admission.

Study Design:

Using deliveries/terminations from 1/1/13-10/31/20, CDC-defined SMM were identified by ICD9/10 codes, audited and time-stamped using pre-specified diagnostic criteria. Patients with SMM prior to/on admission were excluded. Demographic and clinical (including laboratory and vital sign) data informed the model. One variable from highly correlated pairs (Pearson’s r ≥0.6) and those without a univariable signal were removed and logistic regression conducted. VPRSM was built on a discrete time-framework and ran continuously behind the scenes in the EMR (7/21/22-12/26/22) for prospective validation. Area under the receiver operating characteristic curve (AUC) was calculated 1 hour before a SMM (for patients with SMM) or at a randomly chosen time point for those without. SMM was characterized as Composite (any SMM), Minor (transfusion < 3 units of blood) and Major (transfusion ≥3 units or any non-transfusion SMM).  Model data and equity were analyzed by race/ethnicity with White as reference.  Fairness metrics were calculated. False Negative Rate (FNR) was chosen to define fairness, with a tolerance of 1.3 times smaller or larger than reference.

Results:

39,521 patients were included of which 2,940 comprised the prospective cohort (Table). The AUC 1 hour before an event was 0.84 (95%CI: 0.80-0.88) for Composite, 0.87 (95%CI: 0.81-0.93) for Major and 0.83 (95%CI: 0.78-0.88) for Minor SMM. Hispanic and Black met fairness criterion. Asian and Other both had FNR lower than White (better performance) but fell outside the tolerance range (Table).

Conclusion:

VPRSM is a highly predictive equity-based model that incorporates real-time vital sign data to identify patients at highest risk for SMM during delivery admission.