From data to prediction: development and clinical validation of a preterm birth risk assessment tool based on machine learning technologies

Introduction. Preterm birth (PTB) remains one of the most serious complications of pregnancy, being the leading cause of neonatal mortality and contributing to long-term disability along with chronic morbidity in newborns, as well as imposing substantial socioeconomic costs. Despite preventive efforts, the global PTB rate has remained largely unchanged comprising 5–18 %, underscoring a need for developing more effective prediction tools to enable timely prevention.

Aim: using an independent sample to develop and validate a PTB risk-assessment tool based on machine learning (ML) and routinely collected clinical data retrieved from electronic health records (EHRs) of pregnant patients.

Materials and Methods. We analyzed a dataset of 10,000 de-identified EHRs entries containing 54 variables, including historical, clinical, laboratory, and instrumental (diagnostic/imaging) data. The predictive system comprised two interconnected ML components: (1) an NLP model based on RuBERT (а pre-trained ML model for processing Russian texts) for extracting PTB-relevant features from unstructured Russian-language clinical text, and (2) a downstream predictive ML model, for which 14 algorithms were benchmarked.

Results. The NLP model demonstrated high performance with a median sensitivity = 0.998, F1-score = 0.976, and AUC-ROC = 0.974. Among the ML algorithms, the algorithm based on gradient boosting – CatBoost Classifier (Categorical Boosting Classifier) achieved the best risk-prediction results: accuracy = 0.81, sensitivity (recall) = 0.87, precision = 0.76, F1-score = 0.81, and AUC-ROC = 0.82.

Conclusion. The developed model showed performance comparable to that of international counterparts, and validation confirmed its robustness to previously unseen data, indicating strong potential for use in routine clinical practice. This study represents the first step toward an integrated PTB risk-assessment solution combining NLP and ML. Future work will include incorporation of additional predictors (e.g., biochemical markers) and multicenter validation studies.

1. Ivshin A.A., Pogodin O.O., Shakurova E.Yu. et al. Experience of laparoscopic transabdominal cerclage for the correction of cervical insufficiency during pregnancy: a clinical case and literature review. [Laparoskopicheskij transabdominal'nyj serklyazh dlya lecheniya istmikocervikal'noj nedostatochnosti pri beremennosti: klinicheskij sluchaj i obzor literatury]. Obstetrics, Gynecology and Reproduction. 2025;19(1):116–26. (In Russ.). https://doi.org/10.17749/2313-7347/ob.gyn.rep.2025.

2. Serov V.N., Sukhorukova O.I. Effectiveness of preterm birth prevention. [Effektivnost' profilaktiki prezhdevremennykh rodov]. Akusherstvo i ginekologiya. 2013;(3):48–53. (In Russ.).

3. Risnes K., Bilsteen J.F., Brown P. et al. Mortality among young adults born preterm and early term in 4 Nordic nations. JAMA Netw Open. 2021;4(1):e2032779. https://doi.org/10.1001/jamanetworkopen.2020.32779.

4. Jeon G.W., Lee J.H., Oh M., Chang Y.S. Serial long-term growth and neurodevelopment of very-low-birth-weight infants: 2022 update on the Korean Neonatal Network. J Korean Med Sci. 2022;37(34):e263. https://doi.org/10.3346/jkms.2022.37.e263.

5. Gorina K.A., Khodzhaeva Z.S., Belousov D.M. et al. Preterm birth: past limitations and new opportunities. [Prezhdevremennye rody: proshlye ogranicheniya i novye vozmozhnosti]. Akusherstvo i ginekologiya. 2020;(1):12–9. (In Russ.). https://doi.org/10.18565/aig.2020.1.12-19.

6. Stedall P.M., Spencer-Smith M.M., Lah S. et al. Episodic and prospective memory difficulties in 13-year-old children born very preterm. J Int Neuropsychol Soc. 2023;29(3):257265. https://doi.org/10.1017/S1355617722000170.

7. Belousova V.S., Strizhakov A.N., Svitich O.A. et al. Preterm birth: causes, pathogenesis, management tactics. [Prezhdevremennye rody: prichiny, patogenez, taktika]. Akusherstvo i ginekologiya. 2020;(2):82–7. (In Russ.). https://doi.org/10.18565/aig.2020.2.82-87.

8. Thain S., Yeo G.S.H., Kwek K., Chern B., Tan K.H. Spontaneous preterm birth and cervical length in a pregnant Asian population. PLoS One. 2020;15(4):e0230125. https://doi.org/10.1371/journal.pone.0230125.

9. Drukkер N.A., Durnitsyna O.A., Nikashina A.A., Selyutina S.N. Diagnostic value of α-1-microglobulin in the development of preterm birth. [Diagnosticheskaya znachimost' α-1-mikroglobulina v razvitii prezhdevremennykh rodov]. Akusherstvo i ginekologiya. 2019;(1):81–5. (In Russ.). https://doi.org/10.18565/aig.2019.1.81-85.

10. Baev O.R., Dicke G.B. Diagnosis of premature rupture of fetal membranes based on biochemical tests. [Diagnostika prezhdevremennogo razryva plodnykh obolochek na osnove biokhimicheskikh testov]. Akusherstvo i ginekologiya. 2018;(9):132–6. (In Russ.). https://doi.org/10.18565/aig.2018.9.132-136.

11. Clinical guidelines – Preterm birth – 2020 (01.12.2020). [Klinicheskie rekomendacii – Prezhdevremennye rody – 2020 (01.12.2020)]. Moscow: Ministerstvo zdravoohraneniya Rossijskoj Federacii, 2020. 66 p. (In Russ.). Available at: https://cr.minzdrav.gov.ru/schema/331_1. [Accessed: 28.08.2025].

12. Manukhin I.B., Firichenko S.V., Mikailova L.U. et al. Prediction and prevention of preterm birth – current state of the problem. [Prognozirovanie i profilaktika prezhdevremennykh rodov – sovremennoe sostoyanie problemy]. Rossiiskii vestnik akushera-ginekologa. 2016;(3):9–15. (In Russ.). https://doi.org/10.17116/rosakush20161639-15.

13. Khodzhaeva Z.S., Dembovskaya S.V., Dobrokhotova Yu.E. et al. Pharmacological prevention of preterm birth (results of the international multicenter open MISTERY study). [Medikamentoznaya profilaktika prezhdevremennykh rodov (rezul'taty mezhdunarodnogo mnogotsentrovogo otkrytogo issledovaniya MISTERY)]. Akusherstvo i ginekologiya. 2016;(8):37–43. (In Russ.). https://doi.org/10.18565/aig.2016.8.37-43.

14. Barinov S.V., Artymuk N.V., Novikova O.N. et al. Management experience in pregnant women at high risk for preterm birth using an obstetric dome-shaped pessary and cerclage. [Opyt vedeniya beremennykh gruppy vysokogo riska po prezhdevremennym rodam s primeneniem akusherskogo kupoloobraznogo pessariya i serklyazha]. Akusherstvo i ginekologiya. 2019;(1):140–8. (In Russ.). https://doi.org/10.18565/aig.2019.1.140-148.

15. Crockart I.C., Brink L.T., du Plessis C., Odendaal H.J. Classification of intrauterine growthrestriction at 34-38 weeks gestation with machine learning models. Inform Med Unlocked. 2021;23:100533. https://doi.org/10.1016/j.imu.2021.100533.

16. Liu J., Wang C., Yan R. et al. Machine learning-based prediction of postpartum hemorrhage after vaginal delivery: combining bleeding high risk factors and uterine contraction curve. Arch Gynecol Obstet. 2022;306(4):1015–25. https://doi.org/10.1007/s00404-021-06377-0.

17. Melinte-Popescu A.S., Vasilache I.A., Socolov D., Melinte-Popescu M. Predictive performance of machine learning-based methods for the prediction of preeclampsia – a prospective study. J Clin Med. 2023;12(2):418. https://doi.org/10.3390/jcm12020418.

18. Andreychenko A.E., Luchinin A.S., Ivshin A.A. et al. Development and validation of models to predict total and early-onset preeclampsia in the first trimester of pregnancy using machine learning algorithms. [Razrabotka i validatsiya modelei prognozirovaniya obshchego riska preeklampsii i riska rannei preeklampsii s ispol'zovaniem algoritmov mashinnogo obucheniya v pervom trimestre beremennosti]. Akusherstvo i ginekologiya. 2023;(10):94–107. (In Russ.). https://doi.org/10.18565/aig.2023.101.

19. Chen Y., Shi X., Wang Z., Zhang L. Development and validation of a spontaneous preterm birth risk prediction algorithm based on maternal bioinformatics: A single-center retrospective study. BMC Pregnancy Childbirth. 2024;24(1):763. https://doi.org/10.1186/s12884024-06933-x.

20. Zhang Y., Du S., Hu T. et al. Establishment of a model for predicting preterm birth based on the machine learning algorithm. BMC Pregnancy Childbirth. 2023;23(1):779. https://doi.org/10.1186/s12884-023-06058-7.

21. Sun Q., Zou X., Yan Y. et al. Machine learning-based prediction model of preterm birth using electronic health record. J Healthc Eng. 2022;2022:9635526. https://doi.org/10.1155/2022/9635526.

22. Mavrogiorgou A., Kiourtis A., Kleftakis S. et al. A catalogue of machine learning algorithms for healthcare risk predictions. Sensors (Basel). 2022;22(22):8615. https://doi.org/10.3390/s22228615.

23. Hicks S.A., Strümke I., Thambawita V. et al. On evaluation metrics for medical applications of artificial intelligence. Sci Rep. 2022;12(1):5979. https://doi.org/10.1038/s41598-022-09954-8.

24. Liu T., Krentz A., Lu L., Curcin V. Machine learning based prediction models for cardiovascular disease risk using electronic health records data: systematic review and meta-analysis. Eur Heart J Digit Health. 2024;6(1):7–22. https://doi.org/10.1093/ehjdh/ztae080.

25. Khandre V., Potdar J., Keerti A. Preterm birth: an overview. Cureus. 2022;14(12):e33006. https://doi.org/10.7759/cureus.33006.

26. Fomina A.S. Premature birth, modern realities. [Prezhdevremennye rody, sovremennye realii]. Research Results in Biomedicine. 2020;6(3):434–46. (In Russ.). https://doi.org/10.18413/2658-6533-2020-6-3-0-12.