Development of a predicative model of the threat of COVID- 19 unfavorable outcome in hospitalized patients of older age groups by means of artificial intelligence use


DOI: https://dx.doi.org/10.18565/therapy.2024.7.19-27

Kudryavtseva N.A., Chorbinskaya S.A., Devyatkin A.V., Samushiya M.A., Kolpakov E.A., Kuznetsov A.I., Shchepkina E.V.

1) Central State Medical Academy of the Administrative Directorate of the President of the Russian Federation, Moscow; 2) Central Clinical Hospital with a polyclinic of the Administrative Directorate of the President of the Russian Federation, Moscow; 3) Moscow Aviation Institute (National Research University); 4) Russian Presidential Academy of National Economy and Public Administration, Moscow; 5) Scientific and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Department of Healthcare of Moscow
Abstract. High mortality rates among COVID-19 patients hospitalized in 2020–2022, especially among patients over 60 years of age, make necessary the early diagnosis of COVID-19 and assessment of complications and death risk in order to provide timely treatment and supply preventive measures implementation to avoid adverse outcomes of the disease.
The aim: to develop a prognostic model using artificial intelligence (machine learning) to predict adverse outcomes in elderly patients hospitalized with COVID-19 at the initial stage of the disease.
Material and methods. A single-center retrospective cohort study of 263 patients hospitalized with COVID-19 from March 2020 to December 2022 was performed. Multiple logistic regression was used to build a forecasting model.
Results. During the process of multivariate analysis, 23 most important indexes associated with a fatal outcome were identified from 200 different indexes obtained in COVID-19 patients aged 60 years and older. These 23 indexes later were subsequently included in the model for predicting an unfavorable outcome: age, clinical and instrumental examination data (thermometry, pulse oximetry and respiratory rate counting) taking into account polymorbidity (the presence of certain chronic diseases) and anamnestic data (previous anti-COVID vaccination, therapy carried out at the outpatient stage). During the study, a predictive model for assessing the threat of an unfavorable outcome of COVID-19 was developed. The accuracy of the suggested method at the development stage was 80.4 [76.2; 84.6] %, sensitivity – 79.2 [72.6; 84.8] %, specificity – 81.7 [76.4; 87.4] % and ROC-AUC – 88.3 [84.7; 91.5] %. At the testing stage, similar indexes were as following: accuracy – 71.7 [60.4; 81.1] %, sensitivity – 70.6 [50.0; 88.2] %, specificity – 72.2 [59.0; 83.9] %, ROC-AUC – 78.9 [67.0; 88.8] %.
Conclusion. A model for predicting the risk of an adverse outcome of COVID-19 arising in hospitalized patients of older age groups has been developed and tested. It demonstrates high specificity and has undoubted practical value.
Keywords: prognostic model, ROC-AUC, machine learning, COVID-19, artificial intelligence, logistic regression, polymorbidity, sensitivity, specificity, vaccination, diagnostics
Read entire article

Literature

1. Девяткин А.В., Девяткин А.А. Коронавирусная инфекция COVID-19: факты и комментарии: руководство для врачей. М.: ГЭОТАР-Медиа. 2023; 104 с. (Devyatkin A.V., Devyatkin A.A. COVID-19 coronavirus infection: facts and comments: A guide for doctors. Moscow: GEOTAR-Media. 2023; 104 pp. (In Russ.)).


ISBN: 978-5-9704-8067-0. https://doi.org/10.33029/9704-8067-0-CFC-2023-1-104.


2. World Health Organization. Tracking SARS-CoV-2 variants. URL: https://www.who.int/activities/tracking-SARS-CoV-2-variants (date of access – 28.08.2024).


3. Кудрявцева Н.А., Чорбинская С.А., Девяткин А.В. с соавт. Особенности клинического течения COVID-19 у лиц старших возрастных групп. Кремлевская медицина. Клинический вестник. 2023; (4): 28–36. (Kudryavtseva N.A., Chorbinskaya S.A., Devyatkin A.V. et al. Peculiar features of the COVID-19 clinical course in people of older age groups. Kremlevskaya medicina. Klinicheskiy vestnik = Kremlin Medicine Journal. 2023; (4): 28–36 (In Russ.)).


https://doi.org/10.48612/cgma/r69d-vd7v-mura. EDN: BSAOWO.


4. Кузнецов А.И., Щепкина Е.В., Сушинская Т.В. с соавт. Возможности и ограничения применения искусственного интеллекта в медицине. Новости клинической цитологии России. 2023; 27(2): 18–24. (Kuznetsov A.I., Schepkina E.V., Sushinskaya T.V. et al. Possibilities and limitations of artificial intelligence application in medicine. Novosti klinicheskoy tsitologii Rossii = News of Clinical Cytology of Russia. 2023; 27(2): 18–24 (In Russ.)).


https://doi.org/10.24412/1562-4943-2023-2-0003. EDN: FOEYJU.


5. Carr E., Bendayan R., Bean D. et al. Evaluation and improvement of the National Early Warning Score (NEWS2) for COVID-19: A multi-hospital study. BMC Med. 2021; 19(1): 23.


https://doi.org/10.1186/s12916-020-01893-3. PMID: 33472631. PMCID: PMC7817348.


6. Barda N., Riesel D., Akriv A. et al. Developing a COVID-19 mortality risk prediction model when individual-level data are not available. Nat Commun. 2020; 11(1): 4439.


https://doi.org/10.1038/s41467-020-18297-9. PMID: 32895375. PMCID: PMC7477233.


7. Lu J., Hu S., Fan R. et al. ACP Risk Grade: A simple mortality index for patients with confirmed or suspected severe acute respiratory syndrome coronavirus 2 disease (COVID-19) during the early stage of outbreak in Wuhan, China. SSRN Electronic Journal. 2020.


https://doi.org/10.1101/2020.02.20.20025510.


8. Imran A., Posokhova I., Qureshi H.N. et al. AI4COVID-19: AI enabled preliminary diagnosis for COVID-19 from cough samples via an app. Inform Med Unlocked. 2020; 20: 100378.


https://doi.org/10.1016/j.imu.2020.100378. PMID: 32839734. PMCID: PMC7318970.


9. Jin C., Chen W., Cao Y. et al. Development and evaluation of an artificial intelligence system for COVID-19 diagnosis. Nat Commun. 2020; 11(1): 5088.


https://doi.org/10.1038/s41467-020-18685-1. PMID: 33037212. PMCID: PMC7547659.


10. Abdulaal A., Patel A., Charani E. et al. Prognostic modeling of COVID-19 using artificial intelligence in the United Kingdom: Model development and validation. J Med Internet Res. 2020; 22(8): e20259.


https://doi.org/10.2196/20259. PMID: 32735549. PMCID: PMC7451108.


11. Xie J., Hungerford D., Chen H. et al. Development and external validation of a prognostic multivariable model on admission for hospitalized patients with COVID-19. SSRN Electronic Journal. 2020.


https://doi.org/10.1101/2020.03.28.20045997.


12. Zack J.E., Garrison T., Trovillion E. et al. Effect of an education program aimed at reducing the occurrence of ventilator-associated pneumonia. Crit Care Med. 2002; 30(11): 2407–12.


https://doi.org/10.1097/00003246-200211000-00001. PMID: 12441746.


13. Щепкина Е.В., Епифанова С.В., Кузнецов А.И. Stard и tripod: рекомендации по предоставлению результатов диагностических и прогностических исследований. Практические советы для оформления статей в журналы. Педиатрия. Журнал им. Г.Н. Сперанского. 2022; 101(1): 236–248. (Schepkina E.V., Epifanova S.V., Kuznetsov A.I. Stard and tripod: Recommendations for the provision of diagnostic and prognostic research results. Practical advice for submitting articles to journals. Pediatriya. Zhurnal im. G.N. Speranskogo = Pediatrics. Journal named after G.N. Speransky. 2022; 101(1): 236–248 (In Russ.)).


https://doi.org/10.24110/0031-403X-2022-101-1-236-249. EDN: RZWUNE.


14. Moons K.G.M., Altman D.G., Reitsma J.B. et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and elaboration. Ann Intern Med. 2015; 162(1): W1–W73.


https://doi.org/10.7326/M14-0698. PMID: 25560730.


15. Кудрявцева Н.А., Чорбинская С.А., Девяткин А.В., Самушия М.А., Колпаков Е.А., Щепкина Е.В., Кузнецов А.И. Способ прогнозирования наступления смерти у больных COVID-19 в возрасте 60 лет и старше. Патент на изобретение RUS 2805263. Заявка от 13.10.2023. (Kudryavtseva N.A., Chorbinskaya S.A., Devyatkin A.V., Samushiya M.A., Kolpakov E.A., Shchepkina E.V., Kuznetsov A.I. A method for predicting death in patients with COVID-19 aged 60 years and older. Patent for invention RUS 2805263. Application dated 10/13/2023 (In Russ.)).


16. Кудрявцева Н.А., Девяткин А.В., Чорбинская С.А. с соавт. Программное обеспечение для прогнозирования вероятности наступления смерти у больных госпитализированных с COVID-19 в возрасте 60 лет и старше: Свидетельство о государственной регистрации программы для ЭВМ № 2024611345 Российская Федерация. № 2023689822: заявл. 27.12.2023: опубл. 19.01.2024. (Kudryavtseva N.A., Devyatkin A.V., Chorbinskaya S.A. et al. Software for predicting the probability of death in patients hospitalized with COVID-19 aged 60 years and older: Certificate of state registration of the computer program No. 2024611345 Russian Federation. No. 2023689822: application. 12/27/2023: publ. 01/19/2024 (In Russ.)).


17. Assaf D., Gutman Y., Neuman Y. et al. Utilization of machine-learning models to accurately predict the risk for critical COVID- 19. Intern Emerg. Med. 2020; 15(8): 1435–43.


https://doi.org/10.1007/s11739-020-02475-0. PMID: 32812204. PMCID: PMC7433773.


18. Das A.K., Mishra S., Gopalan S. Predicting COVID-19 community mortality risk using machine learning and development of an online prognostic tool. PeerJ. 2020; 8: e10083.


https://doi.org/10.7717/peerj.10083. PMID: 33062451. PMCID: PMC7528809.


19. Ryan L., Lam C., Mataraso S. et al. Mortality prediction model for the triage of COVID-19, pneumonia, and mechanically ventilated ICU patients: A retrospective study. Ann Med Surg (Lond). 2020; 59: 207–16.


https://doi.org/10.1016/j.amsu.2020.09.044. PMID: 33042536. PMCID: PMC7532803.


20. Zhao Z., Chen A., Hou W. et al. Prediction model and risk scores of ICU admission and mortality in COVID-19. PLoS One. 2020; 15(7): e0236618.


https://doi.org/10.1371/journal.pone.0236618. PMID: 32730358. PMCID: PMC7392248.


21. Covino M., Sandroni C., Santoro M. et al. Predicting intensive care unit admission and death for COVID-19 patients in the emergency department using early warning scores. Resuscitation. 2020; 156: 84–91.


https://doi.org/10.1016/j.resuscitation.2020.08.124. PMID: 32918985. PMCID: PMC7480278.


22. Haimovich A.D., Ravindra N.G., Stoytchev S. et al. Development and validation of the quick COVID-19 severity index: A prognostic tool for early clinical decompensation. Ann Emerg Med. 2020; 76(4): 442–53.


https://doi.org/10.1016/j.annemergmed.2020.07.022. PMID: 33012378. PMCID: PMC7373004.


23. Luo Y., Mao L., Yuan X. et al. Prediction model based on the combination of cytokines and lymphocyte subsets for prognosis of SARS-CoV-2 Infection. J Clin Immunol. 2020; 40(7): 960–69.


https://doi.org/10.1007/s10875-020-00821-7. PMID: 32661797. PMCID: PMC7357264.


24. Ma X., Li A., Jiao M. et al. Characteristic of 523 COVID-19 in Henan Province and a Death Prediction Model. Front Public Health. 2020; 8: 475.


https://doi.org/10.3389/fpubh.2020.00475. PMID: 33014973. PMCID: PMC7506160.


25. Urban M.L., Bettiol A., Mattioli I. et al. Comparison of treatments for the prevention of fetal growth restriction in obstetric antiphospholipid syndrome: A systematic review and network meta-analysis. Intern Emerg Med. 2021; 16(5): 1357–67.


https://doi.org/10.1007/s11739-020-02609-4. PMID: 33475972. PMCID: PMC8310508.


26. Allenbach Y., Saadoun D., Maalouf G. et al.; DIMICOVID. Development of a multivariate prediction model of intensive care unit transfer or death: A French prospective cohort study of hospitalized COVID-19 patients. PLoS One. 2020; 15(10): e0240711.


https://doi.org/10.1371/journal.pone.0240711. PMID: 33075088. PMCID: PMC7571674.


27. Rigo-Bonnin R., Gumucio-Sanguino V.-D., Perez-Fernández X.-L. et al. Individual outcome prediction models for patients with COVID-19 based on their first day of admission to the intensive care unit. Clin Biochem. 2022; 100: 13–21.


https://doi.org/10.1016/j.clinbiochem.2021.11.001. PMID: 34767791. PMCID: PMC8577569.


28. Попова К.Н., Жуков А.А., Зыкина И.Л. с соавт. Шкала NEWS2 в практике работы инфекционного госпиталя для больных COVID-19. Внедрение и результаты. Вестник анестезиологии и реаниматологии. 2021; 18(1): 7–16. (Popova K.N., Zhukov A.A., Zykina I.L. et al. NEWS2 score in the practice of infectious diseases hospital in COVID-19 patients. Implementation and results. Vestnik anesteziologii i reanimatologii = Messenger of Anesthesiology and Resuscitation. 2021; 18(1): 7–16 (In Russ.)).


https://doi.org/10.21292/2078-5658-2021-18-1-7-16. EDN: XHCHFS.


29. Yang Z., Hu Q., Huang F. et al. The prognostic value of the SOFA score in patients with COVID-19: A retrospective, observational study. Medicine (Baltimore). 2021;100(32): e26900.


https://doi.org/10.1097/MD.0000000000026900. PMID: 34397917. PMCID: PMC8360480.


30. Пирадов М.А., Супонева Н.А., Рябинкина Ю.В. с соавт. Шкала комы Глазго (Glasgow Coma Scale, GCS): лингвокультурная адаптация русскоязычной версии. Журнал им. Н.В. Склифосовского «Неотложная медицинская помощь». 2021; 10(1): 91–99. (Piradov M.A., Suponeva N.A., Ryabinkina Yu.V. et al. Glasgow Coma Scale: Linguistic-cultural adaptation of the Russian version. Neotlozhnaya meditsinskaya pomoshch’. Zhurnal im. N.V. Sklifosovskogo = Russian Sklifosovsky Journal of “Emergency Medical Care”. 2021; 10(1): 91–99 (In Russ.)).


https://doi.org/10.23934/2223-9022-2021-10-1-91-99. EDN: FIUDEK.


31. Вечорко В.И., Аверков О.В., Супонева Н.А. с соавт. Валидация русскоязычной версии Шкалы оценки смертности 4С (4C Mortality Score) и прогнозирование исходов тяжелой формы COVID-19. Инфекционные болезни: новости, мнения, обучение. 2022; 11(1): 57–63. (Vechorko V.I., Averkov O.V., Suponeva N.A. et al. Validation of the Russian version of the 4C Mortality Score and prediction of outcomes of severe COVID-19. Infektsionnye bolezni: novosti, mneniya, obuchenie = Infectious Diseases: News, Opinions, Training. 2022; 11(1): 57–63 (In Russ.)).


https://doi.org/10.33029/2305-3496-2022-11-1-57-63. EDN: MOFCMX.


32. Вечорко В.И., Аверков О.В., Гришин Д.В., Зимин А.А. Шкалы NEWS2, 4C Mortality Score, COVID-GRAM, Sequential Organ Failure Assessment Quick как инструменты оценки исходов тяжелой формы COVID-19 (пилотное ретроспективное когортное исследование). Кардиоваскулярная терапия и профилактика. 2022; 21(3): 20–27. (Vechorko V.I., Averkov O.V., Grishin D.V., Zimin A.A. NEWS2, 4C Mortality Score, COVID-GRAM, Sequential Organ Failure Assessment Quick scales as outcomes assessment tools for severe COVID-19 (pilot retrospective cohort study). Kardiovaskulyarnaya terapiya i profilaktika = Cardiovascular Therapy and Prevention. 2022; 21(3): 20–27 (In Russ.)).


https://doi.org/10.15829/1728-8800-2022-3103. EDN: WLWYJC.


33. Калькулятор для расчета оценки тяжести состояния пациентов с COVID-19. Доступ: https://ershovlabexpert.ru/test/test_covid19?ysclid=lvwtvqu2fr424055517 (дата обращения – 28.08.2024). (Calculator for calculating the severity of the condition of patients with COVID-19. URL: https://ershovlabexpert.ru/test/test_covid19?ysclid=lvwtvqu2fr424055517 (date of access – 28.08.2024) (In Russ.)).


About the Autors

Natalya A. Kudryavtseva, MD, assistant professor of the Department of family medicine and therapy, Central State Medical Academy of the Administrative Directorate of the President of the Russian Federation. Address: 121359, Moscow, 19/1A Marshala Timoshenko St.
E-mail: natalya_kudryavtseva@inbox.ru
ORCID: https://orcid.org/0000-0003-4019-9598
Svetlana A. Chorbinskaya, MD, Dr. Sci. (Medicine), professor, head of the Department of family medicine and therapy, Central State Medical Academy of the Administrative Directorate of the President of the Russian Federation. Address: 121359, Moscow, 19/1A Marshala Timoshenko St.
E-mail: s.chorbinskaya@mail.ru
ORCID: https://orcid.org/0000-0001-8471-629X
Andrey V. Devyatkin, MD, Dr. Sci. (Medicine), professor of the Department of family medicine and therapy, Central State Medical Academy of the Administrative Directorate of the President of the Russian Federation. Address: 121359, Moscow, 19/1A Marshala Timoshenko St.
E-mail: dav-med@yandex.ru
ORCID: https://orcid.org/0000-0001-9230-1270
Marina A. Samushiya, MD, Dr. Sci. (Medicine), associate professor, head of the Department of psychiatry and psychotherapy, Central State Medical Academy of the Administrative Directorate of the President of the Russian Federation. Address: 121359, Moscow, 19/1A Marshala Timoshenko St.
E-mail: sma-psychiatry@mail.ru
ORCID: https://orcid.org/0000-0003-3681-9977
Egor A. Kolpakov, MD, postgraduate student of the Department of psychiatry and psychotherapy, Central State Medical Academy of the Administrative Directorate of the President of the Russian Federation. Address: 121359, Moscow, 19/1A Marshala Timoshenko St.
E-mail: e.kolpakov0055@yandex.ru
ORCID: https://orcid.org/0000-0003-4229-3545
Anton I. Kuznetsov, student of Moscow Aviation Institute (National Research University). Address: 125993, Moscow, 4 Volokolamskoe Highway.
E-mail: drednout5786@yandex.ru
ORCID: https://orcid.org/0000-0003-2182-5792
Elena V. Shchepkina, PhD (Sociology), deputy head of the Department of consolidated contingent and statistics, Russian Presidential Academy of National Economy and Public Administration, data researcher at Scientific and Practical Clinical Center for Diagnostics and Telemedicine Technologies of the Department of Healthcare of Moscow. Address: 119571, Moscow, 82/1 Vernadskogo Highway.
E-mail: elenaschepkina@yandex.ru
ORCID: https://orcid.org/0000-0002-2079-1482


Similar Articles

Back to Content

Бионика Медиа