Early estimation of a patient’s mortality rate and length of stay is crucial for saving lives. With the launch of Electronic Health Records (EHRs) in the healthcare industry, many studies have been done on clinical decision-making with EHR. The publicly accessible dataset, the Medical Information Mart for Intensive Care IV (MIMIC-IV) database, is used in this study. We propose a prediction of mortality and hospital stay length Mortality Prediction Ensemble Model with XLNet and GRU (MXLPred) framework to address the three main challenges of accurate medical condition prediction tools and data visualisation. First, we have created an ensemble model from eXtreme Language Net (XLNet) and Gated Recurrent Unit (GRU) learning algorithms to predict the first 24-hour and 90-day mortality rates during a patient’s Intensive Care Unit (ICU) stay. We have utilised label-wise FastText embedding to identify medical entities from patient discharge summary notes, which has improved the clinical embedding and mortality estimates. We have also fine-tuned the model for better prediction results. Second, the Extreme Gradient Boosting (XGBoost) model for regression is utilised to predict the length of ICU stay of patients admitted in the ICU. Third, BigQuery is used to visualise the patient’s current and past medical history to aid in quick decision-making. We compare the above prediction estimates with other state-of-the-art algorithms. We demonstrate that the final ensemble MXLPred shows improved outcomes regarding better Area Under the Receiver Operating Characteristic curve (AUROC) and Mean Squared Error (MSE).

[1] Bardak B. and Tan M., “Improving Clinical Outcome Predictions Using Convolution over Medical Entities with Multimodal Learning,” Artificial Intelligence in Medicine, vol. 117, pp. 102112, 2021. https://doi.org/10.1016/j.artmed.2021.102112

[2] Bojanowski P., Grave E., Joulin A., and Mikolov T., “Enriching Word Vectors with Subword Information,” Transactions of the Association for Computational Linguistics, vol. 5, pp. 135-146, 2017. https://doi.org/10.1162/tacl_a_00051

[3] Bourahouat G., Abourezq M., and Daoudi N., “Word Embedding as a Semantic Feature Extraction Technique in Arabic Natural Language Processing: An Overview,” The International Arab Journal of Information Technology, vol. 21, no. 02, pp. 313-325, 2024. https://doi.org/10.34028/iajit/21/2/13

[4] Devlin J., Chang M., and Lee K., “BERT: Pre- Training of Deep Bidirectional Transformers for Language Understanding,” arXiv Preprint, vol. arXiv:1810.04805v2, 2019. https://arxiv.org/abs/1810.04805v2

[5] Elias A., Agbarieh R., Saliba W., Khoury J., Bahouth F., Nashashibi J., and Azzam Z., “SOFA Score and Short-Term Mortality in Acute Decompensated Heart Failure,” Scientific Reports, vol. 10, no. 1, pp. 1-10, 2020. https://doi.org/10.1038/s41598-020-77967-2

[6] Ge X., Huh J., Park Y., Lee J., Kim Y., and Turchin A., “An Interpretable ICU Mortality Prediction Model Based on Logistic Regression and Recurrent Neural Networks with LSTM Units,” AMIA Annual Symposium Proceedings AMIA Symposium, vol. 2018, pp. 460-9, 2018. https://pmc.ncbi.nlm.nih.gov/articles/PMC637127 4/pdf/2974987.pdf

[7] Gupta M., Gallamoza B., Cutrona N., Dhakal P., Poulain R., and Beheshti R., “An Extensive Data Processing Pipeline for MIMIC-IV,” in Proceedings of the 2nd Machine Learning for Health Symposium Conference, New Orleans, pp. 311-325, 2022. https://proceedings.mlr.press/v193/gupta22a.html

[8] Hayat N., Geras J., and Shamout F., “MedFuse: Multimodal Fusion with Clinical Time-Series Data and Chest X-Ray Images,” arXiv Preprint, vol. 182, pp. 1-25, 2022. https://arxiv.org/abs/2207.07027

[9] Hol L., Van Oosten P., Nijbroek S., Tsonas A., Botta M., Neto A., Paulus F., and Schultz M., “The Effect of Age on Ventilation Management and Clinical Outcomes in Critically Ill COVID-19 Patients-Insights from the PRoVENT-COVID Study,” Aging, vol. 14, no. 3, pp. 1087-1109, 2022. DOI:10.18632/aging.203863

[10] Huang K., Altosaar J., and Ranganath R., “ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission,” arXiv Preprint, vol. arXiv:1904.05342v3, pp. 1-9, 2020. http://arxiv.org/abs/1904.05342

[11] Huang K., Singh A., Chen S., Moseley E., Deng C., George N., Deng C., George N., and Lindvall C., “Clinical XLNet: Modeling Sequential Clinical Notes and Predicting Prolonged Mechanical Ventilation,” arXiv Preprint, vol. arXiv:1912.11975v1, pp. 1-9, 2019. http://arxiv.org/abs/1912.11975

[12] Jin M., Bahadori M., Colak A., Bhatia P., Celikkaya B., Bhakta R., Senthivel S., Khalilia M., Navarro D., and Zhang B., “Improving Hospital Mortality Prediction with Medical Named Entities and Multimodal Learning,” arXiv Preprint, vol. arXiv:1811.12276v2, pp. 1-8, 2018. https://doi.org/10.48550/arXiv.1811.12276

[13] Johnson A., Bulgarelli L., Shen L., Gayles A., Shammout A., Horng S., Pollard T., Hao S., Moody B., and Gow B., “MIMIC-IV, A Freely Accessible Electronic Health Record Dataset,” Scientific Data, vol. 10, no. 1, pp. 1-9, 2023. https://doi.org/10.1038/s41597-022-01899-x

[14] Johnson A., Pollard T., Horng S., Celi L., Mark R., “MIMIC-IV-Note: Deidentified Free-Text Clinical Notes,” PhysioNet, 2023. https://physionet.org/content/mimic-iv-note/2.2/

[15] Johnson A., Stone D., Celi L., and Pollard T., “The MIMIC Code Repository: Enabling Reproducibility in Critical Care Research,” Journal of the American Medical Informatics Association, vol. 25, no. 1, pp. 32-39, 2018. MXLPred: A Natural Language Processing Deep Learning Model for ICU Patients’ Data 799 DOI:10.1093/jamia/ocx084

[16] Kaggle-MoA-2nd-Place-Solution/dnn-train.ipynb at Main baosenguo/Kaggle-MoA-2nd-Place- Solution GitHub, https://github.com/baosenguo/Kaggle-MoA-2nd- Place-Solution/blob/main/training/dnn- train.ipynb, Last Visited, 2024.

[17] Khadanga S., Aggarwal K., Joty S., and Srivastava J., “Using Clinical Notes with Time Series Data for ICU Management,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, pp. 6432-6437, 2019. DOI:10.18653/v1/D19-1678

[18] Kormilitzin A., Vaci N., Liu Q., and Nevado- Holgado A., “Med7: A Transferable Clinical Natural Language Processing Model for Electronic Health Records,” Artificial Intelligence in Medicine, vol. 118, pp. 102086, 2021. https://doi.org/10.1016/j.artmed.2021.102086

[19] Lee H., Yoon S., Oh S., Shin J., Kim J., Jung C., and Ryu H., “Comparison of APACHE IV with APACHE II, SAPS 3, MELD, MELD-Na, and CTP Scores in Predicting Mortality After Liver Transplantation,” Scientific Reports, vol. 7, no. 1, pp. 1-10, 2017. https://www.nature.com/articles/s41598-017- 07797-2

[20] Lu Q., Dou D., and Nguyen T., “ClinicalT5: A Generative Language Model for Clinical Text,” in Proceedings of the Findings of the Association for Computational Linguistics, Abu Dhabi, pp. 5436- 5443, 2022. DOI:10.18653/v1/2022.findings- emnlp.398

[21] Mikolov T., Chen K., Corrado G., and Dean J., “Efficient Estimation of Word Representations in Vector Space,” arXiv Preprint, vol. arXiv:1301.3781v3, pp. 1-12, 2013. https://arxiv.org/abs/1301.3781v3

[22] MIT-LCP, MIT-LCP/Mimic-Code: Mimic Code Repository: Code Shared by the Research Community for the Mimic Family of Databases, https://github.com/MIT-LCP/mimic-code, Last Visited, 2024.

[23] Mullenbach J., Wiegreffe S., Duke J., Sun J., and Eisenstein J., “Explainable Prediction of Medical Codes from Clinical Text,” in Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, pp. 1101-1111, 2018. DOI:10.18653/v1/N18-1100

[24] Pang K., Li L., Ouyang W., Liu X., and Tang Y., “Establishment of ICU Mortality Risk Prediction Models with Machine Learning Algorithm Using MIMIC-IV Database,” Diagnostics, vol. 12, pp. 1068, 2022. https://doi.org/10.3390/diagnostics12051068

[25] Parveen H., Rizvi S., and Shukla P., “Disease Risk Level Prediction Based on Knowledge Driven Optimized Deep Ensemble Framework,” Biomedical Signal Processing and Control, vol. 79, pp. 103991, 2023. https://doi.org/10.1016/j.bspc.2022.103991

[26] Pirracchio R., Petersen M., Carone M., Rigon M., Chevret S., and Van der Laan M., “Mortality Prediction in Intensive Care Units with the Super ICU Learner Algorithm (SICULA): A Population- Based Study,” The Lancet Respiratory Medicine, vol. 3, no. 1, pp. 42-52, 2015. DOI: 10.1016/S2213-2600(14)70239-5

[27] Purushotham S., Meng C., Che Z., and Liu Y., “Benchmarking Deep Learning Models on Large Healthcare Datasets,” Journal of Biomedical Informatics, vol. 83, pp. 112-134, 2018. https://doi.org/10.1016/j.jbi.2018.04.007

[28] Shwartz-Ziv R. and Armon A., “Tabular Data: Deep Learning is Not All You Need,” Information Fusion, vol. 81, pp. 84-90, 2021. https://doi.org/10.1016/j.inffus.2021.11.011

[29] Wang H., Wang C., Xu J., Yuan J., Liu G., and Zhang G., “Invasive Mechanical Ventilation Probability Estimation Using Machine Learning Methods Based on Non-Invasive Parameters,” Biomedical Signal Processing and Control, vol. 79, pp. 104193, 2023. https://doi.org/10.1016/j.bspc.2022.104193

[30] Wang L., Wang H., Song Y., and Wang Q., “MCPL-based FT-LSTM: Medical Representation Learning-based Clinical Prediction Model for Time Series Events,” IEEE Access, vol. 7, pp. 70253-70264, 2019. DOI:10.1109/ACCESS.2019.2919683

[31] Wang L., Zhang Z., and Hu T., “Effectiveness of LODS, OASIS, and SAPS II to Predict in-Hospital Mortality for Intensive Care Patients with ST Elevation Myocardial Infarction,” Scientific Reports, vol. 11, no. 1, pp. 1-10, 2021. https://doi.org/10.1038/s41598-021-03397-3

[32] Wang S., McDermott M., Chauhan G., Ghassemi M., Hughes M., and Naumann T., “MIMIC- Extract: A Data Extraction, Preprocessing, and Representation Pipeline for MIMIC-III,” in Proceedings of ACM Conference on Health, Inference, and Learning, Toronto, pp. 222-235, 2019. http://dx.doi.org/10.1145/3368555.3384469

[33] Wu Y., Jiang M., Xu J., Zhi D., and Xu H., “Clinical Named Entity Recognition Using Deep Learning Models,” AMIA Annual Symposium Proceedings, vol. 2017, pp. 1812-1819, 2018. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5 977567/

[34] Xie F., Zhou J., Lee J., Tan M., Li S., Rajnthern L., Chee M., Chakraborty B., Wong A., and Dagan A., “Benchmarking Emergency Department Prediction Models with Machine Learning and 800 The International Arab Journal of Information Technology, Vol. 22, No. 4, July 2025 Public Electronic Health Records,” Scientific Data, vol. 9, no. 658, pp. 1-12, 2022. https://doi.org/10.1038/s41597-022-01782-9

[35] Yin Y. and Chou C., “A Novel Switching State- Space Model for Post-ICU Mortality Prediction and Survival Analysis,” IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 9, pp. 3587-3595, 2021. DOI:10.1109/JBHI.2021.3068357

[36] Zangmo K. and Khwannimit B., “Validating the APACHE IV Score in Predicting Length of Stay in the Intensive Care Unit Among Patients with Sepsis,” Scientific Reports. vol. 13, no. 1, pp. 1-9, 2023. https://doi.org/10.1038/s41598-023-33173- 4