Suitability of low and middle-income country data-derived prognostics models for benchmarking mortality in a multinational Asia critical care registry network: a multicentre study
Gamage Dona DP., Aryal D., Ghose A., Hashmi M., Nath RK., Mat Nor MB., Thwaites L., Tripathy S., Vijayaraghavan BKT., Yen LM., Dondorp A., Haniffa R., Nirantharakumar K., Karwath A., Snell K., Parekh D., Beane A., Collaboration for Research, Implementation and Training in Critical Care; Asia Africa ‘CCAA’ None.
Background: This study evaluates the predictive performance of prognostic models derived from low- and middle-income country (LMIC) data using a multinational Asian critical care dataset. The research also seeks to identify opportunities for improving these models' accuracy and utility in clinical research and for international benchmarking of critical care outcomes Methods This retrospective multicenter study evaluated the performance of four prognostic models: e-Tropical Intensive Care Score (e-TropICS), Tropical Intensive Care Score (TropICS), Simplified Mortality Score for the Intensive Care Unit (SMS-ICU), and Rwanda Mortality Probability Model (R-MPM) using a dataset of 64,327 ICU admissions from 109 ICUs across six Asian countries. The models' discriminative abilities were assessed using ROC curves, and calibration was evaluated with Hosmer-Lemeshow C-statistics and calibration curves. Recalibration was performed to improve model accuracy, and the impact of the COVID-19 pandemic on model performance was also analysed. Results The e-TropICS and R-MPM models showed relatively good discriminative power, with AUCs of 0.71 and 0.69, respectively. However, all models exhibited significant calibration issues, particularly at higher predicted probabilities, even after recalibration. The study also revealed variability in model performance across different countries, with India's data demonstrating the highest discriminative power. Conclusions The study highlights the challenges of applying existing prognostic models in diverse ICU settings, particularly in LMICs. While the e-TropICS and R-MPM models performed relatively well, significant calibration issues indicate a need for further refinement. Future efforts should focus on developing adaptable models that can effectively accommodate the diverse and dynamic nature of ICU populations worldwide, ensuring their utility in global healthcare benchmarking and decision-making.