Optimized Stacking Ensemble of Random Forest and XGBoost for Heart Disease Prediction

Abstract

Heart disease remains a leading cause of death worldwide, highlighting the need for reliable tools to support early diagnosis and clinical decision-making. Although machine learning methods have been widely applied to heart disease prediction, many existing studies rely on default model settings, limited evaluation metrics, and a narrow range of algorithms. This study develops an optimized stacking ensemble model that combines Random Forest and XGBoost, with hyperparameters tuned using Optuna. The models were evaluated using a publicly available clinical dataset comprising 918 patient records with demographic and cardiovascular attributes. An 80:20 train-test split was applied, and performance was assessed using multiple metrics, including accuracy, macro and weighted F1-score, ROC-AUC, PR-AUC, and balanced accuracy. The results show that the optimized Random Forest achieved an accuracy of 89.1% and a ROC-AUC of 93.4%, while XGBoost achieved an accuracy of 88.6% with a similar ROC-AUC. The stacking ensemble provided the best overall performance, with an accuracy of 89.7%, macro F1-score of 89.6%, ROC-AUC of 93.6%, and PR-AUC of 93.5%. These findings suggest that combining optimized ensemble models can improve predictive performance compared to individual classifiers. However, the results are based on a secondary dataset and have not been validated in real clinical settings, which may limit their direct applicability in practice. Overall, this study contributes by demonstrating the value of hyperparameter optimization and model stacking, while adopting a broader evaluation framework for heart disease prediction.