Implementation of SHAP in Ensemble Learning for Interpreting Audit Risk Classification: A Preliminary Study Using a Public Audit Dataset

Abstract

Ensemble learning models can achieve strong predictive performance on structured audit-risk data, yet their complex decision logic can limit transparency and technical accountability. This study develops a proof-of-concept Explainable Artificial Intelligence pipeline using SHapley Additive exPlanations (SHAP) to interpret Random Forest and XGBoost classifiers on the public UCI Audit Data. The experiment processed 776 observations and 27 columns, with one missing value in Money_Value imputed using the median of 0.09 and the non-numeric LOCATION_ID removed, resulting in 25 predictors and one binary target. A stratified 80:20 split with random_state = 42 produced 620 training observations and 156 test observations, with Random Forest achieving 1.0000 across accuracy, precision, recall, F1-score, and ROC-AUC, while XGBoost achieved 0.9936 accuracy, 1.0000 precision, 0.9836 recall, 0.9917 F1-score, and 1.0000 ROC-AUC. SHAP analysis identified Audit_Risk as the dominant global predictor, while its contribution to a representative at-risk prediction reached +5.82, shifting the raw model margin from −0.506 to 5.107 and yielding an estimated probability of approximately 0.994. However, because the dataset's target construction is closely associated with the Audit Risk Score, retaining Audit_Risk as a predictor introduces potential structural target leakage. The findings position the near-perfect performance as evidence of successful pipeline implementation and model interpretability rather than external generalization, supporting future leakage-controlled ablation, cross-validation, and external-data validation.