Leakage-Free, Explainable and Fairness-Aware Early Prediction of At-Risk Students: A Temporal Validation Study on the Open University Learning Analytics Dataset

Abstract

Early identification of students at risk of failing or withdrawing enables timely intervention, yet many published predictors are evaluated with random splits, test-set tuning and accuracy-centred metrics that overstate deployable performance. This study proposes a leakage-free, explainable and fairness-aware early-warning framework evaluated on the Open University Learning Analytics Dataset (32,593 enrolments). Models were trained on three earlier course presentations and tested on an unseen later presentation, using only information available on each prediction day. Eight classifiers were tuned with student-grouped cross-validation. At day 60, LightGBM achieved a ROC-AUC of 0.813 and a PR-AUC of 0.761 and identified 75.3% of at-risk students; ROC-AUC rose from 0.72 at course start to 0.90 at day 150. Assessment behaviour accounted for 47% of SHAP attributions. A fairness audit revealed disparities across deprivation and prior-education groups. A replication on the UCI Student Performance data confirmed that early academic signals outweigh demographic predictors.