Tag
#train-test-split
2 articles
AnalysisModel Evaluation · Analysis
Same Model, Ten Seeds, a 3.5-Point Swing: Why One Split Is Not a Result
Same data, same code, ten split seeds. Logistic regression ranged from 95.8% to 99.3%, and the "winning" model flipped on two seeds. A single split is not a result.
Ibrahim Denis Fofanah·Nov 2, 2026·8 min
Deep DiveFrom Notebook to Production · Deep Dive
79% Accurate, Give or Take 8 Points: What a Bootstrap Interval Says About Your Test Score
One logistic model scored on test sets of 100, 400, and 1,600 rows. The 95% bootstrap interval shrank from 17 points wide to 4 as the test set grew, and 20 fresh test sets of 400 rows landed anywhere from 73.8% to 82.3%. Report the interval with the score.
Ibrahim Denis Fofanah·Oct 29, 2026·8 min