Tag
#scikit-learn
5 articles
98% Accurate and Useless: Six Imbalance Fixes, Measured on a 2% Positive Class
Six models on one 2% positive dataset: the baseline caught 1 of 59 positives, class weights caught 45 at 6% precision, and a balanced random forest predicted zero positives despite the best AUC. Threshold tuning was the only fix that raised F1.
Ibrahim Denis Fofanah·Nov 12, 2026·11 min
Scaling Gave SVM 8 Points and Trees Nothing. I Ran Four Models Raw and Scaled.
SVM went from 88.8% to 97.2% accuracy with one preprocessing line. Random forest moved zero. Logistic regression stopped warning and converged in 20 iterations instead of 1,000.
Ibrahim Denis Fofanah·Nov 7, 2026·10 min
Same Model, Ten Seeds, a 3.5-Point Swing: Why One Split Is Not a Result
Same data, same code, ten split seeds. Logistic regression ranged from 95.8% to 99.3%, and the "winning" model flipped on two seeds. A single split is not a result.
Ibrahim Denis Fofanah·Nov 2, 2026·8 min
Five Leaks That Inflate Your AUC: I Measured Each One
Five controlled experiments, one per leak: honest AUC vs leaky AUC on the same data-generating process. Target leakage bought +0.16, pre-split feature selection +0.26, repeat groups +0.15.
Ibrahim Denis Fofanah·Oct 19, 2026·11 min
Your 0.85 Meant 0.98: A 20,000-Row Calibration Test That Caught a Top Model Misstating Its Own Odds
On 20,000 rows, a random forest with AUC 0.972 had a calibration error of 0.082. Sigmoid calibration cut it to 0.004. Gaussian naive Bayes fell from 0.050 to 0.012 with isotonic calibration, and AUC moved by at most 0.001.
Ibrahim Denis Fofanah·Oct 14, 2026·10 min