Everyday Data Science
Latest
Agentic workflows now power a third of surveyed enterprise automationAfrica's AI startup ecosystem posts record funding yearNew benchmark results reshape the coding-agent leaderboardNigeria launches national AI strategy with major investment planRwanda's sovereign AI cloud enters public betaThe future of AI agents: from tools to teammates

From Notebook to Production

The engineering path from a model that works in a notebook to a system that runs reliably in production: deployment, monitoring, cost control, model routing, and the decisions that keep AI alive after the demo.

The lessons

  1. 1

    Part 1 · 7 min

    Your Model Works in a Notebook. That Is the Easy Part.

    A notebook proves a model can work once. Production requires it to work continuously, on unseen data, within a budget, and to fail safely. The five questions that define the gap, and why deployment is the scarce skill.

  2. 2

    Part 2 · 7 min

    Serving a Model: Endpoint, Batch, or Embedded?

    Three ways to serve a model, endpoint, batch, or embedded, and the one question that decides which. Most people default to the always-on endpoint. It is the priciest and most fragile option, and usually the wrong one.

  3. 3

    Part 3 · 11 min

    98% Accurate and Useless: Six Imbalance Fixes, Measured on a 2% Positive Class

    Six models on one 2% positive dataset: the baseline caught 1 of 59 positives, class weights caught 45 at 6% precision, and a balanced random forest predicted zero positives despite the best AUC. Threshold tuning was the only fix that raised F1.

  4. 4

    Part 4 · 8 min

    79% Accurate, Give or Take 8 Points: What a Bootstrap Interval Says About Your Test Score

    One logistic model scored on test sets of 100, 400, and 1,600 rows. The 95% bootstrap interval shrank from 17 points wide to 4 as the test set grew, and 20 fresh test sets of 400 rows landed anywhere from 73.8% to 82.3%. Report the interval with the score.

  5. 5

    Part 5 · 9 min

    Drift Is Three Different Failures, and One Metric Cannot See Them All

    One logistic model, four test scenarios: a mean shift that broke accuracy but not AUC, added noise that hurt both, and a flipped rule that PSI could not see at all. The table, the code, and the monitoring checklist.

  6. 6

    Part 6 · 10 min

    Scaling Gave SVM 8 Points and Trees Nothing. I Ran Four Models Raw and Scaled.

    SVM went from 88.8% to 97.2% accuracy with one preprocessing line. Random forest moved zero. Logistic regression stopped warning and converged in 20 iterations instead of 1,000.

  7. 7

    Part 7 · 10 min

    Your 0.85 Meant 0.98: A 20,000-Row Calibration Test That Caught a Top Model Misstating Its Own Odds

    On 20,000 rows, a random forest with AUC 0.972 had a calibration error of 0.082. Sigmoid calibration cut it to 0.004. Gaussian naive Bayes fell from 0.050 to 0.012 with isotonic calibration, and AUC moved by at most 0.001.

  8. 8

    Part 8 · 11 min

    Five Leaks That Inflate Your AUC: I Measured Each One

    Five controlled experiments, one per leak: honest AUC vs leaky AUC on the same data-generating process. Target leakage bought +0.16, pre-split feature selection +0.26, repeat groups +0.15.