Everyday Data Science: applied AI, agentic systems, machine learning, and AI in Africa, written by practitioners for people who build.

Agents Find the File and Miss the Cause, and the Gap Grows on Thin Evidence

On 120 real distributed-system bugs, the best agent finds the right file 83.9% of the time but names the causal code block only 67.7% of the time. The gap widens to a 29.8 point drop on reports that name no method and carry no stack trace.

Ibrahim Denis Fofanah·12 min

A True Tool Result Can Still Be Wrong: ReplayBench Found a 64.9% Error Rate

TRAP replayed real, unmodified tool results into the wrong task. Agents acted on them anyway: mean final-state error 0.649, and 0.989 across the full benchmark.

Ibrahim Denis Fofanah·11 min

Scaling Gave SVM 8 Points and Trees Nothing. I Ran Four Models Raw and Scaled.

SVM went from 88.8% to 97.2% accuracy with one preprocessing line. Random forest moved zero. Logistic regression stopped warning and converged in 20 iterations instead of 1,000.

Ibrahim Denis Fofanah·10 min

Same Model, Ten Seeds, a 3.5-Point Swing: Why One Split Is Not a Result

Same data, same code, ten split seeds. Logistic regression ranged from 95.8% to 99.3%, and the "winning" model flipped on two seeds. A single split is not a result.

Ibrahim Denis Fofanah·8 min

Your Email Agent Passed the Phishing Test. Then the Attack Adapted to It

Six email agents caught every old phishing email. Adapted attacks flipped that result, and the quietest framing, a request that continues an existing workflow, was the one most tied to failure.

Ibrahim Denis Fofanah·10 min

Your Agent Sees the Contradiction, Then Answers Confidently Anyway

Four agent systems scored on whether they flag uncertainty when evidence contradicts their memory. Accuracy and humility run in opposite directions, and a one-line prompt buys humility at the cost of accuracy.

Ibrahim Denis Fofanah·11 min