Frontier Models Can Spot a Broken Experiment. They Still Pick It One Time in Five

Why judging an experiment is the skill to test

There is a lot of talk about AI agents that can run research. A less glamorous skill comes first: can a system tell a good experiment from a subtly broken one before anyone spends compute on it? That question matters for alignment research in particular, where the work is hard to supervise and a bad experiment can look productive for weeks.

Tampered ML, by Soroush Ebadian, Parand A. Alamdari, Joe Benton, and Hailey Joren, submitted 23 October 2026 and accepted at EMNLP 2026, tests exactly that skill. It does not ask a model to write a paper, run code, or rate a proposal on a vague scale. It asks a narrower question with a clean answer: of these two near-identical designs, which one supports its claim?

The construction is what makes the benchmark worth taking seriously. The authors started from 981 oral and spotlight papers at NeurIPS, ICML, and ICLR from 2024 to 2026. A language model pipeline extracted one experiment per item, rewrote it as a self-contained proposal, then planted a single flaw with a minimal edit, changing a median of only 2.0% of the text. Automated gates checked that the original stayed faithful to the paper, that the edit really changed the experiment, and that no giveaway cue leaked the answer. A difficulty filter removed items a panel of models already solved, leaving 1,858 item pairs.

Each model sees each pair twice, once in each order, and only scores if it picks the original both times. That rule is the whole point. A single-order score lets position bias hide. A both-orders score exposes it.

Finding 1: The best model is right about four times in five, and that is the ceiling

Claude Opus 5 reached 77.9% both-orders accuracy. Claude Fable 5 reached 79.6% on the items it answered, but 73.5% once its 7.8% refusal rate is counted as errors, which is the honest way to count it for a reviewer that must answer. Per-order accuracy for the leaders ran 86% to 88%, which shows how much the both-orders rule costs: a model can look strong once and still fail the same question presented the other way round.

For context, random guessing scores 25% under the both-orders rule and 50% per order. The leaders are far above chance. They are also far from reliable. A reviewer that is wrong one time in five on a single-flaw comparison, where the two proposals differ by a median of 2.0% of their text, is not a reviewer you can leave alone with go or no-go decisions.

Finding 2: Order bias, not ignorance, explains many misses

The most useful result in the paper is not the leaderboard. It is the diagnosis of the misses.

When a strong model was right in only one order, its reasoning usually named the planted flaw, in over 86% of such cases for the stronger models. The model did not miss the flaw. It weighed it, then let something else win: the order of presentation, the extra specificity of the flawed version, or the fact that the flawed version looked like standard practice.

The order effect is large. Claude Opus 4.6 scored about 66% per order but only about 43% in both orders. Even when models were correct in both orders, several, especially the Opus models and GPT-5.5, were less confident and thought longer when the original came second. That is a signature of a judgment that is unstable, not settled.

The authors catalog seven failure modes, including treating common practice as proof of validity, reading added detail as rigor, echoing the flawed proposal's own justification, contradicting themselves across orders, and recalling the source paper incorrectly. Three of those modes should worry any practitioner: detail as rigor, convention as validity, and self-contradiction across orders are all failures that look like careful reasoning in a single output.

Finding 3: The flaw mix is realistic, and leakage is only a small part of it

The 1,858 items break down by flaw type as follows: confounded comparison 20.5%, blocked mechanism 20.3%, measurement mismatch 18.0%, unfair comparison 17.1%, and four smaller categories, including data leakage at 2.8%.

Flaw type	Share of items
Confounded comparison	20.5%
Blocked mechanism	20.3%
Measurement mismatch	18.0%
Unfair comparison	17.1%
Four smaller categories, including data leakage	24.1% combined

That mix matters because it stops the benchmark from being a leakage quiz. Most planted flaws are about comparison design: what is held constant, what mechanism is actually isolated, what is measured, and whether the comparison is fair. Those are the flaws that survive peer review in real life, because they hide inside a plausible design rather than in an obvious data error.

Finding 4: Rating one proposal at a time is much worse

When models rated proposals one at a time on a 0 to 100 scale, instead of choosing between a pair, accuracy fell 4 to 19 points. The best pointwise score was Opus 5 at 68.8%, followed by Gemini 3.1 Pro at 67.6%.

This is the result with the most direct practical consequence. Most real AI review workflows are pointwise: paste one proposal, ask for a score or a verdict. Tampered ML says that workflow is the weakest version of the task. Pairwise comparison gives the model a contrast to work with. A lone rating gives it room to reward polish.

If your review pipeline only ever shows the model one design at a time, the 77.9% headline does not apply to you. The 68.8% figure is closer, and it is still measured on the easier paired-construction items, not on lone proposals in the wild.

Finding 5: Hints recover errors. More reasoning helps, but with diminishing returns

On a hard subset of 896 items, telling Opus 4.7 and Gemini 3.1 Pro that one design was flawed, and especially naming the flaw category, recovered many errors. More reasoning effort raised accuracy within each model, with diminishing returns. Opus 5 at its lowest effort setting already matched or beat most other models at their best settings.

Read that as two separate lessons. First, the prompt matters more than it should: a reviewer that needs to be told a flaw exists is a second reader, not an independent check. Second, buying more reasoning tokens is not a fix for a judgment problem. It helps at the margin, then flattens, while the order and polish biases remain.

What this means for data and AI practitioners

If you use a model anywhere near experiment review, grant review, or internal design review, build the benchmark's lessons into the workflow:

1. Run both orders, always. Show the model design A then B, then B then A, and treat any disagreement as a warning, not a tie to break by averaging. Disagreement is the signal.
2. Ask for a defect, not a winner. Prompt the model to find the flaw, name its category, and state what claim it breaks, before it picks a winner. In the study, naming the flaw category recovered the most errors.
3. Distrust added detail. When one version is longer, more specific, or more conventional, check whether the extra text adds a control, a measurement, or just confidence. The study shows models reward detail as if it were rigor.
4. Keep a human on go or no-go calls. A 77.9% paired score and a 68.8% pointwise score are both too low to automate a decision that costs real compute or real credibility.
5. Score your own reviewers this way. Take a past experiment, plant one flaw by hand, and run your review prompt in both orders. If it cannot pass that test on your own work, it will not pass it on anyone else's.

Against

The benchmark is model-built, and about one item in seven is estimated not to be clearly valid. Human validation covered 210 items, deliberately focused on ones strong models missed, with a single validator per item, and only 128 of the reviewed items are in the 1,858-item leaderboard set. The post-stratified estimate is 86% valid, with a 95% confidence interval of 81% to 91%. That is honest reporting, and it still leaves a real error bar around every accuracy figure.

The construction pipeline code is not released, which limits exact replication. The rest of the data beyond the public subset is available to researchers on request, not openly.

In favour

The paired, minimal-edit design gives the benchmark something most review benchmarks lack: a clean label. Comparing two independent proposals rarely has a right answer. Here, the tampered design only needs to be worse along one dimension, and the median 2.0% text change keeps the comparison focused.

The both-orders rule is a genuine methodological contribution to evaluation practice, not just a scoring quirk. It turns position bias from a footnote into a measured quantity, and the paper reports it plainly, including the embarrassing Opus 4.6 drop from about 66% per order to about 43% in both orders.

The authors also report the limitations that cut against their own headline, including that their accuracies are optimistic estimates and that judging a flaw is not the same as avoiding it when writing an experiment. A benchmark paper that states its own ceiling clearly is easier to trust than one that does not.

What would make me wrong

Three findings would change my reading of this paper. First, if independent replication on the public subset, by a team with no Anthropic affiliation, reproduced the same ordering and similar both-orders scores, the authorship concern would fade. Second, if a reviewer pipeline that combined both-orders scoring with flaw-category prompting reached the mid-90s on lone, unplanted proposals, not just on paired items, the case for automated review would strengthen considerably. Third, if the 14% of items estimated invalid turned out to concentrate in the items the leaders missed, the true both-orders scores could be several points higher than reported, and the order-bias story would need remeasuring on a cleaned set.

Until one of those happens, the prudent reading stands: frontier models can spot a broken experiment, and still cannot be trusted to pick the sound one every time.

Related on Everyday Data Science

• Switching Agent Harnesses Changed 13% of Tasks. Rerunning the Same One Changed 13% Too
• AI Research Agents Claimed 43% and 71%. Audited, They Got 2% and 15%.
• Your Benchmark Leaderboard Is Measuring the Test, Not the Model. Stanford Checked 56 of Them.
• 79% Accurate, Give or Take 8 Points: What a Bootstrap Interval Says About Your Test Score

Have you ever watched a reviewer, human or model, name the exact flaw in a design and then approve it anyway? What tipped the verdict?

Sources: Tampered ML: Evaluating LLM Judgment of Machine Learning Research, Ebadian et al., arXiv:2610.32114; Tampered ML benchmark writeup, i10X News, 26 October 2026