A different AI model sometimes flagged planted mistakes that a separate review by the model that generated the work missed, according to an Oct. 1 preprint. The 30-item study does not show that changing models alone makes AI reviews more reliable on real-world work.
Tae-Eun Song tested 30 Korean-language items: 10 code modules, 10 technical tutorials and 10 presentation scripts, each seeded with five errors. The paper's run logs identify Claude Opus 4.6 as the generator, though the item files do not record the model. Same-model reviewers inspected items in fresh sessions without the original generation conversation; these reviews were compared with GPT-5.4 and Gemini 2.5 Pro and Flash. Ten conditions produced 900 sessions—three repeated reviews of each item per condition. Because the same items were reviewed repeatedly, statistical comparisons averaged available runs for each item and used 30 paired item-level observations. An automated matcher linked reported problems to planted errors by location and wording.
In the paper's single-review condition results, GPT-5.4 scored 32.3% on F1, compared with 28.6% for same-model fresh-session review. F1 balances precision—the share of flagged problems that matched planted errors—with recall—the share of planted errors found. The comparison is across review conditions, not a single session; repeated runs were averaged within each of the 30 items. The difference was not statistically significant after correction for multiple comparisons, which does not establish that the methods perform equally.
In the first of three runs, a two-call mix of same-model fresh-session review and an information-restricted GPT-5.4 review (which saw the item but not the original requirements) collectively caught 85 of the 150 planted errors; either reviewer could supply a counted error. Two fresh-session reviews collectively caught 64 of 150 in that run. The difference was significant after adjustment under the paper's matcher, but comes from one run on the reused item set and did not significantly outperform two information-restricted GPT-5.4 reviews (76/145 vs. 82/145 across 29 items without a failed call).
Model identity and reviewer capability remain entangled: each capability tier contained one model, and the paper assigned tiers using vendors' positioning rather than a capability test in this study. The analysis excluded 14 failed cross-model calls and one same-model run whose provenance was uncertain; requested model names were not confirmed served checkpoints. The planted errors were not independently annotated, and the matcher was not checked by hand; the same 30 items were all in Korean. The preprint's data and scripts are available from the author on request. The findings do not establish performance on naturally occurring flaws in working code or documents.