An under-review paper from New York University, Salesforce AI Research and Stony Brook University describes an 8B-parameter model trained to propose three research ideas from the same source papers. On the authors’ test, its version trained to optimize the final idea cards without an intermediate written reasoning trace scored higher than Claude Opus 4.8 and GPT-6 Astra on a combined quality-and-diversity measure; AI models judge both parts of that score.
The training reward favors a useful spread of ideas
Instead of asking for one proposal at a time, the system generates three structured idea cards in one response, grounded in summaries of earlier papers. Each card names a research gap, a central claim, a method and the source documents that inspired it.
The researchers first fine-tuned Qwen3-8B to produce the card format, then used group-relative policy optimization, a reinforcement-learning method that updates the model based on how sampled outputs score against one another. Its set-level reward scores each idea for quality and weights that score by how different it is from its most similar earlier card. A promising idea therefore earns less if it repeats a direction already in the set.
A diversity reward raised the model’s AI-judged score
On a test set of 1,740 target papers from 2026 spanning 16 domains, the trained model’s “no-think” version reached a creative-utility score of 0.64. The paper defines that version as optimizing the final idea cards without an intermediate reasoning trace. It reports 0.38 for Claude Opus 4.8 and 0.28 for GPT-6 Astra on the same test. The authors describe creative utility as a combined measure of idea quality and difference among ideas, normalized from zero to one.
An ablation helps show what the diversity reward contributed. When the researchers kept set-level training but removed its diversity component, the model scored 0.38 on creative utility and 0.38 on average idea distance, compared with 0.64 and 0.88 for the full no-think version. This points to the diversity term as a driver of the higher score on this test, although both parts of the metric rely on AI judges: GPT-5.4-mini judges idea quality against a reference proposal, and Gemini 3.5 Flash assesses semantic distance. The score is not a direct count of original discoveries.
Four experts found similar promise counts and more distinct ideas
Four NLP experts each reviewed 15 test instances, for 60 total. For each set of three ideas, they marked which proposals looked worth pursuing and which followed distinct directions. The trained model averaged 2.17 promising and 2.83 distinct ideas per set; Claude Opus 4.8 averaged 2.20 and 2.72. The counts were close for promising ideas, while the trained model’s sets contained slightly more ideas the reviewers considered distinct.
More of the model’s idea sets got positive verdicts in a small coding-agent test
The paper also tested 132 idea cards from 15 instances in a fixed sandbox. A coding agent tried a proof-of-concept experiment for each claim, with a 90-minute and $30 limit and no GPU. “Supports” meant the agent reported a positive result that covered most or all of the claim. At least one of the model’s three ideas got that verdict in 33% of sets, versus 27% for Claude Opus 4.8. This was an agent-run check, not evidence that the ideas produced successful research experiments.
The preprint leaves broader validation open
The paper is marked “Under Review.” Its supervised training examples focus on language research, while the 16-domain test set reaches beyond that field. The expert pilot is small, and the headline benchmark depends on AI judges. The authors’ companion repository says code, data and models are coming soon, so readers cannot yet reproduce the result from those artifacts.
The study tests a way to reward a model for offering a varied shortlist rather than repeating its first plausible idea. Whether that shortlist helps researchers make better discoveries will require larger independent evaluations and evidence from real research work.
Sources
- The authors’ arXiv preprint, submitted October 15, 2026
- Full paper PDF and evaluation tables
- The paper’s companion code and data repository
Image: “Researchers Reading Documentation” by Artem Podrez, via Pexels. Pexels permits use on blogs without required attribution; the image is illustrative and does not depict the paper’s authors.