Global Tech News For people figuring out what to build with AI.
AI

An AI router averaged 75.6% task success across six benchmarks

Diagram of RECAST routing a question to search, database queries or Python code, then passing selected evidence to an answer model

An AI system that chooses among search, database queries and generated code averaged 75.6% task success across six benchmark families, 15.9 percentage points above the strongest tested baseline, according to an Oct. 7 preprint. Success means a frozen Gemini 3.5 Flash judge marked the final answer correct against the reference answer; researchers averaged three runs on 100 previously unseen questions per family.

RECAST is designed for questions where useful information is scattered across records and must be combined or calculated, rather than retrieved as one ready-made passage. Its 9-billion-parameter Qwen3.5-9B model routes each question to keyword or semantic search, or to read-only SQL, which can filter and combine database records. It checks each result and can choose another operation before passing accepted evidence to a separate answer model.

When a task needs a transformation its search and database tools cannot express, RECAST asks a separate code model to turn the router’s specification into Python; the system then runs that program over the source records. Only the router is trained; the code and answer models remain fixed. In the benchmark, Gemini 3.5 Flash generated code for RECAST and produced final answers for every system.

Across the six benchmark families, RECAST averaged 75.6% task success, compared with 59.7% for Gemini-based Interact-RAG, the strongest large-model baseline. The tests covered data tables, financial reports, Wikipedia passages and user-profile histories. On three held-out families excluded from training and checkpoint selection, RECAST scored 79.3%, against 64.3% for the strongest baseline. The authors also rescored one replicate with GPT-4.1 and DeepSeek-V4-Flash; RECAST remained ahead under both judges, although its absolute scores shifted.

These are benchmark results, not tests of a live database service. The authors warn that repeated model calls and occasional code generation can add latency and computing cost compared with one-pass retrieval. Generating the router’s training trajectories also adds overhead. They plan to release the code, data and trained router checkpoint upon acceptance.

Sources