StepGate completed 69% of 100 scripted tasks when it sent about 30% of a small AI model’s tool actions to a stronger cloud model, compared with 48% for the local-only setup. The result suggests that routing decisions made one action at a time may help with multi-step tasks, though the experiment was limited to a scripted environment.
Running an AI agent locally can keep routine work off remote servers, but a small model may choose the wrong tool or produce arguments that do not match what a tool expects. A router that decides once for an entire user request can miss a change in difficulty from one step to the next.
StepGate checks each proposed action using how confident the model is in its output; whether the call and its arguments match what the tool expects; whether separate proposals agree; the task’s current state; and the action’s assigned risk level. It can let a straightforward action run locally, allow the local model one check or retry, or escalate an uncertain action to a stronger model. The handoff carries a reduced context packet, but the paper does not establish a privacy guarantee.
The authors tested Qwen2.5-1.5B as the local model and Qwen2.5-7B as the stronger backend across 100 scripted calendar, email, file and search tasks. Each task took an average of 4.1 agent steps, with a maximum of eight. At roughly 30% of tool actions sent to the cloud, StepGate completed 69% of tasks. Local-only completed 48%, while a router that chose per user query completed 57%. A strong-model-only setup reached 82% while routing every action to the cloud.
The authors’ paired analysis found a statistically significant improvement over query-level routing, but could not draw a firm conclusion about StepGate’s nine-point measured difference from random escalation (60%). The cloud-action figure measures routing frequency; the study does not report a direct cost or latency saving. It also uses one model family and a small scripted task set, and its assigned risk tiers are not formal safety guarantees.
Sources
- StepGate paper and experimental results
- Authors’ software and reproducibility materials
- Illustration: “Cloud Computing Architecture” by Hiren Tat, used unmodified under CC BY-SA 4.0.