Researchers at Southern Illinois University Carbondale have tested a router that chooses whether an AI tool call should run on a small local model or a large cloud model. Across two function-calling benchmarks and four model families, the system reduced estimated operational carbon emissions fourfold on average while keeping success rates close to a cloud-only setup. Function calling is the step in which a language model selects and invokes software tools such as search, calculation or mapping services.
Most AI agents send every request to a powerful cloud model because larger models are generally more reliable. That default treats a one-step lookup like a multi-step geospatial task, even though the two calls require different levels of reasoning. Small models on nearby edge devices use less power but fail more often on difficult calls. The researchers therefore made the routing decision before execution: use the lowest-emission machine likely to complete the call, and reserve the cloud for requests that local models are unlikely to handle.
The test system had three computing tiers. Models with 1 billion to 4 billion parameters ran on an Nvidia Jetson Orin Nano, models with 7 billion to 12 billion parameters ran on a Jetson AGX Orin, and the cloud tier used GPT-OSS-120B on Nvidia H100 GPUs. For each request, a k-nearest-neighbour predictor compared its language and structure with similar calibration examples. It estimated success probability, delay and power for the two edge tiers, then combined those estimates with current electricity-grid carbon intensity. The router selected the lowest-emission tier above a 90% predicted-success threshold and used the cloud when neither edge tier qualified.
The researchers evaluated the router on BFCL V2, which covers general function calls, and GeoEngine, which requires a sequence of dependent geospatial calls. They used 20% of each benchmark for calibration and the remaining 80% for evaluation. Edge models came from the Qwen3, Gemma3, Llama3.1 and Falcon3 families. The comparisons included cloud-only and edge-only execution as well as three existing routing approaches, with success rate, estimated carbon emissions and delay measured for each configuration.
On BFCL V2, the carbon-aware router achieved success rates of 89.1% to 95.7%, compared with 96.7% for cloud-only execution. Its reported emissions were six times lower on average and up to eight times lower in the Falcon3 configuration. With Qwen3 and Gemma3, almost 70% of requests went to the smallest edge tier and only 4.3% reached the cloud. GeoEngine left less room to shift work: success ranged from 55.3% to 59.6%, against 60.9% for cloud-only, while the average emissions reduction was 1.39-fold.
These results come from benchmark workloads rather than a long-running production service. The carbon estimates also depend on the selected hardware, the calibration examples and grid-intensity data for each location. A router that misjudges a difficult request may need another attempt, eroding the savings. The next test is whether the same policy maintains its success rate as user requests, model versions and electricity supplies change over time.
Related coverage
Sources
- Aikaterini Maria Panteleaki and colleagues, “Carbon-Aware Routing for Function Calling in Edge-Cloud LLM Systems,” arXiv:2609.13559, September 11, 2026. Architecture, evaluation design and headline results.
- Full HTML paper. Hardware tiers, routing sequence, benchmark conditions, comparison methods and reported limitations.
- Lead image: “Wikimedia Foundation Servers-8055 09” by Victor Grigas, reused unmodified under CC BY-SA 3.0. The photograph is illustrative and does not show the study's hardware.
