LightOn says its ColGrep-Agent code-search workflow cut Claude API token use by 34.3% versus grep in a small test. The company-reported result covers finding code in repositories; it does not show that the tool makes coding agents better or cheaper overall.
ColGrep-Agent is designed to handle repository exploration locally. It uses ColGrep, which searches code for concepts described in ordinary language and can also filter by path or regular expression. The agent reads promising files through a read-only terminal, then returns file paths, line ranges and code snippets to a larger model such as Claude Code. That larger model still does the subsequent reasoning and editing.
Four search setups, one small test
LightOn tested Claude Code with Sonnet 5.5 on 52 code-location questions from 13 public repositories created in 2026. Each question was run three times under four setups: built-in grep, direct ColGrep search, ColGrep-Agent, and Balanced, which let Claude choose between ColGrep and the agent. That made 624 sessions, but only 52 distinct questions.
Average Claude API input and output tokens per question were 142,000 with grep, 107,000 with ColGrep, 93,000 with ColGrep-Agent and 112,000 with Balanced. LightOn puts the agent’s reduction versus grep at 34.3% (95% confidence interval: 28% to 40%); Balanced cut tokens by 21.2%.
Correct-answer rates were 83.3% for grep, 79.5% for ColGrep, 80.8% for ColGrep-Agent and 83.3% for Balanced. For the agent versus grep, the estimated difference was −2.6 percentage points, with a wide 95% interval from −10.9 to +5.1 points. The interval still allows a meaningful accuracy drop.
The token savings came with longer searches: the agent averaged 21.6 seconds per question, compared with 13 seconds for grep; Balanced averaged 17.7 seconds. LightOn’s estimated cache-aware Claude API cost per question was 7.3 cents with grep, 6.0 cents with ColGrep, 4.4 cents with the agent and 5.1 cents with Balanced. Those figures exclude the local model’s tokens and local computing costs.
The comparison measures code search and localization, not patch writing, bug resolution or end-to-end coding quality. The related arXiv paper is a preprint about multilingual code localization; it reports a separate evaluation and does not validate this Claude Code comparison.
Sources
LightOn’s October 12 release and evaluation; the related arXiv preprint; ColGrep source repository.
Photo: Negative Space/Pexels, via Wikimedia Commons; CC0 1.0. Illustrative image only; it does not show ColGrep-Agent.