Mistral opened an API preview of its Large 4 model on Oct. 6. In an independent software-security test, the model scored 81.7% on its first attempt at each task, the highest result on that test.
Artificial Analysis tested whether an AI agent could find memory-safety flaws in C and C++ projects, reproduce each flaw with an input that crashes the unpatched build, and change the code so the crash stops while the project's existing tests still pass. The evaluator's 131 tasks cover one issue per open-source project, selected from a larger set of vulnerabilities.
Large 4 scored 81.7% on this single-attempt measure, ahead of MiMo-V2.6-Pro at 78.6% and GPT-6 Luna (Max) at 77.9%. Artificial Analysis ran each task in an isolated software-testing environment, allowing up to 90 minutes. Its filtered set and testing setup are not directly comparable with scores from the original CyberGym-E2E benchmark.
The score has a narrower meaning than “vulnerabilities fixed.” A task counted as solved if the test input crashed the unpatched build, stopped crashing after the change, and the project's regular tests still passed. Artificial Analysis says it recorded whether the change also fixed the underlying flaw, but did not count that check in the score. Its public leaderboard does not report the results of this separate check.
Mistral says it will release the model weights and more technical details later this month. Until then, developers can try the preview through its API, but cannot download the weights to run the model on their own systems.
Sources
- Mistral’s Large 4 announcement
- Artificial Analysis’s CyberGym-E2E-AA methods and results
- Illustrative server-room photo by Data Servers on Unsplash, used under the Unsplash License; not Mistral hardware.