AI21 says a specialized version of NVIDIA’s Nemotron 3.5 Super VL reached 72% accuracy on Deel’s eight-country payroll benchmark, eight percentage points above Claude Opus 5.5. It also reports per-query cost at one-thirty-third of an unspecified baseline, without showing the calculation. The post does not disclose how many questions were tested.
Payroll questions can depend on local tax rules, employment contracts and company policies. AI21 says Deel wanted answers that covered those details rather than leaving out part of a response. The company worked with Emet by Deel, which AI21 describes as an evaluation lab that supplied expert-verified training data and benchmark questions across eight countries.
AI21 says initial tests indicated that the base Nemotron model already had most of the needed payroll knowledge. The gap was in the answers: they did not consistently meet Deel’s expected level of detail and style.
Training focused on how the model answered
To address that gap, AI21 says it used supervised fine-tuning on a selected set of model-generated responses. It then applied self-distillation policy optimization, a second training step in which a teacher model received detailed instructions about the desired answer behavior. The company presents the process as a way to adapt an existing model for a specialized task.
AI21’s post does not provide the benchmark’s size, scoring protocol, prompt examples, confidence intervals or cost formula. Without those details, readers cannot reproduce the accuracy comparison or independently assess the reported cost.
The report describes a benchmark, not a live rollout
AI21 says the Deel work was one of three post-training collaborations completed over three weeks, alongside separate work with Gong and Alice. NVIDIA’s NeMo documentation describes Nemotron 3.5 Super VL as a text-and-image model, but it does not document this payroll evaluation. AI21’s post reports training and benchmark results; it does not say the Deel-specific model has been deployed in a live payroll product.
Sources
AI21’s account of its post-training work with Deel; NVIDIA’s Nemotron model documentation.
Photo: Derrick Coetzee, “Rear of rack at NERSC data center - closeup,” CC0, via Wikimedia Commons; unmodified. Photo and rights record; CC0 terms.
