A new system uses AI agents to tune GPU kernels produced by PyTorch's compiler, then keeps a change only if the full model still returns correct results and runs faster. Across 250 KernelBench problems, the system improved 94 and delivered a 1.23× geometric-mean speedup over PyTorch's torch.compile baseline.
Red Hat researchers call the system KernelOPT. Instead of treating a compiled model as one opaque block, it profiles the model's generated Triton kernels and leaves calls to vendor libraries such as cuBLAS and cuDNN intact. Its five-agent workflow uses NVIDIA's Nsight Compute profiler to identify compute, memory, or occupancy bottlenecks, then proposes and measures kernel changes.
A GPU kernel is a small program that performs part of a model's computation. Improving one kernel does not guarantee a faster model: connecting it to the rest of the program can add overhead or change numerical results. KernelOPT therefore checks candidate code in stages, including static validation, output comparisons across multiple seeds, a stricter full-model correctness test with a float64 reference when needed, and an end-to-end speed check. If a change fails, the compiler's original version remains in place.
The paper reports a 1.23× geometric-mean speedup across all 250 KernelBench problems, not only the 94 that improved. Another 58 correct kernels were within measurement noise; 13 attempts produced no valid Triton code and kept the compiler baseline, while 85 candidates were rejected by the verification cascade. The paper attributes 61 of those 85 fallbacks to cuBLAS or cuDNN calls that already ran near peak throughput on the H200. In the separate Level 3 set of 50 full-model architectures, 12 improved, with a 1.07× geometric-mean speedup.
The tests used NVIDIA H200 hardware and Claude Sonnet 4.6, so the reported gains have not been shown to transfer to other chips or production workloads. The paper reports about 1,100 H200 GPU-hours for the benchmark. The paper lists a GitHub repository for the implementation, but that link returned a 404 when checked for this report, limiting independent reproduction.