In a new crystal structure benchmark, a template-based method matched more reference atomic arrangements than any of 12 generative models. The researchers also removed families of familiar patterns from one model's training data. Many test cases that it had previously matched then lost either structure or symmetry agreement.
Crystal structure prediction starts with a material's composition and tries to place its atoms in a repeating three-dimensional arrangement. A template method begins with a known arrangement and substitutes different elements. A generative model instead learns patterns from a structure database and produces a candidate arrangement. The proposed advantage of generation is that it might reach structures absent from the template library.
Researchers at the University of South Carolina compared the 12 models with TCSP 2.0, a template-substitution method, on 180 test structures. Each method submitted one candidate per composition, and the team applied the same structure-matching and symmetry checks. TCSP 2.0 matched the reference structure in 68.3% of cases; the strongest generative model, EquiCSP, reached 66.4%. On a 46-structure subset designed to remove overlap with one training split, the corresponding rates were 56.5% and 51.7%. That subset cannot rule out overlap with every model's training data.
The similar full-set scores conceal where the successes occurred. For each generative model, most structures it predicted correctly were also recovered by template substitution. To test whether learned models needed familiar patterns, the researchers retrained EquiCSP after removing all training and validation structures in each of four composition-ratio families. Across those families, 50% to 78% of each family's test cases lost a previously correct structure or symmetry match. The intervention applies to EquiCSP and these four families, not to every model in the comparison.
Some predictions survived even when their entire composition-ratio family was removed from training. That suggests limited transfer to a withheld family. It does not show that the models can discover stable new materials: the benchmark checks geometric agreement with known structures, not thermodynamic stability.
Sources
- Crystal structure benchmark paper, posted September 22, 2026; see Methods, Results and Limitations. Benchmark data and code are linked by the authors.
- Lead image: table-salt crystal structure illustration by Eyal Bairey, released into the public domain. It illustrates a crystal lattice and is not one of the benchmark results.
