Researchers at AstraZeneca and Swedish universities tested seven large language models (LLMs) against labels in PubChem, a public database of biological screening tests. They found that model evidence can help a curator spot inconsistent records, but the results do not show that models can safely correct a database on their own.
A bioassay measures how a substance affects cells, proteins or organisms. In this study, models inferred an assay’s format and detection method from its description, comments and protocol. Those labels help researchers understand what kind of test was run and how its result was measured.
The researchers counted 1,994,310 entries in PubChem’s bulk BioAssay download. Thirty-six percent had no native field identifying assay format, and fewer than 0.1% had a BioAssay Ontology (BAO) term for assay format or detection technology in downloaded comments. These are gaps in structured metadata; the study does not show that the underlying protocols lack the information.
The team tested four open-weight and three proprietary models. On 1,097 records with ChEMBL assay-format labels, a majority vote across the models agreed with ChEMBL 96% of the time. That is agreement, not independently verified accuracy: the paper calls these labels “silver” because databases can also be incomplete or inconsistent.
The disagreements exposed that problem. In 17 of 43 cases where the model majority differed from ChEMBL, it matched a separate BARD-curated label instead. A senior curator reviewed 17 cases, accepted six model classifications immediately and six more after seeing the models’ cited evidence. In another case, models cited a relevant fact about a single-celled organism and still assigned the wrong assay category.
The study supports using LLMs to flag conflicts and show curators the protocol text behind a suggestion, not to replace expert review. Rare categories had only two or six examples, manual review involved one senior curator, and the paper does not test whether revised labels improve molecular predictions. It is an arXiv preprint accepted to the AIDaR workshop at NeurIPS, with no independent replication reported.
Sources
- Original paper (arXiv, submitted Oct. 1, 2026)
- PubChem BioAssay bulk data
- ChEMBL 37 database used in the study
- Lead image: authors’ BioAssay Ontology detection-method diagram, reproduced from the paper under CC BY 4.0.