Chung-Ang University researchers have presented DescaPE, a method that changes how language models choose their next words. Their September 14 preprint reports improved factual precision in generated biographies for one tested model, with much smaller gains in others.
Word selection has been a target for factuality research before. DoLa, introduced in 2023, compares predictions drawn from earlier and later layers of a language model, using the difference to adjust which word fragment comes next. These fragments are called tokens. The approach sought to draw on knowledge already inside the model without additional fine-tuning or retrieving outside documents.
DescaPE first locates internal layers involved in factual recall by temporarily disabling groups of them. A small trained predictor, called a probe, then estimates the resulting signal without repeating that costly operation during generation. Its scores penalize unusually large signals associated with errors and reward a middle range, helping select among competing continuations while keeping the base model fixed.
To judge biographies, the study uses FActScore. This evaluation method, introduced separately in 2023, breaks generated text into individual factual claims and checks their support against a reference source. A sentence can contain several claims, so one correct detail does not make the whole sentence correct. The metric measures the precision of what was said; it does not measure how much relevant information an answer omitted. That distinction matters when interpreting an improved score.
With Llama-3.1-8B-Instruct generating biographies, DescaPE raised FActScore from 47.1 to 67.2 against greedy decoding, which selects the most probable next token. The evaluation used retrieved evidence and GPT-4o-mini as a judge. Scores barely changed for the other two tested models, and short-answer results were mixed. These are the authors' controlled comparisons, not a ranking of current commercial assistants.
The signal can still penalize correct continuations and does not identify knowledge the model lacks. The authors' repository currently contains a README promising a future code release, despite the paper's code-availability wording. Taken together, the evidence makes biography generation a specific use case worth reproducing. A released implementation and independent tests across models would help determine whether its gain survives changes in model and evaluation setup.