A prompt-only method that allows a large language model to decline a question cut the average rate of incorrect answers it chose to give from 13.1% to 8.9%. The result came from tests on 817 multiple-choice questions across 11 model families. The method answered 87.6% of the questions and withheld the rest, creating a measurable trade-off between answering more often and sending uncertain cases for review. Ali Şenol of Tarsus University reported the work in an unreviewed preprint submitted on September 15.
The method, called Chain-of-Self-Questioning or CoSQ, separates deciding whether to answer from producing the answer. The model first lists the pieces of information it would need, then gives each piece a support score. In the main configuration, it proceeds only when the average score reaches 0.90 and builds its answer from the accepted information. Otherwise it abstains. The procedure requires no fine-tuning or access to internal model probabilities, which makes it usable with hosted systems, but it takes three model calls instead of the single call used by the comparison prompt.
The primary experiment used TruthfulQA, a benchmark designed around questions that can elicit common misconceptions. Answer choices were rearranged so that the correct option did not stay in a predictable position, and every model saw the same 17 prompting conditions. A chain-of-thought baseline had to choose an answer for every question. CoSQ raised accuracy among the answers it gave from 86.9% to 89.7% while lowering wrong commitments across all questions from 13.1% to 8.9%. Both changes moved in the same direction for each of the 11 models at the 0.90 threshold.
The refusals carry a substantial cost. CoSQ abstained on 12.3% of questions at the main operating point. Of those withheld answers, 31.3% matched questions that the chain-of-thought baseline answered incorrectly, while 68.7% matched questions it answered correctly. The gate therefore removed some errors and also declined many questions the baseline could answer. A service using the method would need to price both outcomes: the harm from a wrong answer and the time or labour required to handle a referral.
A separate test on 300 short-answer questions from Natural Questions produced the same general pattern across five models. Wrong commitments fell from 41.2% to 27.4%, accuracy among answered questions rose from 58.8% to 67.0%, and coverage was 83.0%. These figures use open-ended generation and semantic answer matching, so they are evidence from a second setting rather than a direct numerical extension of the multiple-choice result.
The study leaves several deployment questions open. The same model supplies both the confidence scores and the final answer, so the scores are not independently calibrated probabilities. Hosted model endpoints can change, the method triples the number of inference stages, and the main evidence still comes from one multiple-choice benchmark with a smaller secondary test. Domain trials with fixed model versions, explicit cost limits and human review are needed before using the gate in consequential work. The paper's practical contribution is a visible control: system operators can choose how much automated coverage to exchange for fewer wrong commitments.
Related coverage
Sources
- Chain-of-Self-Questioning preprint record, arXiv, submitted September 15, 2026.
- Full Chain-of-Self-Questioning paper, including methods, results and limitations.
- CoSQ public implementation and reproducibility materials, GitHub.
- Lead image: Ajay Suresh, via Wikimedia Commons, CC BY 2.0. The photograph is illustrative and does not show the reported experiment.
.jpg){kind=link}