An October 8 preprint reports that an AI agent improved its task success by learning testable rules about how a simulated environment works, then revising those rules when evidence changed. The system, EvoBelief, kept its underlying language model fixed and stored the learned rules in a separate, persistent model of the environment.
When an action has an unexpected result, EvoBelief proposes possible explanations. A proposed rule cannot yet guide a plan: the system has to test it with later observations or a controlled experiment. If new evidence contradicts a rule, EvoBelief narrows its scope or withdraws it. This lets the agent reuse a verified environmental fact across tasks without changing the language model’s weights.
On ALFWorld, a benchmark of text-based household tasks, the paper reports success rates of 66.42% with Claude Haiku 4.5 and 76.12% with GPT-5.4 in a condition that did not give the acting model records of earlier task transitions. A matched baseline, called Frozen World Model, used the same language model to propose actions and predict their effects, but let no learned rule change a prediction or plan. It scored 42.54% and 44.78%, respectively, a difference of 23.88 and 31.34 percentage points. In a second condition, all methods also received up to four transitions from earlier tasks; each record contained the goal, observations before and after an action, and the observed change. EvoBelief’s margin narrowed but remained positive. Unlike its evidence-checked rules, this shared memory supplied concrete episodes rather than general rules.
A MiniGrid test changed which key opened a door after episode 20, without telling the agent. In one task family, EvoBelief’s success rate was 61.67% before the change and 95.00% after it; the Frozen World Model baseline scored 0% in both periods. The paper says the agent retired a rule that stopped working, experimented again and validated a replacement. Each pre- and post-change result pools 60 episodes from three runs for that task family.
In DiscoveryWorld’s Reactor Lab, EvoBelief completed 10 of 12 Normal tasks and 7 of 12 Challenge tasks. Each of the three evaluated baselines completed none in those two tiers. These small counts are one result inside a set of text-based benchmark experiments, not evidence of performance in a physical lab, robot or deployed software agent. The authors identify those real-world and multimodal settings as future work.
Figure from the authors’ paper, licensed under CC BY 4.0.