An AI agent that reads phone screens and takes actions such as tapping, swiping and entering text scored higher on two benchmark tests with DeltaReplay, a method that reuses or adapts earlier steps only when they still fit the current screen and task. In a preprint posted Oct. 8, researchers at Tsinghua University report that, with the same Qwen3.8-Flash model, task success rose from 49.1% to 56.9% on AndroidWorld-116 and from 41.7% to 58.3% on a 60-task SPA-Bench subset, compared with a no-memory agent.
Mobile graphical-user-interface (GUI) agents read a screen and perform actions such as tapping, swiping and entering text. A saved run can help when a task repeats, but replaying a whole sequence can carry over an old contact or an irrelevant setting. DeltaReplay stores successful runs as connected screen-and-action steps and checks them one at a time. If the current screen and action still match, it replays that step. If the operation still fits but a value must change, it replaces the old task-specific value and checks the current screen. If neither applies, it lets the base agent choose an action. The memory stays read-only; the researchers did not train the models on new data.
The benchmark totals depend on how the memory was built. AndroidWorld-116 has 116 task templates. Before testing, the researchers saved 42 successful Qwen3.8-Flash runs, one from a different instance of each of 42 templates. DeltaReplay could use those records with either tested backbone; the no-memory baseline did not retrieve them. SPA-Bench-60 was a filtered set of Chinese single-app tasks tested on a physical phone. Its memory pooled 50 successful runs from both base agents across 42 tasks, including 27 of the 60 test tasks. The reported gains therefore combine tasks with stored examples and tasks without them; they do not isolate performance on tasks with no matching memory.
Each task was run once per method, and cloud model decoding is nondeterministic. An LLM judged task success on both benchmarks; where AndroidWorld’s automatic checkers worked, they agreed with the labels in 91.5% of episodes. The preprint did not vary app versions, and the authors say their word-matching step can miss implied values such as “tomorrow.” These results are benchmark evidence, not a deployment study.
Sources
- Original DeltaReplay paper, posted Oct. 8, 2026
- SPA-Bench benchmark paper
- Image source and license record
.jpg){kind=link}
Illustrative photo: Android smartphones, not the phones used in the study. Photo: 彭嘉傑 / Wikimedia Commons, CC BY-SA 4.0 (unmodified).