A new attack called TRACE reconstructed simulated indoor scenes and navigation choices from policy-learning gradients sent to a server one step at a time. The study tests what a server could infer from these per-step gradients; it does not show that ordinary robot networks expose camera feeds this way.
In distributed reinforcement learning, an agent can keep raw camera frames on its own device while sending gradients to a central server. TRACE compresses each gradient, uses a causal transformer to carry context from earlier steps, and decodes an image and discrete action at each step. Reconstructed views then provide context for the next one.
Researchers evaluated the attack in AI2-THOR, a simulated indoor-navigation environment. Their main test used policies trained to reach a point-goal across 108 scenes; 12 additional scenes were held out. Across 100 test trajectories of eight frames each, TRACE’s reconstructed images averaged 18.8 decibels for proximal policy optimization (PPO) and 18.9 for advantage actor-critic (A2C) on peak signal-to-noise ratio, a pixel-fidelity measure. It recovered all discrete actions under both methods.
Against Learning to Invert (LtI), another model trained to reconstruct images from gradients, TRACE raised image fidelity from 16.8 to 18.8 decibels under PPO and from 17.7 to 18.9 under A2C. Both methods recovered all actions, so this comparison favored TRACE on image similarity, not action recovery. TRACE took 4.5 milliseconds per frame under PPO and 3.0 under A2C, compared with 1.2 and 0.2 milliseconds for LtI.
TRACE also transferred to scenes excluded from attacker training: before any target-scene fine-tuning, it scored 18.8 decibels and recovered all actions. “Zero-shot” here means no data from those target scenes was used to adapt the attack, not that the attacker had no prior data; it was trained on trajectories from the same navigation task. Fine-tuning on 10% of target trajectories raised image fidelity by 2.1 decibels.
The threat model requires a server that sees ordered, per-step gradients and knows the model checkpoint, policy architecture, action space and training rule. A separate test averaged four successive stored gradients, but the paper did not test clients’ full multi-step model updates or secure aggregation across clients. Its experiments remain in one simulator task, not a real robot fleet.
Eight-bit quantization and pruning that kept only 10% of gradient entries barely changed the main PPO reconstruction. Stronger differential-privacy training reduced image and action recovery, but the authors note added computation and utility costs. They leave open whether aggregation across steps or clients can suppress the temporal signal without undermining learning.
Sources
- TRACE paper (full text)
- AI2-THOR simulator documentation
- Lead image: Figure 1 from Bhujel et al., “Temporal Gradient Inversion for Private Trajectory Reconstruction in Embodied Reinforcement Learning”, CC BY 4.0.