LiFR v2 uses event-camera signals to predict pixel-level object labels and depth estimates before the next standard RGB frame arrives. A revised September 23 arXiv preprint from researchers at the University of Hong Kong and Southern University of Science and Technology adds a completion step for content that was hidden in the most recent image.
Standard RGB cameras capture full-color images at fixed intervals. The paper uses a 20-frame-per-second stream as an example: the scene can change for 50 milliseconds before the next frame. Event cameras instead report pixel-level brightness changes asynchronously. Those signals provide precise timing and motion cues, though not the full appearance of an RGB image. LiFR v2 predicts from the latest RGB keyframe and events received up to each query time, without waiting for a future frame.
The earlier LiFR-Seg method used event-derived motion to carry label-related features from a keyframe forward. That can keep visible objects aligned, but propagation cannot move information for an object the keyframe did not show. The paper illustrates the problem with a road user emerging from behind an obstruction: there is no source feature to carry forward.
LiFR v2 adds an event-guided completion module. During training, its event encoder learns task-relevant features by matching features from the corresponding RGB image. At inference, the model uses only the event stream available up to the query time: it combines those event features with the propagated image features, then uses a learned spatial gate to focus the correction. Completed features are saved for later predictions. The team tests semantic segmentation, which assigns a class label to each pixel, and monocular depth, which estimates distance from one camera view, as well as a combined multitask model.
On SHF-Emerge, a synthetic benchmark designed for rapid object emergence and disocclusion, v2 reached 56.13% mean intersection over union (mIoU) for segmentation, 1.85 percentage points above the propagation-based LiFR-Seg baseline. The metric measures how much predicted object-label regions overlap the reference map. In a separate depth comparison on pixels with ground-truth distances of 1 to 10 metres, root mean squared error fell from 1.564 metres for the propagation-only depth adaptation, which the paper calls LiFR, to 1.118 metres for v2. The paper uses LiFR-Seg for segmentation and LiFR for its depth adaptation; these are separate task-specific comparisons.
The paper reports more than 100 predictions per second for separate segmentation and depth runs at 640 × 440 resolution on one NVIDIA RTX 5090. The batch-one timing used CUDA Graph replay after warm-up, repeated the same five-query streaming sequence and included an amortized RGB-keyframe setup; the combined multitask model ran at 52.63 frames per second. DSEC and M3ED evaluations add real urban-driving and robot-platform data, but remain offline perception benchmarks. The paper does not test closed-loop collision avoidance or performance on embedded vehicle hardware, which remain necessary checks for practical use.
