An Oct. 27 arXiv preprint introduces PORTAL, a controller that uses a limited number of target labels to choose among preset prediction rules for each image batch. In one PACS experiment with classes arriving in blocks, a fixed update scored 22.58% accuracy; PORTAL's choice reached 51.28%, near the unchanged model's 50.88%.
Researchers Jiahao Zhang, Xinling Wen, Liangliang Li and Yu Chen, affiliated with Zhengzhou University of Aeronautics and Xi'an University of Technology, ask when a vision model should apply a test-time update. PORTAL chooses separately for each batch among candidate prediction rules, including the option to leave the model unchanged.
On class-blocked images, one fixed rule fell while PORTAL stayed near baseline
The authors tested a ResNet-50 trained on the Photo portion of PACS, an image benchmark whose domains include photos, art, cartoons and sketches. They arranged incoming images in class blocks. The fixed rule recalculated batch-normalization statistics and averaged predictions from an original image and its horizontally flipped copy. It scored 22.58%, below the unchanged model's 50.88%.
Batch normalization estimates feature statistics from examples in a batch. A class-dominated batch can shift those estimates and hurt predictions. On the same three class-block orderings, using batches of 64, PORTAL selected among the predefined rules and scored 51.28% on the separate deployment subset, whose labels were opened only for evaluation. This is close to the unchanged model, not a large gain over it.
PORTAL uses query labels to select among candidate rules
For each batch, the paper splits images into a query subset and a separate deployment subset. PORTAL acquires target labels for query images, then uses those labels to compare all candidates' predictions on the same examples before choosing one rule for the remaining images. The study treats labels as limited feedback; it does not specify who would supply them in a real deployment.
PORTAL groups images when the candidate rules give them the same pattern of predictions. It assumes queried and deployment images within each group have the same mix of classes. A queried label shows which candidates were correct on that image and updates the comparison. PORTAL directs queries toward rivals that could still overtake the leader, then stops when more labels cannot change the choice or their possible accuracy gain no longer justifies their cost.
Across the three class-block streams, PORTAL used an average of 187.7 queried labels per stream. A full-label oracle reached 56.52%, but that comparison uses the deployment labels themselves and serves only as an evaluation reference.
Unrepresentative query images can make PORTAL confidently wrong
If query images do not represent deployment images with the same prediction pattern, PORTAL can choose the wrong rule; the paper's analysis says biased labels can make confidence in that wrong choice grow. These are controlled image-benchmark results, not a field test or evidence of performance across arbitrary data shifts. The study is an arXiv preprint.
Sources
PORTAL preprint and abstract · full paper · illustrative chart by Dennis, CC BY 4.0; not from the PORTAL study
