Global Tech News For people figuring out what to build with AI.
AI

A Shared Format Lets Image Models Hand Off Mid-Generation

A rhinoceros emerges from noise across successive diffusion-model steps.

An October 9 arXiv preprint proposes a way for one text-to-image model to hand an unfinished image to another partway through generation. Called InstantFusion, it translates the current state into a shared format so a second model can continue refining it without changing either generator’s weights.

Text-to-image models start with noise and gradually turn it into a picture, updating a compressed internal representation called a latent at each step. Those representations differ across generators, so passing one model’s state directly to another can create artifacts. InstantFusion gives each model an encoder and decoder: when model A hands off to model B, A’s encoder maps its current latent into a shared space and B’s decoder converts it into B’s format. Training aligns states from the same image at the same noise level and also aligns the small denoising change each model would make next, helping the receiver continue the image’s trajectory.

That shared space avoids learning a separate bridge for every model pair. The researchers tested it with Stable Diffusion 3 (SD3), FLUX.1, Qwen-Image and Z-Image. They trained the model-specific encoders and decoders on 50,000 image-text pairs while leaving the generators’ original weights frozen.

When Qwen-Image hands an intermediate state to SD3, the authors report a GenEval-v2 score of 35.06, compared with 22.14 for SD3 alone. A Qwen-Image-to-FLUX.1 handoff scores 36.54, versus 22.86 for FLUX.1 alone. GenEval 2 checks whether images match prompts about objects, attributes, counts and spatial or action relationships; higher scores are better. These are results reported by the paper’s authors, not independent evaluations.

The method requires the generators to share an image autoencoder, which compresses images into latents and reconstructs them, and to use compatible denoising schedules so comparable steps correspond to similar noise levels. Hybrid inference also requires multiple frozen generators to be available at once, increasing memory and storage needs. The work is an arXiv preprint.

Sources