Papers › Versatile Diffusion: Text, Images and Variations All in One Diffusion Model

Versatile Diffusion: Text, Images and Variations All in One Diffusion Model

15 Nov 2022ICCV 2023 1arXiv:2211.08332archive 2025-07-28

Xingqian Xu, Zhangyang Wang, Eric Zhang, Kai Wang, Humphrey Shi

Recent advances in diffusion models have set an impressive milestone in many generation tasks, and trending works such as DALL-E2, Imagen, and Stable Diffusion have attracted great interest. Despite the rapid landscape changes, recent new approaches focus on extensions and performance rather than capacity, thus requiring separate models for separate tasks. In this work, we expand the existing single-flow diffusion pipeline into a multi-task multimodal network, dubbed Versatile Diffusion (VD), that handles multiple flows of text-to-image, image-to-text, and variations in one unified model. The pipeline design of VD instantiates a unified multi-flow diffusion framework, consisting of sharable and swappable layer modules that enable the crossmodal generality beyond images and text. Through extensive experiments, we demonstrate that VD successfully achieves the following: a) VD outperforms the baseline approaches and handles all its base tasks with competitive quality; b) VD enables novel extensions such as disentanglement of style and semantics, dual- and multi-context blending, etc.; c) The success of our multi-flow multimodal framework over images and text may inspire further diffusion-based universal AI research. Our code and models are open-sourced at https://github.com/SHI-Labs/Versatile-Diffusion.

PaperPDFConference PDFCode

In Syntology View this paper on Syntology, its page in Syntology's graph. That page lists the repositories linked to the paper, the abstract and the calls for agents.

Open this paper in Syntology's Atlas, the map of the papers in Syntology's graph and their citations.

For agents, on Syntology's MCP service (how to connect):

Code

shi-labs/versatile-diffusion officialmentioned in papermentioned on GitHubpytorchMITSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to CVF open access id Xu_Versatile_Diffusion_Text_Images_and_Variations_All_in_One_Diffusion_ICCV_2023_paper (that paper's own run record: 14 ran (of which 0 constructed an object rather than computing a result; 5 with no instrument failure: 3 honoured, 2 violated, 0 with no contract checked; 9 where Syntology's instrument failed) · 5 unverified); nothing checks that they implement this paper's method. report
huggingface/diffusers mentioned on GitHubjaxSyntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to RefDecoder: Enhancing Visual Generation with Conditional Video Decoding (arXiv:2605.15196; that paper's own run record: 7 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 2 violated, 0 with no contract checked; 5 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report
PaddlePaddle/PaddleNLP paddleApache-2.0Syntology: no sample linked to this paper (harvested for another paper). Syntology's graph links these samples to ERNIE 3.0 Tiny: Frustratingly Simple Method to Improve Task-Agnostic Distillation... (arXiv:2301.03416; that paper's own run record: 2 ran (of which 0 constructed an object rather than computing a result; 2 with no instrument failure: 0 honoured, 0 violated, 2 with no contract checked; 0 where Syntology's instrument failed) · 1 unverified); nothing checks that they implement this paper's method. report

Repository list and official/mentioned flags are the archive's, frozen 2025-07-28. Reachability, where shown, is from one Syntology probe window (2026-09-16 to 2026-09-18); repositories not probed show nothing. GitHub stars are not tracked.

A paper named beside a repository with no sample linked to this paper is shown with that paper's own run record, not this paper's: “ran” means executed on a synthesized input, not that the code is correct, and “where Syntology's instrument failed” counts failures of Syntology's instrument, not of the code. Papers are listed in arXiv-id order, at most three per repository.

Code Syntology ran Syntology

Not run by Syntology. Nothing on this page verifies that the listed code works.

Tasks

AllDisentanglementImage CaptioningImage GenerationImage ManipulationImage to textImage-VariationText-VariationText-based Image EditingText-to-Image Generationtext-guided-image-editing

Results from the paper archive 2025-07-28

No leaderboard rows for this paper in the archive.

Methods

BASEDiffusion

Report a problem or propose a change · a person checks every report against the paper or source before anything changes; decisions are listed on /corrections