Title: Adversarial Flow Models URL Source: https://arxiv.org/html/2511.22475 Markdown Content: Back to arXiv This is experimental HTML to improve accessibility. We invite you to report rendering errors. Use Alt+Y to toggle on accessible reporting links and Alt+Shift+Y to toggle off. Learn more about this project and help improve conversions. Why HTML? Report Issue Back to Abstract Download PDF Abstract 1Introduction 2Related Works 3Method 4Experiments 5Conclusion References License: CC BY 4.0 arXiv:2511.22475v1 [cs.LG] 27 Nov 2025 Adversarial Flow Models Shanchuan Lin  Ceyuan Yang  Zhijie Lin  Hao Chen  Haoqi Fan ByteDance Seed peterlin@bytedance.com Abstract We present adversarial flow models, a class of generative models that unifies adversarial models and flow models. Our method supports native one-step or multi-step generation and is trained using the adversarial objective. Unlike traditional GANs, where the generator learns an arbitrary transport plan between the noise and the data distributions, our generator learns a deterministic noise-to-data mapping, which is the same optimal transport as in flow-matching models. This significantly stabilizes adversarial training. Also, unlike consistency-based methods, our model directly learns one-step or few-step generation without needing to learn the intermediate timesteps of the probability flow for propagation. This saves model capacity, reduces training iterations, and avoids error accumulation. Under the same 1NFE setting on ImageNet-256px, our B/2 model approaches the performance of consistency-based XL/2 models, while our XL/2 model creates a new best FID of 2.38. We additionally show the possibility of end-to-end training of 56-layer and 112-layer models through depth repetition without any intermediate supervision, and achieve FIDs of 2.08 and 1.94 using a single forward pass, surpassing their 2NFE and 4NFE counterparts. 1Introduction Flow matching [49, 80] is a class of generative models that achieves the current state of the art in multiple domains, including image generation [11, 77, 3] and video generation [86, 38, 63, 76, 13], etc. It formulates the generation problem as the transport of samples from a prior distribution to the data distribution. A probability flow is established by interpolating the data and the prior samples, and a neural network learns the gradient of the flow. At inference time, the sample is iteratively transported by querying the network for gradients, incurring a high computational cost. Recent methods accelerate the generation process by formulating the network to directly predict farther positions on the flow instead of the instantaneous gradients. This can be achieved either by distilling from a pre-trained flow-matching model [71, 50], or by training from scratch with consistency objectives as a new class of generative models [81]. However, despite the goal of single-step or few-step generation, consistency-based models must still be trained on all timesteps of the flow for consistency propagation. This wastes model capacity and incurs error accumulation. A more critical problem is that networks in fewer steps have less capacity to predict the exact transformations of the more-step targets, so using pointwise-matching or even moment-matching losses causes some extent of blurriness. For these reasons, many state-of-the-art few-step generation models still rely on distributional matching methods, especially adversarial training, for final refinement [47, 6]. 𝑥 1 step 4 steps 64 steps 𝑧 (a) GANs (b) Flow Matching (c) Adversarial Flow Models (Ours) Figure 1:Models trained on 1D Gaussian mixture. (a) GAN learns an arbitrary transport plan. (b) Flow matching learns a deterministic transport plan but have large discretization errors on low sampling steps. (c) Adversarial flow models support any-step training and generation with a deterministic optimal transport plan. Adversarial training originates from generative adversarial networks (GANs) [15]. It itself is a standalone class of generative models that supports single-step generation. However, adversarial training from scratch often faces stability issues. A handful of recent works exploring adversarial training from scratch adopt non-standard architectural designs [21, 26, 108, 23]. Some further rely on the assistance of additional frozen feature networks [26, 23]. When we switch to a standard transformer architecture [85], the training simply diverges. As shown in Fig. 1, we discover that one of the key reasons GANs are hard to train is that the adversarial objective alone does not present a single optimization target. This is markedly different from other established objectives, such as flow-matching, which has a unique ground-truth probability flow predetermined by the interpolation function, and autoregressive modeling, which has ground-truth token probabilities predetermined by the training corpus. In GANs, the generator is tasked to transport samples from the prior to the data distribution, but the adversarial objective only enforces the matching of the data distributions without constraining the transport plan. Therefore, there exist infinite valid transport plans that the generator may pick, influenced by the weight initialization and the stochastic training process. This causes optimization difficulties as the generator keeps drifting during training. In this paper, we propose adversarial flow models, a class of generative models that unifies the adversarial and flow families of models under the same framework. Our model is trained by the adversarial objective. Therefore, it naturally supports single-step training and generation without wasting capacities on learning the intermediate timesteps needed by consistency methods. Our model also belongs to the flow family of models. Therefore, it has the same deterministic transport mapping for training stability, and it naturally supports multi-step training and generation. Our method can be trained on standard transformer architectures without modifications, which opens the door for wider adoption. On ImageNet-256px, our B/2 model approaches the performance of consistency-based XL/2 models due to the conservation of modeling capacity, while our XL/2 model creates a new best FID of 2.38 under the same 1NFE setting with guidance. Our method also enables fully end-to-end training of 56-layer and 112-layer 1NFE models through depth repetition, and achieves FIDs of 2.08 and 1.94, surpassing the 2NFE and 4NFE counterparts. Furthermore, in the same no-guidance setting, we show that the adversarial objective can even outperform flow matching due to better distribution matching properties. 2Related Works The acceleration of flow-based models. Early distillation works train few-step student models to match the prediction of the teacher flow model [71, 50, 72, 94]. Consistency model (CM) [81, 79] proposes the use of self-consistency constraint and supports standalone training as a new class of generative models. Various follow-up works improve upon CM. Notably, sCM [52] extends the consistency constraint to continuous time to minimize discretization error. iMM [107] incorporates moment matching. Shortcut [12] redefines the boundary condition to allow jumping between arbitrary timesteps. MeanFlow [14] again extends Shortcut to continuous time. However, these methods typically still generate slightly blurry results on large-scale text-to-image/video tasks [54], so distributional matching methods, such as adversarial training [46, 66, 44, 47, 48, 53, 75, 74, 87, 37, 27, 92, 6] and score distillation [97, 96, 75, 53, 55, 105], are often incorporated in practice. Generative adversarial networks. Early research in GANs [15] has developed many techniques to achieve success on domain-specific datasets [65, 100, 28, 29, 31, 32, 30]. BigGAN [4] and StyleGAN-XL [73] have further scaled it to ImageNet [70]. However, GANs have fallen out of favor because of their training instability and limited scalability. A handful of works exploring large-scale text-to-image generation using GANs still employ a convolutional architecture with complex designs [26, 108]. GANs with transformer architectures [85] have been challenging to scale up [24, 40, 22]. Until recently, R3GAN [21] simplifies the adversarial formulation and pushes the state of the art in the ImageNet-64 benchmark using a convolutional architecture. GAT [23] further extends it to a latent transformer architecture. These works revitalize interest in adversarial training. However, GAT still employs a non-standard transformer architecture and uses the assistance of a pre-trained feature network. Our work provides a framework that unifies adversarial models with flow models and improves upon the training stability of adversarial methods. 3Method 3.1Adversarial Training Preliminaries Our method is built on top of generative adversarial networks (GANs) [15], where a generator 𝐺 : ℝ 𝑚 → ℝ 𝑛 aims to transport samples 𝑧 from a prior distribution 𝒵 , e.g . Gaussian, to samples 𝑥 from the data distribution 𝒳 , and a discriminator 𝐷 : ℝ 𝑛 → ℝ 1 aims to differentiate real samples from the generated ones. The adversarial optimization involves a minimax game where 𝐷 is trained to maximize differentiation while 𝐺 is trained to minimize the differentiation by 𝐷 . We adopt the relativistic objective [25] as it has better properties for the loss landscape [82] and achieves the current state of the art [21, 23]: ℒ adv 𝐷 = 𝔼 𝑧 , 𝑥 ​ [ 𝑓 ​ ( 𝐷 ​ ( 𝑥 ) − 𝐷 ​ ( 𝐺 ​ ( 𝑧 ) ) ) ] , (1) ℒ adv 𝐺 = 𝔼 𝑧 , 𝑥 ​ [ 𝑓 ​ ( 𝐷 ​ ( 𝐺 ​ ( 𝑧 ) ) − 𝐷 ​ ( 𝑥 ) ) ] , (2) where 𝑓 ​ ( ⋅ ) = − log ⁡ ( sigmoid ​ ( ⋅ ) ) . Additionally, gradient penalties 𝑅 1 and 𝑅 2  [69] are added on 𝐷 . They prevent 𝐺 from being pushed away from equilibrium [57, 21] and impose a constraint on the Lipschitz constant of 𝐷  [16]. Directly computing the gradient penalties requires expansive double backward and twice differentiation, so we use finite difference approximation [47]: ℒ r1 𝐷 = 𝔼 𝑥 ​ [ ‖ ▽ 𝑥 ​ 𝐷 ​ ( 𝑥 ) ‖ 2 2 ] (3) ≈ 𝔼 𝑥 ​ [ 1 𝜖 2 ​ ‖ 𝐷 ​ ( 𝑥 ) − 𝐷 ​ ( 𝒩 ​ ( 𝑥 , 𝜖 2 ​ I ) ) ‖ 2 2 ] , (4) ℒ r2 𝐷 = 𝔼 𝑧 ​ [ ‖ ▽ 𝐺 ​ ( 𝑧 ) ​ 𝐷 ​ ( 𝐺 ​ ( 𝑧 ) ) ‖ 2 2 ] (5) ≈ 𝔼 𝑧 ​ [ 1 𝜖 2 ​ ‖ 𝐷 ​ ( 𝐺 ​ ( 𝑧 ) ) − 𝐷 ​ ( 𝒩 ​ ( 𝐺 ​ ( 𝑧 ) , 𝜖 2 ​ I ) ) ‖ 2 2 ] , (6) where 𝜖 is set to a small value 0.01. We only compute the penalties on 25% of the samples of a batch and observe no performance degradation. To prevent the discriminator logits from drifting unboundedly in the relativistic setting, a logit-centering penalty is added, similar to prior work [28]: ℒ cp 𝐷 = 𝔼 𝑧 , 𝑥 ​ [ ‖ 𝐷 ​ ( 𝑥 ) + 𝐷 ​ ( 𝐺 ​ ( 𝑧 ) ) ‖ 2 2 ] . (7) Therefore, the final GAN loss is: ℒ GAN 𝐷 = ℒ adv 𝐷 + 𝜆 gp ​ ℒ r1 𝐷 + 𝜆 gp ​ ℒ r2 𝐷 + 𝜆 cp ​ ℒ cp 𝐷 , (8) ℒ GAN 𝐺 = ℒ adv 𝐺 , (9) where 𝜆 gp is a searched hyperparameter controlling the gradient penalty scale of both 𝑅 1 and 𝑅 2 , and 𝜆 cp is universally set to 0.01. The expectation is obtained through Monte Carlo approximation over minibatches of data during training. The 𝐺 and 𝐷 are updated in alternation. For conditional generation, the condition 𝑐 is given to both networks as 𝐺 ​ ( 𝑧 , 𝑐 ) and 𝐷 ​ ( 𝑥 , 𝑐 ) . These are abbreviated in writing. 3.2Single-step Adversarial Flow Models The GAN objective above only enforces 𝐺 ​ ( 𝑧 ) to match the data distribution. However, there are infinite transport plans from 𝑧 to 𝑥 , and the model is free to learn any arbitrary one. Our proposed adversarial flow models learn a deterministic transport plan, which is the same transport plan as flow-matching models with the linear interpolation function. This allows for the existence of a single solution, prevents generator drifting, and stabilizes training. In flow matching, the ground-truth probability flow exists prior to model training and is deterministically defined by the data distribution, the prior distribution, the interpolation function, and the loss objective. It is known that when linear interpolation and the squared distance loss are used, this combination establishes a transport plan that minimizes the squared Wasserstein-2 ( 𝐖 𝟐 𝟐 ) distance between the prior and data distributions [49]. Formally, let 𝝅 ​ ( 𝑥 , 𝑧 ) be a valid transport plan, and 𝐜 ​ ( 𝑥 , 𝑧 ) be a cost function, flow matching with linear interpolation and squared distance objective results in the transport plan 𝝅 ​ ( 𝑥 , 𝑧 ) that minimizes the total transport cost where the cost function 𝐜 ​ ( 𝑥 , 𝑧 ) = ‖ 𝑥 − 𝑧 ‖ 2 2 by the definition of W 2 2 : min 𝜋 ​ ∫ 𝒳 × 𝒵 𝐜 ​ ( 𝑥 , 𝑧 ) ​ 𝑑 𝝅 ​ ( 𝑥 , 𝑧 ) . (10) Therefore, for adversarial flow models, we first restrict the prior distribution to have the same dimensionality 𝑛 as the data distribution, i.e . , 𝑥 , 𝑧 ∈ ℝ 𝑛 and 𝐺 : ℝ 𝑛 → ℝ 𝑛 . This is required by all flow-based models and does not reduce the generality of our method. Then we substitute 𝜋 ​ ( 𝑥 , 𝑧 ) with our generator 𝐺 ​ ( 𝑧 ) : min 𝐺 ​ ∫ 𝒵 𝐜 ​ ( 𝐺 ​ ( 𝑧 ) , 𝑧 ) ​ 𝑑 𝐺 ​ ( 𝑧 ) . (11) Our goal is to find a 𝐺 that is simultaneously a valid transport plan and a plan that minimizes the total transport cost under our defined cost function. Since the adversarial objective already enforces the validity of the transport plan, we only need to add an additional optimal transport (OT) loss on 𝐺 . The loss minimizes the expectation of 𝐜 ​ ( 𝐺 ​ ( 𝑧 ) , 𝑧 ) for all 𝑧 : ℒ ot 𝐺 = 𝔼 𝑧 [ 1 𝑛 ∥ 𝐺 ( 𝑧 ) − 𝑧 ∥ 2 2 ] . (12) The objectives in adversarial flow models (AF) become: ℒ AF 𝐷 = ℒ adv 𝐷 + 𝜆 gp ​ ℒ r1 𝐷 + 𝜆 gp ​ ℒ r2 𝐷 + 𝜆 cp ​ ℒ cp 𝐷 , (13) ℒ AF 𝐺 = ℒ adv 𝐺 + 𝜆 ot ​ ℒ ot 𝐺 . (14) 𝑥 small ← 𝜆 ot → large 𝑧 Figure 2:The effect of using different 𝜆 ot scale. 𝜆 ot = 0 is equivalent to GANs. 𝜆 ot being too small fails to escape local minima. 𝜆 ot being too large forces identity output. In theory, even an infinitesimal 𝜆 ot will break the symmetry in the loss landscape and create a unique global minimum. But in practice, if the value is too small, the optimization may get stuck in local minima, and if the value is too large, it forces the model to output identity, hurting distribution matching, as illustrated in Fig. 2. Therefore, we adopt a schedule to decrease 𝜆 ot over time during training. Unlike consistency-based methods, our model does not need to be trained on other timesteps of the probability flow and can be trained for native one-step generation. This saves model capacity, reduces training iterations, and avoids error propagation. Furthermore, for the one-step training case, this formulation completely removes the hyperparameter of choosing timestep sampling and weighting. Our one-step model also completely avoids teacher-forcing. 3.3Multi-step Adversarial Flow Models Adversarial flow models can be generalized to multi-step generation and allow the model to jump between arbitrary timesteps on the probability flow. We introduce an interpolation function of the same form as flow-matching models: 𝑥 𝑡 = interp ​ ( 𝑥 , 𝑧 , 𝑡 ) := 𝐴 ​ ( 𝑡 ) ​ 𝑥 + 𝐵 ​ ( 𝑡 ) ​ 𝑧 , (15) where 𝑡 ∈ [ 0 , 1 ] . Although 𝐴 ​ ( 𝑡 ) and 𝐵 ​ ( 𝑡 ) can be any continuous monotonic functions satisfying 𝑥 0 = 𝑥 , 𝑥 1 = 𝑧 , we adopt linear interpolation, where 𝐴 ​ ( 𝑡 ) = 1 − 𝑡 and 𝐵 ​ ( 𝑡 ) = 𝑡 , to match the W 2 2 transport behavior. We change the generator to accept additional source timestep 𝑠 and target timestep 𝑡 as 𝐺 ​ ( 𝑥 𝑠 , 𝑠 , 𝑡 ) , and the discriminator to accept additional target timestep as 𝐷 ​ ( 𝑥 𝑡 , 𝑡 ) . During training, 𝑥 𝑠 , 𝑥 𝑡 are obtained by interpolating randomly sampled 𝑥 , 𝑧 . The adversarial loss is extended as: ℒ adv 𝐷 = 𝔼 𝑥 𝑠 , 𝑥 𝑡 , 𝑠 , 𝑡 ​ [ 𝑓 ​ ( 𝐷 ​ ( 𝑥 𝑡 , 𝑡 ) − 𝐷 ​ ( 𝐺 ​ ( 𝑥 𝑠 , 𝑠 , 𝑡 ) , 𝑡 ) ) ] , (16) ℒ adv 𝐺 = 𝔼 𝑥 𝑠 , 𝑥 𝑡 , 𝑠 , 𝑡 ​ [ 𝑓 ​ ( 𝐷 ​ ( 𝐺 ​ ( 𝑥 𝑠 , 𝑠 , 𝑡 ) , 𝑡 ) − 𝐷 ​ ( 𝑥 𝑡 , 𝑡 ) ) ] . (17) The 𝑅 1 and 𝑅 2 gradient penalties are substituted accordingly. We omit the approximation form for simplicity: ℒ r1 𝐷 = 𝔼 𝑥 𝑡 , 𝑠 , 𝑡 ​ [ 𝑤 ​ ( 𝑠 , 𝑡 ) ​ ‖ ▽ 𝑥 𝑡 ​ 𝐷 ​ ( 𝑥 𝑡 , 𝑡 ) ‖ 2 2 ] , (18) ℒ r2 𝐷 = 𝔼 𝑥 𝑠 , 𝑠 , 𝑡 ​ [ 𝑤 ​ ( 𝑠 , 𝑡 ) ​ ‖ ▽ 𝐺 ​ ( 𝑥 𝑠 , 𝑠 , 𝑡 ) ​ 𝐷 ​ ( 𝐺 ​ ( 𝑥 𝑠 , 𝑠 , 𝑡 ) , 𝑡 ) ‖ 2 2 ] . (19) In flow-matching models with linear interpolation, the transport plan between any source and target timestep’s marginal distributions also minimizes the W 2 2 distance, so the optimal transport loss is generalized as: ℒ ot 𝐺 = 𝔼 𝑥 𝑠 , 𝑠 , 𝑡 ​ [ 1 𝑛 ⋅ 1 𝑤 ​ ( 𝑠 , 𝑡 ) ⋅ ‖ 𝐺 ​ ( 𝑥 𝑠 , 𝑠 , 𝑡 ) − 𝑥 𝑠 ‖ 2 2 ] . (20) We empirically find that the following weighting function works well: 𝑤 ​ ( 𝑠 , 𝑡 ) = max ⁡ ( | 𝑠 − 𝑡 | , 𝛿 ) , (21) where 𝛿 = 0.001 is set for numerical stability. During training, the timesteps can be sampled as continuous values: 𝑠 ∼ 𝒰 ​ ( 0 , 1 ) , 𝑡 ∼ 𝒰 ​ ( 0 , 𝑠 ) . In this case, the model is trained to support transporting between any arbitrary timesteps 𝑡 < 𝑠 on the probability flow for generation. When 𝑠 and 𝑡 are close, the model behaves like flow-matching models. When they are far, the model behaves like trajectory models. Since 𝐺 directly learns the target distribution through 𝐷 without needing consistency propagation, the model can alternatively be trained to only learn a discrete set of timesteps that is needed for a specific few-step inference setting, where single-step generation is just a special case. This saves model capacity and training iterations. Our framework unifies single-step and flow generation under the same adversarial training setting. 3.4Discriminator Formulation It is important not to condition the discriminator on the source sample. Specifically, formulating the discriminator as 𝐷 ​ ( 𝑥 , 𝑧 ¯ ) for single-step, or 𝐷 ​ ( 𝑥 𝑡 , 𝑡 , 𝑥 𝑠 , 𝑠 ¯ ) for multi-step, is incorrect. This is because at training, 𝑥 and 𝑧 are randomly sampled and paired. This formulation incorrectly tells 𝐷 that 𝑧 should be paired with all 𝑥 , but 𝐺 can only produce a single mapping, and it is impossible to achieve such a goal, so the training will oscillate or diverge. When searching for hyperparameters, a complication we have encountered is gradient magnitude. Specifically, the 𝐺 objective consists of both the adversarial and the optimal transport losses. However, the adversarial loss received from 𝐷 may have varying magnitudes, influenced by the 𝐷 architecture, its weight initialization, the gradient penalty strength, and the adversarial objective. This creates difficulties in finding 𝜆 ot across model sizes. Formally, we break down the gradient of the generator loss with respect to the generator network parameter 𝜃 following the chain rule: ∂ ℒ AF 𝐺 ∂ 𝜃 = ∂ ℒ adv 𝐺 ∂ 𝜃 + 𝜆 ot ​ ∂ ℒ ot 𝐺 ∂ 𝜃 (22) = ∂ 𝐺 ​ ( 𝑧 ) ∂ 𝜃 ⋅ ∂ 𝐷 ​ ( 𝐺 ​ ( 𝑧 ) ) ∂ 𝐺 ​ ( 𝑧 ) ⏟ discriminator ⋅ ∂ ℒ adv 𝐺 ∂ 𝐷 ​ ( 𝐺 ​ ( 𝑧 ) ) ⏟ adversarial obj. + ∂ 𝐺 ​ ( 𝑧 ) ∂ 𝜃 ⋅ 𝜆 ot ​ ∂ ℒ ot 𝐺 ∂ 𝐺 ​ ( 𝑧 ) ⏟ transport obj. . The boxed terms are the gradient passed down from 𝐷 that has varying magnitudes. In traditional GANs, only the adversarial loss is used, and adaptive optimizers rescale the magnitude per parameter so 𝐺 is invariant to the absolute scale of the gradient. However, in our case, the magnitude matters because it determines the ratio between the adversarial and the optimal transport losses. Therefore, we propose a gradient normalization technique. Specifically, we change the formulation to 𝐷 ​ ( 𝜙 ​ ( 𝐺 ​ ( 𝑧 ) ) ) , where 𝜙 is an identity operator in the forward pass but rescales the gradient in the backward pass: 𝜙 ′ = ∂ ℒ adv 𝐺 ∂ 𝐺 ​ ( 𝑧 ) 𝑛 ⋅ EMA ​ ( ‖ ∂ ℒ adv 𝐺 ∂ 𝐺 ​ ( 𝑧 ) ‖ 2 2 , 𝛽 2 ) . (23) The operator 𝜙 tracks the exponential moving average (EMA) of the gradient norm, normalizes the gradient by the average norm, and rescales by 1 𝑛 where 𝑛 is the data dimension. It can be seen as an extension of Adam [36] into the backward path, so we use the same 𝛽 2 as in Adam for the EMA decay. After normalizing the adversarial gradient to a unit scale, we can find 𝜆 ot that works well across model sizes. 3.5Better Distribution Matching Adversarial flow models enable the complete disentanglement of distribution-matching and optimal-transport objectives. Although the optimal-transport objective matches flow matching, the distribution-matching objective does not. Flow matching’s squared distance loss objective minimizes the Euclidean distance in the ℝ 𝑛 space instead of the semantic distance on the data manifold [43]. As a result, in high-dimensional space, new data samples generated by the model through generalization appear element-wise blended instead of semantically morphed from the training data. This is why flow-matching models without guidance generate perceptually out-of-distribution samples. The effect of classifier-free guidance (CFG) [19] is not only low-temperature sampling, but also perceptual guidance [43]. However, CFG entangles perceptual quality with strong conditional alignment. This limits flow-matching models from representing the full data distribution and can make samples unnatural and canonical [45, 33]. Consistency-based methods share the same pathology. Latest works have explored training on more representational latent spaces [103, 41] and using other guidance types [33, 34, 93] to counter this congenital defect. (More in Appendix E.) Adversarial training is different because of the introduction of a learnable 𝐷 . The distance between real and generated distributions is measured through the semantic network, which captures the perceptual and semantic distance [102] instead of the Euclidean distance. Our experiments find that our model can outcompete flow matching in the guidance-free setting. 3.6Connections to Guidance Having control over sampling temperature and conditional alignment is still a desirable property. We show that guidance can be incorporated into adversarial flow models. We use classifier guidance (CG) [9] for conditional generation as an illustration because of its popularity. Uncond 𝑝 ​ ( 𝑥 𝑡 ) Cond 𝑝 ​ ( 𝑥 𝑡 | 𝑐 ) (a)Original flows Not guided Guided (b)Flow matching + CFG Not guided Guided (c) 𝐶 ​ ( 𝐺 ​ ( 𝑧 , 𝑐 ) , 𝑐 ) Not guided Guided (d) 𝐶 ​ ( interp ​ ( 𝐺 ​ ( 𝑧 , 𝑐 ) , 𝑧 ′ , 𝑡 ′ ) , 𝑡 ′ , 𝑐 ) Figure 3:Conditional models trained on 1D Gaussian mixtures. (a) The conditional flow. (b) The effect of CFG on flow-matching models. (c) Adversarial flow models with simple classifier guidance does not work. (d) Adversarial flow models with flow-based classifier guidance match the behavior of CFG. We visualize an example conditional flow in Fig. 3(a) and the effect of CFG on flow-matching models in Fig. 3(b). Prior adversarial works [73, 26] introduce a classifier 𝐶 ​ ( 𝑥 , 𝑐 ) that predicts 𝑝 ​ ( 𝑐 | 𝑥 ) , and train 𝐺 with an additional loss to maximize the classification probability: ℒ cg 𝐺 = 𝔼 𝑧 , 𝑐 ​ [ − 𝐶 ​ ( 𝐺 ​ ( 𝑧 , 𝑐 ) , 𝑐 ) ] , (24) ℒ cAF 𝐺 = ℒ adv 𝐺 + 𝜆 ot ​ ℒ ot 𝐺 + 𝜆 cg ​ ℒ cg 𝐺 . (25) However, as Fig. 3(c) shows, the guided model yields almost identical results to the original. This is because in this particular example, the classes are well separated, so the classifier has a clear decision boundary and yields no gradient. This example illustrates the important difference compared to CFG, whose transport is influenced by guidance gradients accumulated along the flow instead of at a single timestep. The classifier gradients exist on higher timesteps due to the diffusion of class boundaries by interpolation. To obtain the accumulated guidance gradient along the flow, even for single-step training and generation, we switch to a time-conditioned classifier 𝐶 ​ ( 𝑥 𝑡 , 𝑡 ′ , 𝑐 ) that predicts 𝑝 ​ ( 𝑐 | 𝑥 𝑡 ′ ) on a probability flow. During training, 𝐺 , 𝐷 are still trained for a single step only. The generated samples 𝐺 ​ ( 𝑧 , 𝑐 ) are interpolated to random timesteps 𝑡 ′ ∼ 𝒰 ​ ( 0 , 1 ) before feeding to the classifier: ℒ cg 𝐺 = 𝔼 𝑧 , 𝑐 , 𝑧 ′ , 𝑡 ′ ​ [ − 𝐶 ​ ( interp ​ ( 𝐺 ​ ( 𝑧 , 𝑐 ) , 𝑧 ′ , 𝑡 ′ ) , 𝑡 ′ , 𝑐 ) ] . (26) This maximizes the expectation of classification on all timesteps, giving us similar results to CFG as shown in Fig. 3(d). The timestep 𝑡 ′ can be sampled from a custom range, which corresponds to performing CFG on selected timesteps. Eq. 24 is a special case of Eq. 26 where 𝑡 ′ is always 0. The hyperparameters, i.e. the scale 𝜆 cg and the range of 𝑡 ′ , can be optionally amortized into 𝐺 to allow inference-time adjustment. For clarity, our experiment trains 𝐶 offline as an independent network. If adversarial flow is used for post-training an existing flow-matching model 𝑣 , the gradient of an implicit classifier can be derived from 𝑣 following CFG: ▽ 𝑥 𝑡 ​ 𝐶 ​ ( 𝑥 𝑡 , 𝑡 , 𝑐 ) = 𝑣 ​ ( 𝑥 𝑡 , 𝑡 ) − 𝑣 ​ ( 𝑥 𝑡 , 𝑡 , 𝑐 ) , (27) ℒ cfg 𝐺 = 𝔼 𝑧 , 𝑐 , 𝑧 ′ , 𝑡 ′ ​ [ − 1 𝑛 ​ ∑ 𝐺 ​ ( 𝑧 , 𝑐 ) ​ ▽ ( ⋅ ) ​ 𝐶 ​ ( ⋅ , 𝑡 ′ , 𝑐 ) ] , (28) where ( ⋅ ) is short for interp ​ ( 𝐺 ​ ( 𝑧 , 𝑐 ) , 𝑧 ′ , 𝑡 ′ ) . 3.7Model Architecture We parameterize the single-step or multi-step generator 𝐺 using a neural network 𝑔 . We find that it can work equally well by either formulating it directly: 𝐺 ​ ( 𝑧 ) = 𝑔 ​ ( 𝑧 ) , (29) 𝐺 ​ ( 𝑥 𝑠 , 𝑠 , 𝑡 ) = 𝑔 ​ ( 𝑥 𝑠 , 𝑠 , 𝑡 ) , (30) or as residual: 𝐺 ​ ( 𝑧 ) = 𝑧 − 𝑔 ​ ( 𝑧 ) , (31) 𝐺 ​ ( 𝑥 𝑠 , 𝑠 , 𝑡 ) = 𝑥 𝑠 − ( 𝑠 − 𝑡 ) ​ 𝑔 ​ ( 𝑥 𝑠 , 𝑠 , 𝑡 ) . (32) The latter is closely related to the velocity prediction formulation of existing flow-matching models and consistency-based models. To demonstrate the feasibility of both, we train our single-step models using the direct formulation and multi-step models using the residual formulation. We parameterize 𝐷 directly using a neural network 𝑑 . Both the 𝑔 and 𝑑 networks use standard diffusion transformer architecture (DiT) [62]. For single-step models, the timestep projection is removed. For multi-step models with fixed discretizations, one timestep projection is used. For any-step models, two timestep projections are used in 𝑔 . The condition is injected through modulation for both 𝑔 and 𝑑 following the original DiT. Our 𝑑 is nearly identical to 𝑔 , except for the addition of a learnable [CLS] token prepended to the input. The [CLS] token is used to produce the logit through a final LayerNorm [2] and a linear projection. Our architecture has minimal modifications to the original DiT. 3.8Deep Model Architecture (a)Multi-step (b)Extra-deep single-step Figure 4:Deep architecture through transformer block repetition. Extra-deep models are trained end-to-end using single-step objective without any intermediate supervision. Prior research [47] indicates that the effective model depth is critical for capturing the non-linear transformations required for generating high-quality samples, and the lack thereof is the primary reason causing artifacts on single-step models. We experiment with end-to-end training of extra-deep single-step models. In theory, extra-deep single-step models can outperform their multi-step counterparts because the models can pass hidden states end-to-end without projecting to and reinterpreting from the data space, there is no manual definition of timestep discretizations, and the network is trained without teacher-forcing. As illustrated in Fig. 4, our extra-deep models use transformer block repetition [8]. The hidden state after the initial pass is recycled. Timestep-like embedding is still given to the transformer blocks only to differentiate repetition iterations, but the whole network is trained end-to-end for single-forward generation without any intermediate supervision. This design allows us to exactly match the number of parameters and model behavior with the multi-step counterpart for comparison. 3.9Techniques for Adversarial Training We share additional training techniques that are, though imperfect, very effective. They address the remaining challenges of adversarial training, not introduced by our adversarial flow models. On optimization, minimax optimization with gradient descent is prone to oscillation. The equilibrium is better achieved through the weight average than at the last iterate [7]. We find optimistic optimizer [7] and asymmetrical learning rates [18] ineffective. Our approach is to keep an EMA of 𝐺  [95], which consistently outperforms the online model by a large margin. More importantly, once the performance of the EMA model plateaus toward the end of the training, we replace online 𝐺 with the EMA weights and repeat automatically after every epoch onward. Learning rate is also decreased during this phase. We find this simple technique very effective for approaching peak performance. On training dynamics, vanilla adversarial training needs 𝐺 and 𝐷 to learn at a synchronized pace. If 𝐺 learns too slowly, 𝐷 will separate the real and the generated distributions perfectly, and the gradient vanishes. To address the vanishing gradient problem, WGAN [1] is proposed but requires a K-Lipschitz 𝐷 . DiffusionGAN [90] projects 𝐷 onto a flow in the same spirit as our approach with guidance, but this only guarantees support up to the signal-to-noise ratio that 𝐺 can perfectly fool 𝐷 under each’s capacity. Discriminator augmentation (DA) [30] is another approach to increase the distribution overlap used by recent GANs [21, 23]. However, we find that the choice of augmentation may implicitly inject inductive biases. For example, prior work finds that affine transforms outperform other distortions on image data [30] because it implicitly encourages 𝐷 to recognize affine transforms as a more acceptable generalization for image generation. The last approach is to simply reload 𝐷 from an earlier checkpoint to reset the pace. Our experiment finds that reloading 𝐷 performs surprisingly well while DiffusionGAN has limited effectiveness. Therefore, we reload 𝐷 when training stalls for the no-guidance setting to avoid introducing any additional biases, and additionally use DA for the guidance setting. 4Experiments 4.1Setups and Training Details Experiment setups. We train on class-conditional ImageNet [70] generation to compare with prior works. We follow the protocols to resize the images to 256 × 256 resolution and augment with horizontal flips. We use pre-trained variational autoencoder (VAE)1 [68] and train the models in 32 × 32 × 4 latent space. Evaluations use Fréchet Inception Distance on 50k class-balanced samples (FID-50k) [18] against the entire train set. Training details. We use a learning rate of 1 × 10 − 4 and a batch size of 256 consistent with prior works [62, 12, 14]. We use AdamW [51] optimizer with 𝛽 1 = 0 , 𝛽 2 = 0.9  [47]. We set weight decay to 0.01 and EMA decay to 0.9999. We follow MeanFlow’s definition of model sizes: B/2, M/2, L/2, XL/2, where 2 denotes the patch size. 𝐺 and 𝐷 always use models of the same size. We use separate dataloaders for 𝐺 and 𝐷 . Epochs are measured as the number of images seen by 𝐺 . Since different models reach peak FID at different epochs, we report the earliest epoch that reaches the best FID. Additional training details are in the appendix. Classifier guidance. We train the model without guidance till the best FID, and then continue training it with guidance. Our classifier is trained from scratch on ImageNet using the cross-entropy objective for 30 epochs. It uses the same B/2 transformer in the latent space. We do not amortize the scale of CG into models for a fair comparison with prior works. Extra-deep models. We only repeat depth on 𝐺 while keeping 𝐷 normal depth. The learning rate of 𝐺 is reduced by the factor of repetition. Extra-deep models are trained end-to-end from scratch following the single-step objective. 4.2Ablation Studies on the Hyperparameters The effect of optimal transport loss. Table 1 shows a grid search of the optimal initial 𝜆 𝑜 ​ 𝑡 and 𝜆 𝑔 ​ 𝑝 . Without OT loss, the training diverges regardless of 𝜆 𝑔 ​ 𝑝 , proving the importance of the OT objective on stabilizing adversarial training for transformers. We also observe the phenomenon that an overly small 𝜆 𝑜 ​ 𝑡 fails to regularize the model and an overly large 𝜆 𝑜 ​ 𝑡 hurts distribution matching. Table 2 shows that decaying 𝜆 𝑜 ​ 𝑡 over time is critical for reaching the best FID. FID ↓ 𝜆 ot 0 0.1 \cellcolorgraycellcolor0.2 0.5 𝜆 gp 0.1 178.22 ± 4 60.20 ± 8 70.15 ± 8 178.00 ± 12 \cellcolorgraycellcolor0.25 174.93 ± 2 54.92 ± 2 \cellcolorgraycellcolor53.90 ± 1 157.06 ± 62 0.5 171.81 ± 6 73.85 ± 11 57.51 ± 1 62.38 ± 6 Table 1:Initial 𝜆 ot and 𝜆 gp . B/2 model 1NFE. FID-50k measured on 20 epochs. Average of two runs with variation labeled in gray. Without optimal transport loss, training diverges. 𝜆 ot Decay FID ↓ 0.2 Constant 29.4 \cellcolorgraycellcolor0.2 → 0.01 \cellcolorgraycellcolor\cellcolorgraycellcolorCosine decay over 100 epochs \cellcolorgraycellcolor8.51 Table 2:Decay of 𝜆 ot . B/2 model 1NFE. FID-50k measured on 100 epochs. Decay is critical for achieving peak performance. The effect of flow-based classifier guidance. Table 3 shows that flow-based CG has a minor improvement over CG only at 𝑡 ′ = 0 . The optimal range of 𝒰 ​ ( 0 , 0.1 ) is much smaller than typical flow matching, likely because adversarial models already produce good samples without guidance. FID ↓ 𝜆 cg 0.002 \cellcolorgraycellcolor0.003 0.005 𝑡 ′ 0 2.48 2.40 2.47 \cellcolorgraycellcolor 𝒰 ​ ( 0 , 0.1 ) 2.47 \cellcolorgraycellcolor2.36 2.49 𝒰 ​ ( 0 , 0.2 ) 2.52 2.42 2.48 𝒰 ​ ( 0 , 0.5 ) 2.46 2.45 2.50 Table 3:Classifier guidance scales and timesteps. XL/2 model 1NFE trained till best FID. Flow-based CG is better. 4.3Comparisons to the State of the Art Single-step with guidance. Footnote 5 compares single-step generation with guidance against state-of-the-art consistency-based and GAN models. Under the exact latent space and architectures, denoted by (•), our method achieves new best FIDs with large improvement margins across all model sizes, even when compared to concurrent works [101, 23, 91]. Notably, our B/2 model surpasses many XL/2 consistency-based models, likely because B/2 models are tight in capacity, and our method does not waste capacity on other timesteps. Our XL/2 model reaches a new best FID of 2.38. Table 8 shows the effect of guidance, and our method with only CG is still the best. Note that StyleGAN-XL has a slightly better FID while being smaller, but it operates in the pixel space, while ours is restricted by the VAE and DiT patch size 2. We mainly compare our method against others with the same settings. Few-step with guidance. Footnote 5 shows that our model also achieves better FIDs in the few-step setting. Ours are trained on designated timesteps to conserve capacity. We find that any-step training performs worse due to the dilution of capacity and batch size. This is also observed in our 1D experiments (Fig. 1) where adversarial flow models need a larger batch size and converge more slowly for any-step training. This is often not a real limitation in practice, as achieving the best performance in a designated few-step setting is the priority. No-guidance generation. Footnote 5 shows that without guidance, even our 1NFE and 2NFE models outperform flow matching using 250NFE+ in the same latent space setting denoted by (•). We leave the exploration of combining adversarial flow models with representational methods to future work. However, we are concerned that current representational methods may have an unfair advantage. Since ImageNet is a small dataset where each class only contains about 1000 images, models cannot learn the correct generalizations from limited data on classes with large variations. However, DINO [5, 60] is trained on a much larger dataset, and using it may leak prior knowledge. Extra-deep models. Table 7 shows that our 56-layer and 112-layer models achieve better FIDs of 2.08 and 1.94, surpassing the 2 and 4-step counterparts. This confirms our hypothesis in Sec. 3.8. The results reveal an important insight: the quality of single-step generation may not be bounded by the training method, but by the depth of the generator. Depth scaling is a worthy direction for future research. Computation efficiency. Both consistency and adversarial methods require multiple forward passes per iteration, except adversarial methods commonly use different samples for independent calculations of the expectation of 𝐺 and 𝐷 losses. Counting epochs by 𝐺 is a fair comparison of 𝐺 ’s update iterations against consistency methods. We provide a breakdown of per-update compute in Sec. C.2. Our XL/2 1NFE model takes 1.88 × training compute compared to AlphaFlow but obtains 15% gain in best FID. The additional compute comes from losses and regularizations on 𝐷 , which can be improved in future work. Method Param Epoch Guidance NFE FID ↓ Consistency-based methods • iCT-XL/2 [79] 675M - None 1 34.24 • Shortcut-XL/2 [12] 675M 250 CFG [19] 1 10.60 • MeanFlow-B/2 [14] 131M 240 CFG [19] 1 6.17 • AlphaFlow-B/2 [101] † 131M 240 CFG [19] 1 5.40 • MeanFlow-M/2 [14] 308M 240 CFG [19] 1 5.01 • MeanFlow-L/2 [14] 459M 240 CFG [19] 1 3.84 • MeanFlow-XL/2 [14] 676M 240 CFG [19] 1 3.43 ◦ TiM-XL/2 [91] † 664M 300 CFG [19] 1 3.26 • AlphaFlow-XL/2 [101] † 676M 240 CFG [19] 1 2.81 GANs BigGAN [4] 112M - cGAN [58] 1 6.95 GigaGAN [26] 569M 480 Match-loss [26] 1 3.45 ◦ GAT-XL/2+REPA [23] † 602M 40 DA + cGAN 1 2.96 StyleGAN-XL [73] 166M - CG + cGAN 1 2.30 Adversarial flow (Ours) • AF-B/2 130M 200 CG[9]+DA[30] 1 3.05 • AF-M/2 306M 120 CG[9]+DA[30] 1 2.82 • AF-L/2 457M 120 CG[9]+DA[30] 1 2.63 • AF-XL/2 673M 125 CG[9]+DA[30] 1 2.38 Table 4:Single-step generation on ImageNet 256px.1245 Method Param Epoch Guidance NFE FID ↓ Consistency-based methods • iCT-XL/2 [79] 675M - None 2 20.30 • Shortcut-XL/2 [12] 675M 250 CFG 4 7.80 • IMM-XL/2 [107] 675M - CFG 1 × 2 7.77 • IMM-XL/2 [107] 675M - CFG 2 × 2 3.99 ◦ TiM-XL/2 [91] † 664M 300 CFG 2 × 2 3.61 • MeanFlow-XL/2 [14] 676M 240 CFG 2 2.93 • MeanFlow-XL/2 [14] 676M 1000 CFG 2 2.20 • AlphaFlow-XL/2 [101] † 676M 240 CFG 2 2.16 Adversarial flow (Ours) • AF-XL/2 675M 95 CG + DA 2 2.11 • AF-XL/2 675M 145 CG + DA 4 2.02 Table 5:Few-step generation on ImageNet 256px.1245 Method Param Epoch Guidance NFE FID ↓ Flow-matching and diffusion ADM [9] 554M 400 None 250 10.94 • DiT-XL/2 [62] 675M 1400 None 250 9.62 • SiT-XL/2 [56] 675M 1400 None 250 8.30 • SiT-XL/2+Disperse [89] 675M 1200 None 500 7.43 • SiT-XL/2+REPA [99] 675M 800 None 250 5.90 ℛ RAE-XL [103] 676M 800 None 250 1.87 ℛ SiT-XL/2+REPA-E [41] 675M 800 None 250 1.69 Autoregressive and masking MaskGIT [79] 227M 300 None 8 6.18 VAR [84] 310M 350 None 10 4.95 Consistency-based methods • iCT-XL/2 [79] 675M - None 1 34.24 ◦ TiM-XL/2 [91] 664M 300 None 1 7.11 Adversarial flow (Ours) • AF-B/2 130M 170 None 1 6.07 • AF-M/2 306M 110 None 1 5.21 • AF-L/2 457M 110 None 1 4.36 • AF-XL/2 673M 120 None 1 3.98 • AF-XL/2 675M 90 None 2 2.36 Table 6:No-guidance generation on ImageNet 256px.1235 Method Depth Param Epoch Guidance NFE FID ↓ AF-XL/2 28 (1 × ) 675M 95 CG + DA 2 2.11 AF-XL/2 56 (2 × ) 675M 95 CG + DA 1 2.08 AF-XL/2 28 (1 × ) 675M 145 CG + DA 4 2.02 AF-XL/2 112 (4 × ) 675M 120 CG + DA 1 1.94 Table 7:Deep architectures on ImageNet 256px. Method Param Epoch Guidance NFE FID ↓ AF-XL/2 673M 120 None 1 3.98 AF-XL/2 673M 125 DA 1 3.86 AF-XL/2 673M 125 CG 1 2.54 AF-XL/2 673M 125 CG + DA 1 2.38 Table 8:Guidance type comparison on ImageNet 256px. Additional content. In the appendix, we discuss that our use of CG does not cause data leakage [39]. We also share more details, results, and visualizations. Limitations. Techniques in Section 3.9 can be improved with more principled methods. Current adversarial training uses more compute per iteration. 𝐷 network increases memory consumption. We use CG instead of CFG. We aim to address these in future work. 5Conclusion Our work proposes a framework to unify adversarial and flow modeling. We show that learning a deterministic transport greatly stabilizes training. We propose techniques to normalize gradients and incorporate guidance. Our method achieves new best FIDs and demonstrates training on deep architectures. Despite limitations, this offers exciting prospects for future research. Acknowledgement We thank Chaorui Deng, Peng Wang, and Qing Yan for their valuable discussions on the methodology and manuscript preparation. We also thank Yang Zhao and Qi Zhao for their support in developing the dataloading and evaluation infrastructure. References Arjovsky et al. [2017] ↑ Martin Arjovsky, Soumith Chintala, and Léon Bottou.Wasserstein generative adversarial networks.In International conference on machine learning, pages 214–223. PMLR, 2017. Ba et al. [2016] ↑ Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton.Layer normalization.arXiv preprint arXiv:1607.06450, 2016. Baldridge et al. [2024] ↑ Jason Baldridge, Jakob Bauer, Mukul Bhutani, Nicole Brichtova, Andrew Bunner, Lluis Castrejon, Kelvin Chan, Yichang Chen, Sander Dieleman, Yuqing Du, et al.Imagen 3.arXiv preprint arXiv:2408.07009, 2024. Brock et al. [2018] ↑ Andrew Brock, Jeff Donahue, and Karen Simonyan.Large scale gan training for high fidelity natural image synthesis.arXiv preprint arXiv:1809.11096, 2018. Caron et al. [2021] ↑ Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin.Emerging properties in self-supervised vision transformers.In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650–9660, 2021. Chen et al. [2025] ↑ Junsong Chen, Shuchen Xue, Yuyang Zhao, Jincheng Yu, Sayak Paul, Junyu Chen, Han Cai, Song Han, and Enze Xie.Sana-sprint: One-step diffusion with continuous-time consistency distillation.arXiv preprint arXiv:2503.09641, 2025. Daskalakis et al. [2017] ↑ Constantinos Daskalakis, Andrew Ilyas, Vasilis Syrgkanis, and Haoyang Zeng.Training gans with optimism.arXiv preprint arXiv:1711.00141, 2017. Dehghani et al. [2018] ↑ Mostafa Dehghani, Stephan Gouws, Oriol Vinyals, Jakob Uszkoreit, and Łukasz Kaiser.Universal transformers.arXiv preprint arXiv:1807.03819, 2018. Dhariwal and Nichol [2021] ↑ Prafulla Dhariwal and Alexander Nichol.Diffusion models beat gans on image synthesis.Advances in neural information processing systems, 34:8780–8794, 2021. Esser et al. [2021] ↑ Patrick Esser, Robin Rombach, and Bjorn Ommer.Taming transformers for high-resolution image synthesis.In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021. Esser et al. [2024] ↑ Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al.Scaling rectified flow transformers for high-resolution image synthesis.In Forty-first international conference on machine learning, 2024. Frans et al. [2024] ↑ Kevin Frans, Danijar Hafner, Sergey Levine, and Pieter Abbeel.One step diffusion via shortcut models.arXiv preprint arXiv:2410.12557, 2024. Gao et al. [2025] ↑ Yu Gao, Haoyuan Guo, Tuyen Hoang, Weilin Huang, Lu Jiang, Fangyuan Kong, Huixia Li, Jiashi Li, Liang Li, Xiaojie Li, et al.Seedance 1.0: Exploring the boundaries of video generation models.arXiv preprint arXiv:2506.09113, 2025. Geng et al. [2025] ↑ Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He.Mean flows for one-step generative modeling.arXiv preprint arXiv:2505.13447, 2025. Goodfellow et al. [2014] ↑ Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio.Generative adversarial nets.Advances in neural information processing systems, 27, 2014. Gulrajani et al. [2017] ↑ Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville.Improved training of wasserstein gans.Advances in neural information processing systems, 30, 2017. Guo and Schwing [2025] ↑ Pengsheng Guo and Alexander G Schwing.Variational rectified flow matching.arXiv preprint arXiv:2502.09616, 2025. Heusel et al. [2017] ↑ Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter.Gans trained by a two time-scale update rule converge to a local nash equilibrium.Advances in neural information processing systems, 30, 2017. Ho and Salimans [2022] ↑ Jonathan Ho and Tim Salimans.Classifier-free diffusion guidance.arXiv preprint arXiv:2207.12598, 2022. Ho et al. [2020] ↑ Jonathan Ho, Ajay Jain, and Pieter Abbeel.Denoising diffusion probabilistic models.Advances in neural information processing systems, 33:6840–6851, 2020. Huang et al. [2024] ↑ Nick Huang, Aaron Gokaslan, Volodymyr Kuleshov, and James Tompkin.The gan is dead; long live the gan! a modern gan baseline.Advances in Neural Information Processing Systems, 37:44177–44215, 2024. Hudson and Zitnick [2021] ↑ Drew A Hudson and Larry Zitnick.Generative adversarial transformers.In International conference on machine learning, pages 4487–4499. PMLR, 2021. Hyun et al. [2025] ↑ Sangeek Hyun, MinKyu Lee, and Jae-Pil Heo.Scalable gans with transformers.arXiv preprint arXiv:2509.24935, 2025. Jiang et al. [2021] ↑ Yifan Jiang, Shiyu Chang, and Zhangyang Wang.Transgan: Two pure transformers can make one strong gan, and that can scale up.Advances in Neural Information Processing Systems, 34:14745–14758, 2021. Jolicoeur-Martineau [2018] ↑ A Jolicoeur-Martineau.The relativistic discriminator: A key element missing from standard gan.arXiv preprint arXiv:1807.00734, 2018. Kang et al. [2023] ↑ Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, and Taesung Park.Scaling up gans for text-to-image synthesis.In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10124–10134, 2023. Kang et al. [2024] ↑ Minguk Kang, Richard Zhang, Connelly Barnes, Sylvain Paris, Suha Kwak, Jaesik Park, Eli Shechtman, Jun-Yan Zhu, and Taesung Park.Distilling diffusion models into conditional gans.In European Conference on Computer Vision, pages 428–447. Springer, 2024. Karras et al. [2017] ↑ Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen.Progressive growing of gans for improved quality, stability, and variation.arXiv preprint arXiv:1710.10196, 2017. Karras et al. [2019] ↑ Tero Karras, Samuli Laine, and Timo Aila.A style-based generator architecture for generative adversarial networks.In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4401–4410, 2019. Karras et al. [2020a] ↑ Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila.Training generative adversarial networks with limited data.Advances in neural information processing systems, 33:12104–12114, 2020a. Karras et al. [2020b] ↑ Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila.Analyzing and improving the image quality of stylegan.In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8110–8119, 2020b. Karras et al. [2021] ↑ Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila.Alias-free generative adversarial networks.Advances in neural information processing systems, 34:852–863, 2021. Karras et al. [2024] ↑ Tero Karras, Miika Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine.Guiding a diffusion model with a bad version of itself.Advances in Neural Information Processing Systems, 37:52996–53021, 2024. Kim et al. [2022] ↑ Dongjun Kim, Yeongmin Kim, Se Jung Kwon, Wanmo Kang, and Il-Chul Moon.Refining generative process with discriminator guidance in score-based diffusion models.arXiv preprint arXiv:2211.17091, 2022. Kim et al. [2021] ↑ Hyunjik Kim, George Papamakarios, and Andriy Mnih.The lipschitz constant of self-attention.In International Conference on Machine Learning, pages 5562–5571. PMLR, 2021. Kingma [2014] ↑ Diederik P Kingma.Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014. Kohler et al. [2024] ↑ Jonas Kohler, Albert Pumarola, Edgar Schönfeld, Artsiom Sanakoyeu, Roshan Sumbaly, Peter Vajda, and Ali Thabet.Imagine flash: Accelerating emu diffusion models with backward distillation.arXiv preprint arXiv:2405.05224, 2024. Kong et al. [2024] ↑ Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al.Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024. Kynkäänniemi et al. [2022] ↑ Tuomas Kynkäänniemi, Tero Karras, Miika Aittala, Timo Aila, and Jaakko Lehtinen.The role of imagenet classes in fr \ ’echet inception distance.arXiv preprint arXiv:2203.06026, 2022. Lee et al. [2021] ↑ Kwonjoon Lee, Huiwen Chang, Lu Jiang, Han Zhang, Zhuowen Tu, and Ce Liu.Vitgan: Training gans with vision transformers.arXiv preprint arXiv:2107.04589, 2021. Leng et al. [2025] ↑ Xingjian Leng, Jaskirat Singh, Yunzhong Hou, Zhenchang Xing, Saining Xie, and Liang Zheng.Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers.arXiv preprint arXiv:2504.10483, 2025. Li et al. [2024] ↑ Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He.Autoregressive image generation without vector quantization.Advances in Neural Information Processing Systems, 37:56424–56445, 2024. Lin and Yang [2023] ↑ Shanchuan Lin and Xiao Yang.Diffusion model with perceptual loss.arXiv preprint arXiv:2401.00110, 2023. Lin and Yang [2024] ↑ Shanchuan Lin and Xiao Yang.Animatediff-lightning: Cross-model diffusion distillation.arXiv preprint arXiv:2403.12706, 2024. Lin et al. [2024a] ↑ Shanchuan Lin, Bingchen Liu, Jiashi Li, and Xiao Yang.Common diffusion noise schedules and sample steps are flawed.In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 5404–5411, 2024a. Lin et al. [2024b] ↑ Shanchuan Lin, Anran Wang, and Xiao Yang.Sdxl-lightning: Progressive adversarial diffusion distillation.arXiv preprint arXiv:2402.13929, 2024b. Lin et al. [2025a] ↑ Shanchuan Lin, Xin Xia, Yuxi Ren, Ceyuan Yang, Xuefeng Xiao, and Lu Jiang.Diffusion adversarial post-training for one-step video generation.arXiv preprint arXiv:2501.08316, 2025a. Lin et al. [2025b] ↑ Shanchuan Lin, Ceyuan Yang, Hao He, Jianwen Jiang, Yuxi Ren, Xin Xia, Yang Zhao, Xuefeng Xiao, and Lu Jiang.Autoregressive adversarial post-training for real-time interactive video generation.arXiv preprint arXiv:2506.09350, 2025b. Lipman et al. [2022] ↑ Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le.Flow matching for generative modeling.arXiv preprint arXiv:2210.02747, 2022. Liu et al. [2022] ↑ Xingchao Liu, Chengyue Gong, and Qiang Liu.Flow straight and fast: Learning to generate and transfer data with rectified flow.arXiv preprint arXiv:2209.03003, 2022. Loshchilov and Hutter [2017] ↑ Ilya Loshchilov and Frank Hutter.Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017. Lu and Song [2024] ↑ Cheng Lu and Yang Song.Simplifying, stabilizing and scaling continuous-time consistency models.arXiv preprint arXiv:2410.11081, 2024. Lu et al. [2025] ↑ Yanzuo Lu, Yuxi Ren, Xin Xia, Shanchuan Lin, Xing Wang, Xuefeng Xiao, Andy J Ma, Xiaohua Xie, and Jian-Huang Lai.Adversarial distribution matching for diffusion distillation towards efficient image and video synthesis.arXiv preprint arXiv:2507.18569, 2025. Luo et al. [2023a] ↑ Simian Luo, Yiqin Tan, Longbo Huang, Jian Li, and Hang Zhao.Latent consistency models: Synthesizing high-resolution images with few-step inference.arXiv preprint arXiv:2310.04378, 2023a. Luo et al. [2023b] ↑ Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang.Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models.Advances in Neural Information Processing Systems, 36:76525–76546, 2023b. Ma et al. [2024] ↑ Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie.Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers.In European Conference on Computer Vision, pages 23–40. Springer, 2024. Mescheder et al. [2018] ↑ Lars Mescheder, Andreas Geiger, and Sebastian Nowozin.Which training methods for gans do actually converge?In International conference on machine learning, pages 3481–3490. PMLR, 2018. Miyato and Koyama [2018] ↑ Takeru Miyato and Masanori Koyama.cgans with projection discriminator.arXiv preprint arXiv:1802.05637, 2018. Miyato et al. [2018] ↑ Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida.Spectral normalization for generative adversarial networks.arXiv preprint arXiv:1802.05957, 2018. Oquab et al. [2023] ↑ Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al.Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023. Paszke et al. [2019] ↑ Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.Pytorch: An imperative style, high-performance deep learning library.Advances in neural information processing systems, 32, 2019. Peebles and Xie [2023] ↑ William Peebles and Saining Xie.Scalable diffusion models with transformers.In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195–4205, 2023. Polyak et al. [2024] ↑ Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yao Ma, Ching-Yao Chuang, et al.Movie gen: A cast of media foundation models.arXiv preprint arXiv:2410.13720, 2024. Radford et al. [2021] ↑ Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.Learning transferable visual models from natural language supervision.In International conference on machine learning, pages 8748–8763. PmLR, 2021. Reed et al. [2016] ↑ Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee.Generative adversarial text to image synthesis.In International conference on machine learning, pages 1060–1069. Pmlr, 2016. Ren et al. [2024] ↑ Yuxi Ren, Xin Xia, Yanzuo Lu, Jiacheng Zhang, Jie Wu, Pan Xie, Xing Wang, and Xuefeng Xiao.Hyper-sd: Trajectory segmented consistency model for efficient image synthesis.Advances in Neural Information Processing Systems, 37:117340–117362, 2024. Riba et al. [2020] ↑ Edgar Riba, Dmytro Mishkin, Daniel Ponsa, Ethan Rublee, and Gary Bradski.Kornia: an open source differentiable computer vision library for pytorch.In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 3674–3683, 2020. Rombach et al. [2022] ↑ Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.High-resolution image synthesis with latent diffusion models.In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. Roth et al. [2017] ↑ Kevin Roth, Aurelien Lucchi, Sebastian Nowozin, and Thomas Hofmann.Stabilizing training of generative adversarial networks through regularization.Advances in neural information processing systems, 30, 2017. Russakovsky et al. [2015] ↑ Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al.Imagenet large scale visual recognition challenge.International journal of computer vision, 115(3):211–252, 2015. Salimans and Ho [2022] ↑ Tim Salimans and Jonathan Ho.Progressive distillation for fast sampling of diffusion models.arXiv preprint arXiv:2202.00512, 2022. Salimans et al. [2024] ↑ Tim Salimans, Thomas Mensink, Jonathan Heek, and Emiel Hoogeboom.Multistep distillation of diffusion models via moment matching.Advances in Neural Information Processing Systems, 37:36046–36070, 2024. Sauer et al. [2022] ↑ Axel Sauer, Katja Schwarz, and Andreas Geiger.Stylegan-xl: Scaling stylegan to large diverse datasets.In ACM SIGGRAPH 2022 conference proceedings, pages 1–10, 2022. Sauer et al. [2024a] ↑ Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach.Fast high-resolution image synthesis with latent adversarial diffusion distillation.In SIGGRAPH Asia 2024 Conference Papers, pages 1–11, 2024a. Sauer et al. [2024b] ↑ Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach.Adversarial diffusion distillation.In European Conference on Computer Vision, pages 87–103. Springer, 2024b. Seawead et al. [2025] ↑ Team Seawead, Ceyuan Yang, Zhijie Lin, Yang Zhao, Shanchuan Lin, Zhibei Ma, Haoyuan Guo, Hao Chen, Lu Qi, Sen Wang, et al.Seaweed-7b: Cost-effective training of video generation foundation model.arXiv preprint arXiv:2504.08685, 2025. Seedream et al. [2025] ↑ Team Seedream, Yunpeng Chen, Yu Gao, Lixue Gong, Meng Guo, Qiushan Guo, Zhiyao Guo, Xiaoxia Hou, Weilin Huang, Yixuan Huang, et al.Seedream 4.0: Toward next-generation multimodal image generation.arXiv preprint arXiv:2509.20427, 2025. Shi et al. [2023] ↑ Yuyang Shi, Valentin De Bortoli, Andrew Campbell, and Arnaud Doucet.Diffusion schrödinger bridge matching.Advances in Neural Information Processing Systems, 36:62183–62223, 2023. Song and Dhariwal [2023] ↑ Yang Song and Prafulla Dhariwal.Improved techniques for training consistency models.arXiv preprint arXiv:2310.14189, 2023. Song et al. [2020] ↑ Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020. Song et al. [2023] ↑ Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever.Consistency models.In Proceedings of the 40th International Conference on Machine Learning, pages 32211–32252, 2023. Sun et al. [2020] ↑ Ruoyu Sun, Tiantian Fang, and Alexander Schwing.Towards a better global loss landscape of gans.Advances in Neural Information Processing Systems, 33:10186–10198, 2020. Szegedy et al. [2016] ↑ Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna.Rethinking the inception architecture for computer vision.In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2818–2826, 2016. Tian et al. [2024] ↑ Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang.Visual autoregressive modeling: Scalable image generation via next-scale prediction.Advances in neural information processing systems, 37:84839–84865, 2024. Vaswani et al. [2017] ↑ Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.Attention is all you need.Advances in neural information processing systems, 30, 2017. Wan et al. [2025] ↑ Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al.Wan: Open and advanced large-scale video generative models.arXiv preprint arXiv:2503.20314, 2025. Wang et al. [2024] ↑ Fu-Yun Wang, Zhaoyang Huang, Alexander Bergman, Dazhong Shen, Peng Gao, Michael Lingelbach, Keqiang Sun, Weikang Bian, Guanglu Song, Yu Liu, et al.Phased consistency models.Advances in neural information processing systems, 37:83951–84009, 2024. Wang et al. [2025a] ↑ Jianyi Wang, Shanchuan Lin, Zhijie Lin, Yuxi Ren, Meng Wei, Zongsheng Yue, Shangchen Zhou, Hao Chen, Yang Zhao, Ceyuan Yang, et al.Seedvr2: One-step video restoration via diffusion adversarial post-training.arXiv preprint arXiv:2506.05301, 2025a. Wang and He [2025] ↑ Runqian Wang and Kaiming He.Diffuse and disperse: Image generation with representation regularization.arXiv preprint arXiv:2506.09027, 2025. Wang et al. [2022] ↑ Zhendong Wang, Huangjie Zheng, Pengcheng He, Weizhu Chen, and Mingyuan Zhou.Diffusion-gan: Training gans with diffusion.arXiv preprint arXiv:2206.02262, 2022. Wang et al. [2025b] ↑ Zidong Wang, Yiyuan Zhang, Xiaoyu Yue, Xiangyu Yue, Yangguang Li, Wanli Ouyang, and Lei Bai.Transition models: Rethinking the generative learning objective.arXiv preprint arXiv:2509.04394, 2025b. Xu et al. [2024] ↑ Yanwu Xu, Yang Zhao, Zhisheng Xiao, and Tingbo Hou.Ufogen: You forward once large scale text-to-image generation via diffusion gans.In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8196–8206, 2024. Xu et al. [2025] ↑ Yanbo Xu, Yu Wu, Sungjae Park, Zhizhuo Zhou, and Shubham Tulsiani.Temporal score rescaling for temperature sampling in diffusion and flow models.arXiv preprint arXiv:2510.01184, 2025. Yan et al. [2024] ↑ Hanshu Yan, Xingchao Liu, Jiachun Pan, Jun Hao Liew, Qiang Liu, and Jiashi Feng.Perflow: Piecewise rectified flow as universal plug-and-play accelerator.Advances in Neural Information Processing Systems, 37:78630–78652, 2024. Yazıcı et al. [2018] ↑ Yasin Yazıcı, Chuan-Sheng Foo, Stefan Winkler, Kim-Hui Yap, Georgios Piliouras, and Vijay Chandrasekhar.The unusual effectiveness of averaging in gan training.arXiv preprint arXiv:1806.04498, 2018. Yin et al. [2024a] ↑ Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and Bill Freeman.Improved distribution matching distillation for fast image synthesis.Advances in neural information processing systems, 37:47455–47487, 2024a. Yin et al. [2024b] ↑ Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Fredo Durand, William T Freeman, and Taesung Park.One-step diffusion with distribution matching distillation.In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6613–6623, 2024b. Yu et al. [2015] ↑ Fisher Yu, Ari Seff, Yinda Zhang, Shuran Song, Thomas Funkhouser, and Jianxiong Xiao.Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop.arXiv preprint arXiv:1506.03365, 2015. Yu et al. [2024] ↑ Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie.Representation alignment for generation: Training diffusion transformers is easier than you think.arXiv preprint arXiv:2410.06940, 2024. Zhang et al. [2017] ↑ Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris N Metaxas.Stackgan: Text to photo-realistic image synthesis with stacked generative adversarial networks.In Proceedings of the IEEE international conference on computer vision, pages 5907–5915, 2017. Zhang et al. [2025] ↑ Huijie Zhang, Aliaksandr Siarohin, Willi Menapace, Michael Vasilkovsky, Sergey Tulyakov, Qing Qu, and Ivan Skorokhodov.Alphaflow: Understanding and improving meanflow models.arXiv preprint arXiv:2510.20771, 2025. Zhang et al. [2018] ↑ Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang.The unreasonable effectiveness of deep features as a perceptual metric.In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 586–595, 2018. Zheng et al. [2025a] ↑ Boyang Zheng, Nanye Ma, Shengbang Tong, and Saining Xie.Diffusion transformers with representation autoencoders.arXiv preprint arXiv:2510.11690, 2025a. Zheng et al. [2024] ↑ Kaiwen Zheng, Guande He, Jianfei Chen, Fan Bao, and Jun Zhu.Diffusion bridge implicit models.arXiv preprint arXiv:2405.15885, 2024. Zheng et al. [2025b] ↑ Kaiwen Zheng, Yuji Wang, Qianli Ma, Huayu Chen, Jintao Zhang, Yogesh Balaji, Jianfei Chen, Ming-Yu Liu, Jun Zhu, and Qinsheng Zhang.Large scale diffusion distillation via score-regularized continuous-time consistency.arXiv preprint arXiv:2510.08431, 2025b. Zhou et al. [2023] ↑ Linqi Zhou, Aaron Lou, Samar Khanna, and Stefano Ermon.Denoising diffusion bridge models.arXiv preprint arXiv:2309.16948, 2023. Zhou et al. [2025] ↑ Linqi Zhou, Stefano Ermon, and Jiaming Song.Inductive moment matching.arXiv preprint arXiv:2503.07565, 2025. Zhu et al. [2025] ↑ Jiapeng Zhu, Ceyuan Yang, Kecheng Zheng, Yinghao Xu, Zifan Shi, Yifei Zhang, Qifeng Chen, and Yujun Shen.Exploring sparse moe in gans for text-conditioned image synthesis.In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 18411–18423, 2025. Method Space Param Epoch Guidance NFE FID ↓ sFID ↓ IS ↑ Prec. ↑ Rec. ↑ Flow-matching and diffusion ADM [9] Pixel 554M 400 None 250 10.94 6.02 100.98 0.69 0.63 • DiT-XL/2 [62] LDM [68] 675M 1400 None 250 9.62 6.85 121.50 0.67 0.67 • SiT-XL/2 [56] LDM [68] 675M 1400 None 250 8.30 - - - - • SiT-XL/2+Disperse [89] LDM [68] 675M 1200 None 500 7.43 - - - - • SiT-XL/2+REPA [99] LDM [68] 675M 800 None 250 5.90 - - - - RAE-XL [103] DINOv2 [60] 676M 800 None 250 1.87 - 209.70 0.80 0.63 SiT-XL/2+REPA-E [41] Joint-trained 675M 800 None 250 1.69 4.17 219.30 0.77 0.67 ADM-G [9] Pixel 554M 400 CG [9] 250 × 2 4.59 5.25 186.70 0.82 0.52 • DiT-XL/2 [62] LDM [68] 675M 1400 CFG [19] 250 × 2 2.27 4.60 278.24 0.83 0.57 • SiT-XL/2 [56] LDM [68] 675M 1400 CFG [19] 250 × 2 2.06 4.49 277.50 0.83 0.59 • SiT-XL/2+Disperse [89] LDM [68] 675M 1200 CFG [19] 500 × 2 1.97 - - - - • SiT-XL/2+REPA [99] LDM [68] 675M 800 CFG [19] 250 × 2 1.42 4.70 305.70 0.80 0.65 RAE-XL [103] DINOv2 [60] 676M 800 AG [33] 250 × 2 1.41 - 309.40 0.80 0.63 SiT-XL/2+REPA-E [41] Joint-trained 675M 800 CFG [19] 250 × 2 1.12 4.09 302.90 0.79 0.66 Consistency-based models • Shortcut-XL/2 [12] LDM [68] 675M 250 CFG [19] 1 10.60 - - - - TiM-XL/2 [91] 664M 300 None 1 7.11 4.97 140.39 0.71 0.63 • MeanFlow-B/2 [14] 131M 240 CFG [19] 1 6.17 - - - - • AlphaFlow-B/2 [101] 131M 240 CFG [19] 1 5.40 - - - - • MeanFlow-M/2 [14] 308M 240 CFG [19] 1 5.01 - - - - • MeanFlow-L/2 [14] 459M 240 CFG [19] 1 3.84 - - - - • MeanFlow-XL/2 [14] 676M 240 CFG [19] 1 3.43 - - - - TiM-XL/2 [91] 664M 300 CFG [19] 1 3.26 4.37 210.33 0.75 0.59 • AlphaFlow-XL/2 [101] 676M 240 CFG [19] 1 2.81 - - - - • Shortcut-XL/2 [12] LDM [68] 675M 250 CFG [19] 4 7.80 - - - - • IMM-XL/2 [107] 675M - CFG [19] 1 × 2 7.77 - - - - • IMM-XL/2 [107] 675M - CFG [19] 2 × 2 3.99 - - - - TiM-XL/2 [91] 664M 300 CFG [19] 2 × 2 3.61 6.74 151.79 0.74 0.59 • MeanFlow-XL/2 [14] 676M 1000 CFG [19] 2 2.20 - - - - • AlphaFlow-XL/2 [101] 676M 240 CFG [19] 2 2.16 - - - - Autoregressive, masking, and hybrids MaskGIT [79] VQGAN [10] 227M 300 None 8 6.18 - 182.10 0.80 0.51 VAR [84] MS-VQVAE [84] 310M 350 None 10 4.95 - - - - VAR [84] MS-VQVAE [84] 2.0B 350 CFG [19] 10 × 2 1.92 - 323.10 0.82 0.59 MAR [42] LDM [68] 943M 800 CFG [19] 64 × 100 × 2 1.55 - 303.70 0.81 0.62 GANs BigGAN [4] Pixel 112M - cGAN [58] 1 6.95 7.36 171.40 0.87 0.28 GigaGAN [26] Pixel 569M 480 Match-loss [26] 1 3.45 - 225.52 0.84 0.61 GAT-XL/2+REPA [23] LDM [68] 602M 40 DA[30]+cGAN[58] 1 2.96 - - - - StyleGAN-XL [73] Pixel 166M - CG[9]+cGAN[58] 1 2.30 4.02 265.12 0.78 0.53 Adversarial flow models (Ours) • AF-B/2 LDM [68] 130M 170 None 1 6.07 5.31 169.51 0.72 0.49 • AF-M/2 306M 110 None 1 5.21 5.60 178.48 0.75 0.54 • AF-L/2 457M 110 None 1 4.36 5.39 186.21 0.77 0.53 • AF-XL/2 673M 120 None 1 3.98 5.40 201.85 0.78 0.52 • AF-XL/2 673M 90 None 2 2.36 4.35 235.77 0.81 0.52 • AF-B/2 LDM [68] 130M 200 CG[9]+DA[30] 1 3.05 5.32 269.18 0.81 0.51 • AF-M/2 306M 120 CG[9]+DA[30] 1 2.82 5.20 279.12 0.81 0.50 • AF-L/2 457M 120 CG[9]+DA[30] 1 2.63 5.10 277.96 0.81 0.52 • AF-XL/2 673M 125 CG[9]+DA[30] 1 2.38 4.87 284.18 0.81 0.52 • AF-XL/2 675M 95 CG[9]+DA[30] 2 2.11 4.33 273.84 0.82 0.55 • AF-XL/2 (2 × deep, 56-layer) 675M 95 CG[9]+DA[30] 1 2.08 4.79 298.33 0.79 0.56 • AF-XL/2 675M 145 CG[9]+DA[30] 4 2.03 4.59 259.66 0.78 0.59 • AF-XL/2 (4 × deep, 112-layer) 675M 120 CG[9]+DA[30] 1 1.94 4.54 292.20 0.79 0.56 Table 9:Full metrics on conditional ImageNet 256px generation compared with other notable works. The table only shows methods and their configurations that are the most comparable to ours for comparison. (•) indicates methods using the same latent space and architecture as ours. Please refer to [9] for metric details. Appendix AQualitative Results Qualitative results are provided below. Samples are generated using the same seeds across models. Deterministic transport. Deterministic transport behavior is visible, particularly on bald eagle (class 22, first row) and geyser (class 974, last row), where the sky background (blue or white) is usually consistent on the same seed across models. However, details still vary across models. This is expected because (1) different model sizes have different degrees of generalization, (2) minibatch training has stochasticity, and (3) the OT scale is reduced toward the end of the training. (a)B/2 (FID=3.05, IS=269.18) (b)M/2 (FID=2.82, IS=279.12) (c)L/2 (FID=2.63, IS=277.96) (d)XL/2 (FID=2.38, IS=284.18) Figure 5:Uncurated 1NFE ImageNet-256px generation with guidance. Classes are (22) bald eagle, (29) axolotl, (89) cockatoo, (207) golden retriever, (279) white fox, (992) agaric, (972) cliff, (974) geyser. Same seeds used across models. (a)B/2 (FID=6.07, IS=169.51) (b)M/2 (FID=5.21, IS=178.48) (c)L/2 (FID=4.36, IS=186.21) (d)XL/2 (FID=3.98, IS=201.85) Figure 6:Uncurated 1NFE ImageNet-256px generation without guidance. Classes are (22) bald eagle, (29) axolotl, (89) cockatoo, (207) golden retriever, (279) white fox, (992) agaric, (972) cliff, (974) geyser. Same seeds used across models. (a)XL/2 2NFE (FID=2.11, IS=273.84) (b)XL/2 4NFE (FID=2.03, IS=259.66) Figure 7:Uncurated 2NFE and 4NFE ImageNet-256px generation with guidance. Classes are (22) bald eagle, (29) axolotl, (89) cockatoo, (207) golden retriever, (279) white fox, (992) agaric, (972) cliff, (974) geyser. Same seeds used across models. (a)XL/2 56-layer 1NFE (FID=2.08, IS=298.33) (b)XL/2 112-layer 1NFE (FID=1.94, IS=292.20) Figure 8:Uncurated extra-deep 1NFE ImageNet-256px generation with guidance. Classes are (22) bald eagle, (29) axolotl, (89) cockatoo, (207) golden retriever, (279) white fox, (992) agaric, (972) cliff, (974) geyser. Same seeds used across models. Comparisons with flow-matching models. Figure 9 shows comparisons with SiT [56]. We show that the adversarial objective generates perceptually better-looking samples even without guidance, while the flow matching method generates more perceptually out-of-distribution samples without guidance. (a)Ours XL/2 2NFE No-Guidance (FID=2.36) (b)SiT-XL/2 250NFE No-Guidance (FID=8.30) (c)Ours XL/2 2NFE CG+DA (FID=2.11) (d)SiT-XL/2 500NFE CFG=1.5 (FID=2.06) Figure 9:Uncurated comparisons with SiT [56]. Same seeds. SiT uses Euler sampler. Classes are (207) golden retriever, (360) otter, (387) red panda, (974) geyser, (88) macaw, (979) valley, (417) balloon, (279) white fox, (22) bald eagle, (27) eft, (29) axolotl, (63) Indian cobra, (89) cockatoo, (360) otter, (972) cliff, (992) agaric. Figure 10:Latent interpolation. XL/2 1NFE (FID=2.38, IS=284.18). Uncurated. The figure shows 𝐺 ​ ( 𝑧 𝑡 , 𝑐 ) , where 𝑧 𝑡 = slerp ​ ( 𝑧 1 , 𝑧 2 , 𝑡 ) := cos ⁡ ( 𝜋 2 ​ 𝑡 ) ​ 𝑧 1 + sin ⁡ ( 𝜋 2 ​ 𝑡 ) ​ 𝑧 2 , 𝑡 ∈ [ 0 , 1 ] , 𝑧 1 , 𝑧 2 ∼ 𝒩 ​ ( 0 , 𝐈 ) . Layer visualization. We follow prior works to project every layer’s hidden features onto the top-3 PCA components [23], and through a trained linear projection and decode to images using the VAE [47]. Note that in PCA, because DiT [62] uses absolute positional encoding through input, some PCA of early layers are dominated by sinusoidal encoding. Also, because the PCA of each layer is fitted independently, some layers have a different top-3 order, so the color of the visualization can change abruptly despite the underlying features being similar. Clear imagery is formed in the later layers of the model. Unlike [23], we do not impose any manual supervision losses on the intermediate features, and the models still obtain top FIDs regardless. Notice that for the 112-layer model, a large number of middle layers seem not to be contributing much in the visualization, but they are indeed effective in improving the final FID. The visualizations may not reveal the full contributions of these layers. (a)B/2 (12 layers, FID=3.05) (b)M/2 (16 layers, FID=2.82) (c)L/2 (24 layers, FID=2.63) (d)XL/2 (28 layers, FID=2.38) (e)XL/2-Deep (56 layers, FID=2.08) (f)XL/2-Deep (112 layers, FID=1.94) Figure 11:Layer inspection. Example 1. Class is (22) bald eagle. The top row is hidden features at every layer projected onto the top-3 PCA components [23]. The bottom row is through a trained linear projection layer and decoded by the VAE [47]. (a)B/2 (12 layers, FID=3.05) (b)M/2 (16 layers, FID=2.82) (c)L/2 (24 layers, FID=2.63) (d)XL/2 (28 layers, FID=2.38) (e)XL/2-Deep (56 layers, FID=2.02) (f)XL/2-Deep (112 layers, FID=1.94) Figure 12:Layer inspection. Example 2. Class is (972) cliff. The top row is hidden features at every layer projected onto the top-3 PCA components [23]. The bottom row is through a trained linear projection layer and decoded by the VAE [47]. Appendix BExperimental Details Training. Table 10 provides the details of our architecture and constant hyperparameters. Table 11 lists the training schedule of our models. The hyperparameter patterns are what we find along the training process that produce the best FID. We suppose it can be put on an automatic schedule, but we leave it to future work. When EMA reload is used, it is automatically reloaded after every epoch onward. When 𝐷 reload is used, we manually try a few checkpoints from different epochs and reload only when the training stalls again. Precision. We use TF32 precision to match most prior works. We notice that some prior works [91, 23] train on BF16, but all adopt modified architectures with the addition of QK-normalization and other changes. We also find that QK-normalization is critical for the training stability in BF16 [11]. However, we do not see a major throughput improvement on the 256px latent setting with a patch size of 2. Therefore, we stick to no architecture modification and TF32. Guidance. Flow-based guidance as discussed in Sec. 3.6 is used for 1NFE models. For multi-step models, we find it is sufficient to only apply guidance on selected target timesteps as indicated in the table. We find that multi-step models need a slightly higher guidance scale to get the best FID and are adjusted as such. For the deep models, we keep the exact setting as any other 1NFE models. When training the classifier, we add affine transforms to data augmentation, and it yields better downstream performance. Config NFE Network B/2 M/2 L/2 XL/2 XL/2-Deep Param 1 𝐺 130M 306M 457M 673M 675M 𝐷 129M 304M 455M 671M 671M 2,4 𝐺 - - - 675M - 𝐷 - - - 672M - Depth 𝐺 12 16 24 28 28 × {2,4} 𝐷 12 16 24 28 28 Dim 768 1024 1024 1152 1152 Heads 12 16 16 16 16 Patch size 2 × 2 Activation GeLU MLP expand × 4 Norm Pre-LayerNorm + AdaZero Batch size 256 EMA decay 0.9999 GP scale 𝜆 gp 0.25 GP batch ratio 25% GP approx. 𝜖 0.01 Logit-centering 𝜆 cp 0.01 AdamW weight decay 0.01 AdamW betas (0.0, 0.9) Precision TF32 Data x-flip 0.5 Table 10:Architectures and constant hyperparameters. Guidance 𝜆 cg , 𝑡 ′ 𝜆 ot LR 𝐺 , 𝐷 EMA Reload 𝐷 Reload Epoch FID B/2 None 0.2 → 0.01 1e-4 150 7.30 1NFE None 0.001 8e-5 Yes 155 7.05 None 0.001 3e-5 Yes Yes 170 6.07 0.003, 𝒰 (0,0.1) 0.001 5e-5 Yes Yes 200 3.05 M/2 None 0.2 → 0.005 1e-4 100 6.19 1NFE None 0.001 8e-5 Yes 105 5.54 None 0.001 3e-5 Yes Yes 110 5.21 0.003, 𝒰 (0,0.1) 0.001 5e-5 Yes Yes 120 2.82 L/2 None 0.2 → 0.005 1e-4 85 6.26 1NFE None 0.001 8e-5 Yes 105 5.14 None 0.001 3e-5 Yes Yes 110 4.36 0.003, 𝒰 (0,0.1) 0.001 5e-5 Yes Yes 120 2.63 XL/2 None 0.2 → 0.005 1e-4 90 5.88 1NFE None 0.001 8e-5 Yes 110 4.81 None 0.001 3e-5 Yes Yes 120 3.98 0.003, 𝒰 (0,0.1) 0.001 5e-5 Yes Yes 125 2.38 XL/2 None 0.25 → 0.005 1e-4 75 4.79 2NFE None 0.001 8e-5 Yes 85 4.34 None 0.001 3e-5 Yes Yes 90 2.36 0.02, [0] 0.001 3e-5 Yes Yes 95 2.11 XL/2 None 0.25 → 0.005 1e-4 130 5.28 4NFE None 0.001 8e-5 Yes 135 3.89 None 0.001 3e-5 Yes Yes 140 2.70 0.02, [0,0.25] 0.001 3e-5 Yes Yes 145 2.02 XL/2 None 0.2 → 0.005 5e-5,1e-4 75 4.16 56L None 0.001 4e-5,8e-5 Yes 80 3.41 1NFE None 0.001 1.5e-5,3e-5 Yes Yes 85 2.77 0.003, 𝒰 (0,0.1) 0.001 1e-5,2e-5 Yes Yes 95 2.08 XL/2 None 0.2 → 0.005 2.5e-5,1e-4 90 3.78 112L None 0.001 2e-5,8e-5 Yes 95 3.40 1NFE None 0.001 2.5e-6,1e-5 Yes Yes 100 2.92 0.003, 𝒰 (0,0.1) 0.001 5e-6,2e-5 Yes Yes 120 1.94 Table 11:Training schedules and dynamic hyperparameters. Evaluation. We perform class-balanced evaluation, where we generate 50 images per class for 1000 classes to constitute the total 50k evaluation samples. Recent works [101, 103] find that this approach reduces the stochasticity in the evaluation process and yields more accurate evaluations of the models. Note that class-balanced evaluation may yield a 0.1 FID advantage compared to class-imbalanced counterparts. However, many prior works do not explicitly state the details of their evaluation protocols. We hence report class-balanced FIDs if the work explicitly provides, and otherwise report the original metrics in the work. Our method yields a large-margin improvement on FID, especially surpassing the best consistency-based method AlphaFlow [101], which is evaluated in the same class-balanced setting. The FID and other metrics are computed using the code provided by ADM [9]. The FID is computed against the entire training set using the pre-computed statistics by ADM [9]. Discriminator augmentation. Discriminator augmentation (DA) is used along with classifier guidance (CG). DA is directly performed in the latent space. Let 𝜌 denote the augmentation operation, 𝜙 is the gradient normalization operation, the exact composition for the final adversarial loss is: ℒ adv 𝐷 = 𝔼 𝑧 , 𝑥 , 𝜌 ​ [ 𝑓 ​ ( 𝐷 ​ ( 𝜙 ​ ( 𝜌 ​ ( 𝑥 ) ) ) − 𝐷 ​ ( 𝜙 ​ ( 𝜌 ​ ( 𝐺 ​ ( 𝑧 ) ) ) ) ) ] , (33) ℒ adv 𝐺 = 𝔼 𝑧 , 𝑥 , 𝜌 ​ [ 𝑓 ​ ( 𝐷 ​ ( 𝜙 ​ ( 𝜌 ​ ( 𝐺 ​ ( 𝑧 ) ) ) ) − 𝐷 ​ ( 𝜙 ​ ( 𝜌 ​ ( 𝑥 ) ) ) ) ] . (34) Notice that the same random augmentation 𝜌 must be applied to the real and generated samples as a pair in calculating the expectation of the relativistic loss. We only perform integer translation and cutout with a fixed probability in the latent space. Algorithm 1 shows our implementation using Kornia [67]. Figure 13 shows the visualization of DA. Algorithm 1 Discriminator augmentation. 1Sequential( 2 RandomTranslate( 3 p=0.4, 4 translate_x=(0, 0.3), 5 translate_y=(0, 0.3), 6 resample="NEAREST", 7 ), 8 RandomErasing(p=0.4, scale=(0.1, 0.5)), 9 RandomErasing(p=0.4, scale=(0.1, 0.5)), 10 RandomErasing(p=0.4, scale=(0.1, 0.5)), 11) Figure 13:Discriminator augmentation. The top row is the original samples. The bottom row is what 𝐷 sees after augmentation. The augmentation is performed in the latent space. The visualization is through the VAE decoder. Data leakage. Prior work [39] finds that using a pre-trained ImageNet classifier as 𝐷 backbone when training GANs on FFHQ [29] and LSUN [98] generation tasks cheats the FID evaluation by InceptionV3 [83]. Unlike [21, 23], we do not consider StyleGAN-XL [73] trained on ImageNet fits the data leakage description. Our method further differentiates by only using the classifier for guidance. Since CG is an established approach from ADM [9] and CFG is just CG with an implicit classifier [19], CG should not induce additional advantages. Appendix CLimitations C.1Adversarial Training The need to gradually reduce the scale of the optimal transport objective 𝜆 ot is theoretically justified. The scale cannot reduce too fast, or 𝐺 cannot fully escape local minima. However, the scale also cannot reduce too slowly, or 𝐷 outlearns 𝐺 , and the gradient vanishes when 𝐷 can perfectly separate the real and generated distributions. This is problematic because we would prefer an algorithm that guarantees the eventual convergence of 𝐺 given a long enough 𝜆 ot decay duration, instead of being sensitive to the decay schedule. WGAN. WGAN [1] proposes using a 1-Lipschitz 𝐷 to represent the Wasserstein-1 ( W 1 ) distance between the real and generated distributions, which theoretically guarantees gradients for 𝐺 . WGAN is a special case of the relativistic objective where 𝑓 ​ ( ⋅ ) = − ( ⋅ ) . WGAN-GP [16] proposes adding gradient penalties on linearly interpolated 𝑥 and 𝐺 ​ ( 𝑧 ) samples to limit the Lipschitz constant of 𝐷 . This is theoretically correct for one-dimensional data, but in high-dimensional spaces, linear interpolation only occupies a thin slice of the full space, and thus cannot theoretically guarantee the requirement of WGAN. A stronger approach is to directly limit the Lipschitz constant through 𝐷 ’s architecture. Since in practice, only a 𝐾 -Lipschitz 𝐷 is really needed, which is equivalent to W 1 scaled by a factor of 𝐾 , we experiment using spectral normalization [59] and L2 attention [35] in the transformer architecture of 𝐷 to set an upperbound on the Lipschitz constant of 𝐷 , but we find that this setting is too limiting to the expressiveness of 𝐷 and significantly hinders the learning of 𝐺 . We leave further explorations to future work. DiffusionGAN. An alternative approach is to bridge the gap between the real and the generated distributions. Early approaches add instance noises [57]. DiffusionGAN [90] gives 𝐷 the noise scale as a condition, essentially projecting 𝐷 to a probability flow, similar to our approach with guidance. However, this only guarantees support up to the signal-to-noise ratio that 𝐺 can perfectly fool 𝐷 under each’s capacity constraint. Our preliminary experiments do not find that DiffusionGAN provides benefits. We leave further explorations to future work. Discriminator Augmentation. Non-leaky discriminator augmentation (DA) is an alternative method to increase the overlapping of the distributions [30]. It offers more flexibility to augment samples in the spatial or frequency domains. However, augmentation filters are often modality-specific, limiting the generality. For example, affine transforms can be easily applied in the pixel space, but not in the latent space. Additionally, we suspect that the choice of augmentation filters also injects additional inductive biases that encourage the models to learn toward specific forms of generalization. This can actually be a very useful thing, especially with limited data, because the proper ways of generalization are impossible to deduce from the dataset alone. However, we choose not to use DA in the no-guidance setting. This avoids introducing any additional unfair advantages against flow-matching methods. We only use DA in the guidance setting to achieve the best performance. We only use integer translation and cutout due to the limitation in the latent space. We only use a fixed augmentation probability, though it could be dynamically adjusted [30] for future works. Weaker discriminator. We try gradually increasing dropout in 𝐷 , but find it not very effective. To our surprise, we find that reloading an older checkpoint of 𝐷 is very effective in keeping improving the FID of 𝐺 . However, we acknowledge that this is not a principled approach. Overall, we would like an algorithm to guarantee 𝐺 ’s eventual convergence given an arbitrarily strong and optimal 𝐷 without introducing additional modality-specific augmentations. Although our work improves the stability and formulation of 𝐺 , we believe the distribution-matching aspect of 𝐷 is still worthy of further research. Connections to score distillation. Score distillation [55] is another distribution matching objective that can be used to train one-step generators from a pre-trained flow-matching model. Current score distillation approaches yield one-step generators with arbitrary transport plans, the same problem as GANs. Our optimal transport objective can also be applied to regulating the one-step generator to learn a deterministic mapping when using score distillation. We leave further explorations to future work. C.2Computational Efficiency As discussed in the main text, both consistency and our methods require multiple forward passes per update iteration. The difference is that consistency-based models use the same data samples across all the forward passes, while adversarial methods have traditionally used different data samples to independently calculate the expectation of the 𝐺 and 𝐷 losses. We count the epoch by 𝐺 as a fair reflection of the number of optimizer update steps taken by 𝐺 . Since our model only needs to be trained on selected timesteps, our 𝐺 indeed reaches peak performance using fewer optimizer update steps, given the same learning rate and batch size compared to consistency models. However, when we analyze the computation needed per 𝐺 ’s update step, current adversarial methods are still at a disadvantage due to the excessive computation and regularizations needed by 𝐷 . This is a limitation of the adversarial formulation, not introduced by our adversarial flow. Table 12 shows the breakdown of the computation per 𝐺 update. Current adversarial approach consumes 3.625 × compute per 𝐺 ’s update compared to MeanFlow. Adjusting the reduced iterations needed, on XL/2 models, our method uses 1.88 × compute compared to MeanFlow but obtains 15% FID improvement. For adversarial methods, it may be possible to reduce the batch size on 𝐷 , apply clever result reuse, explore other methods to limit the Lipschitz constant other than using gradient penalties, and explore other objective functions than the relativistic one to save compute. But for our research, we take the most conservative approach and mostly follow the prior state-of-the-art GANs’ formulation [21, 23]. We leave the optimization to future work. Generator Discriminator Total Speed Forward Backward Forward Backward (Adj. by epoch) Flow Matching 𝐺 𝐺 2 - - 3 4.38 × MeanFlow 𝐺 + 𝐺 jvp 𝐺 2 - - 4 1 × Adversarial Flow 𝐺 + 2 ​ 𝐷 𝐺 2 + 𝐷 1 𝐺 + 2.5 ​ 𝐷 2.5 ​ 𝐷 2 14.5 1.88 × Table 12:Computation analysis. No guidance. (1) denotes backward pass only compute gradient regarding to input, which consumes 1 × compute as forward pass. (2) denotes backward pass also compute gradient regarding to parameters, which consumes 2 × compute as forward pass. 2.5=1+1+0.25+0.25, where the 0.25s are the R1 and R2 gradient penalties computed on 25% of samples. Speed is adjusted by best XL/2 epoch: FM: 1400 epochs, MF: 240 epochs, AF: 125 epochs. Appendix DGuidance Details Multi-step guidance. The main text only describes our flow-based guidance approach for single-step generation. Since we find out that we do not need very strong guidance for ImageNet generation, our multi-step models simply apply guidance by setting 𝑡 ′ = 𝑡 : ℒ cg 𝐺 = 𝔼 𝑥 𝑠 , 𝑠 , 𝑡 , 𝑐 ​ [ − 𝐶 ​ ( 𝐺 ​ ( 𝑥 𝑠 , 𝑠 , 𝑡 , 𝑐 ) , 𝑡 , 𝑐 ) ] . (35) But for completeness, we discuss possible ways to still project it onto a flow. When given 𝑥 𝑡 , we may not always be able to derive 𝑥 𝑡 ′ as drawn from the marginal distribution at timestep 𝑡 ′ if the interpolation function is not Markov. Linear interpolation is an example. We can, however, form a new probability flow between 𝑥 𝑡 and 𝑥 1 by taking the marginal distribution at 𝑥 𝑡 as the source distribution. The classifier conditions on both the target 𝑡 and projected 𝑡 ′ as 𝐶 ​ ( 𝑥 𝑡 ′ , 𝑡 , 𝑡 ′ , 𝑠 ) : ℒ cg 𝐺 = 𝔼 𝑥 𝑠 , 𝑠 , 𝑡 , 𝑐 , 𝑧 ′ , 𝑡 ′ ​ [ − 𝐶 ​ ( interp ​ ( 𝐺 ​ ( 𝑥 𝑠 , 𝑠 , 𝑡 , 𝑐 ) , 𝑧 ′ , 𝑡 ′ ) , 𝑡 , 𝑡 ′ , 𝑐 ) ] . (36) Derivation of an implicit classifier. We show the derivation of Eq. 39 in the main text. CFG [19] shows that an implicit classifier can be derived following Bayes’ rule: log ⁡ 𝑝 ​ ( 𝑐 | 𝑥 𝑡 ) = log ⁡ 𝑝 ​ ( 𝑥 𝑡 | 𝑐 ) ​ 𝑝 ​ ( 𝑐 ) 𝑝 ​ ( 𝑥 𝑡 ) (37) = log ⁡ 𝑝 ​ ( 𝑥 𝑡 | 𝑐 ) + log ⁡ 𝑝 ​ ( 𝑐 ) − log ⁡ 𝑝 ​ ( 𝑥 𝑡 ) ∼ log ⁡ 𝑝 ​ ( 𝑥 𝑡 | 𝑐 ) − log ⁡ 𝑝 ​ ( 𝑥 𝑡 ) Flow-matching models predicting velocity 𝑣 ​ ( 𝑥 𝑡 , 𝑡 ) = 𝑥 1 − 𝑥 0 is proportional to the negative score − ▽ 𝑥 𝑡 ​ log ⁡ 𝑝 ​ ( 𝑥 𝑡 | 𝑐 ) . Therefore, the gradient of an implicit classifier is derived: ▽ 𝑥 𝑡 ​ 𝐶 ​ ( 𝑥 𝑡 , 𝑡 , 𝑐 ) = ▽ 𝑥 𝑡 ​ log ⁡ 𝑝 ​ ( 𝑐 | 𝑥 𝑡 ) (38) ∼ ▽ 𝑥 𝑡 ​ log ⁡ 𝑝 ​ ( 𝑥 𝑡 | 𝑐 ) − ▽ 𝑥 𝑡 ​ log ⁡ 𝑝 ​ ( 𝑥 𝑡 ) ∼ ( − 𝑣 ​ ( 𝑥 𝑡 , 𝑡 , 𝑐 ) ) − ( − 𝑣 ​ ( 𝑥 𝑡 , 𝑡 ) ) = 𝑣 ​ ( 𝑥 𝑡 , 𝑡 ) − 𝑣 ​ ( 𝑥 𝑡 , 𝑡 , 𝑐 ) Given the gradient of an implicit classifier, we want to directly pass it down the generator 𝐺 in the backpropagation process. This can be achieved using the constant multiple rule trick, where 𝑓 ​ ( 𝑥 ) = 𝑎 ​ 𝑥 , 𝑓 ′ ​ ( 𝑥 ) = 𝑎 . So we multiply the gradient of the implicit classifier by the generator output: ℒ cfg 𝐺 = 𝔼 𝑧 , 𝑐 , 𝑧 ′ , 𝑡 ′ ​ [ − 1 𝑛 ​ ∑ 𝐺 ​ ( 𝑧 , 𝑐 ) ​ ▽ ( ⋅ ) ​ 𝐶 ​ ( ⋅ , 𝑡 ′ , 𝑐 ) ] , (39) where ( ⋅ ) is short for the flow interpolation process interp ​ ( 𝐺 ​ ( 𝑧 , 𝑐 ) , 𝑧 ′ , 𝑡 ′ ) before passing to the implicit classifier. Summation is taken to produce a scalar for loss backpropagation. Negative is used to invert loss minimization for classification maximization. Implicit guidance in prior GAN works. Many techniques used in prior GAN works are, in fact, classifier guidance. Our work explicitly labels them for clarity. Multiple GAN works use cGAN discriminator architecture [58]. This architecture resembles CLIP [64], where the inner product between the visual embedding and the class embedding is computed at the end and maximized along with the adversarial objective. This is an implicit form of classifier guidance, as this architecture clearly does not apply to unconditional generation tasks. GigaGAN proposes a matching loss [26], which trains the 𝐷 to evaluate class alignment in addition. When training 𝐺 , this implicitly encourages 𝐺 to generate samples that maximize classification. Hence, it is also an implicit form of classifier guidance. Appendix EOn Distribution Matching Why does flow matching generate bad samples? Flow matching defines an interpolation function between samples 𝑥 and 𝑧 and learns the gradient (velocity) of the interpolated sample 𝑥 𝑡 as 𝑣 𝑡 : 𝑥 𝑡 = 𝐴 ​ ( 𝑡 ) ​ 𝑥 + 𝐵 ​ ( 𝑡 ) ​ 𝑧 = ( 1 − 𝑡 ) ​ 𝑥 + 𝑡 ​ 𝑧 , (40) 𝑣 𝑡 = 𝐴 ′ ​ ( 𝑡 ) ​ 𝑥 + 𝐵 ′ ​ ( 𝑡 ) ​ 𝑧 = − 𝑥 + 𝑧 . (41) The model learns the expected velocity of all random pairings. It is trained to minimize the following loss objective with 𝑑 : ℝ 𝑛 → ℝ 1 as the distance function. 𝑑 can be any strictly convex function in principle, and flow matching adopts the squared L2 distance: ℒ FM = 𝔼 𝑥 , 𝑧 , 𝑡 ​ [ 𝑑 ​ ( 𝑣 ​ ( 𝑥 𝑡 , 𝑡 ) , 𝑣 𝑡 ) ] (42) = 𝔼 𝑥 , 𝑧 , 𝑡 ​ [ ‖ 𝑣 ​ ( 𝑥 𝑡 , 𝑡 ) − 𝑣 𝑡 ‖ 2 2 ] . If given enough capacity, the model will learn to memorize the exact mean of 𝑣 𝑡 for all 𝑥 𝑡 . Such a model overfits to the ground-truth probability flow and only generates the training samples at inference. But in practice, models have limited capacities, so they do not memorize the exact 𝑣 𝑡 but regress to a generalized approximation of it. This enables the generation of new data samples through generalization. However, the way of generalization is influenced by the distance function 𝑑 . With squared L2, generalized samples are encouraged to minimize the Euclidean distance to the training data, instead of the semantic distance on the actual data distribution manifold. This causes the generated samples to look out-of-distribution. For images, they appear pixel-blended instead of semantically morphed from the training data, as shown in Fig. 9(b). From the perspective of diffusion. In flow matching, 𝑑 can be any strictly convex function. The use of the squared L2 has a special interpretation from the diffusion model perspective. In DDPM [20], the forward diffusion process is defined as a Markov Gaussian process: 𝑞 ​ ( 𝑥 𝑡 | 𝑥 𝑡 − 1 ) := 𝒩 ​ ( 1 − 𝛽 𝑡 ​ 𝑥 𝑡 − 1 , 𝛽 𝑡 ​ 𝐈 ) . (43) The transition to 𝑥 𝑡 = ∏ 𝑖 = 1 𝑡 𝑞 ​ ( 𝑥 𝑖 | 𝑥 𝑖 − 1 ) has a closed form: 𝑞 ​ ( 𝑥 𝑡 | 𝑥 0 ) := 𝒩 ​ ( 𝑎 ¯ 𝑡 ​ 𝑥 0 , ( 1 − 𝑎 ¯ 𝑡 ) ​ 𝐈 ) , (44) where 𝑎 𝑡 = 1 − 𝛽 𝑡 , and 𝑎 ¯ 𝑡 = ∏ 𝑠 = 1 𝑡 𝑎 𝑠 . The reverse process is also Gaussian, where the ground-truth posterior is given as: 𝑞 ​ ( 𝑥 𝑡 − 1 | 𝑥 𝑡 , 𝑥 0 ) := 𝒩 ​ ( 𝜇 ~ ​ ( 𝑥 0 , 𝑥 𝑡 , 𝑡 ) , 𝛽 ~ ​ ( 𝑡 ) ​ 𝐈 ) , (45) where: 𝜇 ~ ​ ( 𝑥 0 , 𝑥 𝑡 , 𝑡 ) = 𝑎 ¯ 𝑡 − 1 ​ 𝛽 𝑡 1 − 𝑎 ¯ 𝑡 ​ 𝑥 0 + 𝑎 𝑡 ​ ( 1 − 𝑎 ¯ 𝑡 − 1 ) 1 − 𝑎 ¯ 𝑡 ​ 𝑥 𝑡 , (46) 𝛽 ~ ​ ( 𝑡 ) = 1 − 𝑎 ¯ 𝑡 − 1 1 − 𝑎 ¯ 𝑡 ​ 𝛽 𝑡 . (47) This can be modeled by a neural network: 𝑝 𝜃 ​ ( 𝑥 𝑡 − 1 | 𝑥 𝑡 ) := 𝒩 ​ ( 𝜇 𝜃 ​ ( 𝑥 𝑡 , 𝑡 ) , Σ 𝜃 ​ ( 𝑥 𝑡 , 𝑡 ) ​ 𝐈 ) . (48) KL divergence is chosen as the loss objective: ℒ DDPM = 𝔼 𝑥 0 , 𝑡 [ 𝐷 KL ( 𝑞 ( 𝑥 𝑡 − 1 | 𝑥 𝑡 , 𝑥 0 ) ∥ 𝑝 𝜃 ( 𝑥 𝑡 − 1 | 𝑥 𝑡 ) ) ] . (49) The KL divergence between two Gaussian distributions has a closed-form solution. The original DDPM only parameterizes 𝜇 𝜃 , which simplifies to the squared L2 objective if omitting the additional weighting factor: ℒ DDPM ∝ 𝔼 𝑥 0 , 𝑡 ​ [ ‖ 𝜇 𝜃 ​ ( 𝑥 𝑡 , 𝑡 ) − 𝜇 ~ ​ ( 𝑥 0 , 𝑥 𝑡 , 𝑡 ) ‖ 2 2 ] . (50) The DDPM derivation follows the same limitation as that of flow matching. Specifically, the model will overfit to the training data if it fits the expectation of the ground-truth posterior exactly, or it will generalize to new samples influenced by the chosen divergence objective. The divergence objective can be arbitrary, and KL is only chosen for convenience. KL divergence maximizes likelihood, but the generated samples are perceptually out of distribution. Why does adversarial work? If 𝑑 can be an arbitrary function, then it should be learned instead of being manually designed. Adversarial training involves a learnable discriminator as the distance function 𝐷 : ℝ 𝑛 → ℝ 1 . In the adversarial minimax game. 𝐷 is trained to maximize the distance between real and generated samples, so 𝐷 learns to put more weight on the most important and discriminative features. 𝐺 is then updated according to the gradient of 𝐷 , which is highly non-linear unlike the Euclidean distance objective. We believe having a learnable ℝ 𝑛 → ℝ 1 distance function is critical for generating perceptually correct samples. This justifies our experimental results where the adversarial models significantly outperform flow matching in the no-guidance setting. Prior works also find adversarial post-training improves the realism [47, 88]. Why does CG work? Classifier guidance (CG) [9] introduces a learned classifier network 𝐶 : ℝ 𝑛 → ℝ 1 whose gradient is added on 𝑣 ​ ( 𝑥 𝑡 , 𝑡 ) . We believe the presence of a learnable ℝ 𝑛 → ℝ 1 network is critical. Its backward gradient ℝ 1 → ℝ 𝑛 is non-linear and guides the generation toward maximizing 𝑝 ​ ( 𝑐 | 𝑥 𝑡 ) as evaluated by the semantic network, which helps the samples to appear semantically and perceptually correct. Discriminator guidance [34] and reward models are other examples of using a learned ℝ 𝑛 → ℝ 1 networks to help generation quality. On the other hand, only adjusting the sampling temperature alone without CG is not sufficient [93], indicating that the effect of CG is not just temperature sampling. Why does CFG work? Recall in CG, the classifier network ℝ 𝑛 → ℝ 1 predicts 𝑝 ​ ( 𝑐 | 𝑥 𝑡 ) and it is its backward gradient ℝ 1 → ℝ 𝑛 that is added to the ℝ 𝑛 velocity prediction. Classifier-free guidance (CFG) [19] is derived from the Bayes’ rule where ▽ 𝑥 𝑡 ​ 𝐶 ​ ( 𝑥 𝑡 , 𝑡 , 𝑐 ) = 𝑣 ​ ( 𝑥 𝑡 , 𝑡 ) − 𝑣 ​ ( 𝑥 𝑡 , 𝑡 , 𝑐 ) is the gradient of the implicit classifier, as previously discussed in Eq. 38. Therefore, despite not having an explicit ℝ 𝑛 → ℝ 1 classifier, the conditional and unconditional ℝ 𝑛 → ℝ 𝑛 flow models together provide the gradient of an implicit classifier ℝ 𝑛 → ℝ 1 → ℝ 𝑛 . Why does representational latent space help? If flow matching is performed in a latent space where the Euclidean interpolations still yield valid latents that can be subsequently decoded into a perceptual image by the decoder, then it can alleviate the limitation of flow matching. This offloads the responsibility to the decoder, which is often trained with adversarial and LPIPS [102] perceptual objectives. We suspect that representational latent spaces have more semantic features where the Euclidean interpolations can be better decoded. This may be part of the reason, but we defer to future research. Connections to evaluation metrics. FID metric measures the distance between two distributions of images by comparing their representations in the feature space of a pretrained network, i.e. Inception-V3 [83]. Therefore, FID also measures the semantic distance instead of the Euclidean distance. Thus, adversarial, CG, and CFG approaches all help improve FID. Appendix FAdditional Properties of Adversarial Flow (a)Flow (b)FM 64-step (c)AF 1-step (d)AF 4-step Figure 14:Transport between arbitrary distributions. (a) the probability flow between mixtures of isolated Gaussian distributions. (b) flow matching fails to transport samples. (c) and (d) show that adversarial flow can learn the correct transport. Flow matching can fail when transporting between two probability distributions where both are multi-modal. Figure 14(a) shows such an example, where the distributions on both ends of the probability flow are mixtures of isolated Gaussians. Such probability flows lack support in the intermediate timesteps. As shown in Fig. 14(b), flow matching fails to properly transport samples by learning the mean velocity. This problem also exists in consistency-based models. Additional techniques are needed to transport between arbitrary distributions [17, 78, 106, 104]. On the other hand, Figures 14(c) and 14(d) show that adversarial flow models can learn the transports between arbitrary distributions, if the discriminator provides proper support for the distribution matching. Appendix GImplementation The PyTorch [61] implementation is provided in Algorithms 2 and 3 for reference. Algorithm 2 Adversarial Flow. 1from torch.nn import functional as F 2 3 4def interpolate(x, z, t): 5 t = t.view(-1, 1, 1, 1) 6 return (1 - t) * x + t * z 7 8 9def training_step( 10 gen, dis, 11 gen_trainable_params, dis_trainable_params, 12 samples, conditions, noises_src, noises_tgt, noises_r1, noises_r2, timesteps_src, timesteps_tgt, 13 mode="dis", pred_type="x", 14 gp_scale=0.25, gp_bsz_ratio=0.25, gp_eps=0.01, cp_scale=0.01, ot_scale=0.2, 15 augment_fn, grad_norm_fn, 16 cg=None, noises_cg=None, timesteps_cg=None, cg_scale=0.003, cg_flow=True, 17): 18 assert mode in ["dis", "gen"] 19 assert pred_type in ["x", "v"] 20 21 for p in gen_trainable_params: 22 p.requires_grad_(mode == "gen") 23 for p in dis_trainable_params: 24 p.requires_grad_(mode == "dis") 25 26 samples_src = interpolate(samples, noises_src, timesteps_src) 27 samples_tgt = interpolate(samples, noises_tgt, timesteps_tgt) 28 samples_tgt_pred = gen(samples_src, conditions, timesteps_src, timesteps_tgt) 29 30 if pred_type == "v": 31 samples_tgt_pred = samples_src - (timesteps_src - timesteps_tgt).view(-1, 1, 1, 1) * samples_tgt_pred 32 33 samples_tgt_real_aug, samples_tgt_pred_aug = augment_fn(samples_tgt, samples_tgt_pred) 34 weighting = (timesteps_src - timesteps_tgt).abs().clamp_min(0.001) 35 36 if mode == "dis": 37 bsz = len(samples) 38 gp_bsz = max(round(bsz * gp_bsz_ratio), 1) 39 gp_scale = gp_scale * weighting[:gp_bsz] / gp_eps**2 40 41 samples_tgt_real_gp = samples_tgt_real_aug[:gp_bsz] + gp_eps * noises_r1[:gp_bsz] 42 samples_tgt_pred_gp = samples_tgt_pred_aug[:gp_bsz] + gp_eps * noises_r2[:gp_bsz] 43 44 logits_real = dis(samples_tgt_real_aug, conditions, timesteps_tgt) 45 logits_pred = dis(samples_tgt_pred_aug, conditions, timesteps_tgt) 46 logits_real_gp = dis(samples_tgt_real_gp, conditions[:gp_bsz], timesteps_tgt[:gp_bsz]) 47 logits_pred_gp = dis(samples_tgt_pred_gp, conditions[:gp_bsz], timesteps_tgt[:gp_bsz]) 48 49 dis_loss_adv = F.softplus(-(logits_real - logits_pred)).mean() 50 dis_loss_r1 = (logits_real_gp - logits_real[:gp_bsz]).square().mul(gp_scale).mean() 51 dis_loss_r2 = (logits_pred_gp - logits_pred[:gp_bsz]).square().mul(gp_scale).mean() 52 dis_loss_cp = (logits_real + logits_pred).square().mul(cp_scale).mean() 53 54 return dis_loss_adv + dis_loss_r1 + dis_loss_r2 + dis_loss_cp 55 else: 56 logits_pred = dis(grad_norm_fn(samples_tgt_pred_aug), conditions, timesteps_tgt) 57 logits_real = dis(samples_tgt_real_aug, conditions, timesteps_tgt) 58 59 gen_loss_adv = F.softplus(-(logits_pred - logits_real)).mean() 60 gen_loss_ot = (samples_tgt_pred - samples_src).square().mean([1,2,3]).mul(ot_scale / weighting).mean() 61 gen_loss_cg = 0.0 62 63 if cg is not None: 64 samples_tgt_pred_cg = ( 65 interpolate(samples_tgt_pred, noises_cg, timesteps_cg) if cg_flow else samples_tgt_pred 66 ) 67 logits_cg = cg(samples_tgt_pred_cg, timesteps_cg) 68 gen_loss_cg = F.cross_entropy(logits_cg, conditions, reduction="none").mul(cg_scale).mean() 69 70 return gen_loss_adv + gen_loss_ot + gen_loss_cg Algorithm 3 Gradient Normalization. 1import torch 2import torch.distributed as dist 3 4 5class GradientNormalization(torch.nn.Module): 6 def __init__(self, ema_decay=0.9, eps=1e-8, target_scale=1.0): 7 super().__init__() 8 self.ema_decay = ema_decay 9 self.eps = eps 10 self.target_scale = target_scale 11 self.register_buffer("square_avg", torch.tensor(0.0)) 12 13 def forward(self, x): 14 return _GradientNormalizationFn.apply( 15 x, 16 self.square_avg, 17 self.ema_decay, 18 self.eps, 19 self.target_scale 20 ) 21 22 23class _GradientNormalizationFn(torch.autograd.Function): 24 @staticmethod 25 def forward(ctx, x, square_avg, ema_decay, eps, target_scale): 26 ctx.square_avg = square_avg 27 ctx.ema_decay = ema_decay 28 ctx.eps = eps 29 ctx.target_scale = target_scale 30 return x.clone() 31 32 @staticmethod 33 def backward(ctx, grad_output): 34 square_avg = ctx.square_avg 35 ema_decay = ctx.ema_decay 36 eps = ctx.eps 37 target_scale = ctx.target_scale 38 39 # Multiply n here is equivalent to divide by sqrt(n) later in the paper. 40 grad_sq_sum = grad_output.square().sum() * grad_output.numel() 41 42 # Here, we compute avg not sum for distributed training. 43 # This is only to exchange the local grad_sq_sum. 44 # We still want it to be in local scale, not global scale. 45 if dist.is_initialized(): 46 dist.all_reduce(grad_sq_sum, op=dist.ReduceOp.AVG) 47 48 square_avg.lerp_(grad_sq_sum, 1 - ema_decay) 49 scale = square_avg.sqrt() + eps 50 grad_output = grad_output * (target_scale / scale) 51 return grad_output, None, None, None, None Report Issue Report Issue for Selection Generated by L A T E xml Instructions for reporting errors We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below: Click the "Report Issue" button. Open a report feedback form via keyboard, use "Ctrl + ?". Make a text selection and click the "Report Issue for Selection" button near your cursor. You can use Alt+Y to toggle on and Alt+Shift+Y to toggle off accessible reporting links at each section. Our team has already identified the following issues. We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all. Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion, and welcome developer contributions.