Inference-Time Scaling of Diffusion Models via Progressive Seed Pruning

We compare standard sampling ($\bar{N}=1$), Best-of-N (BoN), our main importance-sampling (FK-Steering) and tree-search (DSearch) baselines, and other relevant inference-time compute-scaling methods for diffusion, namely Noise Trajectory Search (NTS), Rollover Budget Forcing (RBF), Breadth-First Search (BFS), and SVDD, and PSP. All results use the public implementation of each method with parameters chosen to match the computational budget, all guided by the same reward signal: ImageReward. We report the guidance reward (IR), HPS, GenEval, and human evaluation scores on final images generated from GenEval prompts, averaged over 3 random seeds. PSP achieves the best results in prompt-alignment as measured by automated scores (GenEval) and human evaluation, across all three backbones.

Across all backbones, PSP improves over standard inference and outperforms all other methods at the same compute in prompt alignment, as measured by GenEval score and human evaluation. Although BFS outperforms PSP in reward maximization with the SDXL backbone, PSP still outperforms BFS and all methods in GenEval and human scores. This is evidence that PSP is less susceptible to reward hacking, since it focuses on seed selection rather than interfering with the inference process through reward-driven resampling. On SD 3.5, results also expose a practical weakness of resampling methods: they require stochasticity to generate diverse children, but stochasticity can degrade flow-matching generation, with Best-of-N outperforming FK-Steering. PSP remains effective with deterministic solvers because it allocates compute to seed exploration and early elimination rather than stochastic branching.

Human evaluation. Results were obtained from 249 online annotators (Prolific, prior AI-evaluation experience) who answered “Does the image align with the prompt?” on 8,295 images (553 per method/backbone pair, the number of GenEval prompts), with majority vote over 3 ratings per image (24,885 ratings total, 80.3% annotator agreement). To our knowledge, this is the first human evaluation of these inference-time scaling methods for text-to-image tasks. PSP was the best-evaluated method on all three backbones.

PSP adds computational overhead over Best-of-N because its intermediate samples must be decoded by the VAE and scored by the reward model. We quantify this overhead on a single H200, comparing PSP, equivalent Best-of-N ($\bar{N}{=}4$), and double-compute Best-of-N ($\bar{N}{=}8$) under the same setup as the main results table. Across all backbones, PSP adds little runtime over Best-of-N ($\bar{N}{=}4$) and is close to $2\times$ faster than Best-of-N ($\bar{N}{=}8$). Relative overhead diminishes with model size, as each diffusion step becomes proportionally more expensive.

Reward models can also be used to finetune generators, amortizing reward optimization across many future inference calls. We compare against DPO-finetuned backbones trained on the Pick-a-Pic dataset. PSP applied to a non-finetuned backbone with a modest compute multiplier already outperforms standard inference on a finetuned backbone, and PSP and finetuning are complementary: applying PSP on a finetuned backbone yields further improvements and outperforms Best-of-N under similar compute. Thus, whenever additional inference-time compute is available, PSP remains beneficial even in the presence of finetuned models.

Although our primary focus is seed search, PSP applies to any discrete pool of candidates available at inference time. A practical example is prompt selection: modern systems often rewrite user prompts to expand and add detail, and different rewrites can substantially change the quality of the outcome. We apply PSP to select among multiple prompt rewrites for a fixed noise seed, generating rewrites once per prompt with an LLM and treating them as the initial candidate set. PSP outperforms Best-of-N at equal compute in this setting, highlighting another practical advantage over resampling-based methods: importance sampling over prompts would require a stochastic “prompt mutation” kernel that repeatedly queries an LLM during inference, whereas PSP only requires a fixed candidate set generated once.