TL;DR
Verifiable Visual Rewards derives image prompts and deterministic constraint checks from the same geometric scene specification. Flow-GRPO training on simple synthetic scenes improves instruction following in Stable Diffusion 3.5 Medium and transfers to natural-image prompts, with measurable tradeoffs.
Why it matters
Counting and spatial relations need reliable feedback, but learned image judges can be noisy. A restricted visual domain makes programmatic verification possible and tests whether the resulting skills transfer beyond that domain.
Key findings
- 01
VVRBench contains 10,000 tasks over 32 constraint types, with a 720-task Challenge set. Task specifications jointly define the prompt and verifier; the system extracts geometric objects from pixels before checking counts, attributes, and relations.
- 02
The authors train Stable Diffusion 3.5 Medium using Flow-GRPO and dense VVR rewards. VVR-Easy and VVR-Matched each contain 100K tasks: the former limits scene complexity, while the latter matches the benchmark’s complexity distribution after decontamination.
- 03
VVR-Easy raises VVRBench task accuracy from 2.81% to 28.27%, a 25.46-percentage-point gain. VVR-Matched reaches 46.60%. Transfer to harder scenes is stronger for individual constraints than for satisfying every constraint together.
- 04
On 160 natural prompts outside VVR, human annotators prefer VVR-Easy over the pretrained model with a reported 71.6% win rate (interval 65.9–77.1%). Mixing VVR-Easy with GenEval2 yields a 58.6% preference win rate over GenEval2-only training, while other mixtures show some metric regressions.
Evidence & limits
The verifier operates on eight colors, three shapes, and plain backgrounds. Deterministic checks in this restricted setting are not a general verifier for arbitrary natural images; pixel-to-object extraction remains part of the verification pipeline.
Post-training is demonstrated on Stable Diffusion 3.5 Medium with Flow-GRPO. Results do not establish transfer across model families, video generation, or image editing.
Human preference results use a 160-prompt external sample. Benchmark task accuracy, partial constraint scores, automated preference metrics, and human win rates measure different properties.
VVR-Easy alone slightly lowers aesthetic and HPSv2.1 scores. Adding it to OCR or five-reward mixtures reduces native OCR scores, so improvements are not uniform across objectives.
The study supports synthetic-to-natural skill transfer, not a demonstrated reduction in total training cost. Matching benchmark complexity is distinct from solving arbitrarily complex scenes.
Scaling dimensions
Models, methods & benchmarks
- Models
- Stable Diffusion 3.5 Medium
- Algorithms
- Flow-GRPO, Verifiable Visual Rewards
- Benchmarks
- VVRBench, VVRBench-Challenge, GenEval, GenEval2, OCR, DrawBench
Topics