MethodologyAnnotator qualityUpdatesHugging Face434.9k downloadsSuggest a model or benchmarkRequest a private benchmark
Benchmarks
Dimensions
License
Models
Categories
Specificity tier
Content
Concept type
Subjects
People
Cultural variability
Medium
Headline set
Style
View
Output Diversity (Mode Collapse) v1updated 2026-10-29

The Human-Preference Leaderboard for T2I Output Diversity (Mode Collapse)

Rapidatabenchmark@rapidata.ai

Abstract

As text-to-image models have improved in prompt adherence, the diversity of their outputs appears to have declined, a phenomenon commonly referred to as mode collapse. This benchmark evaluates output diversity holistically on the basis of large-scale human judgements, over a curated, deliberately non-restrictive set of 299 prompts spanning five tiers of specificity, from a bare subject (“a dog”) to open-ended requests (“something surprising”). For each prompt, every model generates a pool of images from which grids are sampled. Models are then compared in pairwise matches: annotators are shown one grid from each of two models and asked which exhibits greater intra-grid similarity, and the less similar grid is scored as the winner. The 266,060 responses collected across 53,212 such matches are aggregated into ratings with a Bradley–Terry model, such that a higher rating indicates greater diversity; the methodology details matchmaking and score estimation.

LeaderboardCompare OutputsMatchupsAnalysis· DemographicsPromptsDetails

Top model by region

We use crowd intelligence to deliver real, diverse human feedback at scale for model evaluation and post-training.

Benchmarks
Text-to-ImageOutput Diversity (Mode Collapse)Multi-Step Image Editing (Identity Drift)Camera MovementRelative Camera MovementPhysicsPhysicsRealismImage-to-3DSVGLLM-to-3DSuggest a model or benchmark
Method
MethodologyData on Hugging Face
Rapidata
rapidata.ai Private benchmarksContactX LinkedIn
Rapidata, benchmark.ai © 2026