Video and world models are increasingly evaluated on their understanding of real-world physics. Physics-IQ (Motamed et al., 2026), later refined by Physics-IQ Verified, measures how closely generated videos reproduce real physical events using automated motion-based metrics.Building on this work, we investigate how reference-based evaluations relate to human perception. Using 198 real recordings across 66 physical scenarios, we evaluate 26 video and world models through two complementary questions: Blind, which video looks more realistic without seeing the original recording; and Referenced, which video more faithfully reproduces what actually happened.Across 758,940 human judgments and 126,490 pairwise comparisons, reference-conditioned human rankings correlate more strongly with Physics-IQ Verified scores (Spearman 0.81) than blind realism rankings (0.66).These findings highlight two distinct aspects of video generation: reproducing an observed physical event and generating a convincing, plausible continuation. A real recording captures one possible outcome, but not necessarily the only physically plausible one. This distinction raises broader questions for video generation evaluation and post-training: what should automated metrics and reward models optimize for, fidelity to a specific observed outcome or the broader space of physically plausible outcomes? The details cover the setup and the comparison with Physics-IQ, all videos and votes are on Hugging Face, and our blog post walks through examples.
Figure A3. Each model's standing within each prompt category, scored on overall preference. Read a column: who is best in that category. Darkest is first. Hover or tap a cell to read its score, 95% interval and sample size; click a category to sort the rows by it. The number beside each model is its overall rank, so a jump reads as a jump. A cell reports a rank only above 500 matches for that model and is hatched below it, a rank on a handful of matchups is noise, not a result. How many prompts a category rests on is in its header and every tooltip: a narrow category asks a narrower question.
We use crowd intelligence to deliver real, diverse human feedback at scale for model evaluation and post-training.
