Video and world models are increasingly evaluated on their understanding of real-world physics. Physics-IQ (Motamed et al., 2026), later refined by Physics-IQ Verified, measures how closely generated videos reproduce real physical events using automated motion-based metrics.Building on this work, we investigate how reference-based evaluations relate to human perception. Using 198 real recordings across 66 physical scenarios, we evaluate 26 video and world models through two complementary questions: Blind, which video looks more realistic without seeing the original recording; and Referenced, which video more faithfully reproduces what actually happened.Across 758,940 human judgments and 126,490 pairwise comparisons, reference-conditioned human rankings correlate more strongly with Physics-IQ Verified scores (Spearman 0.81) than blind realism rankings (0.66).These findings highlight two distinct aspects of video generation: reproducing an observed physical event and generating a convincing, plausible continuation. A real recording captures one possible outcome, but not necessarily the only physically plausible one. This distinction raises broader questions for video generation evaluation and post-training: what should automated metrics and reward models optimize for, fidelity to a specific observed outcome or the broader space of physically plausible outcomes? The details cover the setup and the comparison with Physics-IQ, all videos and votes are on Hugging Face, and our blog post walks through examples.
Full protocol in the global methodology.
We use crowd intelligence to deliver real, diverse human feedback at scale for model evaluation and post-training.
