Video and world models are increasingly evaluated on their understanding of real-world physics. Physics-IQ (Motamed et al., 2026), later refined by Physics-IQ Verified, measures how closely generated videos reproduce real physical events using automated motion-based metrics.Building on this work, we investigate how reference-based evaluations relate to human perception. Using 198 real recordings across 66 physical scenarios, we evaluate 26 video and world models through two complementary questions: Blind, which video looks more realistic without seeing the original recording; and Referenced, which video more faithfully reproduces what actually happened.Across 758,940 human judgments and 126,490 pairwise comparisons, reference-conditioned human rankings correlate more strongly with Physics-IQ Verified scores (Spearman 0.81) than blind realism rankings (0.66).These findings highlight two distinct aspects of video generation: reproducing an observed physical event and generating a convincing, plausible continuation. A real recording captures one possible outcome, but not necessarily the only physically plausible one. This distinction raises broader questions for video generation evaluation and post-training: what should automated metrics and reward models optimize for, fidelity to a specific observed outcome or the broader space of physically plausible outcomes? The details cover the setup and the comparison with Physics-IQ, all videos and votes are on Hugging Face, and our blog post walks through examples.
Showing the top-ranked models by default. You can pick models in the sidebar to change the lineup. Click a sample to inspect it. Model identities are hidden from annotators.
Outputs for the selected models on shared prompts; annotators saw them unlabeled and in random order. Scroll for more prompts.
We use crowd intelligence to deliver real, diverse human feedback at scale for model evaluation and post-training.
