MethodologyAnnotator qualityUpdatesHugging Face436.7k downloadsSuggest a model or benchmarkRequest a private benchmark
Benchmarks
Dimensions
License
Models
Categories
category
source
LLM to Animation v1updated 2026-11-19

Language Models: Animation Benchmark

Rapidatabenchmark@rapidata.ai

Abstract

When a new language model comes out, one of the most common ways to show it off is a three.js animation: a single clip of a scene the model wrote in code, chosen by the person posting it. This benchmark ranks language models on that task under fixed conditions. Every model receives the same 200 prompts, drawn from requests people post on X and type into chatbots and widened with scene descriptions from video benchmarks, and writes one HTML page per prompt that plays a three.js animation. Each page is recorded as a 7-second clip, and human annotators compare the clips of two models for the same prompt, without seeing the prompt or the code, and say which one they would continue watching and which one they like more. The 925,025 responses collected this way are aggregated into ratings with a Bradley–Terry model; the details describe how the prompts were chosen and the clips made, and the methodology covers matchmaking and score estimation.

LeaderboardCompare OutputsMatchupsAnalysisPromptsModel settingsDetails
1. Motivation
New model releases are often hyped up with X posts with an animation showing off the model’s capabilities. One hand-picked clip is not a fair comparison, and comparing it with other models is also not rigorous. This benchmark gives every model the same 200 prompts and the same generation protocol, and has human annotators compare the resulting animations head to head, so models are ranked across many prompts rather than on their best clip.
2. Prompt sources
The 200 prompts come from nine public sources (Table 1): requests people posted on X, first messages of real chats from LMArena and WildChat-4.8M (Zhao et al., 2024), prompt libraries for 3D and animation work, scene descriptions from VBench (Huang et al., 2024) and VBench-2.0 (Zheng et al., 2025), and 12 prompts written from the three.js examples.
Table 1. Sources of the 200 prompts.
Source
Kind
Prompts
awesome-opus-5-5-video-prompts
X posts
39
awesome-opus-5-5-prompts
X posts
5
awesome-ai-motion-videos
X posts
2
LMArena
Chat logs
36
WildChat-4.8M
Chat logs
34
3D prompt collection
Prompt library
15
prompts.chat
Prompt library
5
VBench and VBench-2.0
Video benchmark
52
three.js examples
Written from code
12
The 200 picks, coloured by source, among the 2,712 other prompts that passed every check (grey).
Figure 1. The 200 picks, coloured by source, among the 2,712 other prompts that passed every check (grey).
3. Prompt selection
We collected 4,252 unique prompts and filtered them using Claude Opus 5.5 as a judge. The judge checked whether each prompt is self-contained, has a clear subject and works as a short silent animation on a web page, and rated how likely it is to separate models. 2,101 prompts passed these checks and were under 4,000 characters. We used GPT-5.5 to sort the prompts into eight categories, and each category gets 25 of the 200 (Figure 2).To avoid near-duplicates, every prompt ppp is embedded with voyage-4-large as epe_pep​, and two prompts are compared by cosine similarity: s(p,q)=ep⋅eq∥ep∥ ∥eq∥\displaystyle s(p,q) = \frac{e_p \cdot e_q}{\lVert e_p \rVert \, \lVert e_q \rVert} s(p,q)=∥ep​∥∥eq​∥ep​⋅eq​​(1) We repeatedly take the most similar pair and drop one of the two until 200 remain. Prompts people actually asked for are kept over benchmark scenes, no category drops below 25, and at most 3 per category come from the two Opus 5.5 collections.The result is more diverse than a random draw with the same sources: a Vendi score of 40.3 against 28.3, and no pair above 0.9 similarity against 16.
The 200 picks by category (blue, 25 per panel) among the 2,912 prompts that passed every check (grey), on a UMAP map of the embeddings: nearby points are similar prompts.
Figure 2. The 200 picks by category (blue, 25 per panel) among the 2,912 prompts that passed every check (grey), on a UMAP map of the embeddings: nearby points are similar prompts.
4. Generation
Every model gets the system prompt below and the prompt as its message, and writes one HTML page that plays a three.js animation. Models run at high reasoning effort with up to 128,000 output tokens (see the Model settings tab). A reply that does not give a playable page is retried, up to three times.Every page is recorded the same way: in headless Chrome, frame by frame, the first 7 seconds at 30 fps and 1280×720.
5. Human annotation protocol
Annotators see the clips of two models for the same prompt, without the prompt or the code, and answer two questions: “Which video would you continue watching?” and “Which one do you like more?”. Each match gets at least 5 votes and goes to the majority.
6. Ranking
Match outcomes are fitted with a Bradley–Terry model. Scores are reported as 1000 + 400 ln π, so the average model scores 1000, with 95% confidence intervals (see Methodology).
7. Limitations
Most X prompts were posted with clips made by Claude Opus 5.5, Claude helped filter the prompts, and Claude models are ranked here. Only the first 7 seconds are judged, with no interaction, so a game or a site is judged on how it looks, not how it plays. Each model gets one clip per prompt. 259 of the 5,800 prompt–model pairs have no clip because every attempt failed; Gemini 4 Carbon misses the most (44).
8. Data and licence
VBench and VBench-2.0 are released under Apache-2.0, WildChat-4.8M under ODC-BY, the LMArena datasets under CC BY 4.0 and Apache-2.0, prompts.chat under CC0 and the three.js examples under MIT. The X collections and the 3D prompt collection carry no licence for the prompts, so they remain their authors’, and every prompt links back to its post or source.
9. References

Full protocol in the global methodology.

We use crowd intelligence to deliver real, diverse human feedback at scale for model evaluation and post-training.

Benchmarks
Text-to-ImageOutput Diversity (Mode Collapse)Multi-Step Image Editing (Identity Drift)Camera MovementRelative Camera MovementPhysicsPhysicsRealismImage-to-3DSVGLLM-to-3DLLM to AnimationSuggest a model or benchmark
Method
MethodologyData on Hugging Face
Rapidata
rapidata.ai Private benchmarksContactX LinkedIn
Rapidata, benchmark.ai © 2026