This benchmark evaluates the 3D understanding of language models: each model writes three.js code that builds an object from a short text prompt, and the object it builds is what gets judged. The rankings come from 135,152 pairwise matches, decided by 780,424 votes. Quality is evaluated in four dimensions: Alignment, Coherence, Creativity and Preference. All rankings are fitted with a Bradley–Terry model and reported on the Elo scale; see the notes for details.
We evaluate language models writing 3D code purely on human feedback. Humans perform inconsistently on absolute judgements, but much better in a comparative setting. Our rating is therefore based on pair-wise matches, the results of which are jointly fit using the Bradley–Terry model where the probability of model m beating model m′ is given by:
A match pairs the 3D objects two models built in code from the same text prompt. As evaluating all permutations is infeasible, matches are sampled, ensuring broad coverage per model. The resulting scores are scaled to the familiar Elo range via Rm = 400 βm + 1000 (2), which stretches standard Elo by ln 10 ≈ 2.3 to better highlight differences in model capability.
We wrote 500 text prompts in five categories of 100: vehicles, animals and creatures, buildings and architecture, furniture and household objects, and instruments and tools. All models share a ceiling of 64,000 output tokens per call, thinking included, and almost no call reaches it: every model's median call stays below 14,000 tokens (see the output and cost table).
To ensure trustworthy ratings we employ several measures, notably annotator pre-filtering via gold-standard questions, and redundancy, collecting a minimum of m responses per match (m = 5) and deciding the winner by majority. Read more
You are an expert 3D modeler who builds models in code with three.js (r186).
Write a JavaScript function that builds the 3D model the user describes:
function build(THREE, addons) {
const model = new THREE.Group();
// create meshes and add them to model
return model;
}
How it will be judged: your model is exported to glTF and filmed on a turntable in a neutral studio with fixed lighting, a soft floor shadow and a camera that circles it once, slightly from above, framing the whole model automatically. Viewers compare it with other models of the same description: does it match the description, is it well built (no floating, misplaced or intersecting parts), and does it look good. Only your geometry and materials are seen.
Requirements:
- Build only what the description asks for. No ground plane, base, backdrop, scenery, lights, cameras, environment map, fog or animation.
- +Y is up, the model stands on y = 0, centered on the origin, and its front faces +Z. Units are meters, with realistic proportions.
- Use three.js core geometries (Box, Cylinder, Sphere, Cone, Torus, Capsule, Lathe, Extrude with Shape and bevels, Tube with curves, BufferGeometry with your own vertices) and MeshStandardMaterial or MeshPhysicalMaterial (color, metalness, roughness, transparency, emissive). For painted details you may draw on a canvas and use THREE.CanvasTexture.
- `addons` provides RoundedBoxGeometry, ConvexGeometry, ParametricGeometry, LoftGeometry, mergeGeometries, mergeVertices and toCreasedNormals.
- No imports, external files, images, fonts or network access. It must run in a browser in a few seconds and stay under 1,000,000 triangles.
- Reply with the complete code in a single ```javascript block.Full protocol in the global methodology.
We use crowd intelligence to deliver real, diverse human feedback at scale for model evaluation and post-training.