Video walkthrough

WorldBench: Evaluating LLMs on Three.js Voxel World Generation

Overview 3:59 CC AI

paperi.ai
0:00 / 0:00

Krish Bakshi

An AI can write an entire interactive world, yet a judge may give credit to a forest, storm, or animal that never appears, and miss one that is really there. The problem is not making the world; it is knowing what exists.

Abstract

Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable. Existing judges take one view of the output: a vision–language model scores a few rendered snapshots, or a language model reads the source. On worlds written by five frontier models we find that the two views disagree on 32% of required items, mostly code that no frame shows, and that fixed views miss small close-up contents. We present WorldBench, a benchmark and judge for open-ended, LLM-generated Three.js worlds. From one prompt describing a floating voxel island with ten biomes, physics, and day/night and seasonal cycles, the judge explores the running world, controlling its clock, orbiting it, and sending a navigator agent to frame each biome, and reads the code for what it sees. Neither channel is trusted on its own: a code quote counts only if it is text the source contains, and visual claims are checked against measured pixels where the property is measurable. A mutation test, in which we remove features by construction, shows that code-only judging gives full credit to four of five removed features, because their code remains in the file. Our judge cuts the points kept on removed features by a third (5.44 to 3.55 of 7.11), and what it still credits is mostly code that exists but never runs. We evaluate five frontier models: Claude Fable 5.1, GPT-6 Astra, Kimi K3, Grok 4.7 and Gemini 3.1 Pro. Code, prompt, tests and judge configuration are available at https://github.com/KrishBakshi/worldbench.

Transcript

An AI can write an entire interactive world, yet a judge may give credit to a forest, storm, or animal that never appears—and miss one that is really there. The problem is not making the world; it is knowing what exists. Large language models can now write complete, interactive three-dimensional worlds as code, but grading those worlds automatically is unreliable.

A source file can promise a feature that the running world never shows. Existing judges take one view of the output: a system scores a few rendered snapshots, or another system reads the source. On worlds written by five frontier models, the two views disagree on thirty-two percent of required items.

WorldBench is a benchmark and judge for open-ended, language-model-written three-dimensional worlds. One prompt asks for a floating block-built island with ten biomes, physical water, lava and weather, and day-and-night and seasonal cycles.

Five tests with fixed maxima score the world item by item, for a total of two hundred eighty points. The judge explores the running world: it controls the world’s own clock, measures which view is day instead of assuming it, circles the island for wide views, and sends a navigator in for close views of each biome.

It also reads the code for what it is looking at, but neither channel is trusted on its own. A code quote must be text the source contains, and measurable visual claims are checked against the pixels. The two views agree on many required items, but they disagree on thirty-two percent.

The disagreement is lopsided: in one hundred and five items, the code contains the feature but no frame shows it, while only ten go the other way. A judge that only read code would credit more items than one that only looked.

But “not seen” can mean either that the feature is absent or that the chosen views never reached it. To test whether exploring matters, the visual judge was run again without the navigator’s close views, leaving the overview and fixed wide orbits—the kind of evidence a snapshot judge has.

Close views find small contents, while wide views find large layout and weather. Exploration matters for coverage, not because any one camera is better. The losses come from different blind spots: some systems miss code that is present but never captured, while others mostly lose points because the required features were never implemented.

A code-only judge gives full credit to all four removed features whose code remains in the file. The combined judge reduces the credit because the frames do not show the removed plants. One removed snowstorm still slips through: the code says it is switched off, while the visual judge reads drifting snow near the peaks as a storm.

Even the combined judge can accept code that never runs, so dead code earns half credit. Across the tested worlds, the overall scores differ, but the order is not uniform across tests. One model leads on contents and physics, another on the island and cycles, and another on placement.

The reported gaps between neighbouring models are only a few points, so the study treats such neighbours as tied rather than claiming a firm ranking. The limits are important: the study grades one world per model, from one prompt, once, and single item verdicts are noisy.

Totals are steadier, but small gaps between neighbouring models are treated as ties. Repeated generations and repeated grading would show variation that this study does not report. So the result is not a permanent league table; it is a warning about how easily an AI world can be misunderstood.

WorldBench shows that judging an AI-built world requires both the program and the running scene, explored from close and wide views. That matters whenever people rely on AI to create interactive spaces they can actually use.

A derivative work by Paperi · AI-generated script, voice and captions · pages and figures unaltered

Made with Paperi.

Drop in a research PDF — get a narrated video walkthrough like this one, with highlights that follow the narration. Free to start.

Try it with your paper →

More in Computer Science

SepGen: Multi-Stem Audio-Video Separation and Generation in a Single Model 4:12

SepGen: Multi-Stem Audio-Video Separation and Generation in a Single Model

Imagine changing the camera angle in a video and having the sound change with it. That only works if every voice and sound exists on its own track, and this system creates those tracks as the scene is made.

On-Chain Archaeology of Bitcoin Oracles: Evidence of Use under Limited Observability 4:17

On-Chain Archaeology of Bitcoin Oracles: Evidence of Use under Limited Observability

Bitcoin was already using outside information to settle valuable agreements before most people had heard of the problem. But much of that history is now missing, and the surviving record can point researchers in the wrong direction.

Autoregressive Retriever: Improving Query Understanding from Item Feedback for Universal Multimodal Retrieval 3:32

Autoregressive Retriever: Improving Query Understanding from Item Feedback for Universal Multimodal Retrieval

Most search engines decide what you mean before they show you anything. This system lets the first results help rewrite the search itself, so a vague request can become clearer while the search is happening.

All 21 papers in Computer Science →