An AI can generate an entire interactive voxel world, but judging that world is surprisingly difficult: the code may promise features no camera ever sees, while a screenshot may show features the code never clearly names. Large language models can now write complete, interactive 3D worlds as code, but grading those worlds automatically is unreliable.

Existing judges take one view of the output: a vision–language model scores a few rendered snapshots, or a language model reads the source. On worlds written by five frontier models, the two views disagree on 32% of required items, mostly code that no frame shows, and fixed views miss small close-up contents.

A language model returns a program: one HTML file that builds terrain from noise functions, places forests from lookup tables, runs water and weather as particle systems, and turns night into day on its own clock. What that world is only exists while the program runs, so grading it means asking whether the code builds the thing and whether the running world shows it.

Neither answer implies the other: code can build an animal at coordinates no camera ever sees, or define a weather system that never switches on. Render-based benchmarks score a handful of fixed views with a vision–language model, but they see the result and not its cause; a still frame cannot show a river flowing.

Code-based evaluation reads the source, with a model or with patterns, but it sees the cause and not the result, and pattern matching is easy to game. WorldCoder-Bench requires each program to expose its state through a fixed interface and checks it against expert-written contracts, but that does not fit an open-ended creative prompt.

On five frontier models’ worlds, the two views disagree on 32% of required items. The code is there and the frames never show it in 105 of 365 cases, while the frames show it but the code was not found in 10. Disagreement alone does not say which view is right, so the evaluation tests that question on features removed by construction.

WorldBench is a benchmark and judge for open-ended, language-model-written 3D worlds. One prompt asks for a floating voxel island with ten biomes in stated relations, physical water, lava and weather, and day-night and seasonal cycles.

Five tests with fixed maxima, 280 points in total, score the world item by item. The judge takes control of the world’s own clock, measures which render is day instead of assuming it, orbits the island for wide views, and sends a navigator agent in for close views of each biome.

The judge also reads the code for what it is looking at, but neither channel is trusted. A code quote counts only if it is text the source contains, a navigator’s sighting is re-checked blind, and measurable properties such as brightness, motion, and colour shift are decided by pixels instead of the visual judge.

The figure’s center is the key evidence: the code channel adds a “Really there?” source-text check, while the visual channel adds a blind “Really shown?” pixel check after exploration. At the right, both arrows converge on the item verdict, where the two half-scores are kept separate until their checks pass.

Every model receives the same prompt, about 2,000 words, asking for a single self-contained HTML file that renders a floating voxel island with ten biomes in a black void. The prompt fixes relationships rather than coordinates, says nothing about how the world will be graded, and provides no ground-truth world to compare against.

The prompt asks for nothing on the grader’s behalf, while grading runs a copy with one clock patch. Figure 2 traces the evaluation from one shared prompt, through an LLM-generated world.html file, to a Three.js world that must load successfully.

The authors gather evidence in two complementary ways: reading the code, with quotes checked against the source, and exploring the running world, with sightings checked against pixels. Each item combines these evidence types into a score, for a total reported out of two hundred eighty.

WorldBench evaluates five models through their developers’ official APIs: Claude Fable 5.1, GPT-6 Astra, Kimi K3, Grok 4.7, and Gemini 3.1 Pro. Each world is one completion, sampled at temperature one, and continued if the response is truncated.

Before grading, a bounded repair loop parses the file, loads it in a headless browser, and sends parse or console errors back to the model for at most five rounds. Repairs see only errors, never renders or scores, so generation never sees the grader’s view.

A world is graded by five tests with fixed maxima, 280 points in total, so totals compare across models. Each test scores many small, checkable items rather than one holistic grade, and every item is decided by the judge. Worked examples throughout the paper use claude-fable-5-1, the highest-scoring world.

The evidence column shows that the ladder mixes code-based checks with visual verification: WC001 combines a code judge and a frame-based bug hunt, while WC003 and WC004 require verified code quotes alongside navigated frames. For motion, the table adds a strict pixel-change condition, and WC005 measures visual change through brightness, frame difference, and chromaticity.

Every world keeps its own clock, written its own way, so a language model patches that clock to obey an externally set time of day and season. The patch allows at most six edits, each under 600 added characters, and each required to read the external time, so it cannot add lighting the model never wrote.

The patch’s mapping from requested time to noon is not trusted: the world is rendered at four requested times, and the brightest render by mean luma is taken as day. The four seasons are rendered at day, and change over time is measured on these frames rather than judged.

The measurements use mean luma for night, frame difference against day, and chromaticity shift for twilight tint and seasons. In the reference world, the visual judge called the four season frames identical, but their measured chromaticity shifts from spring were 0.019, 0.035, and 0.024, and the measurement decides.

In the top row, the second panel is outlined in green as the reference day, while the other requested times appear visibly darker and retain larger differences from that reference. In the bottom row, the season views share the same island composition, but their color bars and subtle shifts in foliage and terrain show that the measurement captures distinctions the eye largely overlooks.

From day, the judge takes wide views—an overview, four orbits, and the four seasons—and close views of each biome. Close views come from a navigator agent using an observe-act-observe loop; each turn receives the accessibility tree and latest frame, then calls one tool within an eight-step budget.

In the reference world’s longest episode, the navigator clicks the swamp’s legend entry, zooms in, orbits away and back to bring water channels into frame, zooms again, and saves in six steps. Figure six compares still-camera bursts for the same rain item in two jungle scenes.

The visual judge reports falling rain in both, but the red overlays reveal changed pixels in only one burst: it measures thirty-eight point four percent changed in claude-fable-five-one and zero point zero percent in gpt-six-astra. This matters because the authors require visible pixel change to validate motion, rather than accepting a motion claim based on appearance alone.

A single frame cannot show motion, so for each saved close view the judge takes a still-camera burst: four frames two seconds apart, then one zoom out and three more. A visual claim that something moves counts only if the burst’s pixels changed.

In one world, rain falls and 38.4 percent of pixels change; in another, the visual judge reports vertical falling rain particles, but zero percent of pixels change, so the claim is refused. Figure seven illustrates three items checked through both source code and rendered frames.

Item A, exposed rock or stone, earns both halves because line one hundred sixty-three is verbatim and the frame shows stone; item B has a valid quote, but mist is absent from every frame, so only its code half is credited. Item C reverses that pattern: the stream is visible, but it comes from a coordinate-based river table rather than biome-naming code, so only the frames half is earned.

The navigator’s claim that it found a biome is not trusted: in one world, it saved the volcano as the swamp with full confidence. Every saved frame is re-taken with the page’s interface hidden, then identified blind in one shuffled call that is never told which biome a frame was meant to show.

A frame counts as a biome’s close view only if the answer matches; otherwise it is dropped and that biome is judged from wide views alone. A code quote counts only if it is text the source contains: matching ignores whitespace, is done line by line, and at least 80 percent of a quote’s lines must be found.

A line whose ending was garbled in copying still matches if its first 85 percent is verbatim, but invented code does not pass. A quote must also come from that biome’s own code, and one quote cannot satisfy two items. When frames show an item but the probe found no code for it, an evidence agent searches the source with read-only tools over a sandboxed copy of the file, within ten tool calls.

The agent is told that a sighting is a lead, not proof, and that inferring a feature from a neighbouring one is not evidence. Only these disagreements are searched again, and this is the one stage that waits for Looking because it needs the frames. An item’s points are split evenly between the two channels, with the code half earned by a quote that passes the checks.

The frames half is earned when the visual judge reports the item in a blind-confirmed frame, or when pixels confirm it where the property is measurable. Items that must not appear in a biome cost points when either channel finds them, through a verified quote or a sighting in the biome’s frames.

WorldBench checks 73 required items per world, 365 in total, and gives each item a code verdict and a frames verdict. The verdicts agree on 250 items and disagree on 115, or 32 percent, with disagreement ranging from 26 to 41 percent per world. The disagreement is lopsided: code is found but no frame shows the item in 105 cases, versus 10 cases in the other direction.

A code-only judge would credit 289 items, or 79 percent, while a look-only judge would credit 194, or 53 percent. Look across the rows, especially the code-found columns: the frames-seen counts decline from forty-seven for claude-fable-5-1 to twenty-four for gemini-3-1-pro, while code-found but frames-not-seen remains substantial.

Claude-fable-5-1 leads with 210.79 of 280, followed by gpt-6-astra at 199.73, kimi-k-3 at 191.36, grok-4-7 at 187.78, and gemini-3-1-pro at 149.00. The order is not uniform across tests: claude-fable-5-1 leads on contents and physics, kimi-k-3 leads on the voxel island and cycles, and grok-4-7 leads on placement.

The paper reads gaps of a few points between neighbours as ties, including kimi-k-3 and grok-4-7 in the totals. WorldBench argues that reliable evaluation needs both channels: read the program for what it builds, explore the running world for what appears, and verify each claim against source text or measured pixels.