AssBench LLM harness benchmark: how it scores models

AssBench: what the test asks

Luigi published AssBench on grigio.org on 24 September 2026 and updated the note on 1 October 2026. The live board is assbench.com. The rule is one prompt, many models, many dates. Each run returns a single HTML file. You open the files and compare two to four of them side by side.

The prompt asks for one self-contained page that runs from disk in Chrome. The model may load one pinned Three.js build from a public CDN and nothing else: no images, fonts, or downloaded models. Geometry and materials have to be drawn in code. The scene is a stylized lower-body figure in a 400 by 400 pixel frame, with drag-to-orbit and a poke that deforms like soft tissue, and it has to hold about 60 frames per second. The author allows screenshot passes until the angles look right, with a two-hour target and room to run to three hours.

Why this prompt is hard to memorize

Most model benchmarks reuse puzzles and games that have sat on the public web for years. A high score there can mean the model remembered a write-up, not that it solved a new constraint. AssBench has no answer key to recall. Three.js examples exist, and so do anatomy references, but no existing file satisfies this exact mix of scene, frame, interaction, and performance. A detail that makes the figure more believable costs frame rate. A mesh that looks right from the back can fall apart from the front. The model has to hold those tradeoffs in one file and judge its own result.

That is the claim Luigi makes for calling it a harness test. The interesting part is not the joke in the name. It is that the file cannot be copied from a known solution, so the run shows whether the model can follow a long spec and still ship something that opens.

What the first model runs showed

The manifest described in the grigio.org note has 19 entries from Anthropic, OpenAI, Google, and xAI. Most of those runs used an effort setting of max or ultra. Claude Haiku 4.5 ran at its default settings and still produced a clean first-try render, the same first-try outcome Luigi records for the other entries. The practical observation is that some models about ten times smaller land close to frontier models on this task, and sometimes ahead of them. On this board, spatial reasoning and constraint handling inside Three.js matter more than raw size.

The board stores the model id, the settings, the date, and whether the file worked on the first try. That record is what makes a cost comparison possible. A frontier run spends more tokens and more time. A smaller model that returns an acceptable file is the cheaper tool when you repeat the job.

How the 40-point score is meant to work

Planned scoring covers realism, interactivity, physics, and reliability, for a total of 40. As of the 1 October 2026 update, those scores were still empty, so there is no ranking to quote from the write-up. The stated point of filling them in is to see which cheaper runs sit within a point or two of the expensive ones. Until the numbers are published, the only comparable fields are model id, effort, date, and the first-try flag. Until those scores exist, treat the board as a log of runs, not a winner list.

What a smaller model changes for a product

AssBench is one narrow test, not a general leaderboard. Luigi's own conclusion is that frontier models still matter when the task needs them, and that many tasks do not. The check before you move a production agent is the recorded model id and whether the file worked the first time, not the parameter count on the box. If a default-settings Haiku run already clears the same first-try bar as a max-effort frontier run, the invoice is the remaining question.

The prompt itself stays on grigio.org. This page uses the published rules and the manifest counts from that note, and the side-by-side viewer on assbench.com, so the comparison is the same one the author set up.

Sources

• Luigi, AssBench is all You need to benchmark LLM harness intelligence, grigio.org, 24 September 2026
• AssBench board at assbench.com