GDPval Explained: Benchmarking Real Professional Work

Almost every benchmark a developer reads measures a slice of software work. GDPval is the odd one out: it asks whether a model can produce the deliverable an experienced professional would produce, across a wide spread of occupations that show up in economic output.

That framing changes what the number means, and it changes who should care about it. It is not a coding benchmark, and treating it as one will lead you to the wrong model.

What the benchmark asks for

Each task is a piece of real professional work with a real artefact at the end. A brief, a plan, an analysis, a set of documents — the kind of thing someone in that occupation would actually be asked to hand over, with the context they would normally be given.

The output is not scored by a test suite, because there is no test suite for a well-argued memo. It is graded by people with experience in that occupation, comparing what the model produced against work of the kind a competent professional would produce.

Occupational coverage is the point. Software is one strand among many, alongside work that is mostly reading, synthesising, structuring and writing under constraints that are not stated explicitly in the prompt.

That last part is what makes it hard. A real brief omits things everyone in the field knows, and a model that does not know them produces something plausible and wrong in ways only a practitioner notices.

How to read the numbers

Results are published as a rating on a paired-comparison scale rather than a percentage resolved, so read a GDPval figure the way you would read an Arena rating: relative to the other entrants, with error bars, and only meaningful inside its own pool.

On the GDPval-AA v2 scale, Claude Fable 5 Max currently leads at 1,815, with GPT-5.6 Sol Max second at 1,747.8 and Claude Opus 4.8 at 1,600. The interesting fact there is not the ordering but the spread: roughly two hundred points between a current frontier model and one that is still perfectly capable.

Do not convert a rating into an intuition about accuracy. It encodes how often expert graders preferred this model's deliverable in head-to-head comparison, which is a statement about relative quality, not an absolute pass rate.

And check which version you are reading. Scales get rebuilt, grader pools change, and ratings from different revisions are not comparable — the same rule that applies to SWE-bench variants.

What it captures that coding benchmarks miss

Deliverable quality under vague instructions. Most software benchmarks hand the model a well-specified objective and a mechanical check. GDPval tasks are specified the way work is actually specified, which is to say incompletely.

Judgement about what to include. A test suite cannot tell you that a document omitted the risk section, or included three paragraphs nobody asked for. An expert grader can, and does.

Breadth outside the training centre of gravity. Models are trained heavily on code and on general web text; a lot of professional work sits in neither, and performance there is a reasonable proxy for how a model behaves on your own unusual internal domain.

That is genuinely useful signal for anyone building an assistant that writes reports, summarises documents or drafts anything a colleague will read rather than a compiler.

What it does not tell a developer

Whether the model can run an agent loop. Nothing here measures tool calling, error recovery across many turns, or working inside a repository. For that, Terminal-Bench is the relevant measurement and GDPval is close to silent.

Whether it is affordable. Deliverable quality and cost per deliverable are separate axes, and expert-graded benchmarks tend to reward long, thorough outputs, which are expensive outputs. A model two hundred rating points ahead may cost several times as much per task.

Whether it fits your domain. Occupational breadth is a strength for the benchmark and a weakness for transfer: a strong aggregate across many occupations says little about the one you operate in.

There is also the honest limitation shared by all judged evaluation — graders are people, preference is partly presentation, and a well-formatted answer carries an advantage that no rubric fully removes.

Using it in a decision

Reach for GDPval when your product turns model output into something a human reads and acts on. Reach for mechanical coding benchmarks when the output feeds a machine. Most teams need both, weighted by which half of their workload is larger.

Use it to set expectations about tiering, too. A large rating spread across models at very different prices is an argument for routing by task rather than standardising on the top of the table for everything.

Then do the thing no published benchmark can do for you. Take ten real deliverables from your own team's output, run the candidates on the same briefs, and have the person who normally reviews that work grade them blind. Blind comparison is what makes the result trustworthy, and it is the only ranking built from your actual standards.