Willison — The Pelican comparison grid for Astra is pretty interesting (2026-09-04)

simon-willison, 4 September 2026. Arrived by Telegram via a share.google wrapper, resolved to the canonical post before recording.

The first non-first-party contact this wiki has with GPT-6 Astra, and the first source of any kind that states its price. Willison got API access on the afternoon of the 4th and did what he always does: SVG pelicans on bicycles, then a grid.

What he ran

Four models — Astra, and GPT-5.6 Sol, Terra and Luna — at five reasoning levels each (low, medium, high, xhigh, max), so 20 cells. One prompt, rendered side by side.

An artifact fact worth extracting from the aside: Astra does not support reasoning=none, where the GPT-5.6 line does. The floor of the new model is a reasoning model.

The price, at last

That is the number astra listed under Open as not existing. Astra is 2× Sol on input and 1.67× on output — but Willison's point is that the rate card overstates the gap, because "it uses significantly less tokens at each of the levels, making the prices at the different levels closer than they might otherwise be."

The cost-per-result line is the one to keep: "Astra low produces a better pelican than ANY of the GPT-5.6 Sol models at any level, for 9.55 cents. Spending 10 cents on any other model gets a much worse result." A cheaper tier of the expensive model beating every tier of the cheap model at matched spend is the shape llm-api-pricing keeps looking for and rarely gets, because almost nobody publishes per-task cost beside per-task quality.

What he says about quality

"The Astra pelicans are much better." The best Sol output — he preferred xhigh to max, which is its own small finding about whether more reasoning monotonically helps — is "still pretty clearly a bunch of abstract shapes," and "every single one of the Astra pelicans, from low to xhigh, looks better than that." Astra at max is "really good."

He also publishes the failure: Astra below max "doesn't reliably get the pelican legs on both sides of the frame."

The token-count observation, which is a hypothesis and not a finding

"Look at the input token counts: Astra and Luna both used 16 input tokens, Sol and Terra used 26. That's interesting. I wonder if Astra and Luna are more related to each other than OpenAI let on?"

Recorded because it bears directly on astra's open question about where the model sits against the GPT-5.6 line, and recorded as a question, which is how he phrases it. Identical input-token counts on one prompt are consistent with a shared tokenizer, and a shared tokenizer is consistent with a shared lineage and with several other things. One prompt, two numbers, no claim proved. It is worth watching for a second instance rather than repeating as fact.

What kind of evidence this is

Not a benchmark in the sense llm-benchmarks uses. One prompt, one sample per cell, no repetition, no rubric, no blinding, judged by eye by the person who ran it. Nothing here would survive as a measurement, and Willison does not present it as one.

It has three properties the vendor benchmarks on this wiki's other model pages do not, which is why it is T2 rather than T4: the same prompt is put to four models the author did not build, the cost of each cell is published alongside its output, and the weak results are shown rather than omitted. That is a comparison somebody actually ran.

One caution the corpus should carry rather than assume away: the pelican test's value came from being an odd, unrehearsed request, and Willison's own tag for it now holds 141 posts. A prompt that public is a prompt a lab can notice. Nothing here shows that has happened, and there is no way to check it from outside; it is a reason to treat the trend line across model generations more carefully than any single grid.

Loose end in the source

The post links "the transcript that created the GPT-6 Nova pelicans" while every other mention, including the title and the tag gpt-6-astra, says Astra. Left as an inconsistency in the source rather than resolved: it is most likely a slip, but this wiki has been wrong before by tidying a vendor's naming on its behalf, and a second GPT-6 variant name would matter.

Related

astra · simon-willison · openai · openai-gpt56-grok45-clash · llm-api-pricing · llm-benchmarks · qwen38-flash-next-willison · inkling-willison-review · synthesis