Anthropic Simon Pelican Test: Claude Fable 5.1 Looks Better, but Only After Thinking Much Longer

Anthropic released Claude Fable 5.1 with a reported 52.6% science benchmark score, but the anthropic simon pelican test exposes a different performance story. Simon Willison asked the model to generate an SVG of a pelican riding a bicycle. It produced its most convincing result only after spending far more time and output tokens at maximum reasoning effort.

That contrast makes the experiment more useful than its absurd premise suggests. Anthropic presents Fable 5.1 as a model for coding, knowledge work, and long-running problems. Willison’s pelican tests what happens when those capabilities meet a small, visual programming task with obvious physical constraints.

The result is not a replacement for Anthropic’s formal evaluations. It is a compact demonstration of the tradeoff behind them. Fable 5.1 can inspect, revise, and improve its work, but more reasoning does not deliver a uniform gain. The best output arrived with much higher computational effort, while lower settings showed little visible deliberation.

Anthropic Shipped a Model Built to Keep Working

Claude Fable 5.1 is less about answering one prompt quickly and more about sustaining a task until the details hold together.

Anthropic introduced Claude Fable 5.1 and Claude Mythos 5.1 on September 1, 2026. The company describes them as the same underlying model with different safeguards. Fable is generally available, while Mythos is restricted to approved research programs.

The company’s launch materials center on coding, research, and long-running knowledge work. Anthropic says Fable 5.1 avoids superficial fixes, checks its own work, and remains effective during projects that span hours or multiple applications.

Those claims matter because many current AI failures happen after a promising start. A model can write a plausible plan, generate functional code, and still lose track of constraints during later revisions. Long-running performance depends on maintaining state, testing intermediate results, and correcting errors without damaging earlier work.

Anthropic’s benchmark results show moderate gains across several established tests. The largest reported jump appears on Terminal-Bench-Science 0.1, an evaluation of scientific tasks completed through a terminal environment.

Fable 5.1 scored 52.6% in Anthropic’s setup. The company reported 24.7% for Fable 5, 29.0% for Opus 5, and 22.4% for GPT-5.6 Sol under the same comparison. Anthropic also discloses a standard error between 3.5 and 4.5 percentage points for each model.

Other results moved by smaller margins. Fable 5.1 reached 55.8% on Terminal-Bench 4.0, compared with 42.0% for Fable 5. It scored 31.4% on AutomationBench, while Fable 5 scored 17.1%.

These are company-reported figures, not universal measures of model quality. Anthropic also notes that safeguards affected some evaluation tasks. Certain flagged requests received zeros, while others were routed to different Claude models.

That disclosure complicates direct comparisons. A benchmark can measure the underlying model, the deployed safety system, the agent harness, or some combination of all three. Enterprise users encounter the combined product, so those operational details belong in any performance discussion.

Anthropic’s early customers supplied supporting examples. MongoDB said the model researched its services and documentation before building a prototype over several days. Millennium said Fable 5.1 traced a rare crash to an external library after earlier models missed the cause.

Those reports are useful, but they come from selected launch partners. They show what successful deployments can look like, not the average result across every codebase or workflow.

Willison’s pelican occupies the opposite end of the evaluation spectrum. It has no enterprise system, research laboratory, or large document collection. Its value comes from making the model’s behavior visible.

Why Simon Willison Still Makes Models Draw Pelicans

The pelican prompt is valuable because success requires many small relationships to work at once, even when the task sounds trivial.

Willison has repeatedly asked language models to generate an SVG of a pelican riding a bicycle. SVG is a text-based image format, so a language model can create the entire picture by writing structured markup and drawing instructions.

A plausible result needs recognizable wheels, a bicycle frame, handlebars, pedals, and a pelican. A good result must also connect them correctly. The bird’s feet should reach the pedals, its wing should meet the handlebars, and its body should sit above the frame.

Those relationships create a compact test of planning and spatial consistency. A model can produce valid SVG code while failing the actual scene. It might draw disconnected legs, misplaced wheels, impossible joints, or an animal that merely floats above a bicycle.

The test became popular because early improvements were easy to see. Better models tended to produce cleaner code, more coherent geometry, and stronger prompt adherence. One image could reveal errors that a long benchmark report might hide behind an aggregate score.

Willison later became less confident in that connection. In his July reassessment, he argued that pelican quality no longer tracked general model quality as reliably as before. Image-making style, training preferences, and reasoning settings increasingly influenced the outcome.

That limitation changes the benchmark’s purpose. It is weak evidence for declaring one model universally better than another. It remains useful for comparing related models, reasoning levels, and repeated runs under similar conditions.

The Claude Fable 5.1 experiment follows that narrower approach. Willison tested five reasoning settings: low, medium, high, extra high, and maximum. Fable 5.1 offered no setting that disabled reasoning entirely.

At low effort, the model produced a clean, recognizable illustration. Willison’s recorded transcript showed no summarized reasoning, although the response contained 1,998 output tokens and took 23.8 seconds.

Medium effort behaved similarly. The output used 1,977 tokens, took 23 seconds, and again displayed no reasoning summary. The final image did not show an obvious advantage over the low result.

High effort finally exposed a small planning trace. The model described the intended scene, including the bicycle, pelican, background, wheels, and body placement. It used 2,612 output tokens and finished in 29.6 seconds.

The visible improvement remained modest. That result matters because reasoning controls are often presented as a simple quality dial. In this experiment, moving from low to high did not create a proportional gain.

Extra high changed the behavior dramatically. The model generated 36,767 output tokens and worked for seven minutes and 51 seconds. Its reasoning discussed the bird’s proportions, feet, pedals, wing position, and visual character.

Maximum effort extended that process further. It used 65,927 output tokens and took 13 minutes and 54 seconds. Willison called the result the best pelican he had seen from an Anthropic model.

The image included a blue helmet, a fish basket, a coherent bicycle, and more careful contact between the bird and the machine. Its visual details reflected repeated inspection rather than one-pass generation.

That progression is the real experiment. The prompt stayed stable while the model’s available deliberation changed. Fable 5.1 did not simply draw a better pelican at every higher setting. It crossed into a different working mode near the top.

The Anthropic Simon Test Exposes an Effort Gap

The best pelican demonstrates stronger self-correction, but it also reveals how sharply quality can depend on inference effort.

The anthropic simon test is most informative when treated as a comparison inside one model family. Low, medium, and high produced acceptable illustrations with limited visible separation. Extra high and maximum triggered much longer reasoning and more deliberate revision.

At maximum effort, the model did more than list objects it needed to draw. It considered how those objects should interact. The reasoning trace examined helmet placement, beak overlap, feather shapes, handlebar details, and the curve of the bicycle’s front fork.

That behavior resembles an iterative design review. The model generated a structure, inspected likely defects, and adjusted individual components. It also rejected unnecessary additions when they threatened clarity.

One trace showed the model reconsidering a bicycle helmet because it might interfere with the pelican’s recognizable crest. Another identified the fork’s incorrect curve and changed a control point to improve its direction.

These decisions are small, but they illustrate a larger capability. Useful long-running agents must detect when their own first attempt is merely plausible. They then need to isolate a weakness and revise it without destabilizing everything else.

That mechanism connects the pelican to real coding work. A software agent might create a feature that passes a basic test yet violates an architectural constraint. A research agent might complete an analysis before noticing that one assumption corrupts the conclusion.

The harder problem is not producing more text. It is spending additional computation on the right checks. Fable 5.1’s maximum trace suggests progress on that behavior, at least within this visual coding task.

However, the experiment also exposes an uneven effort curve. Medium did not outperform low in a meaningful way, despite being a higher setting. High added some planning but did not transform the image. Most of the visible gain arrived much later.

Developers should therefore avoid assuming that every reasoning step buys equal quality. A workload may sit below a threshold where added effort changes little. Another workload may benefit only when the model has enough room to perform repeated inspection.

This creates an evaluation challenge. If a team tests only the default setting, it may underestimate what the model can do. If it tests only maximum effort, it may measure a configuration that is too slow for normal use.

Anthropic says Fable 5.1 defaults to high effort in Claude Code, while other Claude interfaces use medium. Willison’s results suggest that interface defaults can shape a user’s entire impression of the model.

The test also separates output quality from operating efficiency. The maximum pelican was better, but it required substantially more tokens and time than the lower-effort versions. That tradeoff matters even when an article avoids discussing specific prices.

Long-running agents consume compute while they plan, inspect files, call tools, run tests, and revise their work. A successful result can justify that effort for a difficult task. The same process would be excessive for a disposable illustration or routine transformation.

The practical question is not whether maximum reasoning is good. It is whether the extra work changes the decision, reduces downstream review, or prevents an expensive mistake.

That distinction puts pressure on Anthropic and its competitors. OpenAI, Google, and other model providers increasingly expose reasoning controls or automatically allocate computation. Buyers need evidence that those controls map cleanly to workload value.

The pelican suggests that the mapping remains irregular. A higher label does not guarantee a visibly better result, while the top setting can behave like a substantially different product.

The Science Benchmark Raises the Stakes

Anthropic’s science score makes Fable 5.1 important, but the pelican explains why the score requires operational context.

Terminal-Bench-Science 0.1 is designed around scientific workflows that agents perform in terminal environments. Its tasks cover fields including biology, chemistry, physics, earth science, mathematics, and engineering.

The science benchmark launched shortly before Fable 5.1. It contains 70 tasks contributed by scientists, with outcomes that can be checked inside controlled environments.

That structure is more rigorous than judging an illustration. The tasks require agents to navigate software, manipulate data, use scientific tools, and reach verifiable results. They are intended to resemble parts of real research work.

Anthropic’s 52.6% result is striking because it more than doubles the 24.7% score it reported for Fable 5. The gap remains much larger than the disclosed standard error.

The comparison still needs caution. Anthropic reproduced the earlier model results using its own evaluation setup. The public leaderboard reported 30.0% for Opus 5 and 21.4% for Fable 5, while Anthropic measured 29.0% and 24.7%.

The company says those differences fall within expected statistical noise. Even so, readers should distinguish between public leaderboard runs and vendor-run comparisons.

A new benchmark also has limited history. Researchers have not yet observed how rapidly developers optimize prompts, harnesses, and tools around its tasks. They also lack a long record connecting benchmark gains with measurable scientific output.

That does not make the result unimportant. It means the result is an early signal rather than a settled verdict.

The official benchmark and the pelican test measure different things. Terminal-Bench-Science asks whether an agent completes a defined scientific workflow. The pelican reveals how the model allocates effort while solving a visible constraint problem.

Together, they support a narrower conclusion. Fable 5.1 appears better at sustained tool-based work, and its strongest behavior emerges when it has enough computation to inspect intermediate results.

Anthropic provides several scientific examples beyond the benchmark. The company says its models worked on protein binders, planetary mapping, and GPU optimizations for biological models. These claims combine model output with external tools and, in some cases, laboratory validation.

The company also says Fable 5.1 trained a neural network that produced a higher-resolution elevation map covering one-third of Venus. According to Anthropic, the map resolves details at two to three kilometers instead of 10 to 20 kilometers.

Such examples deserve more scrutiny than a one-shot prompt. A scientific result depends on data selection, tool configuration, validation methods, and human oversight. The model may perform important work without being responsible for the entire discovery process.

This distinction matters for enterprise adoption. A buyer does not deploy a benchmark score. It deploys a model inside a system with permissions, proprietary data, reviewers, budgets, safeguards, and failure procedures.

The strongest evidence will come from repeated workflows where teams can compare completion rates, correction time, and human review requirements. A model that scores higher but demands extensive verification may create less value than expected.

Conversely, a slower model can be worthwhile when it prevents a difficult error. Anthropic’s launch partners emphasize examples involving rare failures, multi-service code changes, and unattended research. Those are precisely the cases where additional reasoning has a plausible return.

The science benchmark increases the pressure on OpenAI and Google because it establishes a visible lead on a new agentic evaluation. Yet the larger contest is not one leaderboard. It is whether models can turn extended reasoning into dependable, auditable work.

What the Pelican Does Not Prove

A polished SVG is evidence of one successful run, not proof that Claude Fable 5.1 will reason reliably across unrelated tasks.

The first limitation is sample size. Willison showed one sequence across five effort settings. Model outputs can vary between runs, even when the prompt and configuration remain unchanged.

A stronger comparison would repeat each setting several times. Reviewers could then score physical consistency, code validity, visual quality, execution time, and output length. That would reveal whether the maximum result was typical or unusually good.

The second limitation is subjective judgment. Most viewers can agree that the maximum pelican looks more complete, but visual appeal is not a single measurable property. One person may prefer a minimal illustration while another rewards decorative detail.

The third limitation is contamination. The pelican prompt has circulated publicly for a long time. Model developers can see the examples, and related images may appear in training or evaluation data.

There is no evidence that Anthropic explicitly optimized Fable 5.1 for this prompt. Still, a familiar test becomes less useful as an independent measure once many outputs and discussions are public.

Willison already acknowledges that the benchmark’s relationship with general model quality has weakened. Its best remaining use is controlled comparison, especially within one provider’s model family.

The fourth limitation concerns visible reasoning. An absent reasoning summary does not establish that the model performed no internal reasoning. Products can hide, compress, or selectively display traces.

Willison carefully described low and medium as appearing to skip reasoning. That reportorial distinction should remain intact. The recorded interface behavior is observable, but the model’s internal process is not fully available.

The fifth limitation is that long traces can create false confidence. A model that discusses many details may still make a basic mistake. More deliberation can improve error detection, but it can also produce unnecessary revisions or rationalize a flawed approach.

The animated follow-up makes that risk visible. A Hacker News request asked whether the solved pelican could be animated. Willison fed the maximum SVG back into Fable 5.1 at high effort with the instruction to animate it.

The model produced an animated version using 26,201 output tokens. Willison noted that the wheels appeared to rotate in the wrong direction after conversion to video, although the original SVG seemed correct.

This follow-up is more than a joke. It tests whether a model can preserve a working artifact while adding behavior. That pattern resembles software maintenance, where a new feature can expose problems absent from the original implementation.

Willison’s full pelican experiment also shows why final artifacts need direct inspection. A valid SVG, a coherent reasoning trace, and a successful animation command do not guarantee every visual relationship survived export.

A sixth limitation comes from benchmark alignment. The pelican mainly probes SVG generation, spatial relationships, and iterative design. It says little about factual reliability, security decisions, scientific judgment, or performance on private enterprise data.

Anthropic’s formal evaluations cover some of those areas, but many results remain vendor-reported. Launch-partner testimonials also describe selected successes rather than controlled failure rates.

That leaves an important verification gap. Fable 5.1 appears capable of more sustained work, yet teams still need independent tests built from their own tasks. They should measure how often the model finishes correctly, not how impressive its best session looks.

A useful evaluation should include ambiguous requirements, tool failures, stale documents, and adversarial content. Long-running agents must handle those conditions without silently changing the goal.

The central tension therefore remains unresolved. Fable 5.1 can spend much longer improving an output, but users need reliable ways to decide when that effort is justified and when the model should stop.

Three Signals Will Show Whether Fable 5.1 Delivers

The next test is whether Fable 5.1’s benchmark and demo gains survive repetition, real workflows, and competitive pressure.

The first signal is independent reproduction of Terminal-Bench-Science 0.1. Researchers should run Fable 5.1 across multiple trials using documented harnesses and comparable tool access.

Results close to Anthropic’s 52.6% figure would strengthen the company’s claim. A materially lower public score would suggest that evaluation setup, prompting, or private configuration contributed more than the headline implies.

The variance between runs will matter too. A model that averages well but fails unpredictably presents a different operational profile from one that produces slightly lower, steadier results.

The second signal is workload-level evidence from developers and enterprise teams. The most revealing metrics will include successful completion rates, review time, corrected defects, interrupted runs, and the frequency of human intervention.

Teams should compare high and maximum reasoning on the same internal tasks. They should also record cases where added effort changes nothing or makes the result worse.

That evidence would turn the anthropic simon effort gap into an operational question. If maximum reasoning consistently prevents expensive failures, the longer runtime becomes defensible. If gains appear only in selected demonstrations, the top setting will remain difficult to justify.

The third signal is how competing models respond. OpenAI’s GPT-5.6 Sol trails Fable 5.1 in Anthropic’s science comparison, while Google’s models remain strong at visually expressive SVG generation.

A competitor can answer in several ways. It can outperform Fable on the public science leaderboard, improve automatic reasoning allocation, reduce the time required for comparable results, or publish stronger independent workflow evaluations.

The most meaningful response will combine quality with predictability. Developers do not merely need a model capable of producing a remarkable artifact. They need controls that communicate how much effort a task received and why the system stopped.

For knowledge workers, the same principle applies to research and document analysis. A longer answer is not necessarily a better answer. The useful system is one that checks the evidence, catches missing constraints, and clearly exposes uncertainty.

The anthropic simon pelican test gives Claude Fable 5.1 a memorable demonstration, but its lesson is not that the benchmark has been solved. The lesson is that sustained self-review now produces visibly better work, while the cost of that review remains uneven.

Developers should test the model with artifacts they can inspect and failures they already understand. Run the same task at multiple effort levels, compare the final results, and record where additional reasoning changes the outcome.

The next compelling demonstration should not be another perfect bird. It should show that Fable 5.1 can deliver the same careful correction across repeated, consequential work without demanding maximum effort every time.