Simon Willison Put Blender Under Codex Control, but the Render Is Only Half the Story

Simon Willison turned one prompt into an editable Blender scene in 2 minutes and 39 seconds, despite never operating the application through its visual interface. His coding agent generated Python, launched Blender on macOS, built a pelican riding a bicycle, and saved the result as a native .blend file.

That distinction matters. This was not another text-to-image system producing a flattened picture that becomes difficult to revise. The agent manipulated a programmable creative application, leaving behind source code and structured 3D objects that a person could inspect, edit, rerender, or animate.

The experiment also exposed the real contest surrounding coding agents. The important divide is no longer between writing code and making visual media. It is between software that exposes dependable programmable controls and software that remains trapped behind manual interface operations.

Willison’s example is compact, whimsical, and based on one user’s experience. It is not a controlled benchmark. Yet it offers a useful preview of how agents can extend beyond code repositories without waiting for custom integrations from every application developer.

Simon Willison Made Blender a Coding Agent Target

The notable event was not the pelican image. It was Codex treating an installed desktop application as an executable development tool.

In a September 5 post, Simon Willison described using ChatGPT Codex on his Mac to control Blender. His initial request was direct: use the installed Blender application to render a pelican riding a bicycle.

The agent found a route through Blender’s command-line executable and Python interface. Willison later supplied the more explicit command /Applications/Blender.app/Contents/MacOS/Blender --background --python scene.py, which launches Blender without its normal interface and runs a script.

Background mode means Blender operates without opening its graphical workspace. That makes it suitable for automated rendering, server jobs, test pipelines, and agent-controlled tasks.

Willison reported that the first prompt produced a .blend project and Python script after 2 minutes and 39 seconds. He then requested “a background and a lot of flair,” receiving another version after 3 minutes and 51 seconds.

A final request to make the work “a whole lot better” took 5 minutes and 59 seconds. The resulting coastal parade scene included a boardwalk, ocean, sunset, beach cabins, palm trees, flowers, and a more detailed pelican.

The complete sequence, prompts, output files, and timings appear in Willison’s Blender experiment. That public record makes the example more informative than a polished image posted without its construction history.

The generated files also show what the agent actually did. It did not call a hidden image generator and paste the result into Blender. It wrote scene-building instructions through bpy, Blender’s Python module for accessing objects, materials, cameras, lights, geometry, rendering settings, and project data.

Willison published the final script, which contains 128 lines in its last revision. The scene source creates and modifies individual elements such as the bicycle, bird, boardwalk planks, clouds, beach cabins, sailboat, and woven basket.

That evidence narrows the claim. The experiment shows that one coding agent, one model configuration, and one local Blender installation completed a specific stylized scene. It does not establish general reliability across arbitrary 3D work.

Still, the workflow crossed an important boundary. A conversational instruction became code, the code controlled a mature desktop application, and the application produced both an editable project and a final render.

Why Blender on macOS Was Ready for This Moment

Blender already supplied the automation surface, while the coding agent supplied translation, iteration, and execution.

Coding agents work best when they can inspect a system, write a small program, execute it, and evaluate an observable result. Blender supports each part of that loop without requiring a special agent plugin.

Its Python API exposes scene objects as programmable data. A script can create meshes, adjust coordinates, assign materials, position cameras, configure lights, save project files, and initiate rendering.

The application also accepts command-line arguments on macOS. Once the full desktop application is installed, its internal executable can run from the terminal. The coding agent therefore encounters Blender as another tool available on the local machine.

This changes the integration problem. A developer does not need to wait for a dedicated “Blender connector” that converts a limited set of natural-language commands into interface actions. The agent can instead use the same scripting and command-line mechanisms already available to technical artists.

That approach fits how Codex works in a local environment. According to the Codex documentation, the agent can inspect files, use a terminal, edit code, and run commands within the permissions granted by the user.

Blender contributes deterministic execution at the application layer. The language model contributes an imperfect but flexible planner that converts intent into Python. Neither component provides the entire workflow alone.

The timing is important because current coding agents can sustain longer sequences than simple autocomplete systems. They can create a script, run it, notice an error, revise the file, and repeat the process while preserving the project state.

A conventional chatbot might produce sample Blender Python that a user must copy, debug, and run manually. An agent can close that execution gap by handling those steps in the same working session.

The visual output also gives the agent and user a concrete checkpoint. A render can reveal framing errors, missing geometry, poor lighting, or an overloaded composition more quickly than reading every coordinate in the generated script.

However, visual feedback does not guarantee visual judgment. An agent can successfully render an image that still contains awkward anatomy, inconsistent scale, intersecting objects, or weak composition. Execution success and artistic success remain separate standards.

This is why the local application matters. Blender preserves editable geometry and materials after the initial generation. A human artist can correct defects directly instead of asking the model to regenerate an opaque image from scratch.

For many creative tasks, editability is more valuable than a striking first result. It lets teams keep approved elements, isolate mistakes, and change only the parts that need work.

The Real Contest Is APIs Versus Interface Automation

Willison’s experiment favors applications with scriptable internal models over workflows that depend on simulated clicks.

Computer-use agents generally operate software by interpreting screenshots and controlling a mouse or keyboard. That route offers broad compatibility because almost every desktop application has an interface.

It also introduces uncertainty. Buttons move, dialogs interrupt the sequence, window focus changes, and the agent must infer state from pixels. A missed click can quietly redirect the entire workflow.

Blender’s Python API avoids much of that ambiguity. The agent can address an object, camera, material, or render setting through named operations. The resulting script becomes an inspectable account of its actions.

This is not perfect determinism. Generated code can contain invalid calls, poorly chosen parameters, or logical errors. Blender versions can also change API behavior.

Yet a code failure usually leaves better evidence than an interface failure. The user can retain the script, inspect an exception, compare revisions, and rerun the same command.

The .blend file adds another layer of inspectability. It contains the structured scene rather than only its final pixels. Users can open the project and examine what the agent created.

Willison’s final script illustrates that structure. It programmatically places boardwalk planks, constructs palm leaves, generates foam lines, and adds individual basket elements. These are addressable components, not a single merged picture.

That produces a practical advantage for iterative prompts. “Add a background” can modify the existing scene without discarding the bicycle and pelican. “Make it better” can refine selected components while retaining prior work.

The weakness is that vague language still forces the model to make unspoken design choices. “Better” might mean more detail, clearer composition, improved realism, or simply more decorative objects.

Willison’s result leaned toward a polished, toy-like coastal illustration. Another user might have wanted physical realism or a sparse editorial style. The agent cannot reliably infer every unstated preference.

This creates a new responsibility for creative software vendors. Products with documented scripting surfaces, stable file formats, and headless execution are easier for agents to operate and easier for users to audit.

Applications that only expose visual controls place the agent in a fragile imitation of human interaction. Applications that expose structured commands let the agent work closer to the program’s underlying state.

Blender is especially well positioned because it combines visual editing, Python automation, rendering, animation, and native project files. That combination turns it into both a production tool and an execution environment.

The same principle reaches beyond 3D graphics. Video editors, design applications, data tools, and digital audio workstations become better agent targets when their projects can be created and modified through code.

That does not make graphical interfaces obsolete. It changes their role. The agent can handle repetitive construction, while the interface remains the place where a person reviews, corrects, and art-directs the result.

What the Pelican Render Does Not Prove

A successful demonstration shows workflow feasibility, not dependable creative production.

Willison presented a personal experiment rather than a benchmark. There were no repeated trials, independent evaluators, controlled prompts, or comparisons across models and Blender versions.

The reported timings are useful observations, but they should not become generalized performance numbers. Rendering time depends on the Mac, scene complexity, render engine, resolution, and the agent’s number of attempts.

The example also benefited from a forgiving subject. A stylized pelican on a bicycle can survive exaggerated anatomy and playful proportions. Architectural visualization, product design, medical animation, and engineering work impose much tighter accuracy requirements.

A scene can look convincing while remaining technically poor. Mesh topology may be difficult to edit. Materials might behave inconsistently under different lighting. Objects could intersect outside the selected camera angle.

There is also no evidence here that the agent optimized geometry for animation, real-time rendering, or downstream export. A still image only tests the scene from one viewpoint and one moment.

The final script builds many visual elements procedurally. That gives users a traceable artifact, but generated procedural code can become difficult to maintain if it lacks clear organization.

Successive prompts can compound that problem. An agent might append new operations rather than redesigning an unstable foundation. The project can improve visually while its internal construction becomes more brittle.

Security deserves equal attention. A coding agent that can execute Blender can also execute generated Python with the permissions available in its environment. Users should inspect unfamiliar scripts and restrict access to sensitive files.

The Blender executable itself is not the risk. The risk comes from granting generated code broad access without understanding what it reads, writes, downloads, or launches.

Local agents also produce a more complicated trust boundary than hosted image generators. They may access project directories, reference images, scripts, render outputs, and other resources on the same computer.

Teams need explicit rules about which directories the agent can use and which commands require approval. Those controls become more important when creative projects include unreleased designs or client material.

Licensing introduces a different concern. Blender is distributed under the GNU General Public License, while artistic output generally remains the creator’s property. The Blender license does not resolve rights questions involving generated code, training data, third-party assets, or copied styles.

Users must still track the provenance of textures, models, reference images, and other inputs. An editable output is easier to inspect than a flattened image, but editability does not establish clean provenance.

Quality control therefore remains human work. An experienced artist can identify anatomical, compositional, lighting, and production defects that a general coding agent may overlook.

The strongest interpretation is modest. The test demonstrates that a coding agent can orchestrate a real creative application and produce a useful starting point. It does not show that creative direction has become automatic.

Coding Agents Gain More Than an Image Generator

The deeper change is the creation of a reusable production system rather than a single visual asset.

Willison ended his experiment by asking Codex to create a skill describing how to use the installed Blender application. A skill is a set of operational instructions that helps an agent repeat a specialized workflow.

That final step turned a successful session into reusable knowledge. Future requests no longer needed to rediscover the executable path, background-mode command, or basic approach to scene scripting.

This matters because agent productivity often depends on retained procedure. A model may be capable of finding a solution each time, but repeated discovery wastes time and introduces variation.

A saved skill can document the command to launch Blender, expected file locations, rendering conventions, and validation steps. It can also define when the agent should save intermediate .blend files.

The underlying lesson is familiar to engineering teams. A one-off result becomes more valuable when its process is recorded, reviewed, and reused.

Teams can apply the same pattern to brand renders, product mockups, storyboard scenes, or recurring data visualizations. The agent builds within a documented pipeline instead of improvising every project.

A good reusable workflow would separate generated source files from rendered outputs. It would retain prompt history, name scene objects consistently, and preserve checkpoints before major revisions.

Those practices make agent work easier to review. They also reduce the damage from a vague follow-up request that changes too much.

Willison’s public repository captures part of that history. It includes successive .blend files, Python scripts, and an exported transcript, allowing readers to inspect the path from the first request to the final render.

That record is more valuable than the final image alone. It shows where the agent used code, how the scene expanded, and which artifacts remained editable.

Organizations exploring similar workflows should treat prompts, scripts, project files, and review notes as connected technical knowledge. A searchable engineering knowledge base can preserve why a workflow succeeded, not merely where its files live.

This approach also changes the economics of small creative experiments without requiring a price comparison. A developer can test a visual concept before involving a specialist in detailed production.

That should not be framed as replacing a 3D artist. It shifts the starting point. Artists may receive a rough structured scene instead of a paragraph, while developers can explore ideas that previously stalled before prototyping.

The handoff becomes especially useful when generated objects are clearly named and grouped. A professional can then replace weak geometry, adjust materials, or rebuild the rig without reconstructing the entire scene.

Coding agents can also connect Blender to surrounding tools. They can prepare input data, generate scene scripts, organize renders, and invoke media utilities for output processing.

Willison noted that agents can render image sequences and combine them with FFmpeg. That expands the pattern from a still image into an automated animation pipeline, although his pelican example focused on the rendered scene.

The broader value is orchestration. The agent does not need to become the best modeler, renderer, or video encoder. It needs to coordinate specialized tools while preserving artifacts that humans can inspect.

What to Watch After Simon Willison’s Blender Test

Three signals will determine whether this pattern grows beyond an impressive personal demonstration.

The first signal is reproducibility across models, machines, and Blender releases. Other users should be able to give comparable prompts and receive valid scripts, editable project files, and successful renders.

Repeated tests should track more than whether an image appears. They should examine error rates, retries, scene organization, render consistency, and how well the project survives later edits.

If those results remain stable across different environments, the case for Blender coding agents becomes stronger. If success depends on one model configuration and careful rescue prompts, the workflow remains experimental.

The second signal is whether creative professionals adopt agent-generated scenes as workable starting assets. Their judgment matters because they can evaluate topology, materials, lighting, naming, composition, and downstream compatibility.

A professional workflow must tolerate revision. The scene should remain understandable after multiple prompts, transfer cleanly between people, and support changes beyond the original camera view.

Evidence of artists refining generated .blend files would strengthen the claim that agents can participate in production. A stream of attractive but disposable renders would weaken it.

The third signal is how creative software makers improve programmable access. Blender already exposes a mature Python interface and headless execution. Other applications may respond with better scripting, structured project APIs, agent-specific documentation, or safer permission models.

If vendors invest in those surfaces, the competition will shift away from raw interface control. Agents will increasingly operate applications through explicit commands and inspectable state.

If vendors prioritize closed interfaces, agents will continue relying on screenshot interpretation and simulated clicks. That route can cover more software, but it remains harder to reproduce and audit.

Simon Willison’s example gives developers a practical test today. Choose a bounded scene, keep every script and project revision, and evaluate the editable result rather than only the final render.

Ask whether the agent created a file another person can understand. Check whether the next prompt improves the scene without damaging previous work. Review the generated Python before granting it broader access.

Most importantly, judge the workflow by the quality of the handoff. A delightful pelican image attracts attention, but an editable scene, readable script, and repeatable procedure create lasting value.

That is the conflict this experiment brings into focus. Coding agents can now reach well beyond source repositories, but only software with accessible controls gives them a reliable path.

The next decisive examples will not be the most visually extravagant. They will be the ones where a human can open the project, understand the agent’s choices, correct its mistakes, and confidently continue the work.