RP2040 Drawing Model Executes Exact Programs, but the AI Stays on the Host
The RP2040 drawing model generated compact programs for 12,670 hardware tests, yet its 825,344-parameter transformer never ran on the microcontroller. Instead, a host computer produced drawing bytecode and sent it to a Raspberry Pi Pico for deterministic execution.
That distinction defines both the project's value and its limits. This is not another claim that a tiny device can run a useful generative model locally. It is an experiment in separating uncertain neural generation from exact, constrained execution.
The system challenges the usual pixel-generation route. It asks whether a small transformer can write an executable description, then hand that description to a minimal machine with predictable behavior. The hardware results look unusually clean, while the model's ability to compose unfamiliar structures remains far less certain.
The RP2040 drawing model separates generation from execution
The central result is a division of labor, not on-device neural inference.
According to the project's public repository, an autoregressive transformer with 825,344 parameters runs on a host computer. It generates roughly 100 bytes of drawing bytecode for each example.
Bytecode is a compact instruction format interpreted by another program. Here, it describes operations such as moving a virtual pen, drawing lines, evaluating curves, applying integer transforms, and repeating bounded sequences.
The host transfers that program to a Raspberry Pi Pico. A small virtual machine, or VM, executes the instructions on the Pico's RP2040 microcontroller. It then streams geometric coordinates back through UART, a standard serial communication interface.
The final image comes from those returned coordinates. The Pico does not store transformer weights, run matrix-heavy neural inference, or require a tensor runtime. It only interprets the generated program.
That boundary matters because the phrase "model on an RP2040" would suggest a different technical achievement. A model with 825,344 parameters could require several megabytes with conventional numeric formats before accounting for runtime memory.
The project's author explicitly avoids that claim. The original discussion says the transformer remains on the host, while the Pico stores and executes its output.
The released demonstration model covers five categories: cats, buses, flowers, sailboats, and bicycles. It is not presented as an open-ended text-to-image system.
That narrow scope makes the results easier to interpret. The experiment studies representation, program generation, and constrained execution without claiming broad visual knowledge.
The model outputs instructions rather than a grid of colored pixels. This creates a useful interface between probabilistic AI and deterministic embedded software.
A pixel generator commits directly to the visible result. A program generator instead proposes a sequence that another system can validate, limit, execute, or reject.
That difference creates the article's main tension. Hardware execution appears exact and economical, but exact execution does not guarantee that the generated program represents the intended drawing.
The Pico can perfectly execute a poor bicycle program. It can also reliably reject a malformed or overlong sequence if the VM enforces suitable limits.
In other words, execution correctness and generative correctness are separate properties. The project measures both, and their results point in different directions.
Exact execution is the strongest result
The Pico consistently matched the reference interpreter, although the measurements come from the project's own test artifacts.
The author reports that 12,670 generated programs were executed during a hardware sweep. Every returned trace matched the Python reference VM exactly.
A trace is the ordered geometry produced by a program. Exact matching means the device and reference returned identical coordinates, rather than merely producing pictures that looked similar.
The project's experiment record reports zero-tolerance equality across all 12,670 traces. It also lists 120 of 120 conformance programs passing against a QEMU baseline.
These are first-party results, not an independent replication. Still, the repository includes the interpreter, test structure, captured traces, and documentation needed for technical inspection.
The C interpreter reportedly occupies 1,862 bytes of flash. That figure covers the VM alone, excluding bytecode storage and the surrounding transport harness.
The implementation uses no statically allocated RAM for VM state. Peak stack usage reached 492 bytes under the measured configuration.
At a deliberately reduced RP2040 clock of 12 MHz, the reported average was 7,334 cycles per drawing. That corresponds to about 0.611 milliseconds for the measured QuickDraw programs.
The author also reports 1.959 cycles per executed instruction. Those measurements concern interpreter work, not the time needed to generate a program on the host.
They also exclude host-to-device transfer time and display work. Readers should not treat 0.611 milliseconds as end-to-end generative latency.
The RP2040 is a dual-core Arm Cortex-M0+ microcontroller with 264 kB of on-chip SRAM. Raspberry Pi lists a maximum clock speed of 133 MHz in its RP2040 documentation.
The chip lacks a hardware floating-point unit. That limitation often complicates graphics code because curves and transforms commonly use fractional coordinates.
This VM avoids floating-point arithmetic through a fixed-point representation. Fixed point stores fractional values as scaled integers, producing predictable results across implementations.
The curve evaluator takes advantage of power-of-two step counts. Under that restriction, the relevant cubic Bézier coefficients can be represented as binary fractions with a known denominator.
The implementation selects enough fractional bits to preserve those values during integer calculation. This design removes cross-platform rounding differences from the measured geometry path.
Deterministic arithmetic also makes comparison unusually strict. The test does not need an image similarity score or a tolerance around each vertex.
The reference and device either emit the same trace or they do not. That binary result is easier to audit than a subjective assessment of visual similarity.
Physical testing exposed at least one class of issue that host simulation missed. The project documentation describes peripheral clock initialization races during bare-metal startup.
That observation supports the decision to test on silicon. An interpreter can be mathematically correct while its transport or startup sequence remains unreliable on an actual board.
The current evidence nevertheless has clear boundaries. The author did not measure energy per drawing because the test bench lacked suitable current-sensing equipment.
The repository also does not bundle the trained checkpoint. Users can run the VM and replay recorded captures, but live model generation requires a separately obtained checkpoint.
Those limitations do not erase the execution result. They define what outside reviewers can reproduce immediately and what still depends on the author's materials.
Programs offer control that pixels cannot
Executable output turns model behavior into something a constrained runtime can inspect and govern.
A drawing program exposes operations, control flow, and geometric structure. A raster image exposes only the final arrangement of pixels.
That difference matters on small devices. A runtime can impose a fuel limit, which is a maximum number of instructions allowed before termination.
It can also bound loop nesting, call depth, transform depth, coordinate ranges, and output volume. These constraints make generated behavior finite even when the model produces a flawed sequence.
The project's VM streams vertices instead of storing a complete drawing. This reduces working-memory pressure and fits the behavior of a device designed for real-time control.
The approach resembles other systems that separate planning from execution. A larger machine performs expensive inference, while a smaller controller follows a compact intermediate representation.
That pattern already appears in robotics, computer numerical control, plotters, and embedded interfaces. The unusual element here is using a sub-million-parameter transformer to produce the intermediate program.
Drawing bytecode is especially suitable for the experiment. Lines, curves, and repeated motifs have visible outcomes, yet their execution remains simpler than a general-purpose programming language.
A malformed image prediction produces an unattractive image. A malformed program introduces additional questions about termination, validity, and runtime safety.
The VM answers some of those questions with a deliberately restricted instruction set. It does not offer arbitrary memory access or general operating-system services.
This makes the system closer to a domain-specific language than ordinary generated code. A domain-specific language supports a narrow task with fewer dangerous or ambiguous operations.
The result is a constrained contract. The model proposes a drawing, while the interpreter decides what those bytes mean under fixed rules.
That contract creates opportunities beyond sketches. A similar arrangement might represent toolpaths, pen-plotter commands, LED patterns, simple animations, or bounded interface layouts.
However, those applications would require their own validation. Exact geometry on a Pico does not establish safe motor movement or reliable control of physical machinery.
The target domain would also determine which errors matter. A slightly malformed flower is harmless, while a malformed actuator path can damage equipment.
The project's strongest transferable idea is therefore architectural. Probabilistic generation can sit outside the trusted execution boundary.
The embedded component can stay small, testable, and deterministic. It does not need to inherit the complexity of the model that proposed the program.
This separation also changes how developers can debug failures. They can inspect the generated bytes, replay them in Python, compare traces, and isolate device-specific behavior.
A pixel pipeline often hides structure inside neural activations. A program pipeline leaves an artifact with explicit operational meaning.
That artifact can be logged and versioned. It can also be checked against known rules before a device receives it.
For engineering teams, this resembles a compiler pipeline more than an image generator. The model acts as an uncertain front end, while the VM acts as a strict execution back end.
The analogy should not be pushed too far. Traditional compilers translate well-defined source text, whereas this transformer samples programs from a learned distribution.
Still, the boundary is valuable. It gives a conventional software component authority over what the generated output can do.
Small-model program generation still fails at composition
The interpreter executes exactly, but the transformer does not reliably generate exact unfamiliar relationships.
The project's experiments show that low prediction loss does not automatically produce dependable sampled programs. This gap is the main reason to treat the work as research rather than a finished system.
The baseline flat autoregressive model reports a converged test loss of 489.2 bits per drawing. Test loss measures predictive uncertainty, not whether a sampled drawing satisfies a desired geometric relationship.
The author tested several representations under the same general parameter budget. Those included bytes, individual bits, typed tokens, and relative coordinate deltas.
On a synthetic program corpus, the bit representation performed about as well as bytes. The reported difference was negative 0.67 bits per drawing, with uncertainty of plus or minus 0.77 bits.
The result changed on human sketches drawn from Google's Quick, Draw data. There, bit-level modeling incurred a reported 11.58-bit penalty per drawing, with uncertainty of plus or minus 0.60 bits.
Bits also expanded evaluation sequences by a factor of eight. The experiment processed 254 million bit tokens, compared with 32 million byte tokens.
Reported evaluation time rose from four minutes for bytes to 48 minutes for bits. That is more than an eightfold slowdown in the documented setup.
The contrast weakens any simple claim that a smaller vocabulary always helps a tiny model. A two-symbol alphabet reduces embedding costs but forces the network to recover byte boundaries and field structure.
Synthetic patterns apparently made that recovery manageable. More varied human sketches did not produce the same outcome.
Typed tokens introduced another tradeoff. They bind opcodes and operand roles more explicitly, but their larger vocabulary consumes a substantial share of a small parameter budget.
In one wide model, the embedding table occupied 22 percent of all parameters. That configuration performed 4.87 bits per drawing worse than the comparison arm.
A deep, narrow model absorbed the vocabulary cost more effectively. This suggests that representation and architecture interact strongly at sub-million scale.
The most revealing failures involved repeated geometry. The model learned predictable repetitions within the range present during training.
Its surprise fell by 74 percent when it encountered the second copy of a known motif. A recovery metric reached 0.807 across one reported configuration.
Yet performance collapsed at the fifth copy, exactly one repetition beyond the maximum training count. The model appeared to learn a count distribution rather than an abstract loop rule.
The project also tested geometric compatibility under teacher forcing. Teacher forcing evaluates the next correct element after supplying the true preceding sequence.
Under that setup, compatible continuations received a strong advantage of 4.28 bits per target byte. The reported corrected significance level was 0.001.
Free sampling produced a very different outcome. Exact completion succeeded only about one percent of the time on the tested composed shapes.
Success on simpler flat-step cases ranged from seven to 13 percent. The model could recognize a compatible continuation when shown context, yet rarely construct the entire continuation itself.
That disconnect is central to contemporary generative modeling. Token-level preference can look convincing while small local mistakes accumulate during autonomous sampling.
Every sampled output becomes part of the next prediction's context. One wrong coordinate, opcode, or length decision can move the sequence away from conditions encountered during training.
The RP2040 cannot repair that semantic failure. It can execute the resulting program exactly, but exact execution preserves the mistake.
Hierarchical planning fixes length better than likelihood
Adding explicit structure improved termination, but it made the drawings less probable under the project's primary loss metric.
The author compared the flat transformer with hierarchical designs at the same 825,344-parameter budget. These systems first predicted stroke summaries, then generated detailed bytecode for each stroke.
One planner used autoregression. Another used diffusion, which gradually converts noise into a structured prediction through repeated denoising steps.
Both hierarchical variants lost by roughly 40 to 55 bits per drawing against the flat model. The exact penalty varied across planner type and compute budget.
The experiment therefore rejected the hypothesis that hierarchical planning would improve likelihood at matched scale. More explicit structure carried a measurable modeling cost.
The planners nevertheless controlled output length more accurately. Their length-distribution error ranged from 1.8 to 3.5 bytes, depending on the configuration.
The flat model's corresponding gap ranged from 7.0 to 13.6 bytes. It was more likely to stop prematurely or continue toward the maximum allowed length.
This is not a minor implementation detail. Generated programs must terminate at sensible boundaries before they can become useful commands.
A model with better average likelihood can still produce inconvenient samples if it assigns too much probability to early termination. It can also generate long, repetitive tails.
The hierarchy separated stroke count from local stroke construction. That explicit decision improved the distribution of generated lengths, even as total likelihood declined.
Diffusion did not provide a clear advantage over an autoregressive planner on the shared summary representation. The termination gain came from hierarchy rather than denoising.
Later instruction-set experiments produced a similar pattern. Explicit repeat-and-transform operations delivered limited direct compression because the model already predicted repeated geometry with low surprise.
However, shorter sequences improved context usage and termination. Reported generated-length errors fell into a range of nine to 11 percent.
Comparable flat models showed errors ranging from 41 to 112 percent. These are first-party experimental measurements, but they illustrate a meaningful design tension.
A representation can help generation without winning on conventional test loss. Conversely, a lower loss does not guarantee well-formed programs during sampling.
That tension should shape future evaluation. Researchers need metrics for validity, exact relational completion, termination, novelty, and execution behavior.
Visual quality remains relevant, but it cannot stand alone. Two drawings might look similar while their programs differ substantially in length, structure, or reuse.
Memorization is another unresolved issue. A small model can reproduce familiar motifs without learning the transformations that generate them.
The repository documents controls for position, proximity, frequency, and coordinate sets. Those checks strengthen the relational experiment but do not settle novelty across the full training corpus.
A stronger release would include checkpoints, training manifests, generated samples, nearest-neighbor analyses, and repeatable end-to-end scripts.
Multiple independent training seeds would also clarify which behaviors survive initialization changes. Some out-of-distribution repetition results already diverged noticeably between seeds.
The current project reports negative findings instead of hiding them. That is useful because the failures identify where compact models stop behaving like symbolic reasoners.
What should be verified next
The next milestone is not a larger gallery; it is evidence that explicit relations improve unseen program generation.
The project's current direction adds a copy-or-emit action. The model can either produce ordinary bytes or refer to an earlier source span with an affine transformation.
An affine transformation can translate, rotate, reflect, or scale geometry while preserving straight lines. In this system, supported operations would remain integer-based and executable by the existing style of VM.
This proposal directly targets the teacher-forcing gap. The model already appears sensitive to compatible relational context, but free sampling rarely completes that relationship exactly.
An explicit action might reduce the number of separate decisions needed to reproduce a transformed motif. One correct relation could replace many fragile coordinate predictions.
The first signal to watch is performance on unseen combinations. The model should generate exact relationships that were excluded from training, not merely compress familiar repeated shapes.
Evaluation should compare flat emission with copy-or-emit behavior under matched parameter and training budgets. Exact free-generation success matters more than teacher-forced preference alone.
If unseen relational completion rises materially above the reported one percent level, the mechanism gains credibility. If only likelihood improves, the central generation problem remains.
The second signal is independent reproduction of the hardware sweep. The project provides source code and captured artifacts, but it does not currently bundle the model checkpoint.
An outside developer should be able to rebuild the interpreter, run the conformance suite, send generated programs to a Pico, and reproduce bit-identical traces.
That process should report compiler settings, clock configuration, transport overhead, and complete memory accounting. It should distinguish interpreter flash from total firmware size.
A successful reproduction would strengthen the execution claim. A mismatch would help identify whether the result depends on toolchain, board revision, or undocumented setup details.
The third signal is an end-to-end resource measurement. The current 0.611-millisecond figure covers VM execution at 12 MHz, not host inference or serial transfer.
A practical demonstration should separate generation time, validation time, transfer time, execution time, and rendering time. Energy measurements would also clarify the cost of the embedded stage.
These measurements would not turn the project into on-device AI. They would show whether the split architecture offers a useful systems tradeoff.
The larger lesson already looks credible, even before those tests. Small generative models can produce executable intermediate representations, while tiny deterministic runtimes enforce narrow operational rules.
What remains unclear is whether the model can generate the right structure outside familiar combinations. Hardware exactness solves only the last stage of that problem.
Developers evaluating the RP2040 drawing model should therefore ask two separate questions. Does the Pico execute every valid instruction exactly, and does the transformer reliably write the intended program?
The available evidence gives a strong first-party answer to the first question. It provides a much more cautious answer to the second.
Watch the copy-or-emit experiment, the release of reproducible checkpoints, and an independent Pico run. Those three tests will determine whether this becomes a reusable design pattern or stays an instructive research prototype.
