Anthropic Claude Opus 4.5 Raises Hopes for True Computer Use

Anthropic released Claude Opus 4.5 with new computer control features that let the model move the cursor, click buttons, and type into apps. The update targets tasks that once required human hands, such as filling forms or rearranging files on screen. Early tests showed the model completing simple sequences without constant prompts.

The announcement centers on one shift. Anthropic says the model now keeps state across multiple screen actions instead of resetting after each step. Users report fewer dropped tasks during a single session. The release arrives amid broader industry movement toward agents that can operate entire desktops rather than single websites or code repositories. Finance, legal, and creative teams have quickly begun testing whether the new capability reduces repetitive manual work. Industry observers note that this marks a departure from earlier chatbot-style interfaces that required users to copy and paste outputs manually into other programs. Instead, the model can now act directly on the same visual canvas a human operator would use.

Early feedback from private preview participants highlighted both excitement and caution. One engineering lead at a mid-sized accounting firm described running the agent overnight to reconcile bank statements across three different desktop applications; the process finished with only two intervention points. Other testers emphasized that the model’s ability to reason over live screenshots reduced the need for brittle scripting languages that previously automated the same steps. These early wins, however, came from tightly controlled environments where screen layouts stayed consistent for days at a time.

The broader promise is that knowledge workers could eventually delegate entire end-to-end processes - invoice processing, contract redlining, or campaign asset preparation - without writing a single line of traditional automation code. Yet many practitioners still treat the current release as an experimental tool rather than a drop-in replacement for human labor. The gap between demo videos and daily production use remains visible in the frequency of required human checkpoints.

How the Computer-Use Interface Actually Works

Claude Opus 4.5 introduces an interface layer that captures screenshots at regular intervals, processes them through a vision encoder, and translates recognized UI elements into mouse coordinates and keyboard events. The model then outputs structured actions such as “move cursor to (x, y)” or “type string ‘Q3 revenue’”. This loop runs continuously while the agent remains in control of the desktop session.

Developers can invoke the new capability through the Anthropic API by passing a special “computer_use” tool definition. Once enabled, the model receives both textual instructions and the latest screen capture in the same request. The response contains an action list that the client executes locally before returning the next screenshot. Latency per cycle typically ranges from 800 milliseconds to two seconds depending on network conditions and image compression settings. Details appear in Anthropic’s official computer-use documentation.

Because the system operates at the pixel level rather than through application APIs, it can interact with any graphical software that a human can see. Legacy enterprise applications without modern REST endpoints become reachable. At the same time, the approach inherits every visual ambiguity that affects human users: overlapping windows, dynamic highlights, and anti-aliasing artifacts all require the model to interpret imperfect visual data.

In practice, teams configure the client to resize images to 1024×768 or 1280×720 pixels before transmission, balancing recognition accuracy against token cost. Higher resolutions improve small-text legibility but increase both latency and API spend. The action loop also supports a “thinking” step in which the model can pause and describe its next planned sequence before committing to mouse or keyboard output. This intermediate reasoning trace can be logged for audit purposes or used to debug why the agent chose a particular coordinate.

Implementers also report that adding a lightweight local OCR pass before sending the image to the model sometimes improves text-heavy workflows, although this step is not officially required. The combination of native vision processing and optional OCR yields the most reliable results on dense financial spreadsheets or legal exhibits.

Model Actions on Screen

Early developer testing focused on spreadsheet updates and email drafting. In controlled demonstrations, Claude Opus 4.5 successfully selected ranges, applied formulas, and composed messages when the layout remained constant across iterations. The model learned to locate column headers by reading text rendered on screen rather than relying on fixed coordinates.

Changes in screen size or pop-up windows still caused failures. When a modal dialog appeared unexpectedly, the model often paused and waited for additional human guidance instead of attempting autonomous recovery. Testers also observed that multi-monitor setups introduced coordinate-mapping errors unless the agent received explicit descriptions of each display’s resolution and relative position.

The action space includes left-click, right-click, double-click, drag-and-drop, scroll, and text entry with modifier keys. Timing parameters allow the model to wait for animations or loading spinners before proceeding. These primitives combine into higher-level sequences such as “copy the top ten rows into a new workbook and format the header row in blue.” Advanced users have begun chaining dozens of such primitives into reusable playbooks that can be version-controlled and shared across teams.

Several pilot groups maintain internal libraries of these playbooks, with each entry including the exact software versions and screen-resolution assumptions under which the sequence was validated. When an operating-system patch alters a dialog box, engineers can quickly identify and update only the affected playbooks rather than rebuilding entire workflows.

Where Promises Meet Limits

Anthropic reports that context windows exceeding 200,000 tokens allow the model to retain earlier screen states and typed values across dozens of steps. In internal evaluations shared at launch, the agent completed basic data-entry routines and file-system operations without intermediate prompting, as described in the computer-use technical announcement. One internal test involved processing 47 PDF invoices into a shared spreadsheet; the model maintained row numbering and currency formatting correctly across the entire batch.

Real use introduces variables the model does not control. Network latency spikes, automatic software updates, or user-specific folder structures cause the sequence to diverge from training examples. When a file-save dialog appears in an unexpected location, or when a corporate VPN alters window borders, the agent must either guess or halt. Teams running unsupervised overnight jobs discovered they needed to insert checkpoint prompts every three to five minutes to verify intermediate results. Without those guardrails, error rates rose sharply after the tenth minute of continuous operation. Several pilot groups now schedule the agent only during low-variability windows, such as after nightly batch processes have finished.

Longer sessions also expose token-budget constraints. Even with a 200k context window, each new screenshot consumes hundreds of tokens, causing the model to eventually drop earlier visual history. Teams mitigate this by summarizing completed steps in plain text and feeding that summary back into the prompt, effectively distilling the visual record into a compact narrative the model can carry forward.

Context Supply Problem

The feature presupposes that the model receives complete information about the user’s goal and the current state of every open window. In practice, file hierarchies, naming conventions, and decision history reside only in the user’s mind. Each new session therefore requires the operator to restate project background before meaningful work can begin.

Teams often integrate external retrieval systems so the agent can pull relevant background automatically. Anthropic’s developer guide on long-context agents outlines how retrieved summaries can be injected into the prompt before a computer-use session begins. The operator no longer repeats the same background information on every launch. Similar memory layers exist in other agent platforms, yet most still require explicit integration work before Claude Opus 4.5 can consult them.

General agents without persistent memory force the same setup work on every new task. Claude Opus 4.5 improves action control yet still depends on fresh context supplied by the user. Persistent memory bridges remain an external responsibility rather than a built-in capability. Early adopters therefore treat memory integration as a first-class engineering task rather than an afterthought. Some organizations have begun embedding retrieval-augmented generation pipelines that query internal wikis or ticketing systems before each major agent run, reducing the cognitive load on human operators.

Comparison with Other Computer-Control Agents

Several research groups and startups have explored pixel-based control. OpenAI’s earlier experiments with reinforcement learning agents and Adept’s ACT-1 model demonstrated similar mouse-and-keyboard primitives. Those systems typically required task-specific fine-tuning or large numbers of human demonstrations. Claude Opus 4.5 attempts to bypass heavy fine-tuning through scale and chain-of-thought reasoning over screenshots.

In side-by-side trials, GPT-4o combined with browser automation tools achieved higher throughput on web-only workflows because dedicated DOM access removes visual noise. Claude Opus 4.5, by contrast, shines when the target application lacks an API or runs only as a desktop binary. The tradeoff appears most clearly in long-running enterprise processes where visual robustness matters more than raw speed. Multi-agent frameworks that split planning and execution across separate models also show promise, yet they add orchestration complexity that many teams prefer to avoid.

Community benchmarks published on GitHub show Claude Opus 4.5 achieving 68 % success on a standardized set of 50 desktop tasks, while a fine-tuned open-source baseline reached 54 %. The margin narrows when tasks involve highly dynamic web applications, reinforcing the observation that pixel-level agents still trade generality for the ability to reach legacy software.

Real-World Use Cases

Finance teams have prototyped month-end close procedures that involve copying figures from ERP exports into standardized templates. With sufficient checkpointing, the agent can handle repetitive formatting and cross-referencing. Legal operations groups tested redaction of sensitive clauses across hundreds of PDF exhibits; success depended on consistent document layouts and pre-loaded instructions describing redaction rules.

Creative studios explored batch renaming of asset files and metadata tagging inside design applications. Early results show promise for highly repetitive pipelines but reveal fragility when version numbers or client-specific naming schemes change between projects. Procurement departments have begun testing invoice ingestion workflows that read vendor PDFs, extract line items, and post them into an accounting system, cutting manual processing time from hours to minutes on stable templates.

One healthcare billing company reported processing 1,200 claims in a single weekend using the agent inside a locked-down virtual machine. Staff reviewed only the 4 % of cases flagged by the model as low-confidence, demonstrating measurable labor reduction even before perfect autonomy is reached.

Practical Implications for Teams

Organizations that invest in screen-control agents must also budget engineering time for error-handling scripts and context-preparation routines. The marginal productivity gain appears only after an initial setup period of several days per major workflow. Once established, the agent can reduce manual execution time by 60–80 percent on stable tasks, provided interface changes remain infrequent.

Teams report that pairing the agent with a human reviewer for the first and last steps yields the most reliable outcomes. Full autonomy remains rare outside narrowly bounded environments such as isolated virtual machines with locked-down software versions. Forward-looking departments now include agent-maintenance responsibilities in job descriptions for automation engineers, signaling a permanent shift in how desktop processes are staffed.

Limitations and Risks

Pixel-level control carries security implications. An agent granted desktop access can theoretically read any visible document or execute arbitrary keystrokes. Enterprises therefore run such agents inside sandboxed virtual machines with restricted network egress. Audit logs become essential; every mouse click and keystroke should be recorded for compliance review.

Visual brittleness also creates a new class of maintenance overhead. Whenever an upstream vendor ships a UI update, previously working agent scripts require re-validation or additional training examples. This recurring cost resembles the classic challenge of maintaining large suites of UI test automation. Bias and hallucination risks persist. The model may misread a number on screen and propagate the error into downstream calculations without flagging uncertainty. Human oversight checkpoints remain necessary precisely because the agent cannot reliably communicate its confidence level in visual interpretations.

Future Signals to Track

Watch for updates on error-recovery speed when layouts shift. Anthropic has signaled ongoing work on automatic retry policies that detect common failure modes such as unexpected dialogs. Check whether published accuracy figures from tests longer than ten minutes become available in technical reports.

Observe how teams combine screen control with stored project history. Tools that already hold that history lower the need for repeated setup. Compare results after the next model update to determine whether recovery from small interface changes improves enough to reduce checkpoint frequency. Industry analysts also recommend monitoring open-source projects that replicate the computer-use loop, as community implementations may surface robustness improvements faster than vendor releases.

FAQ

How much human supervision does Claude Opus 4.5 still require?

Most production pilots retain a human in the loop for final verification and for handling unexpected pop-ups. Fully autonomous runs longer than fifteen minutes remain uncommon outside tightly controlled test environments.

Can the model work across multiple monitors?

Yes, once the client provides coordinates and dimensions for each display. Accuracy improves when the agent receives an explicit diagram of monitor arrangement at session start.

Does the feature work inside virtual machines?

It functions inside VMs provided screen-capture permissions are enabled and network latency stays low. Many security-conscious deployments deliberately choose this configuration.

Will future versions reduce dependence on user-supplied context?

Anthropic continues research into long-term memory modules, yet current releases still treat persistent context as an external integration point rather than a native capability.

Teams following fast-moving technology stories often need one place to keep source notes, meeting context, and follow-up questions together. A lightweight AI knowledge base can make those moving pieces easier to revisit after the news cycle changes.