Desktop Automation Without an API: A Practical Playbook

Start with the picking, not the building

The build is the easy part now. You describe the workflow in plain language, the employee works out the clicks, and there are no selectors to maintain. What still goes wrong is choosing a task that was never shaped correctly, then concluding the technology does not work when it was the brief that was broken.

This matters because the underlying capability is sharply uneven. On short desktop tasks, the best agents now score above 85 percent on the standard benchmark, ahead of the roughly 72 percent human baseline researchers measured. On long workflows that take a person about 1.6 hours, the best system finishes around 20 percent. The same technology, pointed at two differently shaped jobs, gives you a reliable colleague or an expensive disappointment.

So the first hour of any desktop automation project should be spent on selection. Run every candidate through the four tests below before writing a single instruction. Anything that fails a test either gets reshaped until it passes or goes back on the shelf.

• Does the software genuinely lack a usable API? If a maintained endpoint covers the exact action, use it. Screen control is for the doors that do not exist, not the ones you have not looked for.
• Is the procedure the same every time? Same screens, same order, same decisions. If a human makes a judgment call halfway through, that call needs to stay with the human.
• Can you write a correct result in one sentence? "Twelve PDFs named by month in the Invoices folder" passes. "The reporting is tidied up" does not.
• Does it repeat often enough to matter? Weekly is a good floor. Something you do twice a year is not worth the ramp, however annoying it is.

Four yeses is a green light. The tasks that pass tend to be deeply unglamorous: exporting a report from a system nobody has budget to replace, downloading invoices from six vendor portals, renaming a folder of files to a convention, pulling one number out of a dashboard so it can be pasted somewhere else. That is fine. Unglamorous work that happens twenty times a month is exactly where this pays.

Write the success condition before the instruction

The one sentence you wrote in test three is the most valuable artefact in the whole project. It is what the employee aims at, what you check against, and what tells you whether a run went well without watching the whole thing.

There is a second reason to insist on this. A condition you can verify at a glance is also a condition the employee can verify mid-run, which is what lets it catch its own mistake at step four instead of confidently continuing to step forty. Vagueness in the brief becomes drift in the execution.

The four-week ramp

From first run to unattended

1. Week one: everything gated — Require approval before file changes, submissions, and messages. Watch each run in the activity feed. Expect to refine the brief twice.
2. Week two: gates on the consequential only — Release approval on the harmless steps and keep it on anything touching money, credentials, or an outside inbox. Keep reading the feed daily.
3. Week three: check the output, not the run — Stop watching step by step and start checking only the success condition. If it holds for a full week, the workflow is real.
4. Week four: add the adjacent step — Extend the workflow by one step, not five, and drop that new step back to week-one treatment while the rest stays where it is.

The temptation is to compress this into an afternoon, and the cost of doing so is not dramatic failure but quiet wrongness: a workflow that looks fine and has been filing the wrong month since Tuesday. The ramp exists so that the first time something goes sideways, you are watching.

One decision worth making early is who owns the workflow. Handing it to a dedicated AI Employee rather than scattering it across tools means the context lives in one place: what this task is for, what went wrong last month, which portal changed its login flow in June. That memory is the difference between a workflow that gets better over time and one you re-explain every quarter. It is also what lets the same employee use a connected integration for half the job and screen control for the half that has no API.

Reach for the terminal more than you think

The single biggest reliability gain in most desktop workflows is not a better prompt, it is doing less of the work through the mouse. Batch renames, format conversions, folder sweeps, and status checks are one command each. Done by clicking, they are hundreds of individually fallible actions.

This is visible in the benchmark data too. Realistic long workflows averaged over three hundred tool calls for a frontier agent, and the systems that scored best were the ones that switched modes rather than clicking through everything. A controller that can only click is working with one hand tied. Sistava's Computer Controller covers desktop control, file management, and terminal commands, and the employee picks per step, which is why a folder of four hundred files is a one-line job rather than an afternoon of clicks.

The practical rule: if you would reach for a shell yourself, tell the employee it may too, and write a tool rule bounding where it can run. "Never run commands outside the project directory" attaches to the tool and holds on every future run, not just the conversation where you said it.

Scaling to the second and third workflow

The second workflow is much faster than the first, because you have already paid the setup cost and, more importantly, you have learned what a well-shaped task looks like in your own business. The trap at this stage is bundling: turning three related tasks into one long run because they happen in sequence anyway.

Resist it. Three short workflows with a checkpoint between them will beat one long one, for the same reason the benchmarks show short tasks succeeding and long ones stalling. Keep them separate, let each produce a verifiable artefact, and chain them. If step two fails you lose step two, not the afternoon.

The other scaling decision is where this runs. Recurring work belongs on a machine that stays on rather than the laptop that travels, ideally one with only the applications that workflow needs installed. That is a reliability choice and a security one at the same time, since an agent cannot be misled into touching software that is not there.

If none of the pre-built roles match the shape of your desktop work, train a custom AI Employee on it directly and brief it the way you would brief a new hire on their first morning. It keeps that context between runs, so the second month starts further along than the first rather than from the same explanation.

The whole playbook compresses to one line: choose narrow, define correct, gate hard, expand slowly. Teams that follow it tend to have three or four boring workflows running unattended within a quarter. Teams that skip straight to the ambitious end-to-end process tend to have a story about how this technology is not ready yet.

For the wider context, the full guide to AI computer control covers where this fits against API automation, the mechanics write-up explains the loop underneath it, and the safety deep-dive covers what to lock down before a workflow runs unattended.