A bounded pilot with an AI agent

Choose one part of the product with regular UI work and real users: the settings pages, an admin table, the marketing page template. It should hold routine screens and a few awkward ones, so the pilot meets the cases that matter. Skip the most political surface in the company, and skip a demo nobody will ship; neither tells you what everyday work will look like.

Write the pilot on one page before any work begins:

• the surface, and the kinds of task in scope;
• the test set of prompts, and the score it has to reach;
• the hours spent fixing agent output on that surface today, measured over a normal week, and the target;
• what each outcome triggers: move to the next surface, run one more round, or stop;
• who makes the call.

Leave the result lines blank until the pilot has run. A pilot whose success is defined afterwards will always succeed.

The design tokens it uses, as data with their purposes. The components it uses, typed and documented in Markdown. Guidance for its patterns. The checker, wired into the agent’s workflow so it runs before work is handed back. If the pilot works, the next surface reuses all of it, so nothing built here is thrown away.

Run the test set on day one, before any change, and again on the last day. Measure fixing hours with the same method both times, ideally by the same person. A pilot that changes how it measures halfway through proves nothing either way.

If the agent still needs hand-fixing on most screens after the guidance has been tuned, the gap is somewhere else: the model, the tool, or the brief itself. Stopping then saves you a larger build on the wrong foundation, and the test set tells you where to look.

Simbolik works the same way from the first contact: a 20-minute call where you show what your AI makes today and who spends the week fixing it, then a one-page proposal that writes down the standard the output has to meet before anything is built. Work with Simbolik (opens in a new tab).