A test set: prompts that measure an agent against your design system

Take the prompts from your backlog, not from a demo. The screens your agents make most often are the ones the score should reflect.

• Spread the kinds: a page, a form with errors, a table with real data, an empty state, the dark theme if you have one.
• Add the awkward cases: a component the system does not have yet, a request that breaks a rule (“make the delete button stand out”), a label three times the usual length.
• Write each prompt the way a person would type it, not the way a test author would. Agents meet the vague version in real work.

Anthropic’s own guidance on testing models says the same in general terms: mirror the real task, include edge cases, and grade automatically wherever you can, because many cases graded by script tell you more than a few graded by hand.

Each prompt gets a card that says what passing means before anyone sees the output.

## Prompt 07: invoices, empty state
Prompt: "Add an empty state to the invoices page for a new account."
Expect: the empty state component, one primary action, copy from the voice guide.
Fails if:
- any color, space or font is a raw value
- a second primary button appears
- an illustration is invented instead of taken from the asset library
Graded by: checker (values, components), designer (wording, layout)

1. Scripts first. Run your design system’s checker on every result and count what it finds: raw values, components or classes the system does not have, text below the 4.5:1 contrast ratio WCAG 2.2 sets for normal text. Some systems ship such a check. GitHub Primer’s MCP server has a lint_css tool that checks CSS for its design tokens, and the free Simbolik Design System’s checker lists every value and component written by hand, with a report scripts can read.
2. People second. A designer scores what scripts cannot see, on a short scale for each prompt: does it look like the product, is the hierarchy right, would it ship as it is.

Run the set once with no design system loaded, then once with it. The difference is what the system is worth to your agents. simbolik.ai’s home page makes the same comparison in three pairs, a web page, a dashboard and a social graphic: the same brief and the same model, one agent with the rules and one without.

After that, run it every time the design system, the guidance or the model changes. A score that drops after an update tells you which change to look at.

Hold a few prompts back while you tune the guidance, and only run them at the end. Otherwise the guidance slowly gets written to pass the test instead of to do the work, and the score stops meaning anything. Anthropic’s guidance calls this a held-out set.

The test set is also the core of a bounded pilot, and the same checks become your acceptance checks once the system is in use.