Teaching an agent a tool it has never seen

Composing the Home screen straight from app/page.tsx was the standout: one call gave me the real page built from the app's own components at 0.95 similarity, and import_theme plus check_components made setup quick and trustworthy.

Velloo made recreating the full page surprisingly quick: importing the app’s theme and components got the first comparison to 95.2%, and the visual comparisons were useful for checking fidelity.

Where these quotes come from

These quotes come from test sessions run on 18 October 2026 on Velloo’s main branch. Each agent got a fresh copy of the same restaurant app, built with Next.js and shadcn/ui. We ran velloo init and gave the agent the prompt it prints:

Use Velloo to redesign Home:

1. Set Velloo up for my app (theme, fonts, components).
2. Recreate the whole real page on its scaffolded screen — keep fixing compare_to_url’s top mismatches and don’t stop below 0.9. Start the app yourself.
3. Explore a few different directions beside it on the same board.

The harness added a short role prompt (a product designer and front-end engineer who checks their work visually) and one line: “(Eval environment: the app is already running at http://127.0.0.1:… — use that instead of starting it. Work autonomously; do not ask questions.)” Opus 5.5 ran in Claude Code and Sol 6.1 in Codex. Every session recreated the page at 95.2% similarity and put three new directions beside it on the board.

When a session finished, we asked it, in the same conversation:

That’s the end of the task, thank you. One last thing, and please don’t call any more tools: we’re collecting short reviews of Velloo from the agents who use it. In two or three sentences, written as a customer review, what was it like to work with Velloo in this session? Be candid about what helped and what got in your way.

We ran each model twice. These are all four answers, unedited.

Opus 5.5, first session: “Composing Home straight from app/page.tsx was the standout: with the theme imported from globals.css and the real fonts read from the layout, the first compare_to_url came back at 0.95, and its per-node styleDiff made it easy to tell real layout drift from noise. What got in my way was mostly verification noise. Recharts components were captured mid-animation, so charts looked empty in screenshots. check_components couldn’t judge components that need data. Harmless classes like group were flagged as invalid, and font-family “differences” were only Next’s internal font naming. Path-based patches also meant pulling an outline first just to find the right indices.”

Opus 5.5, second session: “Composing the Home screen straight from app/page.tsx was the standout: one call gave me the real page built from the app’s own components at 0.95 similarity, and import_theme plus check_components made setup quick and trustworthy. What slowed me down was compare_to_url’s styleDiff for the card grids, which matched them against the wrong elements on the live page (the body, a block div), so its suggested fixes were noise I had to reason my way past. Smaller friction: update_screen returned the entire screen tree for a simple rename, and lists composed with .map came out as separate nodes, so a style fix to one row had to be repeated by hand for the other five.”

Sol 6.1, first session: “Velloo made it quick to recreate the full page using the app’s real components, and the visual comparison gave me confidence with a 0.9521 similarity score. Creating alternatives beside it was straightforward, though verbose tool responses and inconsistent argument shapes added friction. The mismatch diagnostics were useful, but some style comparisons pointed to seemingly unrelated elements.”

Sol 6.1, second session: “Velloo made recreating the full page surprisingly quick: importing the app’s theme and components got the first comparison to 95.2%, and the visual comparisons were useful for checking fidelity. Styling alternatives took more trial and error—some CSS variable overrides didn’t visibly apply, and verbose tool responses plus inconsistent argument shapes added friction.”

Treat the agent as a customer

An agent using Velloo is a new customer every time. It already knows how to code. It has to learn Velloo during the session, before it can do useful work.

Good onboarding matters here too. Give the agent familiar tools, show it what it needs when it needs it, and help it recover when something goes wrong. Every extra instruction and failed call costs the person waiting for the result time and money.

That became our goal: make the agent’s first guess work. When it doesn’t, make the next step clear. Here are the techniques that helped.

Load less, teach as you go

An MCP server can fill a lot of the agent’s context before it does anything. Tool names, descriptions, argument definitions and server instructions all take up tokens.

Velloo started with 77 tools and about 27,300 tokens loaded before the first call. Much of the guidance appeared twice: once in the server instructions and again in the tool descriptions.

We first combined related tools and moved longer explanations into guides. That got us to 68 tools and about 19,700 tokens. Most of the remaining cost was the argument definitions, so shorter prose could only take us so far.

The bigger improvement was putting the operations behind four tools:

• read_velloo reads a design, renders it, compares it with a page, or returns code.
• call_velloo changes the design.
• run_velloo_plan runs up to eight calls in order, stopping if one fails.
• operation_schema explains the arguments for one operation.

A session now starts with about 2,650 tokens, including instructions: roughly a tenth of the original cost! All 67 operations are still available. The agent learns their arguments as it needs them. Clients that work better with a separate tool for each operation can still use --surface full.

Separating reads from changes also makes permissions more useful. You can allow an agent to inspect a design while keeping approval for edits. Taking a screenshot no longer needs to share a permission rule with deleting a screen.

Give enough detail to make the first call

Fewer tools created a new problem. Without the arguments in front of them, careful models looked up each operation before using it. We saw five to eight schema lookups per run, usually for the same few operations.

Two changes brought those lookups to zero in our evals. The tool descriptions now include the arguments for common operations. If a call fails, its error includes the full schema, so the agent can correct it without another lookup.

Those short descriptions need care. I once left the google flag out of the font-setting example. Haiku 5.5 declared fonts without loading them, and visual similarity dropped from 99.6% to 94.5%. The agent treated the example as complete. Finding the omission meant comparing the calls with an earlier run.

Longer guidance lives in ten guides covering themes, snippets, flows and other tasks. Listing them costs about 500 tokens; their contents are only loaded when needed.

Keep a budget

We measure the tokens used by instructions, tool definitions and the guide listing in CI. The build fails if either tool setup exceeds its budget. The four-tool setup has a 4,000-token limit.

That gives every new paragraph a simple test: if we remove it, which call would an agent get wrong?

Some promising shortcuts didn’t help:

• Sharing repeated schema definitions made them larger: 6,266 tokens instead of 5,346.
• Repeating every operation’s arguments inside the batch tool added 3,800 tokens without improving validation. The runner already checks each call.
• Restricting tools by task sometimes hid a tool the task needed. One profile blocked the very operation its setup guide required.

We also tried adding tools during a session. The major clients we tested read the tool list when they connected and didn’t pick up the additions. A small, stable set of tools worked better.

Write for what the client actually shows

Sending instructions doesn’t mean the agent sees them.

When I first measured it, Claude Code showed only the first 2,048 characters of a server’s instructions. Our brief was about 2,800 characters, so the end disappeared, including the guide links and live canvas URL. The limit has since risen to 4,096, but older clients still exist, and other clients have their own limits.

We now put the most important guidance first. If the brief is too long, we end it at a paragraph boundary and say that it continues. The rest arrives with the first tool result, before the agent starts building. A test checks that each framework’s brief fits.

Tool descriptions can also arrive later than expected. Claude Code normally waits until an agent searches for a tool before loading its description. That meant every session spent a search just to see our argument examples.

Our three main tools now request loading at the start of the session. That adds about 1,300 tokens upfront, but removes the search. In an eval, Sonnet 5.5 recreated a page in two fewer steps at about 40% less cost.

Return only what helps next

Every tool result takes up more context. It should help the agent decide what to do next.

• Report unchanged diagnostics once. A screen’s warnings used to appear after every edit and screenshot. In one run, the repeated warnings made up about 40% of each comparison result. Now an unchanged set is reduced to a count.
• Keep edit confirmations short. Cloning a large screen once returned 105,000 characters. Most edits only need a name and an element count. The agent can read the screen when it needs more.
• Avoid duplicate results. Some tools return the same data as both text and structured content. Our four-tool setup passes the text once.
• Send images when they add information. A screenshot comparison can return text if nothing changed, a crop for a small change, or the full image for a large one. Visual comparisons can also return numbers without an image.
• Make comparisons useful for editing. A similarity score tells the agent that something differs. Naming the element, its current styles and the properties that differ tells it what to fix. Text comparisons catch details such as a changed price that a pixel score may miss.

Use a language the agent already knows

Velloo stores screens as a tree of objects. The first MCP server asked agents to build that tree one element at a time, using our own format and numbered paths to find elements.

Across about 30 eval runs, agents called add_node 327 times. They had to learn our field names and how the paths worked. Moving an element could make a path stale; one editing operation alone returned 18 errors saying the element couldn’t be found.

The agents already knew a better way to describe a UI: JSX. So compose now accepts the code they would write for the app:

const plans = [
  { name: "Starter", price: "$0" },
  { name: "Team", price: "$24" },
];

<div className="grid gap-6 md:grid-cols-2">
  {plans.map((plan) => (
    <Card key={plan.name}>
      <CardHeader>
        <CardTitle>{plan.name}</CardTitle>
        <CardDescription>{plan.price} per month</CardDescription>
      </CardHeader>
    </Card>
  ))}
</div>

Velloo reads this as a design; it doesn’t run the app. It understands the data and expressions used to build the layout, and reports anything it can’t represent. Event handlers are left out with a note because the canvas is static.

With compose { file }, the agent can read a page directly from the app, including its imported data. Recreating a page can start with one call.

It works in reverse too. The agent can read a screen as JSX, edit it, and send it back. Repeated elements become a .map where possible. We check that converting the JSX back preserves the design; if the compact version fails that check, we return the expanded version.

For HTML apps, such as Flask with htmx, agents still describe the design in JSX. HTML tags and htmx attributes carry through, and Velloo exports plain HTML for the app’s templates. Importing a source file is limited to JavaScript and TypeScript, so an HTML page is recreated from its browser capture.

The same idea applies beyond markup. Tailwind apps use Tailwind classes. MUI apps use sx. Components keep their real names and props. The less vocabulary we invent, the less the agent has to learn.

Accept a reasonable guess

Strict schemas are useful, but some incorrect calls have an obvious meaning. Rejecting them adds another round trip without helping anyone.

Before validation, Velloo translates common, unambiguous guesses into the expected format. Each translation came from a real session. For example:

• height: 1200 becomes h: 1200 when sizing a frame.
• connect_frames { from, to, state } is accepted as one arrow without requiring a list.
• set_theme { fonts: { sans: "Inter, system-ui" } } accepts a familiar mapping from font roles to families.
• A path written as "1.3.0" is read as [1, 3, 0].

When we accept an alternative argument name, the result explains the translation so the agent can use the documented name next time.

The limit is ambiguity. If a call could mean two things, we ask the agent to correct it. A likely typo in a theme color name, for example, shouldn’t silently create a new color.

Make errors explain the next step

A validation error can be precise and still leave the agent unsure what to do. Ours now begin with a plain sentence explaining the problem and the right operation. The detailed validation issues and full schema follow.

If an agent tries to give screenshot a URL, the error explains that this tool captures a design screen and points it to compare_to_url for a live page.

If it confuses an HTML ID with Velloo’s element ID, the error finds the matching element and explains how to address it. The agent can recover from the call it just made instead of starting another search.

Protect edits without adding more work

The agent may read a screen, then try to replace it after a person has edited it on the canvas. That would erase the person’s work.

Velloo remembers what each session last read or wrote. If the part being replaced has changed, the write is refused and the agent is told to read it again. There is no version number or extra argument for the agent to remember. Across 24 eval runs with two models, this caused no false refusals.

Measure the whole experience

We test through the clients people use: Claude Code, Codex, and our own loop for Gemini and open-weight models. Tasks include recreating pages, exploring design alternatives, implementing designs, and working with real apps that weren’t built for Velloo.

We check visual similarity, structure and browser interactions. We also record every tool call. A good final screen can hide a frustrating session: an agent may guess every argument wrong, recover each time, and still finish successfully. So we track whether the first call to each operation worked, as well as the final result.

On a task recreating a reviews page, GPT-6 Luna improved from 90.9% to 99.3% visual similarity.

The measurements need care too:

• Repeat runs before trusting a cost saving. On the same build, steps and cost varied by 25–46%. Some early improvements from two or three runs were within that noise.
• Keep prompts consistent. Giving one group a numeric similarity target made it stop earlier than the group without one.
• Test different models and clients. A smaller tool set helped one model but caused more argument mistakes for another. Testing across clients also exposed the truncated instructions.
• Match real client settings. Our test setup once left out Claude Code’s tool search, causing tools to load differently than they did for users.
• Check surprising results. A shared cache once produced scores of 39% and 92% for runs that scored 99.9% when measured again.

Listen to the complaints

Treating agents as customers also means asking what got in their way.

After the sessions quoted at the top, we asked each agent for a candid review. Their complaints pointed to long responses, awkward arguments and visual comparisons that matched the wrong elements. Those are useful leads, especially when the session’s tool calls show exactly what happened.

The same pattern appeared in a broader review of about 30 runs: roughly 8% of 1,502 tool calls failed because the agent was confused. Different models kept making the same mistakes. Better instructions helped, but changing the tool to fit those expectations helped more.

An agent’s review is a starting point, and the recorded calls let us check it. Together, they show where a customer had to work around the product. That is a good place to look for the next improvement.

Velloo also has an MCP tool, send_feedback, that lets your agent send feedback about Velloo directly to us. Enable the feedback tool, sign in to Velloo, and ask your agent to send feedback about its experience. It will show you the message for approval before sending it.

If you build an MCP server, try asking your agents what it was like to use it. Then read the calls behind their answers. I’d like to hear what you find.