Frontend Tools
Let your agent interact with and update your application's UI.
See this in Inspector
Open Inspector on localhost. Go to Inspect, then Frontend Tools. Your tool and its schema are listed.
More detail: Inspector.
What is this?
A frontend tool is a function that you register in the browser. Your agent can call it, and the function runs in the user's browser. Because the handler runs in the browser, it has direct access to component state, browser APIs, and any third-party UI library the page already uses. That's how an agent can "reach into" the app: update React state, trigger animations, read localStorage, pop a toast, or steer the user's view.
This page covers the "agent drives the UI" shape of frontend tools. To render a component for a tool call, see Generative UI. To ask the user before a tool runs, see Human-in-the-loop.
When should I use this?
Use frontend tools when your agent needs to:
- Read or modify React component state
- Access browser APIs like
localStorage,sessionStorage, or cookies - Trigger UI updates, animations, or transitions
- Show alerts, toasts, or notifications
- Interact with third-party frontend libraries
How it works in code
Take the forwarded tools off the Flow's state
A Flow owns its own model call, so unlike a chat agent it has to hand the
forwarded tools to the model itself. Type the Flow on CopilotKitState and
read state.copilotkit.actions — that is where a tool registered with
useFrontendTool or a component registered with useComponent arrives.
from crewai.flow.flow import Flow, start
from litellm import acompletion
from ag_ui_crewai import CopilotKitState, copilotkit_stream
class ChartFlow(Flow[CopilotKitState]):
@start()
async def chat(self) -> None:
actions = self.state.copilotkit.actions or None
response = await copilotkit_stream(
await acompletion(
model="openai/gpt-6-luna",
messages=[
{"role": "system", "content": SYSTEM_PROMPT},
*self.state.messages,
],
tools=actions,
parallel_tool_calls=False,
stream=True,
)
)
self.state.messages.append(response.choices[0].message)Wrap the call in copilotkit_stream so the tool call reaches the browser as
it streams. A Flow that returns only when the model is finished renders
nothing until the turn ends.
Decide when the component is required
The Flow controls tool_choice, which is the lever a chat agent does not
have. Forcing the call on the user's turn and leaving it on auto
afterwards is what renders the component immediately and still lets the run
end: the follow-up turn is plain narration once the browser has returned the
result.
on_user_turn = bool(
self.state.messages and self.state.messages[-1].get("role") == "user"
)
tool_choice = "required" if actions and on_user_turn else "auto"Leaving tool_choice on auto for every turn is the usual reason a Flow
answers in prose and the component never appears.
Register a frontend tool with useFrontendTool. Give it a name, a Zod schema for parameters, and a handler. The agent can then call it like any other tool and your frontend runs it in the browser.
import React, { useState } from "react";
import {
CopilotKit,
CopilotSidebar,
useFrontendTool,
} from "@copilotkit/react-core/v2";
import { z } from "zod";
import { Background, DEFAULT_BACKGROUND } from "./background";
import { useFrontendToolsSuggestions } from "./suggestions";
function Chat() {
const [background, setBackground] = useState<string>(DEFAULT_BACKGROUND);
useFrontendTool({
name: "change_background",
description:
"Change the page background. Accepts any valid CSS background value — colors, linear or radial gradients, etc.",
parameters: z.object({
background: z
.string()
.describe("The CSS background value. Prefer gradients."),
}),
handler: async ({ background }) => {
setBackground(background);
return { status: "success" };
},
});The handler receives the parsed, type-safe parameters and can do anything the browser can: update state, call an API, touch the DOM. Its return value is sent back to the agent as the tool result so the model can reason about what happened.
handler: async ({ background }) => {
setBackground(background);
return { status: "success" };
},Ask the agent to change the background to a blue gradient. The page background changes, and the agent gets { status: "success" } as the tool result.
Registering a list of tools
useFrontendTool registers one tool per call, so it cannot be called in a loop
over a list whose length changes between renders. When the set of tools comes
from state, from props, or from a backend response, use
useFrontendTools instead. It takes an
array and runs a single effect over it, so the array can be empty on one render
and hold twenty entries on the next.
useFrontendTools(
reports.map((report) => ({
name: `open_${report.id}`,
description: `Open the ${report.title} report`,
handler: async () => navigate(`/reports/${report.id}`),
})),
[navigate],
);On each render:
- A tool that leaves the array is unregistered.
- A tool that joins the array is registered.
- A re-render that produces an equal list does not re-register anything.
A description built from your data stays current on its own. The second argument
is for values a handler closes over, such as navigate above.
How is this guide?