Frontend Tools

Let your agent interact with and update your application's UI.

Frontend Tools (In-App Actions)

Waking up the demo

See this in Inspector

Open Inspector on localhost. Go to Inspect, then Frontend Tools. Your tool and its schema are listed.

More detail: Inspector.

What is this?

A frontend tool is a function that you register in the browser. Your agent can call it, and the function runs in the user's browser. Because the handler runs in the browser, it has direct access to component state, browser APIs, and any third-party UI library the page already uses. That's how an agent can "reach into" the app: update React state, trigger animations, read localStorage, pop a toast, or steer the user's view.

This page covers the "agent drives the UI" shape of frontend tools. To render a component for a tool call, see Generative UI. To ask the user before a tool runs, see Human-in-the-loop.

When should I use this?

Use frontend tools when your agent needs to:

  • Read or modify React component state
  • Access browser APIs like localStorage, sessionStorage, or cookies
  • Trigger UI updates, animations, or transitions
  • Show alerts, toasts, or notifications
  • Interact with third-party frontend libraries

How it works in code

Declare the component as an external tool

Agno's AG-UI interface does not forward the request's tool definitions to the model. The model only sees tools declared on the Agent, so a component registered with useComponent needs a matching declaration whose body stays empty. Mark it external_execution=True: Agno pauses the run, the browser renders the component, and the run resumes with the result.

The name and the arguments have to match the useComponent registration exactly. The docstring is what the model reads, so describe when to call it there.

src/agents/chart_agent.py
from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools import tool


@tool(external_execution=True)
def render_bar_chart(title: str, data: list[dict]):
    """
    Render a bar chart in the chat.

    Call this whenever the user asks for a chart or a comparison.

    Args:
        title (str): A concise chart title.
        data (list[dict]): Items shaped `{label, value}`.
    """


agent = Agent(
    model=OpenAIChat(id="gpt-6-luna"),
    db=db,
    tools=[render_bar_chart],
    instructions=SYSTEM_PROMPT,
)

Give the agent somewhere to store the paused run

Agno stores the paused run before the browser can return its result, so the Agent that owns the external tool needs a database. An agent with an external tool and no db cannot resume after the browser answers.

src/agents/chart_agent.py
from agno.db.sqlite import SqliteDb

db = SqliteDb(db_file="tmp/agno.db")

SqliteDb needs sqlalchemy installed. For production use durable shared storage such as PgDb, because an ephemeral container file cannot resume a run on another instance.

Register a frontend tool with useFrontendTool. Give it a name, a Zod schema for parameters, and a handler. The agent can then call it like any other tool and your frontend runs it in the browser.

page.tsx
import React, { useState } from "react";
import {
  CopilotKit,
  CopilotSidebar,
  useFrontendTool,
} from "@copilotkit/react-core/v2";
import { z } from "zod";
import { Background, DEFAULT_BACKGROUND } from "./background";
import { useFrontendToolsSuggestions } from "./suggestions";

function Chat() {
  const [background, setBackground] = useState<string>(DEFAULT_BACKGROUND);

  useFrontendTool({
    name: "change_background",
    description:
      "Change the page background. Accepts any valid CSS background value — colors, linear or radial gradients, etc.",
    parameters: z.object({
      background: z
        .string()
        .describe("The CSS background value. Prefer gradients."),
    }),
    handler: async ({ background }) => {
      setBackground(background);
      return { status: "success" };
    },
  });

The handler receives the parsed, type-safe parameters and can do anything the browser can: update state, call an API, touch the DOM. Its return value is sent back to the agent as the tool result so the model can reason about what happened.

page.tsx
    handler: async ({ background }) => {
      setBackground(background);
      return { status: "success" };
    },

Ask the agent to change the background to a blue gradient. The page background changes, and the agent gets { status: "success" } as the tool result.

Registering a list of tools

useFrontendTool registers one tool per call, so it cannot be called in a loop over a list whose length changes between renders. When the set of tools comes from state, from props, or from a backend response, use useFrontendTools instead. It takes an array and runs a single effect over it, so the array can be empty on one render and hold twenty entries on the next.

useFrontendTools(
  reports.map((report) => ({
    name: `open_${report.id}`,
    description: `Open the ${report.title} report`,
    handler: async () => navigate(`/reports/${report.id}`),
  })),
  [navigate],
);

On each render:

  • A tool that leaves the array is unregistered.
  • A tool that joins the array is registered.
  • A re-render that produces an equal list does not re-register anything.

A description built from your data stays current on its own. The second argument is for values a handler closes over, such as navigate above.

How is this guide?