Blog · · 6 min read
Connecting Voice Agents to Other Systems: APIs over MCP Servers
Plugging an MCP server straight into a phone agent means every call runs on the account of whoever signed in. Why APIs (a URL-and-API-key connection or in-call tools) are the safer way to give voice agents tools, what changes on the phone, and how to do it with voicast.
Key points
- Connecting an MCP server directly to a voice agent means one person signs in with OAuth and that token is used by every call in the environment, so anyone who phones in acts with that person's permissions.
- For phone agents, connect through APIs where you decide whose permissions a call uses. In voicast that means a URL-and-API-key connection, or in-call tools that hit your own server.
- Either way, timeouts, a short tool list, read-back and consent before any write, and small tool outputs matter more on the phone than in chat.
To let a phone agent check availability or look up a customer mid-call, you give it tools that reach into other systems. The Model Context Protocol (MCP) is one way to do that, but when the person on the other end of the line could be anyone, it's safer to connect through an API where you decide whose permissions each call runs with. This post explains why, walks through two API-based approaches, and covers what changes when the user is on the phone.
Three ways to give a voice agent tools
There are roughly three ways a voice agent ends up calling external systems:
| Approach | Whose permissions a call uses | Good fit when |
|---|---|---|
| In-call tools that hit your own server | Your server decides, per customer | The target is your own system with a fixed contract |
| A URL-and-API-key connection | An API key issued for the agent | Calling a third-party REST API as is is enough |
| An MCP server connected directly | The account of whoever signed in with OAuth | One person using AI for themselves (chat, coding tools) |
MCP is a standard way for AI models to discover and call tools in other systems. The system publishes a tool list (name, description, input schema) as an MCP server, and the client reads that list and calls tools as needed (MCP spec). That model fits assistants like Claude or Cursor well, where one person uses their own account. Wiring an MCP server straight into a phone agent raises a few problems.
Problems with plugging an MCP server into a phone agent
Every call runs as the person who signed in
Remote MCP servers authenticate with the MCP authorization spec, which is OAuth 2.1 authorization code with PKCE (authorization spec). The person who signs in is whoever set up the connection in the dashboard. If the voice platform holds that token, every call in the environment calls the MCP server with it.
Callers, though, are strangers. The tools the agent can reach cover everything that signed-in account can do, so with the right phrasing a caller may get at data or actions meant only for that person. When that person leaves the company and their account is disabled, every call loses the integration at once. And the target system's audit log shows the signed-in person, not the caller who actually asked for the change.
Tools change without your review
The tool list and descriptions live on the MCP server. When tools are added or descriptions are reworded, the agent's behavior changes even though you never touched its config. Phone agents should be tested before every change goes live (versioning agent configs), so the fewer parts that can shift underneath you, the better.
Annotations are hints, not guarantees
MCP defines tool annotations such as readOnlyHint, but the spec itself says clients should treat annotations as untrusted unless they come from trusted servers (MCP schema). Any tool you can't be sure is read-only has to default to requiring confirmation.
Two ways to connect through an API
With an API, the permissions a call uses are limited to a key you issued for the agent, or to whatever your own server decides.
1. A URL-and-API-key connection
If the other system exposes a REST API, connect it with its base URL and an API key, and define the paths and parameters the agent may call as tools. Use a key issued for the voice agent, scoped down to what it needs (read-only, or create bookings only), rather than a person's account. The model sees only the tools you defined, and if the vendor adds endpoints, nothing changes for the agent until you update those definitions.
2. In-call tools that hit your own server
In a multi-tenant SaaS, each of your customers usually has their own account on the target system. Use in-call tools so your server receives each tool call, works out which customer the call belongs to, and calls the target system with that customer's credentials. Even when a vendor only ships an MCP server, calling it from your own server keeps you in charge of which account and which tools are used. Never put credentials in prompts or tool arguments.
Four things that change on the phone
Whichever approach you use, these matter far more on a call than in chat:
- Timeouts. While a tool runs, the caller hears silence or "let me check that for you." Set a hard limit and have a fallback: if the tool doesn't answer in time, the agent apologizes and offers a callback.
- Tool count and naming. Hand every operation to the model and it picks the wrong one more often, and your prompt grows. Expose only the tools the call needs.
- Confirmation before writes. Anything that changes data, such as creating or cancelling a booking, should be read back to the caller and confirmed before it runs (confirming by reading back).
- Output size. Tool results go straight into the model's context. Return a customer's entire history and replies slow down, and the agent may start reading out details nobody asked for. Return only the fields needed to answer.
Doing this with voicast
voicast does not offer a way to connect an MCP server directly to the AI in calls. Connect other systems with a URL-and-API-key connection or with in-call tools.
Create a URL-and-API-key connection with POST /v1/connections, then save the tools it should call (up to 10):
curl https://api.voicast.jp/v1/connections \
-H "Authorization: Bearer $VOICAST_API_KEY" \
-H "Content-Type: application/json" \
-d '{ "connector": "http", "secret": "<API key>", "input": { "base_url": "https://api.example.com", "header": "Authorization" } }'Reference it from the agent's connectors as { "connection": "con_…", "actions": ["*"] } and the AI can use those tools during calls. If the API doesn't answer within 3 seconds, the call is cut off and the AI gets timeout. Tools set to "confirm before use" are read back, and the AI uses them only after the caller agrees.
In-call tools go in the agent's tools:
"tools": [
{
"name": "create_booking",
"description": "Book an appointment. Use after offering slots and the caller picks one",
"url": "https://saas.example.com/voicast/tools",
"confirm": true,
"parameters": { "properties": { "start": { "type": "string" }, "name": { "type": "string" } }, "required": ["start", "name"] }
}
]Each time the agent uses a tool, voicast sends a signed POST to that URL. The request's call.tenant.external_id tells you which business the call is for, so that's where you pick the credentials for the target system. voicast waits up to 8 seconds and passes up to 4,000 characters of the response to the model. See function calling in live calls for the details.
voicast is also an MCP server itself (https://api.voicast.jp/mcp). That one is for developers who sign in with their own account to create agents and tenants from Claude Code and other MCP clients, covered in setting up voicast from Claude Code over MCP.
FAQ
What if the system I need only offers an MCP server?
Receive the call on your own server with an in-call tool, and call the MCP server from there. Your server decides which account and which tools to use, so callers never inherit the permissions of whoever signed in.
Should I use a URL-and-API-key connection or in-call tools?
If calling the vendor's API as is works and one key per environment is enough, a URL-and-API-key connection is the quickest. If each customer has their own credentials, you want to tidy results before the model sees them, or one tool needs to call several systems, use in-call tools.
What does the agent say when a tool fails?
Tell the model about the failure or timeout and have it apologize and offer a callback from staff. While waiting on a tool, the agent should say something like "Let me check that for you" so the line never goes silent.