How an MCP Gateway Brokers Tool Calls Between AI Agents and Backend Services

An MCP gateway is a proxy that sits between AI agents and the backend tools they call, and it's the single checkpoint where identity gets verified, policy gets enforced, and every call gets logged before a result ever reaches the model. It solves a problem the Model Context Protocol itself refuses to solve: the protocol defines how an agent packages a tool call, but says nothing about who should be allowed to make it. This piece walks through the five jobs a gateway does on every single call, compares how real implementations divide that labor, and lays out what to check before putting one in front of an entire company.

Why the MCP Protocol Creates a Governance Gap

MCP tells an agent how to ask a tool server to run a function and how to get the answer back. Governance never lived in the spec. It has to live somewhere else, and in most companies right now, it lives nowhere.

The shape of the problem is simple arithmetic. Every team ends up managing its own connectors, on its own schedule, with its own notion of what "secure" means.

Shadow MCP is what happens next, and it happens fast. What should worry a security team isn't that the server exists. A firewall built to catch outbound data over HTTP has nothing to catch here, because the data isn't leaving over HTTP in any form the firewall recognizes.

Speakeasy's own definition names the failure mode without softening it: without a gateway, nobody can centrally revoke access, and tool payloads carrying source code, customer records, and credentials move through the system unseen. The gap sits between them, in the space neither side controls.

What an MCP gateway is and where it sits in the stack

An MCP gateway is a proxy that sits between AI agents and MCP servers, and it does something structurally clever to pull that off: it acts as an MCP server when the agent talks to it, and as an MCP client when it talks to the real tool servers upstream. Every call an agent makes passes through this one point, and that is what makes central control possible. So instead of N agents wiring up M servers directly, every agent talks to one gateway, and the gateway talks to every server.

Three terms get used almost interchangeably in casual conversation, and they shouldn't be. The gateway is what you get once you take a bare proxy and add identity, policy, inspection, and audit on top of it.

The gateway is best understood as the tool-call piece of something larger: an AI control plane, the governing layer sitting between every AI agent in a company and every system those agents are permitted to touch. That control plane also covers things like LLM routing and agent orchestration, but the gateway's job is narrower and more specific. Speakeasy's reference architecture frames the gateway as doing two separate jobs on the exact same traffic. For enablement teams, it's how agents reach third-party and internal data without every employee hand-wiring every connector into every client; for security teams, it's where that same access becomes tied to a real identity, inspectable in flight, and auditable after the fact.

None of this means every team needs one tomorrow. When N agents and M servers both grow, the math from the section above stops predicting load and starts predicting a weekly incident.

Step one of brokering: how the gateway handles tool discovery and the agent-facing catalog

The first thing a gateway does, before a single tool call runs, is decide what an agent is even allowed to see. An agent that can't see a tool can't call it, and that restriction closes off an entire category of misuse before it starts.

This filtering is the detail that separates an MCP gateway from an ordinary API gateway. A conventional API gateway routes HTTP paths; it has no concept of a tool list or a tool call as distinct operations. An MCP gateway understands those semantics natively, so it can tailor the catalog itself, per agent, per session, deciding exactly which tools are reachable.

Speakeasy describes the process that builds toward that filtered view as a four-step loop. Fourth, govern: route every call, from every client, through the one control point. That sequence treats MCP servers as managed infrastructure, the way a company would treat any other production service, rather than as ad hoc connectors someone bolted on over a weekend.

There's a cost dimension to catalog filtering that's easy to overlook. Scoping the catalog per identity keeps it lean as a side effect of keeping it secure. agentgateway builds this in as a named capability: discovering MCP servers, versioning them, and keeping an audit trail of what got invoked, treating MCP servers less like static configuration and more like microservices that need their own lifecycle management.

Step two: how the gateway resolves and enforces identity on every tool call

Before any tool call reaches a backend, the gateway has to answer a question the protocol itself never asks: who is this. Once that resolution happens, every action downstream is attributed to someone specific.

On the inbound side, the mechanics vary by context. Interactive users typically authenticate through browser-based OAuth 2.1 flows. Machine-to-machine and automated sessions often rely on JWT and OIDC validation at the edge, and the claims get pulled directly into the routing and authorization decisions the gateway makes a moment later.

A newer approach called EMA uses an Identity Assertion JWT Authorization Grant, or ID-JAG: the enterprise identity provider issues it, then hands it to the MCP server's own authorization server, which exchanges it for an access token. Per-tool scoping still has to happen at the gateway layer regardless of whether EMA is in the mix.

Shared credentials are one of the most common security risks in MCP deployments, mainly because a shared credential can't be revoked for one person without breaking it for everyone else. agentgateway enforces this with explicit tool scoping built into its authentication layer: an agent authenticated into an MCP server is still limited to the specific tools that identity is permitted to invoke, so holding a valid credential doesn't mean holding unlimited reach.

Step three: how policy enforcement decides whether a specific tool call proceeds

Identity answers who is calling. Policy answers whether that particular call should be allowed to happen. Once the gateway knows the identity behind a tools/call request, it checks that request against policy: is this identity, on this client, allowed to call this specific tool with these specific arguments, and if the answer is no, what's the reason.

A reasonable baseline for enterprise MCP governance runs to six controls at minimum: OAuth 2.0 authentication with credentials kept outside the AI's own context, role- and attribute-based authorization evaluated per operation, audit logging detailed enough to attribute every action to a person, path and scope controls limiting what a tool can reach, rate limiting, and evaluation against sensitivity labels on the data involved.

Virtual keys are one mechanism that makes this concrete. A virtual key with no MCP configuration attached exposes no MCP tools at all by default, with the single exception of clients explicitly marked Allow by Default. Execution itself doesn't have to be automatic either: a governance model can require that any tool not on an explicit auto-execute list gets returned for human approval before it runs.

Some organizations don't want the gateway making these calls on its own. External authorization delegation lets the gateway hand the allow-or-deny decision to an existing entitlement service over gRPC or HTTP, so the access logic that already governs the rest of the company's systems doesn't have to be rebuilt a second time inside the gateway. One decision point, every call.

Step four: how the gateway inspects call arguments and responses for security threats in transit

Access control decides who gets through the door. Inspection is what happens once something is already inside the room. The gateway reads both the arguments going out to a tool and the response coming back, before any of it reaches the model, which is the only point in the entire chain where that inspection is even structurally possible.

The sharpest version of this threat is a prompt injection carried inside a tool's own response. The OWASP MCP Top 10 names this prompt injection via contextual payloads, listed as MCP06; tool poisoning under MCP03 covers malicious instructions hidden in a tool's description or schema.

This isn't a hypothetical. A hidden prompt embedded inside a crafted email enabled zero-click data exfiltration from Microsoft 365 Copilot, and no user action was required beyond the email simply being processed. That's what happens when response content reaches a model with nothing checking it first.

PII and secrets embedded in tool payloads exploit the same blind spot from a different angle. The gateway is the only place in the entire pipeline where that traffic passes through a controlled checkpoint.

Supply-chain poisoning is the newer front in the same fight. Researchers have found malicious repositories posing as legitimate AI Skills or MCP servers sitting in public registries, and in testing, agents found and recommended some of those malicious repositories on their own, which led to stolen credentials and installed malware. Allowlisting approved servers at the gateway is the main defense against that pattern. agentgateway treats this as core functionality rather than an add-on, shipping a named PII-shield capability alongside policy enforcement, JWT and OIDC validation, and tamper-evident audit logs.

Step five: how audit logging at the gateway creates the accountability record that agents and backends cannot produce independently

Once a call finishes, the gateway writes a single audit record covering the person, the client, the server, the tool, the outcome, and, where policy allows it, the arguments and the response itself. This is the only point in the whole system where every one of those dimensions sits in the same place at the same time.

Neither side of the transaction can reconstruct that record on its own. Only the gateway holds both halves of the exchange and can tie them back to the identity provider's own record of who that user actually is.

Observability at this layer is what turns agent behavior from something autonomous and unaccountable into something measurable and auditable. Runaway cost is a concrete example: an agent looping through tool calls can burn through a large token budget chasing an answer a single direct call would have produced in one step, and per-run cost telemetry flags that as a budget problem before it reaches the invoice.

agentgateway implements OpenTelemetry by default, tracking token-usage histograms in Prometheus, full traces in Jaeger, and request logs that carry realized dollar cost when a model cost catalog is configured, each one carrying enough attribution to reconstruct what a given agent did and why it did it. Audit logging isn't a reporting feature bolted on after the fact. Without it, every claim about access control is just a policy document. With it, every one of those claims can actually be checked.

How the five brokering functions combine into a governed transit layer

Discovery, identity, policy enforcement, inspection, and audit only amount to governance if they run on every call, at one choke point, without exception. A single call that slips past one of those steps breaks the chain for that call entirely, no matter how well the other four functions performed.

Production gateways tend to implement this as three planes working on the same traffic. Every tool call passes through all three planes, in order, which is what makes the gateway one point of control and one point of audit rather than a loose stack of separate checks that each cover part of the picture.

The functions depend on each other in a way that matters more than it might first appear. Each function covers a gap the others leave open, and none of them substitutes for the one beside it.

This is also why you create a problem, not solve one, when you split governance across separate systems. Speakeasy's framing of the AI control plane makes the organizational stakes explicit: the sanctioned path to tools has to be the easiest path available to employees, and the one security can actually watch. Add friction to the governed path and people will find the ungoverned one, and shadow MCP comes right back.

Architectural trade-offs across representative gateway implementations

No two gateways on the market weight these five functions the same way, largely because they didn't start from the same place. A gateway built by extending an API management platform makes different trade-offs than one built from scratch around MCP semantics, and both differ again from one built by extending an existing integration platform.

Unified open-source AI gateways, Bifrost by Maxim AI among them, govern LLM calls and MCP tool calls inside one gateway, using virtual keys, per-user OAuth, guardrails, and in-VPC deployment. The trade-off is that self-hosting and the operational load that comes with it fall on the platform team running it.

API-platform extensions take a different route. The trade-off is that MCP semantics can end up layered on top of HTTP-path routing rather than built natively into the data plane from the start.

iPaaS bridges extend from the opposite direction. The strength is that it meets a company where its integration layer already lives, rather than asking it to stand up something new alongside it.

What an enterprise-scale gateway needs to cover the five brokering functions

At enterprise scale, thousands of users, hundreds of MCP servers, workflows touching internal APIs and SaaS systems and genuinely sensitive data, the question changes shape.

Shadow AI detection has to surface MCP servers being used outside the sanctioned registry, and it can't just govern the ones already registered and approved. An ungoverned server needs to show up in the inventory as a finding, not stay invisible to it.

The audit pipeline has to feed the organization's existing SIEM, not sit in a proprietary log store that the security team can't query with the tools they already use every day.

Speakeasy's AI control plane is built against this exact scope: it connects, governs, distributes, and observes AI agents, MCP servers, skills, and assistants across an entire workforce, integrating with Okta and Entra ID for role-based access, detecting shadow AI and prompt injection in real time, and surfacing cost and usage telemetry across the organization as a whole, addressing all five brokering functions together at the scale a single team deployment never has to consider.

The five functions were never a checklist to complete once and move on. New MCP servers appear on their own schedule, agent mandates expand as teams find new uses for them, and threat patterns keep shifting under everyone's feet. A gateway earns its place as infrastructure by closing the loop from audit back to discovery, continuously, so that what counts as governed stays current with what the environment actually looks like, not with what it looked like the day the gateway went live.