MCP Server vs API Gateway Decision Framework

Every enterprise architecture review has this argument now: "we already have an API gateway, so MCP is covered." That's a mistake, and a confidently made one. The two technologies sit at different layers of the stack and encode different assumptions about who's calling and what they already know before they call. Mix them up and you don't get redundancy, you get a gap. Someone finds that gap eventually. Better it's your security team than a researcher writing it up for next year's breach report.

What API gateways were actually built to do, and where they still belong

An API gateway governs communication between services over HTTP. Rate limiting, authentication, TLS termination, request routing, all of it works because HTTP hands the gateway everything it needs up front, sitting right there in the header or the path. Every request arrives self-contained. Policy enforcement is deterministic, because someone wrote a spec for the request before it ever showed up.

Decades of production hardening went into this. Audit logging, versioning, developer portals, all mature, all boring, and boring is the compliment here: nobody gets paged at 3 a.m. because the rate limiter misread a request. It parsed a header. That's the whole job.

Gateways still belong anywhere the caller is deterministic code: microservices talking to each other, partner API traffic, a mobile app hitting its backend. None of that changed because your company shipped an AI assistant last quarter. The requester calls the same endpoint the same way every time, and the gateway treats it the same way every time.

The trouble starts once an agent becomes the client. Agents need memory across calls. They need to discover tools they didn't know existed when the code was compiled, kick off a task, and come back later for the result. None of that lives in an HTTP header. Teams that try to bolt MCP support onto an API gateway anyway end up with weeks of custom glue code and a system that still can't natively parse JSON-RPC or track session state, according to MintMCP's review of enterprise deployments that tried it. You can force that fit if you want. Just know you're signing up to maintain duct tape indefinitely.

What MCP servers are actually built to do, and the N×M integration problem they solve

MCP gives AI agents one shared way to reach tools and data, instead of a custom connector for every pairing. Before the protocol existed, every model needed its own bespoke wire to every tool: N models times M tools, each pairing its own thing to build, secure, and eventually fix when it breaks. Ten agents, five tools apiece, and congratulations, you're now the proud owner of fifty integration points that drift out of sync the moment anyone touches one.

MCP collapses that math the same way HTTP collapsed it for browsers a generation ago. Any compliant client talks to any compliant server. The difference this time is the client isn't a person clicking a link, it's a model deciding what to do next.

What sets MCP apart from REST comes down to mechanics. It supports async operations, so an agent can start a long task and check back instead of blocking. It carries state across a session, so the server remembers what happened three steps ago without being told again. And it exposes tools, resources, and prompts as objects a model can reason about directly, rather than forcing it to parse raw JSON and guess at intent.

Dynamic tool discovery is the clever part, and it's also where things go sideways. A server can list its available tools at runtime and let the agent pick. Works fine with five tools. Hand an agent forty of them and watch it stall out trying to pick the right one, a failure mode I've watched happen more than once in production agentic workflows, usually right before someone quietly rolls the change back.

MCP earns its place wherever an LLM makes a genuinely dynamic choice about which tool to call, especially across workflows that touch multiple systems and need to remember what came before. It's also the right layer when a dozen teams inside the same company all need the same tools and shouldn't each be writing their own connector from scratch.

The decision criteria: which signals tell you which architecture fits

Start with one question. Who's the client? Deterministic code calling a known service belongs behind an API gateway. An LLM making a context-dependent choice about which tool to invoke needs MCP. That one question resolves most cases by itself.

State is the second signal. Requests that stand alone and get authorized independently are API gateway territory. An agent that has to remember step two while it's working on step five needs MCP's session model instead.

Third, ask whether the tool set is fixed or discovered on the fly. A versioned schema with a stable, developer-defined contract is a gateway problem. An agent figuring out what's available as it goes is an MCP problem.

Traffic shape is the fourth signal, and it's the one people skip. High-volume, short-lived, stateless HTTP calls are exactly what gateways were tuned for over twenty years of production use. Long sessions with async completions and chained tool calls look nothing like that pattern, and routing them through the same infrastructure just buries the mismatch until it surfaces as an outage.

Fifth, look at how mature the system you're exposing actually is. A stable REST service with a locked contract stays behind its gateway and gets an MCP adapter layered on top if agents need it. A new AI-native service should be built MCP-first, since retrofitting it later means living through the multi-week slog described above.

If the requester can't tell you in advance which endpoint it's going to hit, an API gateway alone can't govern it. That's the exact moment MCP stops being optional and starts being required.

When the answer is both: the layered architecture most enterprises will end up with

Most production shops are stacking both rather than picking one, and the pattern repeats across every company I've seen actually ship this: MCP sits as the AI-facing layer, REST APIs sit behind the API gateway as the execution layer underneath. The agent speaks MCP, the MCP server translates that into calls the backend already understands, and the gateway keeps doing what it always did: rate limiting, auth, routing to the right service.

Kong is a clean example of this bridge in practice. Its platform generates MCP servers directly from existing REST endpoints, no manual rewrite required, which means a company with a decade of REST APIs doesn't have to rebuild any of it just to make it agent-reachable.

This layering protects sunk cost. Existing gateway infrastructure, and the operational muscle behind it, doesn't get thrown out. Non-AI service traffic keeps flowing exactly as it did before. MCP just opens a new door for AI clients without touching what already worked.

What this arrangement doesn't hand you for free is governance at the MCP layer itself. Who's allowed to discover which tools, under what permissions, logged where? Neither the gateway nor a bare MCP server answers that question on its own. That's precisely the gap MCP gateways fill: the API gateway governs service-to-service traffic, the MCP gateway governs agent-to-tool traffic. Gartner has called MCP gateways the missing enterprise layer for registration, discovery, authentication, authorization, and observability across thousands of MCP servers, which is more or less the same role API gateways played for REST a generation back.

Why the ungoverned MCP layer is a security problem enterprises are already living with

Walk into most enterprises today and you'll find MCP servers running in isolation, agents juggling multiple connections with no central discovery point, and no audit trail tying any of it together. One 2026 enterprise security survey put the average organization at 37 deployed agents already, a number that keeps climbing every quarter as teams spin up automation faster than security can review it.

This sits inside a shadow IT problem that predates MCP by years. A 2025 Gartner survey found 69% of organizations suspect, or have confirmed, employees using GenAI tools nobody approved. IBM's 2025 Cost of a Data Breach Report ties shadow AI to one in five breaches and tacks on $670,000 to the average cost of one. Banning the tools doesn't fix it, either: close to half of employees keep using personal AI accounts after a ban comes down, just quietly, on their own devices, where nobody's watching.

MCP introduces risks a gateway was never built to catch, because it was never looking at this part of the conversation. GitGuardian found 24,008 unique secrets sitting exposed in MCP configuration files in 2025, a fair number of them Google API keys and database connection strings in plain text, in a place no scanner thought to check. Then there's EchoLeak, tracked as CVE-2025-32711, disclosed in June 2025 with a CVSS score of 9.3: a zero-click flaw in Microsoft 365 Copilot where a single crafted email got the AI to quietly exfiltrate documents while the user did nothing more suspicious than ask it to summarize their inbox. By March 2026, Unit 42 was documenting the first large-scale indirect prompt injection campaigns running against live commercial platforms, not lab demos.

The confidence gap is the part that should keep a CISO up more than any single CVE. A Beam AI survey found 82% of executives believed their existing policies already covered unauthorized agent actions, while separately, 88% of organizations reported a confirmed or suspected AI agent security incident in the past year. Sit with that for a second: those two numbers coexist in the same market, describing the same companies. Add in that more than half of registered MCP server implementations score 2 out of 5 or worse on published security assessments, and you've got an ecosystem that outgrew its own ability to check its work.

What a governed MCP layer needs to do that a plain MCP server cannot

A bare MCP server sitting there, deployed and left alone, amounts to a working connection, nothing more. Getting from there to something a security team would actually sign off on takes four things.

Policy enforcement has to live at the tool level, not the server level. Knowing who can reach a given MCP server tells you almost nothing useful; you need to know which specific tools inside it a given identity can invoke, tied to whatever identity provider you already run, Okta, Entra ID, or your existing SAML/OIDC setup. Ask any vendor pitching a gateway to describe per-tool policy enforcement in writing. If they can't, you're looking at a proxy with a gateway's name tag stuck on it.

Threat detection has to happen inside the protocol conversation, not around it. Prompt injection needs catching before the tool call executes, and PII or secrets need scanning in both directions, not just on the way in. Responses should get sanitized based on what the requesting user is actually cleared to see, not just whether they authenticated successfully.

Observability has to be session-aware: a full record of which agent, under which identity, called which tool, with what arguments, and got back what response, plus cost and usage broken out per agent and per tool instead of buried in one aggregate nobody reads.

And there has to be real brokering, with pass-through proxying kept honest about what it actually is. Version shielding keeps client agents from breaking every time a backend server ships a protocol update. Tool filtering trims what any given agent sees, both to stop it from choking on too many choices and to shrink the blast radius if that agent gets compromised. Multiplexing stitches several backend MCP servers into one virtual endpoint, so an agent connects once instead of juggling ten separate sessions.

Remote MCP servers, the network-accessible kind rather than local processes, are the fastest-growing slice of this ecosystem, roughly 4x growth through 2025. They're also the ones that need all four of these capabilities most urgently. A local server at least has a physical machine as a boundary. A remote one has nothing.

How to evaluate MCP gateway options across the current vendor landscape

The market splits into three rough tiers, and the one that fits you has less to do with feature lists than with what you've already built.

AWS Bedrock AgentCore Gateway anchors the hyperscaler tier, and it went generally available in October 2025. It's the obvious call if you're already deep in AWS and running workloads through Bedrock, since governance ties straight into IAM and everything else in that ecosystem. The tradeoff is the one you'd expect from any hyperscaler product: the deeper you go, the harder it gets to leave.

Cloudflare and Kong sit in the network-edge tier. Kong's AI Gateway centralizes management of MCP tools, resources, and prompts, and its ability to convert REST endpoints into MCP tools without a manual rewrite makes it a practical fit for any company sitting on a large API estate that wants agent access without a rebuild. Cloudflare brings edge-native distribution, which matters most when latency and global reach are the binding constraints.

The independent tier is where the real differences show up. TrueFoundry, named in Gartner's 2025 Innovation Insight report on MCP gateways, makes sense when you want MCP governance unified with LLM routing and model deployment in one place, assuming you have a platform team willing to own that stack. MintMCP goes the other direction: SOC 2 compliance out of the box, nothing to manage, built for companies that need a compliance story fast and don't have the platform headcount to build one themselves. Composio bets on breadth, aimed at agents that need to reach dozens of SaaS tools quickly, trading some governance depth for that reach. IBM's ContextForge takes a federation approach, built for organizations with enough MCP servers in play that coordinating them under one umbrella beats managing them piecemeal.

None of these is a universal right answer, and anyone who tells you otherwise is selling something. Pick based on what you've already built and how much platform work you're actually willing to own, not on whichever vendor's slide deck has the shiniest logo.