MCP Goes Stateless: No Handshake, No Sticky Sessions, Better AI Gateways

MCP Goes Stateless: No Handshake, No Sticky Sessions, Better AI Gateways

TL;DR

MCP 2026-07-28 removes the initialize handshake and protocol-level sessions, so each request can reach any server replica. For Azure gateways, the useful part is routing and rate limiting per tool from validated headers, without parsing JSON bodies.

Keep application state explicit, protect retry state and check streaming and tracing before removing affinity. Azure’s dedicated AI Gateway is still a preview, and its October resource change matters if you’re planning a pilot.

Running a remote MCP server behind a load balancer used to come with an awkward question: which instance remembers this client? If the server issued an Mcp-Session-Id and kept its session in memory, you needed affinity, shared storage or a deployment small enough to avoid the problem.

The 28 July specification release changes that model. Every request describes itself. A tool call no longer needs a handshake before it can do useful work, and the next call doesn’t have to find the same process.

I spend a lot of my time in the layer between agents and the things they call. From there, the interesting question isn’t just what changed in MCP. It’s what we can simplify in the gateway, what we can finally do per tool, and which responsibilities haven’t gone away.

In a hurry? Jump to the gateway checklist or the eight checks before go-live.

Updated 4 November 2026: the SDK sign-in guidance and AI Gateway migration notes below reflect documentation checked on that date.

Read more
APIM Backend Fan-out: Shared Backend + rewrite-uri vs One Backend Per API (A Real Benchmark)

APIM Backend Fan-out: Shared Backend + rewrite-uri vs One Backend Per API (A Real Benchmark)


🎯 TL;DR: Shared Backend + rewrite-uri is performance-equivalent to one Backend per API.

The question: if you point 100+ APIs at a single APIM Backend entity and use rewrite-uri to reconstruct the path, do you pay a measurable latency / throughput / reliability penalty versus modelling one Backend entity per API?

Two identical APIM Premium instances, same Function App backend, same k6 load. Across 1.67M requests and ~32 minutes of stepped load (50 → 100 → 200 VUs): 882 vs 874 req/s, 146 vs 153 ms p95, 0 errors on both sides. All three pass/fail thresholds satisfied. Pick the pattern on operational simplicity, not performance.

Bonus finding: backend cold-scale will lie to you. The first run made the shared-backend pattern look 2× slower, that was Flex Consumption scaling up during APIM-A’s window, not a pattern difference. Always warm your backend before measuring.

Full benchmark, IaC, k6 scripts, KQL, and interactive dashboard: github.com/Ricky-G/azure-scenario-hub/tree/main/src/apim-backend-fanout-benchmark

Recently I ran into a scenario worth benchmarking properly: an APIM Premium estate with 100+ APIs that all ultimately call the same upstream service. The architectural choice is whether to model those APIs against a single APIM Backend entity and use rewrite-uri to reconstruct the right path on the way out, or to create one Backend entity per API. One backend is one thing to manage, one health probe, one circuit breaker, one place to flip URLs during a migration. One hundred is, well, one hundred.

Intuition splits cleanly down the middle on whether the shared pattern carries a hidden cost (connection pools are per-backend, surely?) or none at all (it’s just a string concat). I couldn’t find a clean controlled benchmark of this specific question, so I built one, fully reproducible in the Azure Scenario Hub. What follows is the methodology, the numbers, and the wrong answer the first run produced.

The Two Patterns

flowchart LR
    subgraph A["APIM-A · Premium 1 unit · SHARED backend + rewrite-uri"]
      A_API1[api-shared-01 /svc01/v1]
      A_API2[api-shared-02 /svc02/v1]
      A_APIn[...10 APIs]
      A_FRAG{policy fragment
set-backend-service
+ rewrite-uri} A_BE[(1 Backend entity)] A_API1 --> A_FRAG A_API2 --> A_FRAG A_APIn --> A_FRAG A_FRAG --> A_BE end subgraph B["APIM-B · Premium 1 unit · PER-API backend"] B_API1[api-perapi-01 /svc01/v1] --> B_BE1[(Backend 01)] B_API2[api-perapi-02 /svc02/v1] --> B_BE2[(Backend 02)] B_APIn[...10 APIs] --> B_BEN[(Backend 10)] end A_BE --> Func[(Same Mock Backend
Function App · 5ms baseline)] B_BE1 --> Func B_BE2 --> Func B_BEN --> Func
Read more

Choose your design

Only the look changes. Your posts, ads, comments and other integrations stay in place, with the same light/dark and cookie preferences.

Choose the look you prefer