Cloudflare Code Mode: Why Your Home Lab AI Is Burning Context (and the Fix)

Every MCP server you connect to an AI assistant quietly dumps its tool definitions into the model's context window — so more tools can mean a slower, pricier, and less accurate assistant. Cloudflare's "Code Mode" flips that: instead of handing the model 200 tool definitions, it lets the model write small snippets of code that call the tools. Cloudflare fit their entire 2,500-plus endpoint API into about 1,000 tokens instead of 1.17 million. Here's the problem, the fix, and what it means if you self-host a pile of MCP servers.

Here's something I noticed the more I leaned on AI in my home lab: the more tools I plugged in, the more sluggish and scatterbrained the assistant felt. Ask it one simple thing and it would sometimes reach for the wrong tool, or just take its sweet time getting there.

I assumed it was me. It wasn't. It turns out there's a real, measurable tax for every tool you connect — and a company you might not expect, Cloudflare, has one of the more clever fixes for it.

Let's walk through what's actually happening, why it slows your assistant down, and what "Code Mode" does about it.

A quick refresher: what is MCP?

MCP stands for Model Context Protocol. It's an open standard that gives an AI assistant a common way to plug into external tools — a "tool" being any action the assistant can take, like "list my Docker containers" or "check GPU usage." One protocol means one AI client can talk to lots of different tools without custom glue for each. If you want the longer version, I wrote a full guide to the Model Context Protocol that goes deeper.

The part that matters here: each MCP server you connect advertises its tools to the model, and every one of those tool definitions has to live in the model's context window.

Two quick definitions so nobody gets left behind. A token is a chunk of text — roughly a word or part of a word — and it's the unit models are measured and billed in. The context window is the model's short-term memory: the total pile of tokens it can "see" at once, including your question, the conversation so far, and yes, every tool definition from every connected server.

The hidden tax: what happens when you add server number seven

Think of the context window as a whiteboard with a fixed amount of space. Everything the model needs for this task has to fit on that board — your actual question included.

Now start connecting MCP servers. Each one writes its tools onto the whiteboard before you've even asked anything. One server for containers, one for your network gear, one for home automation, one for your notes… and suddenly a big chunk of the board is covered in tool descriptions the model has to read every single time.

That costs you in three ways:

• Less room for the real work. Tokens spent on tool definitions are tokens not available for your task, your files, or the conversation.
• More money and more latency. If you're paying per token, you're paying to re-send those definitions on every turn. Bigger context also means slower responses.
• More chances to pick wrong. Hand a model a menu of 200 tools and it can genuinely struggle to choose the right one — the same way you'd struggle to find one item in a 40-page diner menu.

There's a subtler waste too. When the model has to chain several tool calls together, the traditional approach feeds the result of each call back through the model just to copy it into the input of the next call. As Cloudflare put it in their original Code Mode write-up, "wasting time, energy, and tokens." The model is basically acting as a very expensive clipboard.

Why code beats menus

Here's the insight that makes Code Mode click. Cloudflare's team argued that LLMs are better at writing code to call tools than at calling tools directly.

Why? Because of what these models learned from. They've read millions of open-source projects — mountains of real-world code. But "tool calls" are a special, relatively rare format that models mostly saw in contrived training examples. So asking a model to juggle a big menu of tool calls is playing to its weakest hand.

Cloudflare had a great line for it: making an LLM do tasks through tool-calling is "like putting Shakespeare through a month-long class in Mandarin and then asking him to write a play in it. It's just not going to be his best work." Let him write in English — let the model write in code — and you get its best work.

The other win is composition. When the model writes a little script, it can chain several calls, filter the results, and hand back only the final answer you asked for — instead of dragging every intermediate result back through its own brain.

What Code Mode actually does

So instead of exposing every tool to the model, Code Mode turns the tools into a typed code API (basically a documented set of functions the model can call in code), and gives the model just two tools:

• search() — the model writes a bit of code to search through what's available and narrow thousands of options down to the handful it needs.
• execute() — the model writes code that actually calls those functions, chains them, and returns just the result.

Conceptually, the model goes from picking off a giant menu to writing a two-line note:

// Instead of the model reading 200 tool definitions,
// it writes a little code against a compact API:
async () => {
  const rules = await search("waf firewall rules");
  return execute(rules[0]); // returns only the final result
}

(That snippet is illustrative — the point is the shape of it, not exact syntax.)

The generated code runs in a locked-down sandbox — specifically a V8 isolate, the same lightweight JavaScript sandbox that powers Cloudflare Workers. It spins up in milliseconds, runs the snippet, and gets thrown away.

Now the numbers, because they're the headline. Cloudflare built an MCP server that exposes their entire API — over 2,500 endpoints — through just those two tools, using about 1,000 tokens. The equivalent traditional MCP server, with every endpoint as its own tool definition, would burn roughly 1.17 million tokens — more than the whole context window of today's top models. That's a 99.9% cut, measured with OpenAI's tiktoken counter. And the footprint stays fixed no matter how many endpoints exist.

There's a nice security side effect, too. The sandbox has no filesystem, no environment variables, and no outbound internet by default. Your API keys stay with the supervisor that dispatches the calls, not in the code the AI writes — so the model literally can't write a script that leaks your keys. For anyone who has ever worried about handing credentials to AI-generated code, that's a genuinely reassuring design.

Worth noting: this isn't just a Cloudflare idea. Anthropic explored the same pattern independently in their code execution with MCP work, which matters if — like a lot of us — you drive your setup with Claude.

What it means if you self-host a pile of MCP servers

Time for an honest caveat: Code Mode as Cloudflare ships it is a hosted, Cloudflare-flavored feature. I don't run it in my lab, and pointing everything at Cloudflare isn't the goal for most self-hosters. So treat what follows as principles you can apply, not a setup guide for something I'm running.

The underlying lesson is portable. If you've wired several MCP servers into an assistant — the same way I connect tools to help manage my home lab with AI — you're paying the tool-definition tax whether you notice it or not. A few things that hold true regardless of Code Mode:

• Connect fewer servers at once. You don't need every MCP server live for every task. Turn on the ones this session actually needs.
• Prefer well-scoped tools. A server that exposes 12 focused tools is kinder to your context window than one that throws 90 at the model.
• Watch for the slowdown. If your assistant got sluggish or started fumbling tool choices right after you added a server, that's your signal — not a coincidence.
• Expect this pattern to spread. "Let the model write code instead of drowning it in tool definitions" is showing up across the ecosystem (Anthropic's programmatic tool calling is the same idea). It's likely coming to a client you already use.

Final thoughts

I run a fairly modest local AI stack — Open WebUI, LiteLLM, and Ollama — and I've felt the "too many tools" drag firsthand. Finding out it was a measurable design problem, and not just my lab being cranky, was oddly satisfying.

Code Mode isn't something I've deployed, and I'm not telling you to run out and adopt it. But the core move — let the model write a little code instead of forcing it to read a phone book of tools every turn — is clever, and the direction MCP tooling is clearly heading. If your AI assistant has been feeling a little slower with every server you bolt on, now you at least know why.