Writing

A shell, not a schema

How a local 7B model went from failing every tool call to driving almost any process — by meeting it where it already spoke the language.

A 7B Qwen coder model, running locally on my own machine, started out useless for anything agentic and ended up able to drive almost any process I put in front of it, without the constant failures. The model never changed. The interface did — and the lesson generalizes.

Looking inside its head, cheaply

Before I trusted the model with anything, I wanted to know what it actually knew, as opposed to what it could plausibly produce once. So I did the dumbest thing that could work: I wrote a tiny HTTP server that asked the model to generate a web page, and then I just hit refresh. Same prompt, fresh generation, over and over.

That refresh is a window into the model's head. A real, reliable skill comes back the same every time — the model isn't guessing, it knows this. A lucky guess scatters: every refresh is a different shape, a different bug, a different story. You're not reading weights or attention maps; you're reading the variance, and variance is certainty turned inside out. It isn't mechanistic interpretability — it's just sampling the same question enough times to tell a skill from a coincidence. For almost no effort, it's startlingly effective.

What it told me was that the model's competence is wildly uneven, and the unevenness tracks one thing: how often it saw something in training. Rock-solid on the idioms it had seen a million times. Shaky on anything bespoke. Hold that thought.

The schema tax

The standard way to let a model act is tool calls: you hand it a JSON schema of functions and ask it to emit structured calls you parse and execute. I built that. Qwen choked on it — malformed JSON, wrong shapes, arguments in the wrong place, calls that almost-but-didn't conform. It spent its capability fighting the format instead of doing the task, and a small model has none to spare. Every token spent conforming to my schema was a token not spent thinking. The protocol was a tax, paid in the one currency the model couldn't afford.

And of course it was — a bespoke JSON tool schema is exactly the kind of bespoke thing the refresh test had already shown it was weakest at. I'd built the interface in the one place the model was sparse.

The flip

So I threw the protocol out. Instead of inventing a language and forcing the model to speak it, I gave it an environment it was already fluent in: a sandboxed UNIX shell over a virtual filesystem. It has read millions of shell sessions. ls, cat, grep, a pipe, a redirect, a heredoc to write a file — that's not a protocol it has to learn, it's an idiom it's drowning in from training. The model acts by emitting one sh block; the commands run against the sandbox; the combined output comes back on the next turn. No schema to conform to. Just a language it already knew.

The same 7B that couldn't reliably emit a tool call now reads files, transforms them, writes them back, and chains multi-step work with ; and && and | — without the constant stumbling. Making the environment native to the model did what reaching for a bigger model or a fine-tune would have, for free, on the model I already had.

The principle

Meet the model where its training is dense. The instinct is to design the interface where it's convenient for you — a tidy, well-typed schema you can validate. But the model didn't read your schema a million times; it read the shell. Build on the idioms the model is already saturated with, and a small model stops fighting and starts working. The environment should be native to the model, not the model forced native to your framework.

This is why the agent in my editor acts through a shell instead of a tool API, and why I treat a local 7B as a real workhorse rather than a toy. It was never short on ability. I'd just been asking for it in a language it didn't speak.