Your Agent Is Not the Model: Harness vs Inference Service Explained

People often talk about Claude as if it's a single entity, but an AI agent is a stack of three distinct layers. Knowing the difference saves you hours of debugging — when something goes wrong, you need to know whether to blame the model, the service, or the logic around it.

The Three Layers

• Model — the mathematical function that transforms input tokens into output tokens. Examples: Sonnet, Opus, Gemini.
• Inference service — the hosted service that runs the model and tracks usage. Examples: AWS Bedrock, Anthropic's API.
• Harness — the logic that shapes inputs, interprets outputs, and touches the outside world. This includes MCP and Skills — the model doesn't inherently know about them.

Put together, an agent system is a harness that calls an inference service, which runs a model. That's it.

Real-World Breakdown

The same model (say, Sonnet) can behave differently depending on the harness — because the harness shapes inputs and interprets outputs. That's why your prompts work in Claude CLI but not in Cursor.

Debugging With This Mental Model

• Bad answers? Look at the harness — maybe it's not providing enough context.
• Too slow or expensive? Check the inference service — pricing and performance live there.
• Unexpected output? Is the model wrong, or is the harness feeding it garbage?

The article also notes that as models get smarter, some harness logic (like MCP or Skills) might become obsolete — so the way we build harnesses now may not age well.

Next time you say "my model did X," stop and ask: was it the model, or was it the harness orchestrating it?