If you have ever watched an AI coding agent nail one task and completely botch the next, you have probably blamed the model.
I did too. I would swap from Claude to GPT, try a smaller local model, chase the latest release notes. Sometimes that helped. Often it did not.
Then I watched Aditya Bhargava's talk. He goes by "Aditya," pronounced like a tax audit, and he made a case I had been ignoring: maybe the harness matters more than the model.
Bhargava works at Etsy and wrote Grokking Algorithms. His talk is not a product pitch. It is a first-principles walk through what a harness actually is, why it changes outcomes, and how you can evolve one step by step.
This post is my notes from that talk, written for anyone building or using AI agents, especially coding agents in tools like Cursor.
What is a harness, anyway?
Start with three words people mix up constantly: model, harness, and agent.
The model is the brain. It takes text in, produces text out. By itself, it cannot read your files, run your tests, or open a browser. It just predicts the next token.
The harness is everything wrapped around that brain. Tools, permissions, safety checks, loops, memory, sub-agents, how errors get handled. The harness decides what the model can touch and what happens when something goes wrong.
The agent is the combination: model plus harness, working together on a goal.
Think about Claude Code, or Cursor's agent mode, or any coding assistant that can edit files. You are not talking to raw Claude. You are talking to Claude inside a harness someone built. That harness decides whether the agent can run shell commands, which directories it can write to, whether it asks before deleting files, and how it recovers when a test fails.
Same model, different harness, different behavior.
That distinction sounds obvious once you see it. But most of us, me included, still reach for a better model before we ask whether the harness is the real bottleneck.
Why does the harness matter so much?
Bhargava pointed to a benchmark called HarnessBench. It runs 106 realistic agent tasks and measures how much the harness, not just the model, affects outcomes.
The spread was striking. On the same task suite, one harness configuration scored 52.4%. Another scored 76.2%. More than a 20-point gap, with the harness as the main variable.
That is the gap between "this is unusable" and "this actually works for my team."
The effect gets stronger with weaker models. A great harness can pull a mediocre model up to something useful. A bad harness can make a strong model look dumb.
If you have tried running a local model through an agent setup and felt disappointed, the model might not be the whole story. The harness might be leaving performance on the table.
Bhargava's broader point is that if we keep betting everything on proprietary frontier models, we stay dependent on a handful of companies. If harness quality is what really changes outcomes, any of us can build and improve harnesses, including ones that make local open-source models good enough for real work.
What goes into a harness?
Before the seven-step ladder, it helps to name the parts. A harness is not one thing. It is a stack of decisions:
- Tools — what the agent can do (read files, write files, run commands, search the web)
- Permissions — what it is allowed to do without asking
- Interrupts and handlers — how the harness pauses for human approval
- Constraints — scoping tools to specific directories, APIs, or operations
- Feedback loops — running tests, checking output, trying again
- Delegation — sub-agents that handle specialized work
- Optimization — measuring what works and tuning the harness over time
You do not need all of this on day one. The talk walks through them in order, using one running example: fixing a bug in a Python file called median.py.
The bug: a median function that breaks on even lists
The example is deliberately small. You have a median.py file with a function that computes the median of a list. It works for odd-length lists. It fails for even-length lists. When the list has an even number of elements, you need the average of the two middle values, not just one middle index.
You ask the agent to fix it. There is a failing test to prove it.
Simple problem. But how you set up the harness completely changes whether the agent succeeds, stalls, or does something dangerous.
Step 1: Model only
The most basic setup: just the model. No tools. No file access.
You tell it: "The Python function median in demo/median.py has a bug. Please fix the bug." It responds: "In order to help, I'll need to see the existing code."
Of course it does. It cannot read the file. It cannot write anything. It can only talk.
Problem it solves: nothing, really. This is the baseline.
Problem it creates: no grounding in reality. The model guesses.
Step 2: Tools (read and write)
Add tools. Now the agent can read median.py and write changes back.
Better. It sees the real code. It can apply the fix directly.
But you just gave an autonomous process write access to your filesystem. It can edit any file it can reach. It can overwrite something important. It can introduce a fix that breaks three other things.
A well-designed harness should treat unconstrained read/write as unsafe by default, not something you hand out casually and hope for the best.
Problem it solves: the model can act on the real world instead of just talking about it.
Problem it creates: safety. Capability went up. So did risk.
Step 3: Handlers and interrupts (human in the loop)
Add handlers. Now when the agent wants to read or write a file, the harness pauses and asks you to approve.
You see what file it wants to touch. You type yes or no. Safe again, mostly.
But you are now the bottleneck. Every file read and write needs you. For a one-line fix in median.py, fine. For a refactor across twenty files, you will click approve until your hand cramps.
An interrupt here is not just a popup. It is a pause in execution. The harness stops, asks for input, and only continues if you approve.
Problem it solves: safety without giving up tool access.
Problem it creates: speed. You traded autonomy for control.
Step 4: Constrained tools (scoped autonomy)
Instead of giving the agent a generic read(file_name, directory) tool where it can pick any directory, you lock the directory argument to ./demo/:
# Conceptually: lock the directory so the LLM only sees file_name
read_file = partial(read, directory="./demo/")
write_file = partial(write, directory="./demo/")This pattern is called partial function application. You take a general-purpose tool and pre-fill the arguments you do not want the model to control.
The model does not even know the directory parameter exists. It passes a filename. The harness always reads and writes inside demo/.
Now the agent can read and write freely, but only inside that folder. No approval popup needed. It cannot touch your home directory. It cannot overwrite your SSH keys.
For the median.py example, the agent reads the file, figures out the fix, and describes the corrected code. But in this version it still stops short of writing. It asks whether you want it to update the file.
Problem it solves: the capability vs safety tradeoff. You get autonomy without full trust.
Problem it creates: you have to think about scope upfront. Wrong scope means the agent cannot reach what it needs, or can still reach too much.
This is the step that clicked for me. Most coding agents already do something like this: sandboxing, workspace roots, allowed commands. But naming it as constrained tools makes the design intent clear. You are not removing capabilities. You are narrowing them.
Step 5: ReAct — reason, act, observe, repeat
Tools and constraints are not enough if the agent cannot check its work.
ReAct stands for reason and act. The agent reads the file, acts (writes a fix), observes (runs the test), and reasons about what happened. If the test still fails, it tries again.
# Conceptual loop
while not tests_pass():
code = read_file("median.py")
fix = model.reason_about(code, test_output)
write_file("median.py", fix)
test_output = run("pytest test_median.py")For the median bug, the agent reads median.py and the test file, runs tests, sees the failure, writes a fix, runs tests again, and confirms success. No human in the loop. The harness closed the loop.
Problem it solves: one-shot fixes that look right but are not. The agent gets feedback from the real environment.
Problem it creates: context bloat and cost. Every read, write, test result, and retry goes into the conversation. Longer tasks mean bigger prompts, more tokens, more noise.
If you have used Cursor's agent and watched it run tests in a loop, you have seen this pattern. It works. It also eats context fast.
Step 6: Sub-agents as tools
Instead of one agent with every tool dumped into one context, you give it specialized sub-agents.
Bhargava's example: fix the failing test in test_median.py, then research Jensen's inequality for medians using a research agent and explain it in limerick form. The main agent does not get raw read/write tools. It gets two sub-agents as tools: a coding agent (with file tools and tests) and a research agent (with search).
The main agent picks which specialist to invoke. The specialists run in parallel when possible, research and code fixing at the same time.
Problem it solves: context pollution and tool confusion. When one agent has too many unrelated tools, it picks the wrong one or gets lost. Grouping tools inside focused sub-agents keeps each context smaller and the job clearer.
Problem it creates: orchestration complexity. Who talks to whom? How do results get merged? What if a sub-agent fails silently?
This maps directly to how modern coding setups work. MCP servers, specialized tools, separate research steps. Same idea, different packaging.
Step 7: Self-optimization
The last step is measure, then tune.
Instead of hand-tweaking prompts and hoping, you define what success looks like ("fix the bug in median.py using TDD") and let the harness run experiments against that goal.
The optimizer runs the agent, scores the result (a baseline might be 0.2 on some objective), and adjusts the prompt or configuration to do better. You are not guessing. You are measuring.
# Conceptual: optimize the harness, not the model
for iteration in range(N):
score = run_agent(harness_config, task="fix median.py with TDD")
harness_config = improve(harness_config, score)Problem it solves: hand-tuned harnesses that work on demos but fail in production. You stop guessing and start measuring.
Problem it creates: you need examples and a metric. Garbage examples produce a garbage harness. This is engineering work, not magic.
The capability vs safety tradeoff
Every step on that ladder is really a decision about two things: what the agent can do, and what you are willing to let it do unsupervised.
Step 2 gives capability. Step 3 pulls it back for safety. Step 4 finds a middle ground. Step 5 adds verification. Step 6 manages complexity. Step 7 makes it repeatable.
If you are building an agent for yourself, you might accept more risk. If you are building for a team at Etsy, safety is not optional. The harness is where that tradeoff lives, not in the model weights.
Safety and control flow should be first-class
Most agent frameworks bolt safety on as an afterthought. Check permissions in middleware. Hope the loop stops when it should. Wrap everything in try/catch and pray.
Bhargava argues that a good harness treats pauses and approvals as real control flow, not a side feature you wire up at the edges. The agent should be able to pause anywhere: inside a loop, inside a tool call, inside a sub-agent. Resume cleanly from the exact same point. Serialize execution state and come back to it later.
That matters because agents take real actions. The harness is not just routing API calls. It is the layer that decides when the agent is allowed to act, when it must stop and ask, and when it should try again.
Whether you build that in Cursor hooks, custom middleware, or your own runtime, the design question is the same: is safety part of the execution model, or something you glued on afterward?
The controversial part: bet on the harness, not the model
Proprietary models from Anthropic, OpenAI, and Google are strong. They will probably keep getting stronger. But they are also expensive, rate-limited, and outside your control.
The industry wisdom right now is that models are so good you can keep the harness simple. Give it a few tools and the model does the rest. Bhargava pushes back on that. Simple harnesses plus expensive proprietary models lock you into paid APIs and make local models feel hopeless.
If the harness is what really determines agent quality, then a good harness with an open or local model might beat a bad harness with a frontier model. HarnessBench suggests that is not wishful thinking. Weaker models gained more from a better harness.
For coding agents specifically, this changes the calculus. Instead of waiting for the next model release, you could invest in:
- Tighter tool scoping
- Better test feedback loops
- Sub-agents that keep context clean
- Measured optimization instead of prompt guessing
The gap between a local model and Claude might be smaller than we think, if the harness is doing its job.
The harness ladder, at a glance
Bhargava walks through seven stages with the same model and the same median.py task. Only the harness changes:
- Model only — can talk, cannot act
- Tools — can act, but unsafe
- Human-in-the-loop handlers — safe, but slow
- Constrained tools — autonomous within a safe scope
- ReAct feedback loops — verifies its own work
- Sub-agents — more capability without context bloat
- Self-optimization — measure and improve systematically
Simple things should be simple. Complex things should be possible. Fixing median.py should not require a research project. Running a multi-agent workflow in production should not mean fighting your framework at every step.
Things you will forget
A few practical reminders, because I will need them too:
- A better model will not fix a harness with no test feedback loop
- Unconstrained write access is how agents delete the wrong file at 2 AM
- Human approval on every action does not scale. Scope the tools instead.
- Sub-agents help with context until you lose track of which one did what
- Optimization without a clear metric is just automated guessing
Where this leaves you
Next time an agent disappoints you, ask which step on the harness ladder you are missing.
Maybe it needs constrained file access. Maybe it needs a test loop. Maybe it needs a research sub-agent so the main context stays clean.
Watch the full talk by Aditya Bhargava. It is one of the clearest explanations of agent architecture I have seen, and it changed how I think about the agents I build and use.