All guides

AI agent harnesses: a field guide

What an AI agent harness is, how the generator-evaluator loop and check gates decide when a change is done, and why the harness is the part you trust.

Field guide3 articles35 min read

An AI agent harness is the layer around a coding model that turns one-shot prompting into a repeatable loop: plan, generate, then check the result against gates like tests, type checks, or a reviewer model. The model writes the code. The harness decides whether to ship it, retry, or stop.

Three articles, for anyone letting an agent write code they will have to maintain. Read them in order: the loop itself, then why the harness is what you trust, then the pipeline that ships what it produces. The build log of one specific harness is its own guide.

In this guide

  1. What a ralph loop actually is, and how to run one without burning your tokensA ralph loop is a coding agent in a while loop. What it is, why it resets context each pass, and how to run one without it spinning forever or costing $200 overnight.13 min read
  2. Trusting AI-generated code: the harness, not the modelTrusting AI-generated code you didn't read line by line: trust the harness you built, not the model. AI now assists ~42% of committed code.8 min read
  3. The delivery harness: your agent writes code, but who's shipping it?Agent harnesses orchestrate code generation. Delivery is a longer pipeline. Here's the mental model I've been using to plug team, tools, context, and models into one value stream, and where the agent harness fits inside it.14 min read

Key ideas

  1. The loop matters more than the prompt. A harness owns the whole cycle (plan, generate, evaluate, verify), so its output answers to the same checks a human engineer would run.
  2. A ralph loop is the bare mechanism: an agent in a while loop, state on disk in git history and files, resetting to a clean context every iteration instead of dragging a filling window forward.
  3. Generator and evaluator are separate roles. One model proposes a change; a second model, or a deterministic gate, critiques it and sends it back for a supervised retry. An agent grading its own homework gives itself an A.
  4. Check gates decide done, not the model's confidence. Tests, type checks and linters are deterministic, inspectable and yours, which is why trust belongs to the harness rather than to the model inside it.
  5. An agent harness gets a correct change produced and verified. A delivery harness is the longer pipeline around it: requirement, plan, ship, review, operate.

Frequently asked questions

What is an AI agent harness?

An AI agent harness is the layer around a coding model that turns one-shot prompting into a repeatable loop: plan, generate, evaluate, verify. Check gates like tests, type checks, and linters decide when a change is actually done. The model writes the code; the harness decides whether to ship it, retry, or stop.

What is a "ralph loop", and how does it relate to an agent harness?

A ralph loop is the bare technique: a coding agent in a while loop, running a fresh context each iteration with state on disk (git and files) instead of the model's context window. An agent harness is what you wrap around that loop to make it stop safely: a generator-evaluator gate, check gates, and crash recovery. The full explainer is at /blog/ralph-loop/.

How is an agent harness different from a coding assistant?

A coding assistant answers a prompt; a harness owns a workflow. It controls the loop, feeds the model context, runs the gates, and retries when something fails, so the result answers to the same checks a human engineer would run instead of to a single model reply.

What is a generator-evaluator loop?

A generator-evaluator loop splits the work between two roles: a generator proposes a change, and a separate evaluator (another model, or just a deterministic gate) judges it against explicit criteria. The change only moves forward when the evaluator passes it; otherwise it goes back for another supervised attempt. That's how it catches the plausible-but-wrong output single-pass generation lets through.

What is the difference between an agent harness and a delivery harness?

An agent harness handles code generation: getting a correct change produced and verified. A delivery harness is the longer pipeline around it: turning a requirement into a plan, shipping the change, then running review and release. The agent harness is one stage inside it.

More guides

All guides