All guides

Building ralphctl: an agent harness, release by release

The build log of one agent harness: how ralphctl went from a v0.0.4 sprint CLI to a cross-provider loop I trust to run coding agents overnight.

Field guide5 articles70 min read

ralphctl is the agent harness I build and use daily. It wraps an AI coding CLI in structure: one model generates the code, a second model checks it against the spec, and the state lives in files rather than in a context window.

These five articles are the build log, in order, from v0.0.4 to v0.21.0. Read them start to finish for the arc, or jump to whichever release solved the problem you have. For the concepts underneath, without the version numbers, start with the field guide.

In this guide

  1. Building RalphCTL: a sprint CLI for AI-assisted codingI built a sprint CLI for Claude Code and GitHub Copilot. The problems I didn't see coming and what it took to make AI-assisted coding feel like a real workflow.17 min read
  2. From sprint CLI to agent harness: how ralphctl got an evaluatorAnthropic's first harness article inspired ralphctl. Their second one on generator-evaluator loops pushed me to build an evaluator for v0.2.0, and the tool changed identity.11 min read
  3. The harness era caught up: ralphctl and the convergence I bet onWhen I started ralphctl, 'harness' was a word from one Anthropic post. Now it's the third phase of AI engineering, there's an arXiv paper, and the whole field has standardized on the patterns I built early.13 min read
  4. Earning the overnight run: ralphctl from 0.8 to 0.13Last post I confessed ralphctl broke users' data four times in a week with no clean fix. Ten releases later, it has one. The unglamorous month that took the harness from 'works in the demo' to something I trust to run overnight: safe migrations, a memory, parallel tasks, a real budget.10 min read
  5. What ralphctl found when it finally measured itselfThe first time ralphctl graded itself it drew a line through its own history: unmeasurable before the instrument shipped, fully graded after.19 min read

Key ideas

  1. ralphctl is terminal-only and file-backed: state lives as JSON under ~/.ralphctl/, no server and no database. Two-phase planning splits requirements (the refine flow) from dependency-ordered tasks (the plan flow), both launched from the TUI.
  2. A per-repo verifyScript, typically pnpm typecheck && pnpm lint && pnpm test, runs before and after every task and gates completion. I added it after shipping broken code twice.
  3. v0.2.0 added the evaluator, and by v0.2.3 it graded each task against explicit verificationCriteria on correctness, completeness, safety and consistency; robustness made it five floors in v0.15.0. The retry loop is bounded and non-blocking.
  4. From v0.8.4 the loop crossed labs by default: Claude Opus 4.8 generated, OpenAI Codex on GPT-5.5 evaluated, so a rival lab graded the work. Parity across providers is real, and keeping it that way is constant work.
  5. The overnight-trust work was unglamorous: parallel tasks in isolated git worktrees and an NDJSON learning ledger (v0.9.0), cost-tiered presets that climb one rung per stall, and a consent-gated crash-safe migration (v0.13.0).
  6. v0.20.0 gave the harness a report on itself. The first run said 84.3% first-pass and zero failed acceptance criteria, off 274 of 400 criteria that were never graded at all.
  7. Design decisions now cite their evidence in the source: thirteen arXiv papers at 42 sites across 23 files at v0.21.0, including the comments that say where the code departs from the paper.

Frequently asked questions

What is ralphctl?

ralphctl is an open-source agent harness for long-running AI coding tasks. It takes a ticket, refines it into structured requirements with AI, plans dependency-ordered tasks, executes them through a coding CLI, and gates each one on a check script plus a second model's review. State lives as files under ~/.ralphctl/, so there is no server and no database. It is on npm under the MIT license.

Which AI coding providers does ralphctl support?

Four backends: Claude Code, OpenAI Codex CLI, GitHub Copilot CLI and OpenCode, selectable per flow, so a different backend can sit on refine, on plan, and on each side of the implement loop. OpenCode is the vendor-neutral one: it fronts other providers and local runtimes on your own keys, or runs on its free tier with none. OpenCode joined in v0.19.0.

How does ralphctl decide that a task is done?

Two gates in sequence. First a per-repo check script has to pass, for example pnpm typecheck && pnpm lint && pnpm test. Then an evaluator model grades the change against the task's explicit verificationCriteria across five floor dimensions, correctness, completeness, safety, consistency and robustness, and the overall result fails if any single dimension fails. A failure resumes the generator with the critique, within a bounded per-attempt turn budget.

Can you leave ralphctl running unattended?

That is what the v0.8.4 to v0.13.1 releases were for. Tasks can run in parallel in isolated git worktrees that fold back onto one sprint branch, a per-project learning ledger carries what one sprint discovered into the next, cost-tiered presets start on a cheap model and climb one rung per stall, and an upgrade runs a consent-gated, crash-safe migration with a full backup instead of asking you to start fresh.

More guides

All guides