Harness, Loop, Graph: The Three Layers Behind Reliable AI Agents

The real reason AI agents fail

AI agents often look impressive in a demo: give the model a goal, expose a few tools, and watch it complete a task. Production is less forgiving. Tools time out. State disappears between sessions. A nearly-correct result gets accepted without verification. A workflow branches in ways no one can inspect, and a retry simply repeats the same mistake.

When that happens, teams often upgrade the model first. That is frequently the wrong diagnosis. A serious agent is not just a model; it is a system built around the model. That system has at least three distinct engineering layers: the harness, the loop, and the graph.

“The harness gives the model a place to work. The loop gives the work a feedback cycle. The graph gives the process an explicit route.” - Adapted from the source thread by rari 1

These layers overlap, and one codebase may contain all three. They are nevertheless different levers for reliability:

The shortest version is worth memorizing: harness equals environment, loop equals feedback, and graph equals flow. The distinction becomes important as soon as an agent touches real files, APIs, customers, money, or production code.

1. Harness engineering: build the environment around the model

A raw model can transform an input into an output. It cannot independently maintain project state, run a test suite, inspect a browser, write files safely, enforce permissions, or resume tomorrow where it stopped today. The harness supplies that machinery.

A useful mental model is this: the model provides intelligence, while the harness turns that intelligence into an operational capability. If you remove the model from the architecture diagram, everything that remains is probably part of the harness.

A production harness usually has six responsibilities:

Harness engineering earns its keep when a task lasts longer than one conversation. A coding agent that works for hours cannot rely on chat history alone. It needs durable artifacts that another session-or another person-can understand.

A practical long-running setup might include an initializer that inspects the workspace, a progress file that records what is complete and what remains, commits that preserve working states, checkpoints before risky actions, and verification tools that produce clear evidence. That is not a better prompt. It is a better working environment.

Start with the harness when the agent cannot access the right capability, loses progress between sessions, has permissions that are too broad, behaves differently across environments, cannot be paused or resumed, or produces failures nobody can reconstruct.

2. Loop engineering: make the work iterative and verifiable

Every tool-using agent already has a small internal cycle:

model → action → observation → model

Loop engineering begins when you deliberately design the cycles around that behavior. The goal is not to make the agent repeat itself forever. The goal is to turn a one-shot attempt into a managed process with evidence, useful feedback, and a bounded exit.

The most valuable outer loop is a verification loop:

BUILD
  ↓
CHECK AGAINST EVIDENCE
  ↓
PASS? ── yes ──> STOP
  │
  no
  ↓
RETURN SPECIFIC FEEDBACK
  ↓
RETRY WITH A LIMIT

The check may be deterministic: tests pass, a schema validates, links resolve, numbers reconcile, or files compile. It may also require a reviewer who evaluates whether the argument is complete, the tone fits the audience, the evidence supports the conclusion, or the change is correctly scoped.

The rule is the same in both cases: do not loop on confidence; loop on evidence. “The agent says it is finished” is not proof. Passing tests, resolving sources, reconciling numbers, or receiving approval on a diff is proof.

A useful production loop has seven parts:

Loops can stack. An agent loop performs the work. A verification loop checks the work. An event loop wakes the system when new work arrives. A trace-improvement loop studies production behavior and changes the harness itself.

This is why loop engineering is larger than prompt engineering. A prompt defines what should happen during one model call. A loop defines what the system does after that call.

Loops have a cost: every retry, grader, and reviewer adds latency and spend. Add a loop when the expected cost of failure is higher than the cost of verification. A high-impact publishing workflow may justify several checks; a low-risk formatting task may not.

3. Graph engineering: make control flow explicit

Graph engineering asks a different question: not “How should the agent work?” but “What is allowed to run next?” Work becomes nodes, allowed transitions become edges, and state moves through the graph.

A graph can represent fixed sequences, conditional branches, parallel fan-out, joins, bounded cycles, recovery paths, and human interrupts. The important decisions are architectural rather than cosmetic:

A graph is worth the ceremony when the process contains meaningful branches, parallel specialists, approvals, recovery routes, or stateful handoffs. It is not worth adding merely because a workflow has several steps. If one capable agent with three tools can solve the task, a graph may add structure without adding value.

The opposite mistake is also common: teams formalize the workflow before they understand the work. The result is a beautiful diagram that encodes the wrong assumptions. Start with a simple harness, study real traces, and formalize the paths that remain stable.

How the three layers nest in a real system

Consider a research-and-publishing agent that produces a factual industry briefing. The harness provides browser and search tools, source storage, a writing workspace, citation checking, permissions, approval rules, checkpoints, and traces.

The graph controls the route:

RESEARCH
   ↓
DRAFT
   ↓
FACT CHECK ── fail ──> RESEARCH
   │
  pass
   ↓
EDITORIAL REVIEW ── fail ──> DRAFT
   │
  pass
   ↓
HUMAN APPROVAL
   ↓
PUBLISH

The loops live inside that route. The research node may search until source coverage is sufficient. The drafting node may revise until a style grader passes. The fact-check node may return the exact unsupported claims instead of a vague rejection.

This nesting is the key idea. The graph runs inside the harness. The loops run inside parts of the graph. The harness supplies the tools, state, and evidence those loops need. The layers overlap because real software layers overlap, but they still give you three different levers when the system fails.

Diagnose the failure before changing the architecture

When an agent fails, first identify which layer owns the failure. Changing the model or adding orchestration before doing that usually increases complexity without addressing the cause.

This diagnostic approach is more useful than arguing about terminology. Find the layer that owns the failure, then fix that layer first.

The expensive mistakes behind weak agent systems

Building the graph too early. Do not convert an imagined business process into dozens of nodes before watching a strong agent perform the work. Trace first; formalize second.

Letting the maker grade itself. Self-review is useful, but it shares many blind spots with the original attempt. Prefer deterministic checks where possible, use an isolated reviewer context for subjective checks, and require human approval for high-impact actions.

Defining the loop as “keep trying.” An unbounded retry is not reliability; it is a cost leak. Every cycle needs fresh evidence, a maximum attempt count, and a named escalation path.

Turning the harness into a warehouse. More tools do not automatically create a better agent. A crowded toolset increases selection errors, noisy context increases confusion, and broad permissions increase risk. Give the agent the smallest environment that can complete the job.

Blaming the model for orchestration failures. A stronger model cannot reliably repair stale state, broken APIs, ambiguous tool schemas, or missing exit conditions. Prove that the model is the problem before upgrading it.

A production-ready checklist

Before shipping an agent, review each layer independently and then review how the layers interact.

The framework in one sentence

Harness engineering makes the model operational. Loop engineering makes the work iterative and verifiable. Graph engineering makes complex execution explicit and controllable.

None replaces the others. A perfect graph cannot save an agent that loses its state. A perfect harness still wastes money if the loop has no evidence or stopping rule. A strong loop becomes difficult to operate when branches, parallelism, and approvals are hidden inside ad hoc code.

Reliable agents appear when all three layers are designed together - and when every layer has one clear job:

ENVIRONMENT → HARNESS
FEEDBACK    → LOOP
FLOW        → GRAPH

That is the practical shift from prompt engineering to agent-systems engineering.

Source note: This article is an original adaptation of rari’s thread, “LOOP vs GRAPH vs HARNESS ENGINEERING”, published July 28, 2026. The structure and wording have been edited for a blog audience; credit remains with the original author for the framework and source ideas.

References