Skip to content
HN On Hacker News ↗

Harness Engineering for Coding Agents: 9 Patterns for More Reliable AI-Assisted Development

▲ 12 points • 1 comments • by nvmdbljstm • 2d ago • HN discussion ↗

Pangram verdict · v3.3

We believe this text is mainly AI, with some human-written content.

95 %

AI likelihood · overall

AI
5% human-written 95% AI-generated
SEGMENTS · HUMAN 4 of 16
SEGMENTS · AI 12 of 16
WORD COUNT 1,477
PEAK AI % 100% · §3
Analyzed
Nov 3
backend: pangram/v3.3
Segments scanned
16 windows
avg 92 words each
Distribution
5 / 95%
human / AI fraction
Verdict
AI
Pangram v3.3

Article text · 1,477 words · 16 segments analyzed

Human AI-generated
§1 Human · 0%

Home Blog Harness Engineering for Coding Agents: 9 Patterns for More Reliable AI-Assisted Development Harness Engineering for Coding Agents: 9 Patterns for More Reliable AI-Assisted Development read • Table of contents TL;DR: Software testing vs AI testing: what changesThe testing pyramid still holds – but everything inside it changesThe failure modes that actually matterWhat an AI testing strategy should includeWhen the system type changes, so does the testing strategyRetrieval-augmented systemsAgent systemsExternal service integrationsThe part nobody talks about: validating with the clientA more honest definition of “done” Code is cheap.

§2 AI · 100%

Human time and model context are not.

§3 AI · 100%

That is the core shift behind Harness Engineering: instead of trying to write better prompts, teams getting more reliable value from coding agents increasingly design better systems around their agents – documentation, context loading, validation, and feedback loops that let models safely own more of the work.

§4 Human · 0%

Table of contents TL;DR: Software testing vs AI testing: what changesThe testing pyramid still holds – but everything inside it changesThe failure modes that actually matterWhat an AI testing strategy should includeWhen the system type changes, so does the testing strategyRetrieval-augmented systemsAgent systemsExternal service integrationsThe part nobody talks about: validating with the clientA more honest definition of “done” Earlier this year, our team won an OpenAI Codex hackathon with HyperCode, a repository intelligence layer that prepares relevant context for coding agents before they start working.

§5 AI · 100%

Building it gave us a concrete way to explore how the system around an agent shapes its work. For engineering leaders, the goal is more predictable delivery: less time spent reconstructing context and fixing avoidable agent mistakes, with consistent checks before changes reach review. We also tested harness patterns in our own setups with Codex and Claude Code, bringing together people from across deepsense.ai with different backgrounds, workflows, and levels of experience with AI‑assisted development. We explored agent‑readable repo docs, just‑in‑time context, validation loops, and ways to turn recurring mistakes into durable guardrails.

§6 AI · 100%

Some patterns proved useful. Others needed adjustment.

§7 AI · 100%

This post brings together those findings and the practical recommendations that emerged.

§8 AI · 100%

This article is our practical playbook: ten harness patterns you can apply to a real repo this week.

§9 AI · 100%

We focus on three concrete layers: how your repo becomes an agent‑readable knowledge base, how context is compiled just‑in‑time for each task, and how validation and feedback turn agent mistakes into durable system improvements. TL;DR: Make the repo the knowledge base: use a short AGENTS.md, linked docs, and tests that explain why. Load context just in time: assemble relevant files, tests, history, and past decisions for each task. Start with durable tools: prefer familiar commands and a few focused, task-relevant skills. Make validation discoverable: provide a fast command menu shared by humans, agents, and CI. Close the feedback loop: let agents validate, observe, and fix—with explicit retry limits. Use visual feedback selectively: start with text; use targeted screenshots when pixels matter. Prompt for intent: explain the desired outcome and decision process; reserve rigid rules for true invariants. Keep planning proportional: use short plans and low-risk experiments; add detail when mistakes are expensive. Turn mistakes into guardrails: capture friction at checkpoints and promote recurring issues into docs, tests, or linters. Build for model changes: keep the workflow and validation contract stable as models and tools evolve. What we mean by “harness” In this post, harness means everything built around the model: the repo structure, AGENTS.md and specs, tools and skills, sidecars and hooks, validation menus, observability, and the “garbage collectors” that keep the system from decaying. Humans steer objectives and design the harness.

§10 AI · 100%

Agents execute inside it.

§11 AI · 100%

A harness is the system around the model that lets it work on your codebase. It supplies context, runs tools, and feeds results back so the agent can decide what to do next. Repo docs, skills, hooks, tests, and browser automation all contribute to that system. Take a bug fix: the agent reads the relevant code, makes a change, runs a test, inspects the failure, and tries again. The harness makes that loop possible and sets its limits.

§12 AI · 100%

Humans define the goal and constraints; agents work inside them.

§13 Human · 19%

1.

§14 Human · 1%

Repo as knowledge base – document the why, not just the what (AGENTS.md) Your coding assistant can read your code.

§15 AI · 100%

What it cannot infer on its own is why the code is structured the way it is, what tradeoffs were made, which abstractions are intentional, and which parts are legacy landmines. The fix is to give agents access to the engineering knowledge your team already relies on. Start with an AGENTS.md file at the repo root, so the assistant can work from the same assumptions and constraints as the people maintaining the code. What goes into AGENTS.md Think of it as the document you would hand a strong contractor or staff‑level engineer on day one – but written for a model that “craves text” about your architecture, rules, and scars. It should cover: Architecture overview – main modules, their responsibilities, and how they interact Key constraints – “never bypass the auth middleware,” “all DB writes go through the repository layer” Conventions – naming patterns, file structure, preferred libraries Entry points – where to start reading for common task types Out-of-bounds areas – parts of the codebase the agent should not touch without explicit instruction Use AGENTS.md as a table of contents for the codebase: a stable map the agent can orient itself with before starting any task. It does not need to contain every detail. Instead, point to deeper docs in docs/ and domain‑specific READMEs, so the model can progressively discover more context when it needs it. Without this, the agent either invents assumptions or wastes context trying to infer structure from file trees. Minimal AGENTS.md skeleton # AGENTS.md ## What this repo is [One paragraph: product purpose, main user flows, tech stack.] ## Architecture - `src/api/` - HTTP layer, no business logic here - `src/services/` - all domain logic lives here - `src/db/` - repository pattern, direct DB access only here ## Key constraints - Auth is handled by `AuthMiddleware` - do not bypass or replicate it - All external API calls go through `src/clients/` with retry logic ## Conventions - Commit messages: `type(scope): what changed - WHY it changed` - New endpoints need an integration test in `tests/api/` - Feature flags via `FeatureFlag` enum in `src/config/flags.py` ## Where to start - Adding a feature: read `docs/architecture/feature-checklist.md` - Fixing a bug: run `make test-unit` first, reproduce before changing anything - Touching infra: tag @platform-team for review ## Do not touch without asking - `src/legacy/billing/` - scheduled for migration, fragile - `infra/terraform/prod/` - requires human approval Keep this file short enough to load at the start of each task, and update it when the architecture or working conventions change. The commit message as the “why” store AGENTS.md covers structure. But the deeper why – reasoning behind individual decisions – lives best in commit history and test docstrings. Both are already in the repo, versioned, and readable by the agent with a simple git log or grep. Good commit message discipline is not new advice – explaining why in the body, not just what in the subject, has been a best practice for years. The diff shows what changed; the message captures the reasoning that will never be visible from the code alone – the constraints, the tradeoffs, the alternatives you rejected. What changes now is who drafts them. As more of your diffs are generated by Codex or Claude Code, the agent can write the commit message too – including the why – if you instruct it to follow your template. Humans still own the final check: skim for correctness and nuance, especially on risky changes, and feed recurring corrections back into AGENTS.md so the next commit comes out better. ## Commit message convention When committing, always include a body explaining: - Why this change was made - What problem it solves or risk it removes - Any tradeoffs or constraints that shaped the approach Format: type(scope): short summary of what changed Reasoning here - what was the previous behavior, why was it a problem, and what should a future engineer know before touching this. Closes #<issue> With this in place, every agent-generated commit becomes a documentation artifact – a durable explanation of why the model produced it that way, readable by the next agent that picks up a related task. Regression tests as living rationale A test docstring is one of the best places to record why a guard exists – especially for invariants that have already bitten you in production. Keep this for real incidents and non‑obvious constraints so tests stay readable. def test_token_validated_before_db_lookup(): """ Regression: tokens must be validated before any DB call. See commit a3f9c12 - DB-first validation caused race conditions under load and unnecessary read pressure. This test ensures expiry check stays upstream of the repository call.

§16 AI · 100%

""" ...