Validating agent work

A skill tells an agent to check the record. It does not make the agent do it. Validation that holds needs three layers, and each one catches what the previous one misses.

The problem

The decision-librarian skill fires when a person states a consequential choice: “let’s move to Elasticsearch” triggers a search. But an agent producing work makes dozens of small choices nobody states. It can write code that contradicts a decided architecture without ever noticing a choice was involved. Instructions are advisory; a model under context pressure can skip them, and nothing fails when it does.

Layer 1: the skill (advisory)

Keep it. It handles the conversational cases well: stated choices, “why did we” questions, drafting. It costs nothing and catches most conflicts at the moment they are cheapest to resolve, before any work is done. Just do not treat it as a guarantee.

Layer 2: a harness hook (deterministic, per machine)

Most agent harnesses can run a script at fixed points, outside the model’s discretion. Two placements work well with Precedent, using Claude Code as the example; the Hooks page has the full setup for every hook point, tokens included.

On every prompt, inject the record. A UserPromptSubmit hook queries search_decisions with words from the prompt and adds the top hits to context. Hybrid search does the semantic matching, so wording does not have to match. This guarantees the relevant decisions are in front of the model; it does not guarantee compliance.

When the agent finishes, check the diff. A Stop hook (or a PreToolUse hook on git push) summarises what changed, searches the record with it, and blocks with the conflicting refs if anything decided comes back. The block reason is fed to the model, which must reconcile before it can finish:

// .claude/settings.json
{
  "hooks": {
    "Stop": [{ "hooks": [{ "type": "command",
      "command": "./scripts/precedent-check.sh" }] }]
  }
}

# scripts/precedent-check.sh (sketch)
# 1. summary=$(git diff --stat && git log -1 --format=%s)
# 2. POST the summary to search_decisions on your /mcp endpoint
# 3. hits above threshold? exit 2 and print:
#    "This change touches decided topics: AR-064, AR-065.
#     Cite them or raise a supersession before finishing."

The hook uses embedding search, not judgement: it tells the agent that a decided topic is touched and makes it look. Whether the work actually contradicts the rationale is settled by the model reading the decision, which the block forces.

Layer 3: CI (universal)

A hook only governs one harness on one machine. The check nobody can skip runs where the work lands: a CI step that takes the pull request diff, searches the workspace’s decisions with it, and comments the matching refs on the PR, or fails when a decided topic is touched and no decision is cited. That covers every agent and every human, whatever tools they used. The same read-scoped OAuth client an agent uses works for CI; audit exports record its calls like any other client’s. The GitHub Actions page has the complete workflow.

What to expect

Layer 1 catches stated choices in conversation. Layer 2 catches your own agents’ produced work before it leaves the machine. Layer 3 catches everything at the merge, later but universally. None of them decides anything: every path ends in a person reading the cited decision and confirming, revising, or superseding it in the app, which is the point.