---
title: Reviewing The Agent's Own Session
type: playbook
level: L2
status: live
revision: 1
updated: 2026-08-14
systemVersion: 4.2
tags: [quality, review, playbook]
rating: 7.55
ratingAxes: useful 8 · evidence 6 · pull 7 · original 9 · form 9
ratingKind: derived
source: session review skill, in production
---

# Reviewing The Agent's Own Session

_Written 2026-08-14 · last verified 2026-08-14 · system v4.2 · live_

**TL;DR** — A session review checks the agent's behaviour against its own written rules rather than against a feeling about output quality. It loads the rule set dynamically, verifies each rule against the transcript, and every failure becomes a counter entry — which turns a retrospective into an escalation mechanism instead of a diary.

## Prerequisites

A written rule set the agent is supposed to follow. A transcript of the session. A repeat-mistake ledger for the output to land in.

Without the third, this produces observations that go nowhere.

## Steps

**1. Load the rules dynamically, do not hardcode them.** The review reads the current rule files at run time. A review with a baked-in checklist drifts from the rules it is supposed to enforce, and the drift is invisible — it keeps passing.

**2. Check each rule against the transcript, not against memory.** For each rule: did the situation it governs occur, and was it followed? Three outcomes — `n/a`, `followed`, `violated` — and `n/a` must be available, or every review inflates into a list of technically-unmet rules that never applied.

**3. Check the categories separately.** They fail differently:

> protocol — was the required output emitted, in order · routing — was the right skill or agent used · memory — was what should have been written, written · verification — were claims checked at the level they live · autonomy — was anything done that should have been asked

**4. Score it, and keep the scale coarse.** A number makes trends visible across sessions. Keep it blunt — false precision on a subjective scale invites arguing about points instead of about behaviour.

**5. Route every violation into the ledger.** This is the step that converts a retrospective into a mechanism. A violation becomes a counter entry; the counter escalates on its own thresholds. Without it, the same finding is rediscovered monthly and nothing changes.

**6. Name the 1 to 3 decisions you were least sure about.** Not violations — judgement calls. These are the highest-value output of the whole pass, because they surface the places where the rules were silent rather than broken.

**7. Compare against the previous review, not against zero.** A single score is nearly meaningless — the useful signal is direction. Two consecutive reviews with the same finding mean the fix did not take, which is a stronger result than either review alone.

**8. Keep one section the agent cannot pass.** A field for *what I would do differently* with no rubric and no score attached. Everything scored gets optimised toward the score; an unscored field is the only place where an honest answer has no incentive working against it. In practice it produces the most useful line in the whole review.

## Verify

The review works when its findings are **actionable without further investigation**. `Response was too long` is not actionable. `Emitted the suggestions block after an explicit request for brevity — rule X` is.

Check the false-positive rate. A review that flags rules which did not apply will be dismissed, and a dismissed review is worse than none because it consumes the slot a real one would occupy.

## Troubleshooting

**Every session scores well and nothing improves.** The review is checking what is easy to check. Add the categories that are uncomfortable — autonomy and verification usually.

**Findings repeat forever.** Step 5 is missing. Observations without counters do not escalate.

**The review takes longer than the session.** Scope it to rules that had a chance to fire. A full pass over every rule every time is thorough and will be abandoned within a month.