Evaluation and improvement

“Continually improves” is easy to say and usually means nothing. This is what it means here.

The loop

Each step names durable evidence in the system, not a vague aspiration:

1. Outcome—what actually happened. The refund was issued; the customer churned anyway; the ticket was reopened twice. Outcomes come from events your tools already emit.
2. Feedback—a human judgment attached to something specific: this output was wrong, this decision was right, this email should not have gone out. Async, low-ceremony, and attributable.
3. Evaluation—feedback generalized into something automatic. A set of cases with expected behavior that any future version must still satisfy.
4. Change—a proposed alteration, usually written by an agent, carrying the evaluation results as evidence.
5. Gate—the rule that decides whether it ships.
6. Release—into an environment, versioned and reversible.

Feedback is a first-class input

Feedback has to be cheap to give and hard to lose.

Cheap: a reviewer rejecting an output says why in a sentence. Someone reading a report flags a number. Nobody files a ticket.

Hard to lose: that sentence is attached to the relevant event, object, tool invocation, output, and source version. It is not a message in a channel that scrolls away. When someone later asks “why does the system do it this way,” the feedback that caused it is still attached.

Evaluations

Feedback tells you about one case. An evaluation makes sure that case never regresses.

Evaluations can run against curated cases, production examples, and historical decisions. A change that would have decided past cases differently deserves closer review, and the evidence should be visible before it ships.

Agent-authored changes are safe at volume for this reason, not because the agent is trusted. A change has to hold up against everything you have already decided is correct.

Improvement is never silent

The industry is loose about this, so to be plain:

• Nothing retrains itself.
• No behavior changes without a versioned change passing a gate.
• Every improvement traces to feedback, an evaluation, a human correction, or an updated policy—all of which you can read.

The system proposes and you decide. It gets better over time because your decisions accumulate as evaluations instead of evaporating.