---
id: con-evals
schema: concept/1
layer: garden
status: emerging
name: Evals
aka: ["evaluations", "test cases", "benchmarks", "AI testing", "golden examples"]
summary: >
  A small, reusable set of test tasks with known good answers that you run an AI setup against, so
  you can tell whether it works and whether a change made it better or worse.
first_seen: unknown
relations:
  builds_on: [con-verification]
  evolved_from: []
  related: []
  parallels:
    - idea: "quality control and acceptance testing"
      field: "operations"
      note: "Check samples against a standard before the batch ships, and after any change to the process."
    - idea: "a grading rubric"
      field: "teaching"
      note: "Writing down what good looks like before marking makes judgments consistent."
implementations: []
competencies: [cmp-check-the-result, cmp-make-it-repeatable]
origin:
  kind: ai-proposed
  by: "seed agent 2026-10-16"
  rationale: "Usually framed as engineering; this card makes it a plain-language habit for anyone who reuses a prompt, skill or assistant."
---

An eval is everyday quality control for AI. If you rely on an AI setup repeatedly, such as a saved prompt, a skill, a custom assistant or an automation, collect five to ten real examples where you know what a good result looks like. Each time you change the setup, or the tool updates its model, run those examples again and compare.

Without a test set, you judge by whatever result you saw last. With one, you can say "the new prompt fixed the tone problem but started missing deadlines in two cases out of eight."

Good test cases include the tricky ones: the email with no clear ask, the document with a buried exception, the request that should be refused. Write down what "good" means for each before you look at any AI output.

## Example

A support team uses an AI assistant to draft replies. They keep eight real (anonymised) customer emails with ideal replies written by their best agent. After editing the assistant's instructions, they rerun all eight and catch a new habit of promising refunds it shouldn't.

## How it connects

- **Verification**, made repeatable and comparable.
- Needed for **skills**, **custom assistants** and **automation** that others rely on.
- The best way to settle **model choice** for your own work.

## Try it

Pick a prompt you reuse. Write down three test inputs and, for each, one sentence on what a good answer must include. Run the prompt on all three and score them yourself.