Every LLM feature starts the same way: you try a few prompts, the outputs look good, and you ship. Then you tweak the system prompt to fix one annoying case, and three other cases quietly break — and you have no idea, because "looks good" does not have a test suite. Evaluations are how you turn a vibe into something you can measure and defend.

An eval is just a test for non-deterministic output. You give the system inputs, you check the outputs against some standard, and you get a score you can watch over time. The trick is that "check the output" is harder when the correct answer is a paragraph of prose rather than 42. There are a few ways to handle that, and good eval suites use more than one.

Why eyeballing does not scale

Manual review has three problems. It does not repeat: you cannot re-run last week's judgment against today's prompt. It does not cover: you look at the cases you happen to think of, not the ones that actually break. And it hides regressions: a change that improves nine cases and silently ruins the tenth looks like a pure win until a user finds the tenth. Evals fix all three by making the check automatic, comprehensive, and comparable across versions.

Three kinds of checks

Golden datasets pair an input with an expected output (or an acceptable set). Best when there is a right answer — classification, extraction, routing. Assertion-based checks do not demand an exact answer; they assert properties: the output is valid JSON, contains the order id, stays under a length, never leaks a system-prompt phrase. LLM-as-judge uses a model to score an output against a rubric, which is the only practical option for open-ended text like summaries or tone. Each has failure modes; using all three keeps any one from lying to you.

A small harness

You do not need a framework to start. A list of cases, a runner, and a scorer is enough to catch real regressions.

type Case = { name: string; input: string; check: (out: string) => Promise<boolean> };

const cases: Case[] = [
  {
    name: "extracts an order id",
    input: "where is order A-4471?",
    check: async (out) => /A-4471/.test(out) && !/system prompt/i.test(out),
  },
];

async function judge(output: string, rubric: string): Promise<number> {
  const res = await model.complete(
    `Score 0-5 how well the response meets the rubric.
Rubric: ${rubric}
Response: ${output}
Return only a number.`,
  );
  return Number(res.trim());
}

for (const c of cases) {
  const out = await runFeature(c.input);
  const passed = await c.check(out);
  console.log(passed ? `PASS ${c.name}` : `FAIL ${c.name}`);
}

Assertions give you a hard pass or fail; the judge gives you a graded score for the cases where "correct" is a spectrum. Store both so you can track pass-rate and average score separately.

Metrics that matter

Pick metrics that map to what users feel. Exact-match and parse-success for structured output. Semantic similarity when wording can vary but meaning must not. Rubric scores for quality dimensions like helpfulness or tone. The single most useful number is pass-rate over your case set, tracked per prompt or model version — it turns "did that change help?" into a chart instead of an argument.

LLM-as-judge is not neutral. A judge model tends to favor outputs that match its own style, and it can be talked into a high score by confident-sounding nonsense. Pin the judge to a fixed model and prompt, spot-check its scores against your own, and never let it grade in a way you would not.

Run them in CI

Evals only prevent regressions if they run before the change lands. Wire the suite into CI and set a floor: a prompt or model change has to clear the existing pass-rate to merge.

jobs:
  evals:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run evals -- --min-pass-rate 0.9
        env:
          MODEL_API_KEY: ${{ secrets.MODEL_API_KEY }}

One caution as the suite grows: do not overfit. If you keep tuning the prompt until every case in the set passes, you have taught it your test, not the task. Keep a held-out set you do not tune against, and add new cases from real failures as they show up. Evals are a ratchet — every production bug becomes a case that can never regress silently again.