Writing · AI evaluation

The AI Reliability Checklist: 10 Checks Before You Ship an LLM Feature

Robert Morong · 2026-10-22 · Drafted with AI assistance (Claude)

Print this, run it against your feature, and you'll know whether you're shipping something that works or something that merely demoed well. Each check is a question with a measurable answer — "measure, never assert."

Before you test

1. Define the task and the ground truth. Write down what a correct answer is and how you know. If you can't state how you'd check correctness, you can't measure accuracy — you can only measure consistency and calibration. Say which.

2. State the decision and the bar. "Ship to users" needs a higher bar than "internal draft." Pick the accuracy threshold before you see the numbers, so you're not grading on a curve after the fact.

3. Build a real test set. Cover the typical path and the edges: ambiguous inputs, long inputs, adversarial phrasings, out-of-distribution cases, and your known failure modes. Record the sample size prominently — it governs everything that follows. Make sure the model hasn't seen these items (no contamination).

Accuracy, honestly

4. Report accuracy with a confidence interval. Wilson for rates, bootstrap for F1/RMSE/AUC. A bare percentage with no interval is not a result. If the interval is too wide to clear your bar, add test cases.

5. Don't call a difference real unless the stats support it. Comparing two prompts or two models? Use a paired test and check the interval of the difference. If it crosses zero, the "improvement" is noise.

Stability

6. Test robustness to trivial changes. Re-run under meaning-preserving perturbations — paraphrase, case, whitespace, and especially option-order shuffle for multiple-choice. Report the fraction of verdicts that survive. A big drop means your accuracy is partly an artifact.

7. Test run-to-run consistency. Run the same inputs several times (vary only the seed). Measure the flip rate. Contradictory answers to identical questions are a defect even when the average looks fine.

Trust

8. Check calibration. Does stated confidence match observed correctness? Compute Brier score and Expected Calibration Error. Flag overconfidence explicitly — high confidence plus lower accuracy is the failure that erodes user trust fastest.

For claims and results

9. Reproduce, don't trust. If you're relying on a benchmark result or a paper's number, re-derive it from the stated inputs. Record provenance: data version, code commit, seed, environment. A result you can't reproduce from its own description is unverified — and never tune your reproduction to force a match.

The verdict

10. Write one honest verdict with its caveats. Ship / Caution / Not-ready, each line carrying its number and uncertainty, plus a short "what this audit did not test" note. An honest Caution beats a flattering Ship every time.

Keep reading

• How to Tell If Your LLM Feature Is Actually Reliable (Before You Ship)
• Your LLM Benchmark Score Is Probably Lying to You
• Shuffle the answers and small models change their minds (ARC-Challenge audit)