THE STARTING POINT
Prepare an AI pilot evaluation dataset with representative tasks, difficult cases and review criteria. Separate development examples from the final evaluation.
Collect realistic inputs.
Select examples that reflect the task’s usual inputs and difficult cases. Include incomplete information and requests outside the intended scope. Check that you have permission to use the material and remove unnecessary personal details. A convenient set of clean examples can hide the conditions that matter in operation.
Describe acceptable responses.
Write down what a useful result must contain and what would make it unacceptable. Some tasks need a correct value; others need a supported explanation or a clear request for more information. Ask the person responsible for the workflow to review these expectations before evaluating outputs.
Record failures as learning material.
Keep the input, output and review decision together when an example fails. Look for patterns rather than relying on a single overall impression. Use the findings to adjust the task boundary or review process, and check those changes against the wider example set before expanding the pilot.
Build an evaluation set with traceable judgments
Describe the intended task and collect examples from the conditions in which it will operate. Use approved, appropriately protected material or carefully constructed fictional examples. Include common inputs, uncommon but important cases and requests outside the task boundary. Record why each example belongs in the set.
Write an expected result or a review rubric before testing. Have reviewers compare a small selection and resolve disagreements about correctness. Keep development examples separate from the evaluation used to make a continuation decision. When the prompt, model or source material changes, record the version and rerun relevant checks. Preserve individual failures instead of reporting only an overall score that can conceal a serious error.
An evaluation case and result log
A spreadsheet can be enough for an early pilot if permissions and versioning are appropriate. Use stable identifiers so a result can be traced to the original case.
- Case ID and category: common task, ambiguous input, exceptional case or out-of-scope request.
- Input and context: approved material, relevant source version and information available to the system.
- Expected behaviour: required facts, permitted uncertainty and outcomes that must be rejected.
- Review rubric: dimensions such as factual accuracy, completeness and suitability for the task.
- Observed output: system version, result, reviewer judgment and explanation of any disagreement.
- Disposition: correction, additional investigation, acceptance or a reason to stop the pilot.
Do not let the demonstration become the test
If the team repeatedly adjusts a system against the same examples, those examples no longer provide an independent view of performance. Keep a separate evaluation set and describe its limits.
Avoid forcing subjective outputs into a single unexplained pass/fail score. Review disagreement and significant failure categories separately. Do not treat an absent error in a small sample as proof that the error cannot occur. The final decision should reflect both the evidence and the consequences of getting a result wrong.
How many evaluation examples are enough?
There is no universal number for every task. Begin with coverage of meaningful input types and failure cases, then expand where results are uncertain. Consider the variety of work and consequences of errors rather than choosing a sample size solely because it is convenient.
ILLUSTRATIVE PLANNING EXAMPLE
Example: evaluating document summaries
Choose documents representing the lengths, structures and language the team expects to handle. Include a document with conflicting dates and one with an important exception. Write down the facts a useful summary must preserve and the errors that make it unacceptable. Remove confidential details before sharing examples outside the approved environment.
Keep a separate evaluation set and record reviewer judgments consistently. A few impressive summaries are not enough to establish suitability for the full workflow.
TAKE THE NEXT STEP
Your discovery checklist
- Select usual, difficult and out-of-scope examples.
- Write acceptable-output criteria with the workflow owner.
- Record failed examples and review their patterns.
FURTHER READING
Official resources to explore
Explore the related official guidance below. The templates and examples are practical starting points to adapt to your own project.