AI equipment manual retrieval
Answer document questions with the correct equipment and revision context, including an explicit not-found response when the supplied manual does not contain the answer.
Synthetic evaluation input
Synthetic manual MAN-22, revision D, section 2: inspection records are stored in register REG-8. No service interval is provided in the supplied excerpt. Questions: Where are inspection records stored? What service interval is stated?
Original Inzonex training example, version 1.0. No customer document or real equipment data. No model has been scored on this page.
Expected fields
| Field | Reference answer |
|---|---|
| manual_id | "MAN-22" |
| revision | "D" |
| evidence_section | "2" |
| record_register | "REG-8" |
| service_interval | null |
Check a structured response
The comparison runs in your browser against this answer key. It checks exact fields and values, not the accuracy of an explanation or a model's general ability.
Recommended workflow
Retrieve bounded evidence first and require a source identifier for every answer. Test unanswered questions as deliberately as answerable ones.
A manual assistant needs to know which document applies before it answers. Similar equipment names and old revisions can produce plausible but inapplicable responses. Include the manual identifier and revision in the question context and keep the evidence available to the reviewer.
The packet supplies one administrative fact and omits a requested service interval. The expected response includes the stated document-storage location and an explicit null for the interval. Guessing a typical interval is a failure, even if it happens to be common elsewhere.
This is a retrieval and abstention exercise, not an instruction to service equipment. A real system also needs document access controls, revision retirement and tests that unrelated or malicious document text cannot redirect the assistant's task.
Failure checks
- Use MAN-22 revision D, not a similar manual.
- Return the evidence section with the answer.
- Abstain on the unstated service interval.
What this exercise does not prove
Passing this sample does not validate retrieval across a full manual library or permit maintenance decisions without the applicable approved documentation.
For an actual evaluation, keep a separate held-out set, record tool/model version and settings, and log raw outputs, corrections, elapsed time and actual charges. Do not compare tools tested on different inputs as though they ran the same benchmark.
What is checked in AI equipment manual retrieval?
Answer document questions with the correct equipment and revision context, including an explicit not-found response when the supplied manual does not contain the answer.
What does AI equipment manual retrieval not prove?
Passing this sample does not validate retrieval across a full manual library or permit maintenance decisions without the applicable approved documentation.
Related tasks
Match this workflow to your data requirements
Methodology and reuse
These packets and answer keys are original Inzonex educational material, licensed CC BY 4.0. Attribute Inzonex and link to this task page when reusing the packet. The licence does not cover third-party material linked from this site.
NIST AI 600-1: background on generative AI evaluation and risk. This exercise is not NIST-certified.