Do not include every enquiry at the start
If the team cannot say how it will judge the result, the scope is probably too broad. Choose one type of question, such as document requirements for a lane. Record who answers it, what they look up and where errors occur.
Prepare examples you can compare
Include common, unclear, incomplete and out-of-scope questions. Use the same set to compare current work with the pilot rather than repeatedly choosing the easiest questions.
- Keep the question, existing reply and reference material.
- Ask a business reviewer to mark errors, omissions and unnecessary follow-ups.
- Check whether a colleague receives enough information when taking over.
Count the maintenance work too
Who updates the information? When a rate or policy changes, how quickly can the old answer stop being used? Record this work during the pilot; a faster reply may not justify difficult maintenance.
Stopping is a valid result
Discuss expansion when results are repeatable, colleagues would use the tool and preparation and maintenance effort are acceptable. The next step might be one read-only lookup or better information, rather than a replacement system.
