Agree what you are comparing
Choose one completed outcome and the team responsible for it. For a case briefing, measure the effort required to produce a briefing that the receiving colleague can use. Do not compare manual completion with an AI draft that still needs checking.
Use similar task types and include difficult examples. Record differences that could explain the result: document length, missing information, reviewer experience or a changed process. A before-and-after average without that context can mislead.
- Baseline effort
- Assisted effort
- Quality and exceptions
- Decision with evidence
Track a small, useful set of measures
| Measure | Why it matters |
|---|---|
| Accepted output quality | Speed has little value if the output cannot be used. |
| Total staff minutes per item | Includes preparation, inspection, correction and exception handling. |
| Unresolved or abandoned items | Prevents failed attempts from disappearing from the result. |
| AI consumption per completed item | Connects measured usage to useful work. |
| Elapsed turnaround | Includes queue and waiting time; differs from hands-on effort. |
NIST’s work on AI evaluation warns that performance estimates depend on what is being measured and the assumptions behind the evaluation. State those assumptions beside your result rather than presenting a small pilot as a universal benchmark. [1]
Work through a transparent result
Illustrative example. For 200 comparable tasks, baseline effort is 10 minutes each: 2,000 minutes. Assisted handling averages 6 minutes including corrections: 1,200 minutes. Another 120 minutes is spent investigating failures and maintaining the configuration. The net difference is 680 minutes, or about 11.3 hours.
That is a capacity estimate for this sample, assuming quality meets the agreed standard. It is not an annual saving, a cash benefit or a measured REDA customer result. If the sample excluded difficult tasks, repeat the exercise before extrapolating.
Explain where value would actually appear
The team might use released time to reduce a backlog, respond sooner or absorb more work. A cash-saving case needs a credible connection to a cost that will actually change, such as avoidable overtime. Record that mechanism separately from the time calculation.
Include implementation, subscription, usage and operating costs when making the investment decision. Avoid converting the same benefit into both salary savings and additional capacity in one total.
Decide whether to continue, revise or stop
A pilot can be useful even when it identifies a poor candidate. The data may be inaccessible, review may remain expensive or the task may be better handled by ordinary automation. Those findings prevent a larger mistaken investment.
REDA’s sandbox approach is designed to support a review before production planning. Bring the baseline and the business outcome you care about; the business-case guide provides a fuller worksheet for the decision.
Sources
- NIST — Expanding the AI evaluation toolbox with statistical models. Accessed 22 September 2026.