An automation running successfully is a technical result. A customer receiving the right answer sooner, or a team spending less time correcting records, is a business result. Measure both, so a green run history does not hide an awkward workflow.
Choose the outcome before the build
Pick a small set of measures linked to the reason for the change. For a quoting process, these might be time from complete enquiry to sent quote, active handling time and the percentage returned for missing information. Give each measure a precise definition.
Record the starting point using a representative sample. Include the busy days and ordinary exceptions. If you cannot establish a reliable baseline, describe that limitation instead of manufacturing a precise before-and-after claim.
Compare similar work
Check volumes and case complexity. A quiet week after launch is not directly comparable with the busiest week before it. Use rates or time per item where appropriate, and keep the sample size visible.
In a hypothetical pilot, 40 straightforward requests might complete faster while ten unusual requests take longer. Report both groups. An average alone could conceal the fact that the people handling exceptions now have more work.
Include the effort around the automation
Count review, correcting outputs, maintaining rules and handling failed integrations. Ask whether a removed spreadsheet has become a set of private messages. The whole workflow is the unit of improvement.
Track technical reliability separately: failed runs, repeated processing, recovery time and unresolved items. Define what happens when an alert is missed and who checks the backlog. A system that fails clearly can be easier to operate than one that appears successful but leaves missing records.
Ask the people using it
Combine measurements with short conversations. Can they find current status? Do they trust the result? What happens when something unusual arrives? Those answers can explain why the numbers changed and reveal costs that a timer misses.
GOV.UK's service-performance guidance describes selecting measures that show whether a service meets its goals. For a business workflow, use the same principle: choose evidence about the intended result rather than counting features shipped.
Make the next decision explicit
Set a review date and decide whether to keep, adjust, expand or stop the change. Keep the baseline and assumptions, so later comparisons remain understandable. Report observations as observations and estimates as estimates.
Use a realistic admin-cost model if you need a financial estimate. If the result is disappointing, return to the actual work before adding more automation.
