AI Aimaiaim.org

How can a team distinguish AI value from ordinary workflow improvement?

A team should treat AI value as an evidence question, not a label. It should separate three matters: whether the workflow improved, what contribution the intended AI capability could make, and whether the observed gain can reasonably be separated from other process changes. Improvement after introducing AI does not, by itself, establish that AI created the improvement.

Start with the user’s problem

The GOV.UK Service Manual advises discovery teams to “focus on the user’s problem rather than possible solutions.” A use-case description should therefore establish:

  • what the user is trying to accomplish;
  • where the current process fails or creates unnecessary effort;
  • what outcome would be better; and
  • which part of that problem an AI capability is intended to address.

This order prevents the proposed technology from defining the use case. If the description begins with a desired model or tool but does not explain the user problem, the team cannot yet tell what meaningful improvement would look like.

Document the intended AI contribution

NIST’s AI Risk Management Framework calls for potential benefits of intended AI-system functionality and performance to be examined and documented. That makes the expected contribution a testable hypothesis rather than an assumed result.

A useful use-case record connects four elements:

Element Question
User problem What needs to improve?
Intended AI functionality What is the AI capability expected to contribute?
Potential benefit What user or process outcome could that capability improve?
Remaining workflow What steps, rules, roles, or constraints will still exist?

The team should also consider whether the expected benefit could be produced without AI—for example, by changing the sequence of work, introducing a rule, removing an unnecessary step, or redefining a role. That does not prove AI is unnecessary. It identifies the non-AI alternative against which the proposed use case should be compared.

Establish a baseline before evaluating the change

The GOV.UK Service Manual states that a team can know whether it has improved things only if it knows where it started. A baseline should capture the current user outcome and the conditions under which the work occurs.

Depending on the use case, the baseline may include the outcome being examined, the current workflow, the effort or delay users encounter, and relevant quality conditions. The supplied sources do not prescribe a universal metric or threshold; those choices must be confirmed from the team’s own context.

Without a credible baseline, a later result is difficult to interpret. It may reflect an AI contribution, a workflow redesign, changing operating conditions, or some combination of them.

Separate observed change from attribution

The evidence should be read in sequence:

  • Better outcome after a workflow change: supports the conclusion that the workflow improved, but not yet that AI caused the gain.
  • Documented AI benefit: establishes what the intended AI capability is expected to contribute; it remains a potential benefit until tested.
  • Comparison with a baseline: shows how the observed outcome differs from the starting point.
  • Record of other process changes: reveals alternative explanations for that difference.
  • Plausible non-AI alternative: helps the team judge whether the proposed AI capability offers a distinct contribution.

Where practical, staging the workflow redesign and the AI-enabled process can make attribution easier. If several changes occur together, the team should describe the result as an outcome associated with the combined intervention rather than claim that AI alone produced it.

For internal decision-making, the team can use three evidence levels:

  1. Workflow improvement: the user outcome improved after a process change.
  2. Potential AI value: the intended AI functionality and its expected benefit have been documented but not established in use.
  3. Supported AI-attributable value: the evidence connects the intended AI contribution to the observed improvement while considering baseline conditions and plausible non-AI explanations.

These are decision categories, not guaranteed outcomes.

What the team must still confirm

Before selecting a model or provider, the team must confirm the baseline, the measures that represent the user problem, and whether the AI contribution can be distinguished from other changes. It must also examine data, integration, operational, legal, privacy, security, cost, and control requirements relevant to its own context.

The cited guidance does not settle those project-specific questions or provide a universal rule for declaring AI value. That responsibility remains with the team and its designated reviewers.

Sources