Review three things when an upstream data source changes: the affected AI system components’ functionality and behavior in production, the monitoring applied to those components, and the context used to interpret their outputs. NIST AI RMF states that component functionality and behavior are monitored in production and that AI output is interpreted within its context. The cited material does not define a source-change trigger, review interval, or decision threshold.
What needs review?
Production functionality and behavior
The first target is any component identified through the system map that could be affected by the source change. The review should ask:
- Which components depend on the changed source?
- Could their expected functionality or behavior change in production?
- What evidence would show such a change?
These are practical review questions, not a source-change checklist prescribed by the cited NIST material.
Production monitoring
Monitoring should be checked for every affected component. The relevant question is whether the existing production monitoring would expose a change in functionality or behavior.
NIST AI RMF supports monitoring component functionality and behavior in production, but the cited section does not specify monitoring tools, metrics, comparison periods, or alert thresholds.
Output interpretation
The review should also examine whether outputs are still being interpreted within the correct mapped context. A change in the upstream source could affect the meaning or relevance of that context even when no failure is immediately visible.
NIST AI RMF says AI system output is interpreted within its context. It does not prescribe a particular interpretation method or define which contextual elements must be documented.
How to check it
A team can apply the guidance through the following sequence:
- Map the impact: Trace the changed source characteristics to the components and outputs they may affect.
- Separate expected from observed behavior: Record what each affected component is expected to do in production, then compare that expectation with available monitoring evidence.
- Recheck the output context: Determine whether the context used to interpret the output remains valid after the source change.
- Document uncertainty: Distinguish observed changes from possible effects that monitoring has not established.
The trace, evidence record, and uncertainty log are implementation practices. They are not requirements stated in the cited NIST section.
What must still be confirmed
The cited guidance does not answer several decisions a team may still need to make:
- Whether a particular source change is significant enough to require review
- Which components, outputs, or contextual assumptions are affected
- Whether existing production monitoring is sufficient
- What review cadence, threshold, owner, or approval process applies
- Whether any contractual, legal, regulatory, or internal policy imposes additional requirements
Those points must be confirmed against the team’s authoritative documentation. The cited NIST material alone is not enough to declare a source change material or immaterial, or to establish a mandatory review cadence.