Business Rule Application — Documentation & Override Logging

Many high-risk AI systems apply business rules after model inference: minimum score thresholds, hard rejection criteria, fairness calibrations, and score adjustments. Each rule modifies the model’s raw output and changes the outcome that the affected person experiences. The post-processing layer is where these rules are applied, and each one must be transparently documented.

The Technical SME documents every post-processing rule in the AISDP with three elements: the rule itself (what it does), the rationale (why it exists), and the fairness impact assessment (how the rule affects different subgroups). A rule that automatically rejects applicants without a university degree, for instance, may disproportionately affect certain demographic groups; the Technical SME acknowledges and assesses this impact rather than treating the rule as a neutral business requirement.

Override logging at this layer is essential. Every instance where a business rule or fairness calibration changes the model’s raw output is logged with the original output, the modified output, and the specific rule that triggered the modification. This log enables retrospective analysis: if the fairness profile shifts in production, the organisation can determine whether the shift originates in the model’s predictions or in the post-processing rules.

Key outputs

• Documented catalogue of all post-processing business rules
• Per-rule rationale and fairness impact assessment
• Override logging capturing original output, modified output, and triggering rule
• Module 3 and Module 6 AISDP entries

Threshold Stability Monitoring

Where the post-processing layer applies decision thresholds (for example, “shortlist candidates with scores above 65”), the proportion of inputs crossing each threshold must be monitored over time. Changes in the crossing rate may indicate that the model’s score distribution has shifted, even if the threshold itself has not been altered.

A stable threshold applied to a shifting score distribution produces shifting outcomes. If the model begins producing higher scores on average, a fixed threshold will admit more candidates, potentially changing the system’s effective behaviour without any configuration change. Conversely, a downward score drift may cause the system to reject more candidates than intended. Both scenarios represent outcome drift.

Threshold stability monitoring tracks the crossing rate, identifies statistically significant changes, and triggers alerts when the rate deviates beyond configured bounds. The alert feeds into the post-market monitoring framework and may prompt a threshold recalibration exercise. The monitoring results are documented in AISDP Module 12 as part of the continuous surveillance record.

Key outputs

• Threshold crossing rate monitoring per decision threshold
• Statistical significance testing and alerting configuration
• Integration with post-market monitoring dashboards
• Module 12 evidence of threshold stability surveillance

Fairness Re-Evaluation on Production Data

Fairness metrics computed during development reflect the system’s behaviour on the validation and test datasets. Production data may differ from those datasets in distribution, composition, and temporal characteristics. The fairness profile observed during development is therefore not guaranteed to hold in production.

The post-processing layer periodically recomputes the fairness metrics that were evaluated during development, using production data that has passed through the complete pipeline. This re-evaluation catches drift that affects the final outputs, not just the model’s raw predictions. A subgroup that received equitable treatment during testing may experience disparate outcomes in production if the input population has shifted or if business rules interact differently with production data patterns.

Fairlearn’s ThresholdOptimizer can be applied at this layer to find subgroup-specific thresholds that satisfy a fairness constraint while maximising accuracy. The optimised thresholds should be re-validated periodically, because distributional shifts in the model’s raw outputs can render previously optimal thresholds sub-optimal. The re-evaluation results feed into AISDP Module 5 (as updated validation evidence) and Module 12 (as post-market monitoring records).

Key outputs

• Periodic fairness re-evaluation schedule and methodology
• Production fairness metrics compared against development baselines
• Threshold re-optimisation records where applicable
• Module 5 and Module 12 AISDP evidence

Outcome Drift Control — Periodic Threshold Recalibration

Threshold recalibration is the corrective action triggered when threshold stability monitoring or fairness re-evaluation identifies that the system’s post-processing thresholds are no longer producing the intended outcomes. Recalibration adjusts the thresholds to restore alignment between the system’s actual behaviour and its documented intent.

The recalibration process follows a structured workflow. The Technical SME analyses the drift signal to determine whether the root cause lies in the model’s raw output distribution, in the production input distribution, or in both. The appropriate corrective action depends on the root cause: if the model’s outputs have shifted, retraining may be required; if the input distribution has shifted, threshold adjustment may suffice.

Any threshold change is a material modification to the system’s behaviour and must pass through the governance framework. The change is documented in the AISDP with the analysis that prompted it, the old and new threshold values, the expected impact on fairness and accuracy metrics, and the approval record. The updated thresholds are validated on recent production data before deployment, and post-deployment monitoring confirms that the recalibration achieved its intended effect.

Key outputs

• Root cause analysis for threshold drift
• Updated threshold values with supporting evidence
• Governance approval for the threshold change
• Post-deployment validation of recalibration effectiveness
• Module 6 and Module 12 AISDP entries