v2.4.0 | Report Errata
docs operations docs operations

Shared Monitoring Infrastructure & Cross-System Analysis Monitoring infrastructure, evidence repositories, document management systems, and CI/CD pipelines are designed as shared services supporting multiple AI systems. The marginal cost of adding a new system to the monitoring infrastructure should be low. Shared infrastructure (Prometheus/Grafana with multi-tenant configuration, or Datadog with per-system tags) enables cross-system analysis: detecting patterns (a common vulnerability across systems using the same GPAI model) that individual system monitoring would miss. Each system’s metrics are labelled with the system identifier, enabling aggregate views (how many systems have open non-conformities?) and per-system drill-down (what is system X’s current fairness drift status?). Key outputs

  • Shared monitoring infrastructure as multi-system service
  • Low marginal cost per additional system
  • Cross-system pattern detection capability
  • Per-system isolation within shared infrastructure

Tiered Oversight (Risk-Based Assignment) Not all high-risk systems require the same oversight intensity. A credit scoring system affecting millions of consumers warrants more intensive oversight than an internal document classification system. The AI Governance Lead defines oversight tiers based on the system’s risk profile, deployment scale, and affected population sensitivity. Higher-tier systems receive more frequent reviews, dedicated oversight personnel, and more granular monitoring. Lower-tier systems receive scheduled reviews, shared oversight personnel, and standard monitoring configurations. Tier assignments are documented and reviewed annually. Key outputs

  • Oversight tiers based on risk, scale, and sensitivity
  • Resource allocation calibrated to tier
  • Annual tier assignment review
  • AI Governance Lead documentation

Centralised Governance, Distributed Execution The AI Governance Lead provides central coordination: maintaining the portfolio-level risk register, ensuring consistent standards, and reporting to executive leadership. Day-to-day oversight execution (monitoring, escalation handling, operator training) is distributed to the teams closest to each system. This model ensures governance standards are consistent across the portfolio while operational knowledge remains local. A centralised team that tries to handle daily oversight for twenty systems will lack the domain expertise needed for each; distributed teams without central coordination will drift into inconsistent practices. Key outputs

  • Central coordination by AI Governance Lead
  • Distributed execution by system-proximate teams
  • Consistent standards with local operational knowledge
  • Portfolio-level risk register maintained centrally

Portfolio-Level Compliance Dashboards Portfolio compliance dashboards aggregate the compliance posture across all systems into a single executive view. For each system, the dashboard shows conformity status (green/amber/red), number and severity of open non-conformities, PMM metric status, evidence currency status, and date of last formal assessment. Credo AI and Holistic AI provide built-in multi-system views. For organisations using the open-source stack, Grafana or Metabase dashboards aggregate per-system metrics into portfolio views. The dashboard enables the AI Governance Lead and executive leadership to allocate resources, set priorities, and identify systems approaching compliance risk. Key outputs

  • Per-system compliance status in single executive view
  • Five status indicators per system
  • Credo AI, Holistic AI, or Grafana/Metabase implementation
  • Decision support for resource allocation and prioritisation

Standardised Processes & Cross-System Learning A common AISDP template, evidence taxonomy, non-conformity workflow, and assessment checklist reduce per-system governance overhead. The governance team applies the same process to every system, learning from experience across the portfolio. A finding in one system (a monitoring gap, a documentation deficiency) is applied as a preventive check across all others. Standardised processes are documented as a portfolio-level governance procedure maintained by the AI Governance Lead. This documentation demonstrates to a competent authority that the organisation applies consistent compliance standards, not ad hoc approaches that vary by system. Key outputs

  • Common templates and workflows across portfolio
  • Cross-system preventive learning from individual findings
  • Portfolio-level governance procedure documentation
  • Consistent compliance standards demonstrated
On This Page