Model Cards (Per Build)

Model cards are auto-generated from the evaluation metrics stored in the experiment tracker and model registry as part of the CI pipeline’s documentation generation stage. When a new model version is registered, the pipeline generates a model card containing the model’s architecture summary, training data version, evaluation metrics disaggregated by subgroup, intended use statement, and known limitations.

The model card template is version-controlled and maintained by the Conformity Assessment Coordinator. Template changes require review to ensure continued alignment with AISDP module requirements and Annex IV expectations. Each generated model card references the template version used in its creation.

Google’s Model Cards Toolkit provides an established framework for this generation. The data that populates the card is drawn from the pipeline’s own outputs (evaluation metrics, registry metadata, configuration values), ensuring that the model card always reflects the actual model. Auto-generation eliminates the risk of model cards being written from memory weeks after training, which introduces inaccuracies. The generated model card is stored as a pipeline artefact with a ten-year retention policy.

Key outputs

• Auto-generated model card per model version
• Version-controlled template with governance review for changes
• Pipeline artefact storage with ten-year retention
• Module 5 and Module 3 AISDP evidence

Test Reports (Per Build)

Test reports covering all unit, integration, regression, fairness, and robustness tests are auto-generated as part of each CI pipeline execution. The report aggregates the results from every test category and the model validation gates into a single, navigable document.

The report should present results at two levels: a summary view showing pass/fail status per test category (suitable for the AI Governance Lead’s review), and a detailed view showing individual test results, failure messages, and metric values (suitable for the Technical SME’s investigation). The report carries a timestamp, the pipeline execution ID, the composite version identifier, and the data version used for evaluation.

The currency mechanism ensures that each auto-generated report carries a source reference linking it to the specific pipeline run that produced it. A governance dashboard should display, for each AISDP module, the date of the last auto-generated update and the date of the last human review. Modules where the auto-generated content is newer than the last human review are flagged for attention.

Key outputs

• Auto-generated test report per pipeline execution
• Summary and detail views for different audiences
• Timestamp, pipeline execution ID, and composite version linkage
• Module 5 AISDP evidence

SBOMs (CycloneDX/SPDX, ML-Specific Components)

Software Bills of Materials (SBOMs) list all third-party dependencies with versions and licence information. For AI systems, the SBOM must extend beyond traditional software dependencies to include ML-specific components: model framework versions, pre-trained model provenance, dataset processing library versions, and any third-party model APIs or embedding services.

CycloneDX and SPDX are the two standard formats for SBOMs. The SBOM is auto-generated as part of the CI pipeline using tools such as Syft, CycloneDX CLI, or SPDX tools. The generated SBOM provides the foundation for dependency scanning and licence compliance scanning, ensuring that the vulnerability and licence assessments operate on an accurate and complete inventory.

The SBOM is retained as both Module 3 evidence (documenting the system’s technical composition) and Module 9 evidence (supporting the cybersecurity assessment). Each deployment should reference the SBOM version that corresponds to the deployed container image. Over the system’s lifecycle, the sequence of SBOMs provides a history of the system’s dependency evolution, useful for investigating supply chain incidents and for demonstrating proactive dependency management.

Key outputs

• Auto-generated SBOM per pipeline build (CycloneDX or SPDX format)
• ML-specific component inclusion (frameworks, pre-trained models, APIs)
• SBOM linkage to the deployed container image version
• Module 3 and Module 9 AISDP evidence

AISDP Section Updates

The CI pipeline should generate draft updates to the AISDP modules most affected by a model or system change. Using Jinja2 templates or equivalent templating engines, the pipeline populates draft AISDP module sections from pipeline metadata. A Module 5 draft, for instance, draws from the model registry for model architecture and training configuration, the experiment tracker for evaluation metrics, the fairness evaluation for subgroup metrics, and the deployment ledger for deployment date and version.

The draft is not the final AISDP; it is a starting point that the AI Governance Lead reviews, augments with narrative context, and approves. This review-and-augment workflow reduces the human effort from writing documentation from scratch to verifying and enriching an already-accurate draft. The generated drafts carry the pipeline execution ID and timestamp, linking them to the specific build that produced them.

Documentation quality assurance should verify four properties: completeness (no missing sections or empty fields), accuracy (metrics match the raw evaluation data), consistency (terminology aligns with AISDP style conventions), and currency (the generation timestamp matches the artefact it describes). A quarterly manual review of auto-generated documentation against the source data catches systematic generation errors.

Key outputs

• Auto-generated AISDP module drafts per pipeline execution
• Version-controlled Jinja2 templates maintained by the Conformity Assessment Coordinator
• AI Governance Lead review and approval workflow
• Module 10 documentation