Security
Threat modelling, OWASP LLM Top 10, cybersecurity foundations, DevSecOps, incident response, and cross-regulatory mapping.
103 articles in this section
1.
Threat Modelling (S.8.3, S.8.4)
Threat modelling for AI systems combines traditional software security analysis with AI-specific threat taxonomies.…
2.
Methodology
STRIDE (Traditional Software Threats — Six Categories) STRIDE is a threat classification framework that categorises…
3.
Attack Surface Identification
Attack Surfaces (Eight Categories) The first stage of threat modelling(
4.
Threat Actor Profiles
Threat Actors The threat model(
5.
AI-Specific Threat Categories
This section covers the following topics: OWASP LLM01: Prompt Injection(
6.
OWASP LLM01: Prompt Injection
Prompt Injection — Attack Vectors (Direct, Indirect, Multi-Turn) Prompt injection is the most widely discussed threat…
7.
OWASP LLM02: Sensitive Info Disclosure
Information Disclosure — Attack Vectors (Memorisation, Membership Inference, Property Inference) The model may leak…
8.
OWASP LLM03: Supply Chain
Supply Chain — Attack Vectors (Dependencies, Pre-Trained Models, Third-Party Services) The AI system's supply chain has…
9.
OWASP LLM04: Data and Model Poisoning
Data Poisoning — Attack Vectors (Targeted, Untargeted, Label Flipping, Backdoor) An attacker who can manipulate the…
10.
OWASP LLM05: Improper Output Handling
Insecure Output — Attack Vectors (XSS, SQL Injection, Command Injection via Output) Model outputs that are passed to…
11.
Plugin Security (cf. LLM06 Agency)
Plugin Security — Attack Vectors & Controls (Allowlists, Validation, Human Approval, Logging) For systems where the AI…
12.
OWASP LLM06: Excessive Agency
Excessive Agency — Attack Vectors & Controls (Least Privilege, Permission Inventory, Access Reviews) When the AI system…
13.
OWASP LLM09: Misinformation
Note: This topic is covered within the parent article. See the full AI-Specific Threat…
14.
OWASP LLM10: Unbounded Consumption
Model DoS — Attack Vectors & Controls (Rate Limiting, Timeouts, Cost Caps) An attacker submits inputs designed to…
15.
Model Theft
Model Theft — Attack Vectors (Extraction via Querying, Infrastructure Compromise, Artefact Exfiltration) Model theft…
16.
Beyond OWASP Top 10
Adversarial Examples — Attack Vectors & Controls (Adversarial Training, Input Validation, Ensemble Methods) Adversarial…
17.
Threat Modelling Artefacts
Threat Model Document (Living) The threat model(
18.
Cybersecurity Foundations (S.8.2)
Cybersecurity foundations establish the baseline security posture for the AI system's infrastructure and operations.…
19.
Network Security
Dedicated VPC with Segmentation The AI system's infrastructure should be deployed within a dedicated Virtual Private…
20.
Zero Trust Architecture
Independent Service Authentication & Authorisation A zero trust architecture assumes no implicit trust based on network…
21.
Authentication & Access Control
MFA, RBAC & Service-to-Service mTLS Multi-factor authentication (MFA) should be mandatory for all operator accounts and…
22.
Encryption
Data at Rest (AES-256) & Data in Transit (TLS 1.3) The engineering team encrypts all data at rest using AES-256 or…
23.
Vulnerability Management
Continuous Scanning & Remediation SLAs Continuous vulnerability scanning(
24.
Patch Management
Documented Schedule, Zero-Day Process & Staging Testing Operating system, framework, and dependency patches are applied…
25.
API Security (S.8.2.2)
Authentication — API Keys & Per-Consumer Identity(
26.
Authentication — API Keys & Per-Consumer Identity
Every inference endpoint should require authentication, even for internal consumers. API keys or OAuth tokens identify…
27.
Rate Limiting
Rate limiting on inference endpoints prevents denial of service and model…
28.
Input Validation & Sanitisation
Model endpoints validate inputs against a strict schema before they reach the model. Input dimensions, data types,…
29.
Output Filtering
Model outputs pass through a filtering layer before reaching the consumer. For classification models, confidence scores…
30.
Inference Timeout Enforcement
Inference timeout enforcement sets a maximum execution time per request, terminating any request that exceeds it. This…
31.
API Versioning & Deprecation
When a model is updated, the API version should change to prevent consumers from unknowingly receiving outputs from a…
32.
API Logging & Audit
Every inference request and response is logged with sufficient detail for forensic analysis. The log record includes…
33.
Data Security in ML Pipelines (S.8.2.3)
Training Data Security(
34.
Training Data Security
Training datasets often contain the most sensitive data in the ML pipeline. Access is restricted by the security team…
35.
Feature Store Security
Feature stores aggregate and serve pre-computed features for model training and inference. They can become single…
36.
Model Artefact Security
Trained model files are valuable intellectual property and a potential attack vector. Model artefacts are stored in…
37.
Inference Log Security
Inference logs contain the system's production inputs and outputs, which may include personal data, commercially…
38.
Vector Database Security — Write/Read Separation
Systems using retrieval-augmented generation, semantic search, or embedding-based matching store dense vector…
39.
Vector Database Security — Adversarial Document Injection
Adversarial document injection is a novel attack surface introduced by vector databases. An attacker who can insert…
40.
Vector Database Security — Bulk Extraction Monitoring
An attacker with query access to the vector database could systematically probe the embedding space to reconstruct or…
41.
DevSecOps Integration (S.8.3)
SAST (Bandit, SonarQube, Semgrep)(
42.
SAST (Bandit, SonarQube, Semgrep)
Static Application Security Testing (SAST) scans source code for common vulnerability patterns, including injection…
43.
DAST
Dynamic Application Security Testing (DAST) tests running application instances for vulnerabilities by sending requests…
44.
SCA/Dependency & Container Image Scanning
Software Composition Analysis (SCA) scans the project's dependency tree against known vulnerability databases (CVE,…
45.
IaC Security Scanning
Infrastructure-as-code security scanning validates that infrastructure definitions follow security best practices…
46.
AI-Specific Rules
Standard security scanning tools do not detect AI-specific security risks. AI-specific rules augment the SAST, DAST,…
47.
Manual Security Code Review
Beyond automated scanning, the Technical SME conducts manual security code review for security-critical components.…
48.
SBOM Generation — CycloneDX/SPDX with ML Components
SBOM(
49.
Cybersecurity Testing Programme (S.8.5)
The cybersecurity testing programme validates the security controls documented throughout the…
50.
Penetration Testing
Annual Independent Penetration Testing Penetration testing is conducted annually by an independent firm with expertise…
51.
Vulnerability Scanning
Continuous Automated Scanning — Four Layers The vulnerability scanning programme operates across four layers, each…
52.
Adversarial ML Testing
Evasion/Adversarial Examples — White-Box &…
53.
Evasion/Adversarial Examples — White-Box & Black-Box
Adversarial example testing evaluates the model's susceptibility to input perturbations designed to cause incorrect…
54.
Adversarial Testing by Modality
Adversarial testing methodologies vary by the model's input modality. For tabular models, the testing protocol perturbs…
55.
Data Poisoning Simulation
Data poisoning simulation tests the model's resilience to corrupted training data. ART's poisoning modules provide…
56.
Prompt Injection Testing (LLM Systems)
For systems incorporating LLMs, prompt injection(
57.
Model Extraction Testing
Model extraction testing evaluates whether an attacker can reconstruct the model's decision boundaries through…
58.
Membership Inference Testing
Membership inference testing evaluates whether an attacker can determine if a specific individual's data was included…
59.
Adversarial ML Testing Frequency
The Technical SME conducts the full adversarial ML testing(
60.
Additional Threat-Specific Testing
Output Validation Testing For systems where model outputs are consumed by downstream components (web interfaces,…
61.
Red Team Exercises
Annual Red Team — Scenarios Annual red team exercises simulate realistic threat scenarios combining technical attacks…
62.
Test Result Mapping
Summary Table per Test Type All cybersecurity testing(
63.
Incident Response (S.8.6)
Incident response for high-risk AI systems must account for multiple overlapping reporting obligations under the AI…
64.
Integrated Incident Response Plan
Detection & Triage — Cross-Regime Assessment(
65.
Detection & Triage — Cross-Regime Assessment
At the triage stage of any cybersecurity incident, a multi-regime decision tree is activated. Four questions are…
66.
Fundamental Rights Dimension Assessment
Every incident triage must separately assess the fundamental rights dimension, regardless of whether the incident is…
67.
Parallel Reporting Streams — DORA
For financial entities subject to DORA, major ICT-related incidents are reported to the competent financial authority…
68.
Parallel Reporting Streams — NIS2
For entities subject to NIS2, significant incidents are reported to the national CSIRT or competent authority. The…
69.
Parallel Reporting Streams — CRA
For products with digital elements within the CRA's scope, actively exploited vulnerabilities are reported to ENISA…
70.
Parallel Reporting Streams — AI Act Art. 73
Serious incidents meeting the Article 3(49) definition are reported to the market…
71.
Pre-Drafted Reporting Templates
Pre-drafted dual-regime and multi-regime reporting templates are maintained by the incident…
72.
Art. 73(9) Simplification for NIS2/DORA Entities
Article 73(9) provides a simplification for sectors where equivalent reporting obligations exist. Entities subject to…
73.
Evidence Preservation — No System Alteration Prior to Notification
Article 73(6) explicitly prohibits altering the AI system in a way that could affect subsequent evaluation of the…
74.
Regulator Contact Register
Per-Jurisdiction Authority Contacts A regulator contact register is maintained by the AI Governance…
75.
Supply Chain Security (S.8.7)
Supply chain security addresses the risks introduced by third-party components: open-source libraries, pre-trained…
76.
SBOM
SBOM — Standard & ML-Specific Components The AI system's supply chain extends well beyond traditional software…
77.
Dependency Management
Version Pinning & Private Repositories Every dependency is pinned to an exact version in a lock file: Poetry lock, pip…
78.
Third-Party Model Provider Assessment
AI Act Model Origin Risk Assessment Every third-party model component undergoes a vendor risk…
79.
DORA Third-Party Register
All AI-Related Service Providers Every AI-related service provider should appear in the third-party register. The…
80.
Cross-Regulatory Mapping (S.8.1)
This section covers the following topics: Applicable Regimes(
81.
Applicable Regimes
Regime Determination The first step in the cross-regulatory mapping(
82.
Consolidated Mapping
Mapping Table The consolidated mapping table maps seven cybersecurity domains across all applicable regimes,…
83.
CRA Deemed Compliance Pathway
CRA Scope & Product Classification The CRA scope determination addresses whether the AI system qualifies as a product…
84.
NIS2 Interaction
NIS2 Scope, Dual Reporting & Simplification NIS2 applies to essential and important entities across sectors including…
85.
DORA Interaction
DORA ICT Risk Extension DORA's requirements overlap substantially with the AI Act's cybersecurity requirements,…
86.
Emerging Interactions
EU Data Act & EHDS Overlays Two additional regulatory instruments have cybersecurity relevance for specific system…
87.
Security Artefacts
Threat Model (Living Document)(
88.
Threat Model (Living Document)
The threat model(
89.
Cross-Regulatory Mapping Tables
The cross-regulatory mapping(
90.
CRA Scope Determination & Product Classification
The CRA scope determination and product classification is retained as a standalone Module 9 artefact. The artefact…
91.
DORA Third-Party Register & Risk Assessments
The DORA third-party register is retained as a standalone Module 9 artefact for financial entities. Each entry contains…
92.
Adversarial ML Test Results
The adversarial ML(
93.
Penetration Test Reports
The penetration test(
94.
Vulnerability Management Register
The vulnerability management(
95.
SBOM (Per Build)
The SBOM(
96.
Red Team Exercise Reports
The red team report archive contains the full reports from each annual red team…
97.
Incident Response Plan with Decision Tree
The incident response plan(
98.
Pre-Drafted Reporting Templates
The pre-drafted reporting templates are retained as standalone Module 9 artefacts. The template set includes the shared…
99.
Regulator Contact Register
The regulator contact register is retained as a standalone artefact. For each member state where the system is…
100.
Security Code Review Records
The security code review records archive contains the documentation from each manual security code review. Each record…
101.
Supply Chain Risk Assessments
The supply chain risk assessment(
102.
Module 9 Test Summary Table
The Module 9 test summary table is retained as the primary navigation artefact for Module 9's testing evidence. It maps…
103.
Threat Modelling — Attack Surfaces & Actors
This section covers the following topics: Attack Surface Identification(