Compliance Artifacts: What You Need to Produce

Building trustworthy artificial intelligence within the European Union is fundamentally an exercise in evidence. The regulatory frameworks governing high-risk AI systems, medical devices, machinery, and data processing do not merely ask organisations to be responsible; they require organisations to prove their responsibility through a structured collection of documents, records, and procedures. These materials, collectively known as compliance artifacts, form the backbone of regulatory oversight and market surveillance. For engineers, product managers, and legal counsel working across AI, robotics, biotech, and data systems, understanding the precise nature of these artifacts is not a bureaucratic exercise—it is a prerequisite for lawful operation and market access. This article examines the practical set of compliance artifacts required for AI-enabled systems, detailing their content, interrelationships, and the procedural context in which they must be generated and maintained.

The Regulatory Context: From Principles to Paperwork

European regulation of AI and data-driven systems translates high-level principles—such as lawfulness, fairness, transparency, accountability, and robustness—into concrete operational requirements. The primary instruments in this domain are the Artificial Intelligence Act (AI Act), the General Data Protection Regulation (GDPR), the Medical Device Regulation (MDR) and In Vitro Diagnostic Regulation (IVDR), the Machinery Regulation, and the Cybersecurity Act. While the AI Act introduces a horizontal framework specifically for AI systems, other regulations impose vertical requirements on specific applications or functionalities. Crucially, these frameworks are not isolated; they overlap and interact. An AI-based medical imaging device, for example, must satisfy the conformity assessment procedures of the MDR, the specific risk management and data governance obligations of the AI Act, and the data protection principles of the GDPR.

Compliance artifacts are the tangible outputs of the processes mandated by these regulations. They are the evidence that a manufacturer or deployer has established, implemented, and maintained the required systems. In the event of an audit by a market surveillance authority, a notified body assessment, or a data protection authority inquiry, the absence or inadequacy of these artifacts can lead to significant penalties, product recalls, or prohibitions on processing. Therefore, treating documentation as a mere administrative burden is a strategic error. It is the operational record of due diligence.

The Interplay of EU Regulations and National Implementation

It is essential to distinguish between EU-level regulations and their national implementations. The AI Act, for instance, is a regulation directly applicable in all Member States without the need for national transposition. However, it requires the establishment of national competent authorities (NCAs) and a European AI Office to coordinate enforcement. Similarly, the GDPR allows Member States to legislate on specific areas, leading to national derogations and interpretations. For example, the legal basis for processing personal data for scientific research under Article 89 of the GDPR is implemented with nuances in national laws of countries like Germany (BDSG) or France (Loi Informatique et Libertés).

When producing compliance artifacts, an organisation must therefore consider both the harmonized EU requirements and the specific interpretations adopted by the national authority in the jurisdiction where the system is placed on the market or where processing occurs. This is particularly relevant for regulatory sandboxes and real-world testing environments, which are encouraged by the AI Act but operationalised at the national level. Documentation produced within a sandbox must align with the specific conditions set by the national authority, which may include enhanced transparency or data protection safeguards beyond the baseline.

Core Compliance Artifacts: A Taxonomy

The compliance artifacts required for AI-enabled systems can be grouped into several functional categories. While specific documents may carry different names depending on the sector (e.g., Technical File under MDR vs. Technical Documentation under AI Act), the underlying logic is consistent: establish the system’s design, assess its risks, verify its conformity, and monitor its performance.

1. Governance and Accountability Policies

Policies are the high-level strategic documents that define an organisation’s commitment to compliance. They are not technical specifications but statements of intent and procedure that guide operational activities.

AI Strategy and Governance Policy

This document outlines the organisation’s approach to AI development and deployment. It should define the scope of AI systems covered, the roles and responsibilities of key personnel (e.g., AI Officer, Data Protection Officer), and the principles guiding AI use (e.g., human oversight, non-discrimination). Under the AI Act, this policy should reference the organisation’s approach to human oversight and the measures taken to ensure, as far as possible, that the system is robust against manipulation or errors.

Data Governance Policy

Data is the lifeblood of AI. The AI Act explicitly requires that high-risk AI systems be trained, validated, and tested on data sets that are relevant, representative, free of errors, and complete. A Data Governance Policy documents how these requirements are met. It should cover:

• Data Collection: Lawful basis for data acquisition, including GDPR compliance (consent, legitimate interest, public task).
• Data Preparation: Procedures for labeling, cleaning, and augmenting data. This is crucial for addressing bias.
• Data Bias Mitigation: Methods for detecting and mitigating bias in training and testing data. This is a core requirement for high-risk systems.
• Data Security: Measures to protect data integrity and confidentiality.

For biotech and healthcare applications, this policy must also align with regulations on the secondary use of patient data for research, ensuring that data minimization and anonymization techniques are properly documented.

Quality Management System (QMS) Policy

High-risk AI systems, particularly those integrated into medical devices or machinery, require a QMS. The QMS policy documents the organisation’s commitment to quality and compliance with standards like ISO 9001 or ISO 13485. It outlines procedures for design controls, risk management, document control, and corrective actions. The AI Act reinforces the need for a QMS by requiring that manufacturers establish a system for the continuous collection, documentation, and analysis of post-market monitoring data.

2. Risk Management Files

Risk management is the cornerstone of compliance for high-risk systems. It is an iterative process, not a one-time event. The Risk Management File is the central artifact that records this process.

Risk Identification and Analysis

The process begins with identifying hazards associated with the AI system. For AI, this goes beyond traditional safety risks. It includes risks of bias, discrimination, lack of transparency, and loss of human oversight. The AI Act requires an analysis of risks to fundamental rights, which is a novel requirement. The Risk Management File must document:

• The identified hazards (e.g., algorithmic bias leading to discriminatory loan decisions).
• The estimated severity and probability of harm.
• The affected stakeholders (e.g., individuals, groups, society).

Risk Estimation and Evaluation

Once risks are identified, they must be estimated. This involves analyzing the interaction between the AI system and its environment, including potential reasonably foreseeable misuse. The file must document the criteria used for risk evaluation and the acceptable risk levels. Under the AI Act, if a risk is deemed unacceptable, the system cannot be placed on the market.

Risk Control Measures

For each identified risk, the file must specify control measures. These can be:

• Inherently Safe Design: Modifying the algorithm to reduce risk.
• Protective Measures: Implementing technical safeguards, such as confidence thresholds or human-in-the-loop mechanisms.
• Information for the User: Providing clear instructions and warnings about the system’s limitations and intended use.

The effectiveness of these measures must be evaluated and documented. This is a continuous loop: as the system is monitored post-market, new risks may emerge, requiring updates to the Risk Management File.

3. Technical Documentation (The “Technical File”)

Technical documentation is the evidence of conformity. It must be detailed enough to allow a notified body or authority to understand the system’s design, functioning, and compliance with essential requirements. The AI Act and MDR have similar but distinct requirements for technical documentation. A robust technical file for an AI-enabled system typically includes:

General Description of the System

This includes the intended purpose, the context of use, the user profile, and the hardware/software environment. It should also describe the principles underlying the AI system (e.g., machine learning, logic-based).

Development Process and Design Specifications

This section details the lifecycle of the AI system. It should document:

• The methodologies used for development (e.g., Agile, CRISP-DM).
• The data used for training, validation, and testing (see below).
• The design choices and architecture (e.g., neural network topology, feature selection).
• The metrics used to measure performance, accuracy, and robustness.

Data Specifications

As required by Article 10 of the AI Act, the technical documentation must contain information about the data sets used. This is a critical artifact:

• Provenance: Where did the data come from? How was it collected?
• Labeling: How was data labeled? By whom? What instructions were given to labelers?
• Cleaning and Pre-processing: What steps were taken to ensure data quality?
• Bias Mitigation: What techniques were used to assess and mitigate bias (e.g., re-weighting, adversarial debiasing)?

For systems trained on data generated by the user (e.g., federated learning), the documentation must explain how this data is handled and protected.

Information Supplied to the User

Transparency is a key requirement. The technical file must include the instructions for use, the user interface design specifications, and any information required to enable human oversight. This includes details on the system’s capabilities and limitations, and the circumstances in which it may fail. It is a common mistake to treat user instructions as marketing material; in the context of high-risk AI, they are a safety-critical compliance artifact.

Harmonised Standards and Conformity Assessment

The technical file must list the harmonised standards applied (e.g., ISO/IEC 23894 for AI risk management, ISO 13485 for medical devices). If no harmonised standards exist, the manufacturer must describe the solutions adopted to meet the essential requirements. This includes records of the conformity assessment procedure followed (e.g., internal control, third-party assessment by a notified body).

4. Testing and Validation Reports

Testing is the verification that the system meets its requirements and is safe. The AI Act requires rigorous testing of high-risk systems throughout their development lifecycle. The reports generated are critical artifacts.

Pre-market Testing Reports

These reports document the results of testing in controlled environments. They should cover:

• Performance Metrics: Accuracy, precision, recall, F1-score, etc., reported for relevant subgroups to detect bias.
• Robustness Testing: Results of stress testing, adversarial attacks, and testing against out-of-distribution data.
• Safety Testing: Verification that the system fails safely and that risk control measures are effective.
• Human Oversight Testing: Evidence that the system can be effectively overseen by a human operator.

Post-market Testing and Monitoring Plan

The AI Act introduces a specific requirement for a Post-market Monitoring (PMM) plan. This is not just a reactive process; it is a proactive plan for systematically collecting data on the system’s performance in the real world. The PMM plan documents:

• What data will be collected (e.g., user feedback, performance logs, incident reports).
• How this data will be analyzed to identify emerging risks or performance degradation.
• The frequency of review and reporting.

The output of this monitoring is a stream of data that must be fed back into the risk management process. The reports generated from this monitoring are essential artifacts for demonstrating ongoing compliance.

5. Logs and Records of Operation

Logs provide the “black box” record of what happened. For AI systems, particularly those with high stakes (e.g., autonomous driving, medical diagnosis), logs are essential for traceability, accountability, and incident analysis.

System Logs

These are technical logs recording the system’s internal state, inputs, outputs, and errors. They are crucial for debugging and for understanding the cause of failures. The level of logging must be sufficient to reconstruct events.

Event Logs (for Human Oversight)

The AI Act requires that high-risk systems enable human oversight. This implies that the system must log the actions taken by the human overseer. For example, if a system recommends a medical diagnosis and a doctor overrides it, that override must be logged. This record is vital for:

• Accountability: Determining who made a decision.
• System Improvement: Understanding why humans override the system to identify potential flaws.
• Regulatory Audit: Demonstrating that human oversight was actually exercised.

Data Protection Logs (GDPR)

Under the GDPR, controllers must be able to demonstrate compliance. This often involves maintaining records of processing activities (Article 30). For AI systems, this includes logs related to:

• Lawful basis for processing personal data.
• Data subject access requests and responses.
• Automated decision-making and the logic involved (the “right to an explanation”).

These logs are a direct response to the accountability principle and are often the first thing requested by a Data Protection Authority during an investigation.

6. Change Control Records

AI systems are not static. They are continuously updated, retrained, and patched. A robust change management process is essential to ensure that these updates do not introduce new risks or invalidate previous conformity assessments. The Change Control Record is the artifact that tracks this evolution.

Version Control and Configuration Management

This is the baseline. Every artifact (code, model, dataset, document) must be versioned. The change record must link specific versions of the model to specific versions of the training data and code. This is critical for reproducibility and for identifying the root cause of issues discovered post-market.

Impact Analysis

Before deploying an update, an impact analysis must be conducted. This document answers:

• What is the nature of the change (e.g., retraining on new data, architectural change, hyperparameter tuning)?
• What are the potential impacts on performance, safety, and bias?
• Does the change require a new conformity assessment?

Under the AI Act, a substantial modification to a high-risk AI system may trigger the need for a new conformity assessment. The change control record provides the evidence for this determination.

Deployment and Verification Records

Once a change is deployed, the record must be updated to confirm:

• That the deployment was successful.
• That post-deployment monitoring has been initiated.
• That any required updates to user documentation or risk management files have been completed.

This creates a closed loop, ensuring that the compliance artifacts remain a living record of the system’s state.

Practical Considerations for Artifact Management

Managing this volume of documentation is a significant challenge. It requires a systematic approach that integrates documentation into the development lifecycle, rather than treating it as an afterthought.

Tooling and Automation

Manual document management is prone to error and obsolescence. Modern organisations should leverage tools to automate the generation and maintenance of compliance artifacts. For example:

• Model Cards and Datasheets: Standardised formats for documenting AI models and datasets (e.g., from Google’s Model Cards or Microsoft’s Datasheets for Datasets) can form the basis of the technical documentation.
• MLOps Platforms: These tools automatically track data lineage, model versions, and performance metrics, providing a rich source of evidence for the technical file.
• Document Management Systems (DMS): A DMS with version control and audit trails is essential for managing policies, risk files, and change records.

However, automation does not replace human judgment. The content of these artifacts—especially the analysis of bias and the design of human oversight—requires expert input from legal, ethical, and technical stakeholders.

The Role of the Notified Body and Market Surveillance

For high-risk AI systems in regulated sectors like medical devices, the technical documentation and QMS will be reviewed by a Notified Body. This is a third-party conformity assessment body designated by a Member State. The Notified Body will scrutinize the Risk Management File, the technical documentation, and the test reports. They will issue an EU Certificate of Conformity if all requirements are met. The artifacts are the primary basis for this assessment.

For other high-risk AI systems (e.g., in education, employment), the conformity assessment is generally based on internal control (the manufacturer self-certifies). However, this does not mean no oversight. Market Surveillance Authorities have the power to request all documentation at any time. If the documentation is incomplete or does not demonstrate compliance, the authority can impose fines, require corrective actions, or ban the system from the market. The artifacts are the first line of defense in such an inquiry.