The EU AI Act: What It Means for Software Engineers

The EU AI Act came into effect on 1 August 2024. Although its provisions are phased in through 2027, the engineering implications are already tangible: traceability, explainability, and audit requirements are now legal obligations for certain AI systems, not just optional quality practices.

This article focuses on what the Act requires from the people who build and deploy AI‑integrated software, not just from legal or compliance teams, but from engineers themselves.

The Risk-Based Framework

The Act categorizes AI systems into four risk tiers, with each tier dictating specific obligations. Higher tiers impose more rigorous documentation and oversight because the potential harm is greater.

Tier	Examples	Engineering obligations
Unacceptable risk (banned)	Social scoring by governments, real‑time biometric surveillance in public spaces	Cannot be built or deployed in the EU
High risk	CV screening, credit scoring, medical devices, critical infrastructure, law enforcement	Full conformity assessment, technical documentation, human oversight, logging
Limited risk	Chatbots, deepfake generators	Transparency obligations (disclose AI interaction)
Minimal risk	Spam filters, AI in video games	No specific obligations

A critical compliance milestone for engineering teams is determining whether an AI subsystem qualifies under Annex III high-risk classifications. Architectural prudence dictates designing for high-risk governance standards whenever model inferences affect credit scoring, recruitment filtering, critical infrastructure, or biometric verification, ensuring complete lineage tracking and human-in-the-loop audit controls from day one.

The Act defines high‑risk systems in Annex III, which covers eight domains including employment, education, essential services, law enforcement, and critical infrastructure. If your system makes or materially influences decisions in these domains, it is likely to be classified as high‑risk.

What High‑Risk Systems Must Have

For high‑risk AI systems, the Act mandates specific technical measures, which are engineering requirements rather than mere policy statements.

Technical documentation

Before a high‑risk system is placed on the market or put into service, the provider must produce documentation covering:

• a general description of the system and its intended purpose;
• the design specifications, including the training methodology, data used, and known limitations;
• the risk management process and its outcomes;
• the accuracy, robustness, and cybersecurity measures; and
• the post‑market monitoring plan.

This documentation must be kept up to date throughout the system's lifecycle. Any model update that changes behavior requires updated documentation.

Logging and audit trails

High‑risk systems must automatically log events sufficient to enable post‑hoc investigation. Without such logs, we cannot reconstruct the decision path when a user challenges an outcome.

The Act specifies that logs must cover:

• the period of each use of the system;
• the reference database against which input data was checked;
• the input data that led to a given output; and
• the identity of the natural persons involved in verification.

// Logging structure for a high‑risk AI decision
interface AIDecisionLog {
  sessionId: string;
  timestamp: Date;
  systemVersion: string;        // model version + config hash
  inputDataHash: string;        // hash of input, not raw PII
  outputDecision: string;       // the decision made
  confidenceScore: number;
  humanReviewerId?: string;     // if human oversight was applied
  overrideApplied: boolean;     // if human overrode the AI decision
  retentionUntil: Date;         // per data retention policy
}

Logs must be retained for a period appropriate to the system's purpose, though the Act does not specify a universal retention period.

Human oversight

High‑risk systems must be designed so that natural persons can effectively oversee them. Effective oversight means the user can understand why a decision was made and intervene if it seems erroneous.

• the system must be interpretable enough for an overseer to understand its outputs;
• overseers must be able to intervene, override, or halt the system;
• the system must not create automation bias, it must not present outputs in a way that discourages human review.

Automation bias is an engineering problem, not just a UX one. When the interface hides uncertainty, engineers inadvertently hand over too much authority to the algorithm.

Accuracy, robustness, and cybersecurity

High‑risk systems must achieve appropriate levels of accuracy for their intended purpose, be robust against errors and inconsistencies in input data, and be resilient against adversarial manipulation. These requirements must be documented and tested, not merely asserted. A signed test report becomes part of the technical file required for conformity assessment.

General‑Purpose AI Models

The Act introduces specific obligations for providers of general‑purpose AI (GPAI) models, foundation models that can be used for a wide range of tasks. Because these models can be repurposed for high‑risk applications, regulators treat them with extra scrutiny.

Providers of GPAI models must:

• maintain technical documentation;
• comply with EU copyright law regarding training data;
• publish a summary of training data; and
• for models with systemic risk (above a compute threshold of 10^25 FLOPs), conduct adversarial testing, report serious incidents to the European AI Office, and implement cybersecurity measures.

If your organization fine‑tunes a GPAI model and releases it, you may have obligations under this provision.

The Conformity Assessment Process

Before a high‑risk AI system can be deployed in the EU, it must undergo a conformity assessment. This assessment proves that the system satisfies every technical requirement outlined earlier.

For most high‑risk systems, this is a self‑assessment against the Act's requirements, documented in a technical file. For certain high‑risk systems (biometric identification, critical infrastructure), third‑party assessment by a notified body is required.

The conformity assessment is not a one‑time event. It must be repeated when the system is substantially modified, including when the model is retrained, when the training data changes significantly, or when the intended purpose changes.

What This Means in Practice

For teams building AI features in existing products

If your product operates in a high‑risk domain (HR, credit, healthcare, education, critical infrastructure), audit whether your AI features fall under Annex III. Mapping every decision point to the Annex III list is essential.

If they do, you need technical documentation, logging, and human oversight mechanisms before deployment in the EU. Skipping these steps would expose the company to fines and product bans.

For teams building AI‑native products

Design for auditability from the start. Logging hooks and explainability modules are embedded in the core architecture so they cannot be tacked on later.

Retrofitting logging, explainability, and human override mechanisms into a deployed system is significantly harder than building them in. That is why 15 % of sprint capacity is allocated to compliance engineering from day one.

The Act's requirements are a useful forcing function for engineering practices that improve reliability regardless of regulatory obligation. Even if we later sell outside the EU, these practices raise the overall quality of the product.

For teams using third‑party AI APIs

If you integrate a third‑party AI system into a high‑risk application, you are the deployer and you share obligations with the provider. A data‑sheet from the API vendor that details model limitations and training provenance is therefore required.

You cannot outsource compliance to the API vendor. Ultimately, the team must verify that the API's behavior aligns with our risk assessments. Wrapper code records each request, response, and confidence metric.

Timeline

Date	Obligation
February 2025	Prohibited AI practices banned
August 2025	GPAI model obligations apply
August 2026	High‑risk AI system obligations apply (Annex III)
August 2027	High‑risk AI systems embedded in regulated products

The August 2026 deadline for Annex III systems is the most significant for most engineering teams. Systems that require conformity assessment, documentation, and logging infrastructure need that work completed and tested before the deadline, not started at it. Otherwise we risk rushed implementations that miss critical safety checks.

A Note on the February 2025 Deadline

The prohibition on unacceptable‑risk AI practices (social scoring, real‑time biometric surveillance in public spaces) took effect in February 2025 and is already in force. If your system falls into that category, the question is not compliance planning, it is whether the system can legally operate in the EU at all.

What the Act Does Not Specify

The Act sets requirements but leaves implementation choices to engineers and organizations. This flexibility allows us to choose tools that fit our stack while still meeting the law.

It does not mandate a specific logging format, a specific ML framework, or a specific explainability technique.

This is intentional: the Act is technology‑neutral. The obligation is to achieve the outcome (auditable, overseen, accurate systems), not to use a particular tool to get there. Our compliance evidence consists of test reports, design reviews, and risk registers.

This means engineering teams have latitude in how they meet the requirements, but they must be able to demonstrate that they have met them. Documentation of design choices, test results, and risk assessments is what makes that demonstration possible.