Why Annex IV documentation needs runtime evidence, not code scans

What Annex IV actually requires

Annex IV of the EU AI Act lists what technical documentation for a high-risk AI system has to contain: a general description of the system and its intended purpose, details of the design and development process, information about how the system is monitored and controlled once deployed, performance and validation metrics, risk-management measures, and a record of significant changes across the system's lifecycle. None of that is a description of what the code is capable of doing in theory. It's a description of what the system does, has done, and how it behaves in production.

What traces capture that code can't

A static scan of a repository can enumerate which SDKs are imported, which model endpoints are called, and roughly how the code is structured. That's useful for a different question — “what could this system do?” — but Annex IV, alongside Article 14 (human oversight) and Article 9 (risk management), is asking a different question: what did this system actually do, and how was it overseen?

Human oversight (Article 14). Providers have to demonstrate that a human can and does intervene in the system's operation. A code scan can see that an “approve” endpoint exists. It can't see whether a human actually used it, how often, or under what circumstances — that only shows up as an event in a runtime trace.

Post-market monitoring (Article 72). Providers must monitor a deployed system's performance and log significant incidents. Runtime traces are, by definition, the record of what happened after deployment: tool calls, errors, latency, and behavior. Source code, deployed once and then read statically, cannot describe events distributed over the weeks or months after that deployment.

Risk management (Article 9). This requires identifying risks that materialize during actual use — not just risks that are theoretically possible given the code. An error trace, a rejected tool call, or a human override are all risk signals that live in execution data, not in a function signature.

There's also a structural gap: many AI agents call tools and services the provider didn't build — third-party APIs, MCP servers, other teams' internal tools. A code scan of your repository simply can't see behavior that happens outside your own source tree. A runtime trace captures it regardless of where the tool call was implemented, because it records what happened, not where the code that caused it lives.

How Attestly maps trace events to Annex IV sections

Attestly ingests normalized trace events — tool calls, model calls, human interventions, errors, and deployment/version changes — from OpenTelemetry, LangSmith, AgentOps, MCP logs, or generic pre-normalized JSON. Each Annex IV requirement is mapped against the specific events relevant to it: a monitoring-measures section is drafted from tool-call and error events; a human-oversight section is drafted from intervention events; a change-log section is drafted from deployment-version events. Every generated sentence carries a link back to the exact trace event that justified it, so a reviewer can verify the claim in seconds instead of re-deriving it from raw logs.

This doesn't replace human judgment or legal review — every draft is explicitly marked AI-generated, user-provided, or missing, and has to be reviewed and approved by a person before export. What it replaces is manually reconstructing that evidence trail from scratch every time documentation needs updating.