AI Compliance-by-Design: A CTO Guide to Model Inventory, RAG Sources, and Decision Logging under the EU AI Act
The EU AI Act is now in force and rolling out in phases; several obligations arrive before full application in 2026–2027. The European Commission’s AI Office will oversee providers — with a special focus on general-purpose AI (GPAI) — and may request information or evaluate models. For GPAI in particular, the Commission has published obligations fact pages and a voluntary GPAI Code of Practice to reduce ambiguity while Article 53/55 duties phase in.
Start with Scope: Are You a Provider or a Deployer?
The Act differentiates between providers (who place on the market or put into service) and deployers (who use AI systems). If you provide or fine-tune GPAI models, you face specific duties such as technical documentation, a copyright policy, and a summary of training content. For “systemic-risk” GPAI (the most capable models), additional requirements include risk assessment/mitigation, incident reporting, and robust cybersecurity.
To manage these obligations effectively, maintain a live registry that tags each model integration by role (provider/deployer), category (GPAI vs. task-specific), and jurisdiction exposure. This approach drives compliance with relevant obligations and controls.
Inventory First: Models, Datasets, Prompts, and RAG Sources
The fastest path to non-compliance is not knowing what you run. Inventory should be treated as a product:
- Model catalog: Version, provider, fine-tuning status, eval scores, intended use, and contact owner.
- Data contracts for every RAG source: lineage, freshness, completeness rules, retention, and allowed uses.
- Prompt & tool registry: Approved prompts, tool call scopes, and high-risk actions requiring manual checks.
- Decision logs: “Who-what-when-why” for changes, refusals, and overrides.
This aligns with the NIST AI Risk Management Framework and its implementation guidance.
Build the Technical Documentation Once — and Keep It Living
Article 53 expects providers of GPAI to “draw up technical documentation” and share what downstream users need without disclosing IP. The Commission’s GPAI Code of Practice provides a model-documentation form that can be adopted now.
Maintain a single documentation bundle per model integration, which includes:
- Model card: Capabilities, limits, evals, safety scope.
- Data sheet / RAG card: Sources, contracts, copyright policy, summary of training content when applicable.
- Safety case: Refusal policy, escalation paths, abuse channels.
- Operational runbook: SLAs, rollback, change approvals, incident playbooks.
This unified documentation approach helps to streamline legal, security, and product reviews, ensuring updates do not diverge across teams.
Treat Copyright & Training-Data Transparency as Production Requirements
The Act mandates a copyright policy and, for GPAI, a summary of training data. The GPAI Code of Practice provides templates for these requirements. Downstream deployers benefit from clearer provenance, which reduces takedown risk and support burden.
For content-generating assistants, embed citation hints and disallow outputs that cannot be traced to permitted sources to ensure compliance.
Log Decisions, Not Just Predictions: Auditability by Design
Transparency obligations extend beyond user disclosure; you must demonstrate that humans can oversee and trace system decisions. Log decision context (inputs, retrieved sources, tools called), policy gates triggered, human approvals, and reasons for refusal. This satisfies audits and shortens incident investigations.
In practice, ensure that every security answer includes linked sources and a policy-decision record to maintain compliance and speed during RFP reviews.
Control Access and Risk Like You Would for Money Movement
With tightening GPAI oversight, treat tool calls and data access as financial transactions:
- Short-lived identities for agents; least privilege scopes; JIT elevation for high-risk tasks.
- Session recording/logging for sensitive actions.
- Change previews & rollbacks for batch operations.
- Jurisdictional separation: Maintain regional vector indexes and storage.
Implement these practices to enhance security and reduce legal review time.
Use a Stage-Gate Cadence the CFO and CISO Can Support
Run GenAI initiatives on a 15/45/90 rhythm with explicit cost caps and quality thresholds:
- 15 days: one workflow, one metric, cost ceiling.
- 45 days: baseline vs. after, error costs captured, quality above threshold.
- 90 days: either integrate or shut down based on performance.
This approach mirrors the shift from pilots to production, ensuring compliance obligations are met.
When to Use the GPAI Code of Practice
If you are a GPAI provider (or fine-tune GPAI) and seek a lower-friction path to demonstrate compliance, the Commission’s GPAI Code of Practice offers a voluntary route now, covering Transparency, Copyright, and Safety & Security for systemic-risk models.
Borrow the Code’s documentation form for internal use to prepare for potential regulatory scrutiny.
What to Brief the Board On
- Scope & role: Distinguish between provider vs. deployer roles.
- Obligations & timing: Identify applicable Article 53/55 duties.
- Controls in place: Inventory, data contracts, decision logging, copyright policy.
- Stage-gates: 15/45/90 cadence with cost and quality thresholds.
- Assurance: Alignment with NIST AI RMF for consistent language across audits.
