Governing Technology We're Still Learning
Recent lessons learned from conference we attended, discussing how we address changing technology
Step
Building an AI Agent Risk Register
Addressing Technical Failure Systematically towards Business Risks
As organisations give AI agents more authority to act within core business processes, they face a fundamental governance problem: how do you translate the technical ways an agent can fail into business risks that can actually be owned, controlled and managed?
Imagine a refund agent that receives customer claims, checks eligibility and, where appropriate, issues payment automatically. One day it incorrectly pays a customer who was not eligible. Security may identify a prompt injection. Operations may see an unauthorised transaction. Finance may see a financial loss. All are describing different aspects of the same incident.
An effective risk framework needs to connect those perspectives without treating them as interchangeable — distinguishing why the failure could happen, what the agent actually does wrong, and what harm follows.
IN THIS ARTICLE
- Why technical failure modes (like prompt injection) aren't the same as business risks
- A simple structure — Cause → Event → Impact — for describing what can go wrong
- Six business-event families you can use to spot the risks an AI agent creates
- How to trace those events back to causes and forward to business impacts
- What a complete risk-register entry looks like, using a refund agent as a worked example
Why technical failure modes aren't enough
Technical issues such as hallucination, prompt injection or excessive privileges tell us how an AI agent might fail. In a business-process risk scenario, these act as the conditions or mechanisms that can lead to something going wrong. On their own, they don't tell us what happens in the business process, or what harm follows.
The same cause can lead to very different outcomes depending on what the agent can do. A prompt injection may have limited consequences in an assistant restricted to non-sensitive information — but in an agent connected to payments or customer data, it could trigger an unauthorised transaction or data disclosure, leading to financial loss or customer harm.
This is why stopping at the cause isn't enough. Linking the technical cause to a business event and its consequences makes the business risk easier to assess, and accountability clearer.
For example, when assessing a refund agent before deployment, listing “prompt injection” (cause) and “financial loss” (impact) leaves out the event — the observable business failure that connects the two. A more useful risk statement reads like this:
A prompt injection tells the refund agent to ignore the eligibility rules, and weak transaction controls allow the instruction to reach execution (cause). The agent then issues a refund to an ineligible customer (event), resulting in financial loss and customer disputes (impacts).
This gives us a simple structure:
For each material risk scenario, ask three questions: why could it happen, what would actually go wrong, and what harm would follow?
Refund-agent example
The goal is to preserve the technical cause while translating it into the operational event and business impact it can produce — giving engineering, security, risk and business teams a common description of the exposure they can actually assess and govern.
For a practical risk assessment, it's usually best to start with the event — what goes wrong in the business process — then work backwards to the causes and forwards to the impacts. That's the order the rest of this article follows.
The question
Refund-agent example
Cause
Why could it happen? The technical vulnerability, control failure, or external trigger that starts the sequence.
A prompt injection attack or compromised third-party dependency.
Event
What actually goes wrong? The operational failure where the agent acts incorrectly or fails to act. Must be observable and classifiable.
The agent issues a refund that was not properly authorised.
Impact
What harm follows? The business consequences of the event.
The company loses money, the customer may face a dispute, and the incident requires investigation and recovery work.
Building the risk register
One agent may fail in several materially different ways. The refund agent, for example, could issue an unauthorised payment, pay the wrong amount, disclose customer information, or fail to process an approved refund. Each of these should be assessed as a separate scenario wherever the causes, impacts or controls differ materially.
Good practice in operational risk management is to rely on mutually exclusive and collectively exhaustive (MECE) scenarios:
- Causes may overlap. A single event can be triggered by multiple technical failures or threat vectors.
- Events are mutually exclusive. The incident itself must be classified under a single primary event (e.g. “Unauthorised Execution”) to avoid double counting, even where multiple causes are present.
- Impacts are collectively exhaustive. All material loss channels should be captured, to support aggregation and reporting.
The example below illustrates one risk-register entry. All names, volumes, limits and ratings are illustrative. Likelihood refers to the event occurring over a one-year assessment period; the residual rating is conditional on effective implementation of the listed controls, rather than evidence of a completed assessment.
Illustrative risk-register entry
Register field
Illustrative refund-agent entry
Identity and scope
Refund Agent — customer-service refund process. Agent ID: AG-12457. Scenario ID: RS-004.
Risk scenario
Cause: a malicious instruction causes the agent to ignore refund eligibility rules
Event: the agent issues a refund to an ineligible customer
Impacts: financial loss, customer disputes and investigation costs
Event: the agent issues a refund to an ineligible customer
Impacts: financial loss, customer disputes and investigation costs
Event classification
Unauthorised execution
Causal tags
OWASP ASI01: Agent Goal Hijack
Potential business exposure
Baseline assessed before the listed controls: autonomous refunds of up to €2,000 per transaction, with approximately 1,200 refund requests per day. Coordinated exploitation could create material aggregate financial loss before detection, alongside customer and reputational impact.
Risk owner
John Smith, Head of Customer Service
Control owners
Payments Platform Owner; Customer Identity Product Owner; AI Platform Owner.
Inherent risk
High — Likelihood: Possible (3/5); Impact: Major (4/5); score: 12/25, before considering the listed controls.
Key controls
Refund eligibility independently enforced by a deterministic rules engine; maximum autonomous refund €250; refunds above €250 require human approval; payment API verifies transaction/customer match; agent cannot modify eligibility rules.
Control assurance required
Test eligibility and payment controls before deployment; include a prompt-injection test suite in release testing. Retain evidence of the results.
Residual risk (conditional on effective implementation)
Medium — Likelihood: Unlikely (2/5); Impact: Major (4/5); score: 8/25. Impact remains Major because repeated lower-value refunds could still produce material aggregate losses.
Treatment decision
Accept with restrictions, subject to control assurance. Autonomous refunds permitted up to €250 once the controls are implemented and their effectiveness evidenced. Higher-value refunds remain subject to human approval. Reassess before increasing the limit.
Monitoring and review
Daily reconciliation of executed refunds against approved claims; automated alerts for duplicate refunds, unusual recipient patterns and control overrides; quarterly scenario review or immediate reassessment following a material incident or system change.
“Prompt injection” is no longer the risk record. It's one possible cause within a broader scenario that connects the technical failure to the business event, its consequences, and the governance decisions required around it.
What comes next: from risk register to ongoing governance
The risk register provides a basis for ongoing governance. Once material scenarios have been identified, organisations can use them to drive several downstream activities:
- Control design and testing — what prevents the event, detects it or contains its consequences, and is there evidence those controls actually work?
- Scenario analysis — how likely is the event, and how severe could the consequences become?
- Reverse stress testing — what combination of failures would allow an unacceptable outcome to occur?
- Monitoring — what indicators could show the exposure is increasing after deployment?
- Incident reporting — when something does go wrong, how does the actual event compare with the scenarios identified beforehand?
- Independent assurance — are the assumptions, controls and risk assessments credible when challenged independently?
Each of these deserves deeper treatment. Subsequent articles in this series will look at how to design and test controls for AI agents, use scenario analysis and reverse stress testing to assess material risks, and monitor agent behaviour once systems move into production.
The important first step is simpler: describe the risk correctly. Start with what could go wrong in the business process, work backwards to understand what could cause it, and assess the consequences if it happens. Then record that scenario in a form that someone can own, challenge and act on.
About the author
Andrea Politano is an AI risk and governance specialist with senior experience in model risk management, independent validation and regulated financial services.
Read next one
Meet us
Schedule a working session with us to identify how we can support you on your journey adopting AI tools and managing your processes with focus on return on your investments.
Address
