A worker made of software: what an AI agent actually is
The term “AI agent” turns up in board papers and supervisory conversations every day, but often without a definition. For people who test controls or answer for them, that matters, because it is hard to govern something nobody has described. Here is the definition I use in risk and compliance work.
An AI agent is software that you give a task and a written procedure. It reads the evidence, works in the systems you give it access to, takes each case through to a conclusion and records what it did at every step. In a risk function, the closest comparison is a new hire.
You don’t need to understand the technology deeply to govern an AI agent. You need a clear picture of what it does. You already do much of the rest when a new tester joins your team.
RPA repeats recorded steps, while an AI agent judges each case
Three terms are often used in the same conversation: robotic process automation (RPA), the AI model and the AI agent. They do different jobs. Important to remember is that only the AI agent works through a case on its own.
RPA repeats steps that someone recorded. It works for as long as the information arrives laid out in the same way. It does not easily recognise a change in the layout, so RPA needs more maintenance.
The AI model, usually a Large Language Model (LLM), sits inside the AI agent and does the reading and writing. It is the same technology as the public chat assistants. On its own, a model answers questions. It does not log into a system or finish a piece of work.
An AI agent puts that model to work. Give it a test procedure and it works through the documents the way your tester would: the contract, the policy, the approval email. It reads each document for its meaning, not for where things sit on the page, so a change in layout does not stop it. It logs into the systems it has credentials for, such as your GRC platform and document archive, and uses them as its tools. The AI agent takes a case from instruction to conclusion without anyone steering each step. It records each step as it goes.

Testing every contract lists each exception instead of inferring from a sample
Many second-line teams test a control like this one: new customer contracts must be activated on the terms that were approved.
Your test procedure likely reads something like this:
- Pull a sample, say 25 of the 6,000 contracts activated in the period
- For each contract in the sample, retrieve the approval
- Compare the approval with the activated record
- Apply the tolerances in your procedure to any differences, and conclude
- Document the result in the workpaper.
An AI agent is onboarded onto that procedure, whatever yours actually says, and follows the same steps for all 6,000 contracts. For every case it records the retrieval, the comparison and the decision. Your reviewer gets a workpaper with a readable conclusion. The case-by-case record behind it serves as the audit trail.
With a sample, a clean result on 25 contracts lets you conclude that the control operated effectively. How much assurance that gives depends on your sampling methodology. It cannot tell you whether any particular contract outside the sample was activated on the approved terms.
With full-population testing, every contract in the extract has been tested and each exception is listed by contract. Your methodology then needs to decide how many exceptions in a full population still support an effective rating. You also need to test that the extract is complete, as a step in its own right. Finally, you need evidence that the AI agent performs the test accurately, which I come back to below. A person still reviews every exception, so the effort moves from testing to reviewing what the test found.

Risk work suits AI agents because it is recurring, capacity-bound and evidence-based
Risk functions are cautious by design, so they can look like an unlikely place to start with AI agents. Three features of the work make them a good fit.
First, a lot of it is recurring casework built on evidence. Control testing, due-diligence file reviews, incident write-ups and monitoring follow-ups all run the same way: gather the evidence, apply a defined standard, record the conclusion, and repeat next quarter. The work also tends to arrive in large batches of near-identical cases, which suits an AI agent well.
Second, coverage is limited by tester hours. Sample sizes come from your methodology, but how many controls make the plan each cycle, how often they rotate and how far each test reaches depend partly on the hours available. An AI agent eases that particular limit.
Third, the work is judged on what you can show. An AI agent documents as it works, so the workpaper comes out of the test as it runs. If you have ever spent the last week of a testing cycle writing up work done in the first, you will see the appeal.
Show supervisors that your controls operate as designed
Testing evidence matters because supervisors look closely at how banks govern themselves. In ECB banking supervision, breaches of internal governance were the largest single category of sanctioning proceedings in both 2023 and 2024, at 57 and 55 per cent.¹
Sweden sits outside ECB supervision, but one case from the Swedish regulator Finansinspektionen (FI) shows how closely a supervisor can read an institution’s framework. In December 2024 FI gave a financial institution a remark and a SEK 500 million administrative fine. FI did not claim that any money laundering had taken place. It found that the general risk assessment had not assessed how the institution’s own products could be misused. The customer due diligence procedures also did not cover every situation that required them.²
Testing cannot write that framework for you. It can show that the controls built on the framework operate as designed, quarter after quarter. A bank can put that evidence in front of a supervisor.
A named person answers for an AI agent’s work, backed by checks and supplier oversight
Paying someone to do work you remain answerable for is nothing new. Banks do it with employees, consultants and outsourced service providers. An AI agent differs because it cannot carry accountability. It holds no mandate of its own, so the accountability sits with the people and the process that govern it.
One way to handle that would be to have a person re-perform every case the AI agent touched. The work would be limited by how many people you have, just as before. What’s worse, is that you would also pay for the AI agent on top of the cost of human labour.
The other way works like managing a team whose work you do not check line by line. A named person is accountable. That person relies on a process built on AI agent:
- A written, versioned test procedure, so the AI agent follows the same instructions every time and every change is recorded.
- Checks that do not depend on the AI agent. They cover the attributes that can be calculated, such as reconciling a value against the source record.
- An answer key: a set of cases where people have already agreed the correct outcome. It measures how often the AI agent misses a real exception and how often it raises a false one.
- An escalation route that sends ambiguous cases to a person.
If you run a testing team, you already ask who wrote the procedure, who reviews the work and who signs the conclusion. An AI agent also works inside your systems. That brings in the questions you ask of any outsourced service, such as what it can access and how you would exit. Your DORA register of information should list the provider. Where the service supports a critical or important function, it may also need to list subcontractors such as the model provider.

Better models will not remove the need for human judgement and independent checks
Language models keep improving. It is tempting to expect the remaining limits of AI agents to disappear with them. Two of those limits will stay.
First, some cases should always go to a person, such as the ambiguous ones and the first of a kind. Human effort then goes to the exceptions and the hard cases instead of the routine ones.
Second, an AI agent cannot be the only check on its own work. A natural next idea is to have a second AI model check the first one. That helps less than you might expect. In a 2025 study of more than 350 language models, different models got the same benchmark questions wrong far more often than chance would predict. The more accurate models shared more of their errors.³ A useful check rests on something the AI agent did not produce, such as the calculated checks and the answer key described above.
Testers know a similar idea from automated application controls. Once the logic is tested, assurance moves to the general IT controls around it, such as who can change the logic and how changes are approved. An AI agent’s judgement is less predictable than fixed logic. The model underneath it can also change. For that reason the calculated checks keep running on every case. The judgement calls are covered by escalation and by measuring the AI agent against the answer key for as long as it runs. Each conclusion should be traceable to the model version that produced it.

Build on a platform or buy completed work, but keep the procedures and records
There are broadly two ways to put AI agents to work. You can buy a platform, then build and run the AI agents yourself. Or you can buy completed work from a provider that runs the AI agents for you, with an operating team behind them.
A platform you may give you more control over the build. But it comes at the expense of carrying the engineering, the monitoring and the model changes.
Completed work leaves you less to build and run. While you have less say over how the AI agent is built, you receive finished tests with the full work shown. You govern the provider as you would any outsourced service.
Either way, the written procedures and the records should stay with you. You can then change course without losing them.
A parallel run on one control is the sensible way to start
On day one, you have not yet gained evidence of how well an AI agent performs your test. That evidence comes from a parallel run. The AI agent re-performs a test your team already runs, alongside the people who run it, until its output has been benchmarked. By then you have also agreed what good looks like.
Control testing is a good first candidate because it is bounded and recurring. It is also easy to judge against the team doing it today. Once that runs well, the same approach can extend to neighbouring casework such as due-diligence file reviews and incident write-ups.
Govern an AI agent like a new hire, with independent checks added
Strip away the technical vocabulary and you are left with a familiar management problem. When a new tester joins, you give them written procedures and check their early work against people you trust. You widen their scope as the evidence comes in. A named manager answers for how they perform. An AI agent needs the same treatment, plus checks that do not depend on it.
If this raised questions for your own testing programme, I’m happy to compare notes.
Sources
- 1. ECB Banking Supervision, SSM sanctioning reports for 2023 and 2024: breaches of internal governance as a share of sanctioning proceedings (57% of 371 proceedings in 2023; 55% of 441 in 2024). https://www.bankingsupervision.europa.eu/press/other-publications/publications/sanctioning-report/html/ssm.sr2024~fa110879af.en.html and https://www.bankingsupervision.europa.eu/press/other-publications/publications/sanctioning-report/html/ssm.sr2025.en.html
- 2. Finansinspektionen, sanction decision of 11 December 2024, FI dnr 22-11505 (published on fi.se).
- 3. Kim, E., Garg, A., Peng, K. and Garg, N., “Correlated Errors in Large Language Models”, ICML 2025 (PMLR 267). https://proceedings.mlr.press/v267/kim25e.html
About the author
Berry van Leijen, Go to Market Lead, Agent Workforce
Berry is helping banks and insurers across the Nordics apply Agentic AI to Risk & Compliance. He enables them to deploy AI agents across control testing, KYC and CDD, financial controls, and credit and lending assurance. Through Agent Workforce, Berry helps them increase assurance coverage, reduce manual testing effort and introduce AI in a controlled way, while keeping human accountability and decision-making in place.

Share this post