Governing Technology We're Still Learning

An honest starting point

‍

The most useful sentence at a recent conference Agentic AI for leaders summit I attended came in its opening minutes: none of us in the room is an AI expert. It was a curated group of business leaders, technologists and governance practitioners, and the admission set the tone for everything that followed.

It was not false modesty. The technology has moved so fast in the past few years that expertise has a short shelf life. We are all learning, and we are learning while we deploy. That is exactly why governance has become the question to address right now across multiple domains, even though nobody in the room claimed to have the final answers.

‍

Searching for the value

‍

A year or two ago, most conversations about AI were technology questions. Which model? Which vendor? What can it do? Those questions have not disappeared, but they are no longer the main ones. The harder question now is how we apply the technology, and how we manage the value it creates.

Value that essentially can be visible in company’s P&L. It is the most pressing aspect of AI adoption today: where exactly does it show up, and can we prove it? Answering that requires to look at two unglamorous aspects. First, well-documented processes and deliberately designed workflows, because you cannot improve what you have never mapped. Second, honest before-and-after measurement, agreed before the pilot starts rather than reconstructed after it ends.

Definition of adoption metrics matters a lot. Because tracking usage counts can tell you that people logged in, however it will not tell you whether a process got faster, cheaper or better.

‍

From chatbots to agents, from experiments to habits

‍

Last year the conversations were mostly about generative AI and chatbots. Yet chatbots are now mainstream. This year the conversations are more about agents: systems that orchestrate among themselves and execute multiple end-to-end processes, possibly with no human involved at all. Agentic assistants resolving IT service tickets end-to-end was one of the clearest real-world examples discussed. However, how do we ensure that in the chain of multiple tasks executed by agents something will not go wrong and potentially set risks in the IT systems of the company?

That example raised an uncomfortable question in the room. Should we slow down deployment? Are we actually ready to govern this?

The second uncomfortable question was about experimentation. Pilots are often short-lived. Do employees genuinely change and adapt to new ways of working, or do they (sometimes silently) drift back to old habits once the novelty fades and the value is unclear?

Real adoption means changing the company's processes, not just its toolset. It works like an enterprise muscle: if you do not exercise it, it does not grow. (A lighter observation from the day: people still have far too many meetings. That alone is a workflow worth redesigning.)

‍

Foundations first: strategy, data and risk-adjusted value

‍

AI is not a strategy on its own. It is an enabler of your business strategy. Your technology strategy must incorporate chosen approach towards data, cloud and AI, and they all must sit inside the overall business strategy rather than beside it.

Data quality itselfremains the unavoidable foundation. Without good-quality data sets, your AIwill produce nonsense, however capable the model.

The conversations also introduced a useful angle of adoption: the risk-adjusted cost of AI. Any business case should assume that something will go wrong, and price that in. It reframes adoption as, to a significant degree, a question of model risk management. Some of that risk can be reduced very early, at the prompt level, through clear instructions, constraints and guardrails, long before it reaches a process or a customer.

‍

Governance that keeps pace

‍

The central tension of the day was following: governance is essential, yet it is very hard to fix in place while we are still testing what measurable outcomes AI brings to a business. Governance frameworks will keep changing as the technology changes and as we observe what it actually delivers.

The goal is clear even if the final form is not. Governance should make sure the technology is deployed correctly, and that the risks of adoption are identified and addressed, without limiting the technology's potential to raise productivity and competitive advantage. Four points stood out on what to pay attention in times of adoption:

1.     Governance must be agile and speak to developers. If a governance process stops agent development for months, the problem it was meant to solve has often moved on by the time a decision arrives. Governance has to work at the speed and in the language of the teams building the systems.

2.     Human-in-the-loop needs rethinking. We often assume a person will review everything an agent does. In reality, nobody can review every task an agentic system executes at scale. Human oversight stays essential, but its meaning must change: from a human in the loop to wisdom in the loop, where people design the boundaries, the escalation points and the checks.

3.     Accountability stays human. Decisions can be delegated to technology. The consequences cannot. They remain fully owned by people and by the organisation.

4.     Check your cover. A practical question every leadership team should ask: does your insurance cover the usage of agentic AI and the risks that come with it? I am sure you are answering that question as you read...

‍

Future of consulting and people's involvement

‍

Agentic systems will do more than automate tasks. They will change organisational cultures: how work is divided, who decides what, and how trust between people and systems is built.

That shift reaches consulting too. A topic that is very close to me and what we do in PheonIQ. Standard delivery of reports, documents and recommendations is likely to shrink, because much of it can now be produced faster by the technology itself. Where consultancy will however still add real value is in transformation and change management. Giving clients access to accumulated knowledge, so clients can actually benefit from what has been learned across many engagements will be the main advantage.

The main outcome of the day was a shared position: We understand the potential of this technology. We understand that people must learn to work alongside it. But we also understand we are still experimenting and still measuring, so no one can honestly claim to know how every risk will be addressed or what governance will ultimately look like.

That is precisely why the role of people in defining it matters more now than at anytime before. Governance will not be handed to us finished. It will be shaped by those willing to learn, test, measure and adjust, openly admitting that none of us is an expert yet.

‍