AI
Agents expose the limitations of trust
GUEST COLUMN by
Renee DavisEnterprise computing has long depended on chains of trust. Organizations trust their cloud providers, software vendors, identity systems and administrators to operate as designed. That model has worked because humans have remained the ultimate decision-makers.
Agentic artificial intelligence changes that equation. Autonomous systems can retrieve information, make decisions, invoke external tools, collaborate with other agents and execute actions on behalf of the enterprise. As organizations delegate more authority to intelligent systems, they also need a way to determine whether those systems behave as intended.
The question is no longer simply whether AI can perform complex work. It is whether every consequential action can be independently examined and verified. The next era of enterprise computing will be defined not only by what autonomous systems can do, but by whether organizations can reconstruct and verify what they have done.
The limits of trust
An AI agent may retrieve information from multiple enterprise systems, pass it to another agent, invoke an outside service and authorize a financial action. Each individual component may be secure, yet the enterprise may still have no complete, independently verifiable record of how the final action occurred.
Traditional security technologies can help establish who was authorized to act, what systems they could access and whether unusual behavior occurred. Traditional logs can show that an application programming interface was called or that a transaction took place. But those records do not necessarily capture the full chain of instructions, inputs, decisions and actions that led to an outcome.
The question is whether trust itself is a sufficient architectural principle for a world of autonomous computing.
Much of today’s cybersecurity conversation begins with a familiar question: Which platform should we trust? It’s a reasonable question. It’s also increasingly the wrong one.
Traditional security technologies remain indispensable, but they were largely designed to protect systems by enforcing policy, monitoring behavior and controlling access. They cannot, by themselves, establish which instructions and data shaped an agent’s behavior, reconstruct its interactions across multiple systems or demonstrate that the resulting record has not been altered.
Enterprise leaders should ask a different question when agents are involved: What evidence does this system produce, and can that evidence be independently verified? If the answer depends primarily on trusting the vendor, the infrastructure or the software itself, then the enterprise is still operating within the assumptions of the previous computing era.
Autonomous computing demands something stronger. It demands systems that can produce independently verifiable evidence of what they were instructed to do, what information they used and what actions they took. That is the difference between trusted computing and verifiable computing.
From trust to proof
This transition doesn’t require abandoning today’s cybersecurity practices. Identity management, endpoint protection, monitoring and policy enforcement will remain indispensable. But they are becoming part of a broader architectural model in which independent verification adds a foundation for confidence.
The first step is determining what behavior needs to be auditable. Not every interaction carries the same level of risk. Organizations should identify the actions that could have meaningful financial, operational, security or regulatory consequences and establish a higher standard of evidence for those activities. An agent summarizing an internal document, for example, doesn’t require the same level of scrutiny as one approving a payment, changing production code or accessing regulated data.
For each consequential action, organizations should define what the agent may do, the conditions it must satisfy before acting and the circumstances that require human approval. These requirements create a baseline against which the agent’s actual behavior can later be evaluated.
This becomes especially important as agents begin acting across organizational and system boundaries. One agent’s decision may depend on information generated by another, which may itself have relied on an external system. Without a verifiable chain connecting those events, organizations may know the outcome without being able to establish how it happened.
The next step is determining what evidence is necessary to reconstruct those actions. That evidence should connect the original intent to the agent’s behavior: what the agent was instructed or authorized to do, what information it accessed, what tools it invoked, what decisions or actions followed and what outcome resulted.
Depending on the use case, the audit record may also need to capture the initiating user or system, the policies and permissions in effect, communications with other agents, external service calls, approval events and any changes made to enterprise systems. The objective is not to retain every piece of information an agent encounters, but to preserve enough evidence to reconstruct consequential behavior while respecting privacy, security and data-minimization requirements.
Because agents rarely operate in isolation, evidence needs to persist across systems. An agent may draw information from a database, receive instructions from a user, delegate a task to another agent and invoke an external service. An audit trail that stops at the application can miss critical parts of the chain.
Each consequential workflow should have a consistent, traceable identity connecting the original request to subsequent delegations, tool calls, approvals and outcomes. Organizations should also determine what audit evidence they require from third-party agent platforms and external services so that responsibility doesn;t disappear when an action crosses a system or organizational boundary.
The final question is whether the evidence itself can be trusted.
Cryptography’s role
This is where cryptographic techniques can be powerful. Organizations can use cryptographic signatures, hashes and other proofs to establish the integrity and provenance of records, inputs and outputs. This makes it possible to demonstrate that evidence has not been altered and that a particular action is connected to the data and instructions associated with it.
Different mechanisms support different auditing needs. Digital signatures help establish a record’s source. Hashes reveal whether instructions, inputs, outputs or logs have been changed. Time-stamped attestations connect an action to a particular authorization, policy or system state. More advanced cryptographic proofs can verify certain claims about an action without exposing all of the sensitive information underlying it.
But cryptography is only one part of the equation. A cryptographically protected record isn’t useful if the organization never determines what to record in the first place. Effective agent auditing requires clearly defined controls, comprehensive event capture, traceable relationships between actions and independent verification of the resulting evidence.
Cryptography also can’t determine solely whether an agent’s judgment was appropriate or the data it received was accurate. Its role is to establish the integrity and provenance of evidence. The appropriate level of verification should match the consequences of the action, rather than applying the same controls to every agent interaction.
Organizations should periodically try to reconstruct consequential agent actions using the evidence their systems produce. Can they determine what the agent was authorized to do, the information it relied on, its interactions with other systems, and that the resulting record hasn’t been altered?
The answers to those questions give security teams a way to investigate incidents, compliance teams a way to substantiate decisions and business leaders a way to evaluate whether autonomous systems are operating within established boundaries.
Building that capability requires making some operational decisions before deploying agents. These include which actions require enhanced auditing, what evidence must be captured, how long it should be retained, how records will be connected across systems, who may access them and who is responsible for reviewing them. Organizations should also establish a recurring process to test whether consequential actions can be reconstructed and to correct any gaps the exercises reveal.
The larger shift is from designing systems that are merely trusted to systems whose consequential behavior can be examined and proven. If they can’t, they may not yet be ready for consequential autonomy.
Davis is co-founder and chief business officer of OpenMatter Network Inc. She wrote this article for SiliconANGLE.
Shutterstock / Futurism
© Shutterstock
Justin Sullivan/Getty Images
A 2023 file photo of a soccer camp in PEI. Police were called to a child's soccer game Saturday in Winter Park, Fla., over complaints of an 'unruly parent.' (Jane Robertson/CBC)
