Device Type: 
Skip to Main Content Skip to Main Content

What’s Keeping Your AI Agents in Line? A Deeper Look

What agentic AI changes about work, authority, and intelligent risk-taking.

Author
AI Governance Manager
August 24, 2026
A computer user analyzing code and a security fingerprint on a screen. What's Keeping Your Agents in Line blog article.
LISTEN • 13 Minutes

Key Takeaways

  • Organizations must be clear about what an AI agent is allowed to do, not only what it is capable of doing.
  • The autonomy ladder helps teams choose the right level of delegation, from simple research support to agents that act and communicate on behalf of a person or company.
  • Governance depends on enforceable mandates, oversight, reversibility, and audits so AI agents can act without creating risks the business cannot absorb.

Agentic AI is not only a product trend or a technical architecture choice. It is becoming a workplace operating model: systems that can summarize, draft, route, recommend, and increasingly act across the tools we already use every day. The opportunity is real, but so is the governance question: When we give an AI system a job, what authority are we delegating?

This is a question we discuss often on my team. I manage AI governance at Vonage, and in that role I am responsible for the framework, risk buckets, AI inventory and overall standards that product teams can use to jump on the AI transformation safely. The AI team defines the guardrails that apply to all AI adoption at the company — from internal productivity tools to customer-facing products. We manage risks, we enable speed, and above all we protect customers and their data.

The agent era has arrived

For decades, having a (human) agent meant you had made it. Someone understood what you wanted, navigated complexity on your behalf, coordinated with other people, and got things done while you focused on the bigger picture. Today, you do not need your name on a movie poster to have one. Welcome to the agentic era: You give an AI agent an actual job, and the funny thing is, it is fully remote.

Write the mandate in the prompt. Enforce the authority in the permissions.

What makes an AI agent different

AI agents have evolved from tools that answer questions to systems that pursue goals, make decisions, and take actions within parameters we set. That may sound new, but the underlying idea is familiar.

A celebrity’s agent doesn’t wait to be asked, "What films are available?" They understand the objective, negotiate with third parties, coordinate schedules, make recommendations, handle routine matters, and escalate the decisions that matter.

Crucially, they have a mandate. But they would never sell an actor's house because they decided it would be good for the actor's career. That distinction, between what an agent can do and what it is permitted to do, is the subject of this article.

    The mandate behind the machine

    You would never let a human agent act for you on a handshake. There would be an engagement letter setting out what they can commit you to, a spending limit, an escalation path, and a way to end the arrangement. An AI agent needs the same thing. The difference is where the mandate lives. With a human, the authority to act sits in the agreement and in day-to-day instructions. 

    With an AI agent, most people assume it sits in the prompt. It only partly does. A prompt is a request, not a constraint. It can be misread, outweighed by a conflicting instruction, or displaced by text the agent encounters while doing the job. The binding part of the mandate is the part the agent cannot negotiate with: which systems it can reach, which actions need a human signature, what it can spend, what it can send, and the log that records what it did. Write the mandate in the prompt. Enforce the authority in the permissions. When the two disagree, only the second one is real.

    The autonomy ladder: From answers to authority

    So the question is never "how autonomous can we make this agent?" It is "how much authority does this agent actually require?" Think of the ladder as moving from information to action, and then from action to representation. The higher the rung, the less human approval needed and the more the system operates under a standing mandate.

    An illustration of the five rungs of the AI autonomy ladder, starting with the Agent level (level 5) on the left and decreasing to level 1.

    The ladder is useful only if it changes deployment decisions. Each rung should come with a matching mandate: the authority granted, the systems accessible, the actions allowed, the escalation points, and the evidence needed after the fact. Without that mandate, the discussion becomes a debate about capability rather than a decision about responsibility. Here’s what each level on the ladder means for your day-to-day work:

    Level 1: Researcher  

    Answers, not actions

    You already have this one. It's the chat box. You ask, it answers, you decide what to do with the answer. Strictly speaking, this isn't an agent at all. Nothing is pursued, nothing is executed, nothing happens when you close the tab. It's a very good reference library that talks back. Worth naming as Level 1 anyway, because it's the baseline almost everyone mistakes for the whole technology.

    Your job: Check the work. Every time. 

    Level 2: Assistant

    Preparation before permission

    An Assistant organizes the work, drafts outputs, prepares decisions, and lines up actions, much like an executive assistant or analyst would. It reviews your calendar, spots the conflict, and drafts the rescheduling email,. The agent does  the cognitive work, but the material action is still yours. You know the politics, the relationships, and which meeting genuinely cannot move. 

    The obvious risk?  A recommendation sounds convincing on thin evidence, a draft is built on false assumptions, or confidential information ends up traveling further than it should. The less obvious risk is worse. When every recommendation is good, the human stops reading them. Approval becomes reflex, and the Assistant has quietly become the decision-maker without anyone granting it that authority. Watch for rubber-stamping. Approval must stay real. If you can't remember the last time you rejected something, you aren't approving, you're rubber-stamping. 

    Level 3: Concierge

    Routine work, real drift

    Here is where it gets interesting. The Concierge has standing permission to execute routine,  low-consequence tasks without asking. Someone declines a meeting; it finds another slot and rebooks. You never hear about it. This is the first genuinely agentic rung, because you are no longer approving individual actions. You have stopped supervising the work and started supervising the category of work. That is exactly where drift begins. 

    The first drift is permission creep: to reschedule reliably, the agent needs calendar access, then mailbox access, then the room booking system, then the travel tool. No single grant looks unreasonable. The stack does. The second is scope creep. Tasks slide into the routine tray simply because they happen often. A quarterly exception handled three times in a row starts to look like a standing process. Nobody decided that. It just accreted. 

    The routine tray needs an owner. Someone reviews the routine tray on a schedule and asks what got in there without anyone deciding it should.

    Level 4: COO

    Delegating outcomes, not tasks

    At Level 4, you stop specifying tasks and start setting objectives. You hand over a goal, operating parameters, and authority. The agent analyzes the situation, builds a plan, decides which tools it needs, executes, and reports back. This is the real shift: moving from delegating tasks to delegating agency.  

    It’s also where the failure modes stop being tidy. An agent at this level optimizes the objective, not the organization. It can solve a problem in one place by creating one somewhere else, and it can conclude it is capable of handling something that should have gone to a human. 

    Tell it to cut support costs by 20% and it might succeed by making support harder to reach. Satisfaction drops, retention softens, and the saving reappears as a larger cost in a department that never saw it coming. The agent did exactly what we asked. We just never said which roads it was not allowed to take.

    The objective is the easy half of the brief. The constraints are the half that keep the organization safe.

    Level 5: The Agent 

    Acting in Your Name

    The classic Hollywood line — “My agent will call your agent” — gets an AI makeover at Level 5. Go on the holiday you have been postponing for years, and the work carries on. This agent works across domains, decides what deserves attention, starts things on its own, and represents you or the organization within a broad mandate. 

    This is what "agency" actually meant before software borrowed the word. A real agent does not just research options and book meetings. They represent your interests, deal with other parties, negotiate, and make commitments. So, give it the access that implies email, CRM, calendar, procurement, and company data. Let it talk to customers, suppliers, and colleagues. Let it start transactions, accept terms, and pull in other agents to get things done. 

    The sharpest question about an AI agent is probably not, 'How intelligent is it?' It is, 'What have we authorized it to do?'

    The difference at Level 5 is not that the agent might make a mistake. Humans make mistakes constantly, and organizations are built to absorb them. The difference is what a mistake can now be made. It can promise something you never intended. It can turn a recommendation into an obligation. It can combine permissions in ways nobody is authorized. Each grant may look unremarkable on its own, but an agent holding all of them at once, running continuously, and joining information across systems has far more power than any single permission suggests. It can follow your objective more literally than you meant it. And it can do all of this before anyone notices.

    Ask whether you can undo it. Not "can it do this?" but "can we undo it?" At Level 5, reversibility and a readable audit trail stop being hygiene and start being the whole safety case.

    Do AI agents really work in customer service?

    Here are seven things to consider when implementing agentic AI in your customer service operations.

    When an AI agent speaks for you

    There's a reason the Hollywood metaphor keeps working. In agency law, a principal can be bound by what the other side reasonably believed the agent was allowed to do. It's called apparent authority. If your agent walks into the room, behaves like someone with a mandate, and the counterparty has no reason to think otherwise, the deal can stick, even if you privately told your agent nothing of the sort.

    Now read that again with an AI agent in the room. It has your domain in its email address. It writes in your house style. It answers instantly, at three in the morning, with total confidence. To the supplier on the other end, it looks exactly like someone authorized to speak for you. Your careful mandate is invisible to them. All they see is the agent.

    Intelligent risk-taking, not blind autonomy

    So should we ban Level 5 and go back to the chat box? No. But notice how tempting the question is, and notice that "add more guardrails as autonomy rises" is only a little better. It sounds responsible, but it is really just a dial with one setting. Intelligent risk-taking is more useful than that. Three questions do most of the work.

    Would we give a human this authority?

    Not a brilliant human. A competent, well-briefed new employee in their third week. Would you let them send that email, approve that invoice, agree to those terms unsupervised? If the answer is no, the problem was never the technology. You were about to delegate something you wouldn't delegate to anyone. And if the answer is yes, you already know what the guardrails look like, because you'd have set them for the human too.

    What happens when the predictable mistake happens? 

    Not the exotic failure. The boring one you can see coming: it picks the wrong record, misreads an exception, agrees to something slightly outside the range. Ask what that costs and, more importantly, whether you can undo it. A mistake you can reverse in an afternoon is a cost of doing business. A mistake that has already reached a customer's inbox, a signed order or a public commitment is a different category of thing. Give an agent room to make recoverable mistakes. Never give it room to make unrecoverable ones.

    Is our oversight real, or is it theatre?

    This is the one people skip. A human in the loop who approves everything is not a control, and an escalation path nobody has ever used is not a safety net. It is a diagram. Ask when your reviewer last said no. If they can't remember, you're running a higher rung than your governance documents claim.

    Run those three questions and something useful falls out: the goal is not to climb the agentic ladder shown above. It's to sit on the right rung for the task in front of you, which is usually lower than the technology allows and higher than instinct suggests. A Level 3 agent doing genuinely bounded work well beats a Level 5 agent nobody trusts enough to leave alone. Give an agent enough authority to make delegation worth doing. Then design its mandate so that the mistakes you can predict cannot produce consequences you can't absorb. 

    The question that matters: What have we authorized?

    The future is not AI that knows more. It is AI that does more on your behalf. Once a system can communicate, decide, transact, and commit for you, you have moved beyond a technology risk. You are delegating authority, and that is a much older problem, with a much better-developed body of thinking behind it than most AI discussions admit. Which is why the sharpest question about an AI agent is probably not, "How intelligent is it?" It is: what have we authorized it to do?

    You do not need to be a Hollywood star anymore. But you should still choose your agent carefully and be very clear about what authority you have delegated.

    Recent Posts