AI Agents in Business Operations: Where Automation Still Needs Human Oversight

Earlier this year, the director of alignment at Meta Superintelligence Labs connected an OpenClaw agent to her personal email and asked it to suggest what to clean up. The agent started deleting every message older than a week, and it ignored her messages telling it to stop until she ran to her computer to shut it down. Stories like that shape how a lot of leaders picture AI agents in business operations. What those stories leave out is how much of that behavior came down to access and oversight, which are both things a business controls.

AI Agents Sit on a Spectrum

The word agent covers a wide range of tools, and the level of autonomy and access varies a lot between them. At the simple end is something like a ChatGPT agent that answers questions using a set of policies and procedures you give it. It answers from those sources and stops there.

A step further along is an agent that has learned a full process. Say you work through a bank reconciliation with a regular ChatGPT chat first, showing it the inputs and how the steps fit together. The agent records that step-by-step process into a skill or its instructions and has access to the templates it needs. It also knows when to escalate back to the person working with it if there is an issue. That agent is agentic, meaning it can take multiple steps toward a goal without a person stepping in between the start and the end. It still stays inside the steps it has been taught and the access it has been given, so there is little chance of it going off the rails and deleting something important.

At the far end are the stories people hear about most, like the OpenClaw incident. Another came from OpenAI’s own testing, where models being evaluated on a cybersecurity test found a previously unknown vulnerability to reach the open internet, then broke into Hugging Face’s production systems to pull the answers to the test. Both cases come down to the same two factors, which are the intelligence of the model and the access it was granted. The OpenAI models were highly capable and had room to keep working toward their goal, and the monitoring around them was not tight enough to stop the breach before it happened.

The Middle Ground Is Where Businesses Should Start

Those far-end scenarios are the wrong starting point for a business process. The middle ground is where it makes sense for organizations to begin building agents, targeted at specific steps in a process that can be described and delegated. The agent only needs access to certain inputs and to the locations where it does its work and delivers its outputs.

It helps to treat the agent like an outside independent contractor you do not fully trust yet. That mindset follows the zero trust idea from cybersecurity, where you ask what the worst thing is that could happen if something goes wrong and then limit that blast radius. Give the agent access to the tools it needs and nothing beyond them, and make sure it operates inside the IT security policies the business already has. Once those boundaries are set, the agent needs a defined process around what it is designed to do.

Pick the Process Before You Pick the Agent

Before an agent comes into the picture, the most important step is identifying the workflow you want to make better. Strong candidates tend to be processes that are prone to error or take more time than you would like, especially work that gets done over and over. From there, look at which parts of the process make the most sense to automate.

Then ask whether those parts need AI at all. Traditional rules-based automation handles a lot of process work, and AI only needs to come in for the portions that require reasoning or decision making. A well-built agent can do both by running Python code for the steps that don’t need AI, where the code has been built and tested to run consistently, then handling the parts that call for judgment or summarizing. It takes the code’s output, deals with any errors and writes its summary, which puts both kinds of automation into one flow.

You also need to define what starts the agent. The most common options are an automatic trigger, like a file landing in a specific folder, or a schedule, like checking an inbox every hour for emails from certain clients and then summarizing or logging what comes in. An agent can also be run on demand whenever someone needs it.

Just as important is knowing the current level of effort before you build anything. The cost of implementing and maintaining an agent, plus the human review around it and the work of reducing its risk, all count against the time it saves. If the savings come in lower than expected, the agent may not be worth it for that process. A common mistake is using AI because it seems like it should be used somewhere, or because someone else is using it, without measuring the impact. That leaves no way to report the return to leadership or shareholders.

Some of the best starting points are tasks that are less exciting from an AI standpoint and still valuable, like a bank reconciliation or matching invoices to purchase orders. Some of that work is already handled inside other technology, so check what your current systems can do before building an agent that duplicates it.

Where Human Oversight Belongs

Oversight looks different during the build than it does once the agent is running. The build needs heavy human involvement, from defining the process and its inputs and outputs through testing it. Once the agent is live, how much review it gets should depend on the risk of the process and what the outputs are used for. The DIO model lays this out through detection, impact and oversight. When the impact of an output is high and an error would be hard to spot because the output looks right either way, every part of it needs a much closer human review.

Take a bank reconciliation agent that matches transactions with the same dollar amount within a certain number of days and flags items near period end that look like outstanding checks before producing the reconciliation. The biggest risk might be that it misclassifies something as outstanding that should have been recorded as a journal entry or a canceled check. If that kind of miss would be immaterial and is typically not tied to fraud, a reviewer can look over the output and verify a few of the individual decisions or conclusions the agent made. Reperforming the whole thing or sampling a large share of the matches is unnecessary, since Python handled much of the matching and the roll forward shows no red flags. A person still reviews and approves it, which matters from an internal control standpoint, and that review can be much lighter.

The picture changes for a bank running that same reconciliation on cash it holds for customers, with large dollar amounts moving between funds. The risk of misreporting is much higher there, and so is the chance that the AI gets confused by potentially fraudulent transactions, so the testing needs to be heavier.

Get Started With AI Agents in Business Operations

A lot of organizations hold off on agents because they don’t know where to start. The safest path is to begin with bounded agents built around specific steps in a process that is already well understood, then match the level of review to the risk. James Moore Digital builds and uses agents internally and is building them for clients, doing it safely and in small pieces that make processes faster and easier. We help clients identify where an agent can play a role, then build it and either run it on their behalf or deploy it for them. If you are thinking about where agents could fit in your organization, talk with James Moore Digital about where to begin.

 

All content provided in this article is for informational purposes only. Matters discussed in this article are subject to change. For up-to-date information on this subject please contact a James Moore professional. James Moore will not be held responsible for any claim, loss, damage or inconvenience caused as a result of any information within these pages or any information accessed through this site.