How to Supervise a Team of AI Agents

Yaniv ZadokOrganizational consultant to large companies and industrial plants · Automation and AI specialist

At RIPA, dozens of AI agents now work alongside the human team. They write, summarize, check, and prepare drafts.

I've spent years guiding large companies and industrial plants in managing people and processes. Even so, the agents taught me a new lesson. We found out very quickly that the hard question isn't "what can the agent do?" The hard question is: how do I know what it actually did?

🧭 An agent isn't ordinary software

Ordinary software does the same thing every time. An agent gets a task, decides on its own how to carry it out, and reports back. And sometimes the report sounds great, and the work wasn't done. Not out of malice. It simply checked something close by, and not the thing itself.

Anyone who has ever managed a team of people knows this from the factory floor: "I took care of it" isn't the same as "it's done." The difference is that agents work faster and at higher volume, so the mistakes pile up faster too.

Big companies solve this with a product. A small business doesn't need to wait for one. Over time, we settled on five habits. Each one is simple, and together they do most of the work.

1️⃣ A pulse line at the end of every answer

Every agent we run signs every answer with one fixed line: who I am, what I did, when, and how much of my working capacity is already used up.

That sounds technical, but it's exactly like an employee who says at the end of a task, "Done, moving on to the next thing, and I'm getting a bit tired." In one second I know who's talking, whether the information is fresh, and whether it's time to refresh the agent. Because an agent whose memory is filling up works less well, just like a person at the end of a double shift.

2️⃣ An open-items table that counts, not remembers

Every round of work starts with one table: what's open, what came in since last time, and what got closed. And the table is counted from scratch every time. Not copied from yesterday.

Why does that matter? Because a number passed from hand to hand goes stale. "Three tasks left," written yesterday, is no longer true today. When you count again, you see right away whether the team is closing more than is coming in, or whether the pile is just growing.

3️⃣ Every agent gets an inbox, and silence is not a status

Every agent we run has one inbox where its tasks arrive. And every task gets a reply: first "got it," and then "done," or "stuck, waiting on...". With a name and a reason.

Silence is not a status.

If I haven't heard anything, I don't assume everything is fine. I assume I don't know. That's the difference between a team you manage and a team you hope for the best about.

4️⃣ What can't be undone, a human approves

Our agents do a lot on their own, and that's fine. But any action with no way back, like sending something to a client, deleting, paying, or changing something that already works, waits for a human's approval. A written approval, in words. Not something the agent inferred from context.

It isn't a lack of trust. It's the same logic that has an orderly business put two signatures on a large check.

And there's another benefit here I didn't expect. When the boundary is clear, you can let the agent work freely on everything else. I don't need to go over every draft and every summary. I need to be available exactly at the moments when there's something to lose. That saves me time instead of adding work.

5️⃣ A green check without a reason is a false check

This is the hardest habit to instill. An agent reports "The site is up ✅." But what did it check? Sometimes it turns out it checked that the server responds, not that the page the client sees actually works. The check passed. The goal didn't.

So our rule is: every green check comes with its reason. Not "it works," but "I opened the link on my phone, filled in the form, and the message arrived." And a check has to touch the real thing, not something standing next to it.

🛠️ How to start tomorrow morning

You don't need to build a system. You can start with three small steps, even with a single agent:

1️⃣ Add one sentence to the agent's instructions: "At the end of every answer, write one line: what you did, when, and what's still open." That's it. From today, you have a log.
2️⃣ Write a short list of what it's not allowed to do on its own. Sending to a client, deleting, paying. Everything on the list goes through you.
3️⃣ Once a week, check one green check yourself. Pick one "done" report and open the real thing. You'll be surprised how much you learn from it.

After a month like that, you'll know more about your agent than most businesses know about theirs.

🏢 Why this applies to your business too

Maybe you're thinking: "I've got one agent answering clients on WhatsApp, this isn't for me." Actually, now is exactly the time. These habits are cheap when you have one agent, and very expensive when you have ten and nobody remembers who does what.

Every business that brings in agents will reach this question. The only question is whether it gets there before the first mistake, or after.

The giants are already building products for this. Google, for example, recently launched a service that detects when an AI agent behaves abnormally. These products will be good, and they're worth following. But a product doesn't replace discipline. Anyone who doesn't know today what their agent did yesterday won't know it with a nice dashboard either.

📌 Three things to take with you

✅ To check this week

Bringing AI agents into your business?

We supervise a team of agents of our own every day. I'd be glad to sit down with you, see what's already running in your business, and build supervision together that fits the size of your business. No expensive product, just habits that work.

Talk to Yaniv 💬Or visit my digital business card

Source: Google Cloud · Agent Anomaly Detection (Google's documentation; the service runs inside its agent platform).
The images in the illustrations were created with AI.