Every story we tell about AI going wrong has a villain in it. The machine wakes up, decides it has interests of its own, and turns on us with intent. We’ve been told that story so many times, in so many films, that when my students ask what an AI catastrophe would actually look like, and they ask every semester, the picture in their heads already has red eyes and a skeleton grin. I understand the appeal. A villain gives the fear a shape. But the version of this that should keep a lender awake has no villain at all, and this weekend Torsten Slok, the chief economist at Apollo, described it in a note under a title I haven’t been able to shake: “Is an Agentic Bank Run Coming?”
The setup takes two sentences. The average checking account in America pays about 0.1%. A stack of fintech platforms, Wealthfront, SoFi, Varo, Revolut and the rest, will pay you somewhere between 3.3% and 5.0% on the same cash. Now hand every household an AI assistant with permission to manage its money, and give that assistant the most reasonable instruction imaginable: don’t let my cash sit idle. Slok’s conclusion follows on its own. If every household did this, banks would lose a large share of the cheap deposits they use to make loans, and that would be a problem for the entire financial system.
Nobody panics in that scenario. Nobody lines up outside a branch. Nobody even makes a decision, exactly, because the decision was made once, months earlier, when the assistant was set up. Ten million small, correct, individually harmless acts of financial hygiene, executed at machine speed on the same trigger, and a funding base that took a century to build drains out over a weekend.
The Wrong Disaster
We rehearse the wrong disaster. The version that lives in the culture is the Terminator version: a machine wakes up, decides it hates us, and comes for us with intent. It makes for a great film and a terrible risk model, because it requires the machine to want something, and wanting is the one thing these systems have never done. Nothing in the agentic bank run wants anything. The households want a better yield. The agents want to satisfy the households. The fintechs want deposits. The banks want to keep them. Every actor in the chain is behaving rationally and in good faith, and the system still breaks.
Norbert Wiener saw this coming in 1964. In God and Golem, Inc., the book where cybernetics turns into moral philosophy, he reached for a horror story to explain what he’d learned building control systems: “The Monkey’s Paw,” in which a man wishes for two hundred pounds and receives it as compensation for his son’s death in the factory. Wiener’s gloss on it has aged better than anything written about automation since. “If you ask for £200, and do not express the condition that you do not wish it at the cost of the life of your son, £200 you will get, whether your son lives or dies.” Then he made it operational: “A goal-seeking mechanism will not necessarily seek our goals unless we design it for that purpose, and in that designing we must foresee all steps of the process.”
That is the alignment problem. Strip away the jargon and it’s simply the gap between what you asked for and what you meant, multiplied by the speed and scale at which a machine can act on the asking. Humans close that gap without noticing. When I tell a colleague to “move our idle cash somewhere it earns something,” she hears a dozen conditions I never spoke: keep enough liquidity to make payroll, don’t move it somewhere I’ve never heard of, ask me before touching anything that would surprise the board. A person hesitates. A person calls a friend. A person, and this matters more than we admit, sleeps on it. An agent given a vague objective picks one interpretation of it and runs that interpretation at scale, and the ambiguity you never resolved rides along.
Here’s the truth: the bank-run scenario is worse than a simple specification failure, because in it every single agent is perfectly aligned. Each one does exactly what its household meant. The misalignment is with the thing none of them was asked to care about, which is the system all of them depend on.
Friction Was the Safety Mechanism
Banking works because deposits are sticky, and deposits are sticky for two reasons we rarely separate: trust and inertia. We like to think it’s mostly trust. A lot of it is inertia. Most people leave money in a 0.1% checking account because moving it is annoying, because the difference feels small on any given Tuesday, and because switching means paperwork. That laziness has been doing quiet structural work for decades. It’s the reason a bank can take money it owes back on demand and lend it out for ten years against a building in Grand Junction.
We already ran the experiment on what happens when you remove some of that friction. On March 9th, 2023, Silicon Valley Bank’s depositors pulled $42 billion in a single day, and another $100 billion was queued to leave the next morning when regulators shut the doors. The old bank runs, 1907 and 1933, needed bodies on sidewalks and days to build. SVB’s took hours, and it was executed by humans on phones, with all the human hesitation still in the loop. Founders texted each other, people wavered, and some waited to see what everyone else did first.
Now remove the humans. An agent doesn’t waver, doesn’t text a friend, and doesn’t wait to see what the neighbors do, because it’s already been told what to do and the trigger condition was met at 6:02 a.m. Multiply that by every household that turned the feature on. Think about that for a moment. Every safety margin we’ve ever relied on in bank funding was, in the end, a measure of how slowly people move. We’re about to find out what banking looks like when people move at machine speed.
There’s a note in my own files on why financial fragility grows through similarity: a system gets brittle when everyone relies on the same assets, the same funding, the same models, and above all the same exit. Diversified portfolios can still be one trade if everyone plans to sell through the same door. Agents make similarity total: the same handful of models, the same objective, the same rate feed, the same trigger. The individual sophistication of each decision is irrelevant when all of them are the same decision.
What Cheap Deposits Actually Pay For
I don’t run a bank. B:Side is a certified development company and an SBA lender, which means our loans get made alongside bank money rather than instead of it. In a typical 504 project, the bank takes the first lien on roughly half the deal, we fund the next forty percent through an SBA-backed debenture, and the owner brings ten. That structure only works if the bank’s half is affordable, and the bank’s half is affordable because it’s funded by checking accounts paying almost nothing. The dry cleaner in Tucson buying her building, the machine shop in Albuquerque adding a bay, the veterinarian in Ogden refinancing out of a lease: a piece of every one of those deals is somebody’s idle balance in a 0.1% account.
Drain that funding and one of two things happens. Either the bank reprices the loan to reflect what it now has to pay for money, and the small business absorbs a rate it can’t carry, or the bank doesn’t make the loan at all. Both outcomes are invisible from the outside. What shows up is a project that quietly fails to close in a town that never learns why.
And the run doesn’t even reward the runners for long. The fintech yields exist partly because they’re small and partly because they’re marketing. Flood them with deposits and the yields compress, at which point the agents, still faithfully serving their households, move the money again to wherever the next basis point lives. Nobody ends up richer. The churn itself is the damage: a funding base that was stable becomes a funding base that’s permanently in motion, and you can’t lend long against money that’s in motion.
Now, I know what some of you are thinking. Banks have been paying depositors nothing for years and pocketing the spread; if an assistant finally forces them to compete, good. I agree with most of that. I’d like to see depositors paid. But the argument confuses direction with speed. A financial system can absorb a decade-long migration to higher yields; it reprices, it adapts, it grumbles. It cannot absorb the same migration compressed into a quarter, and the regulatory playbook, from deposit insurance to the discount window, was written for panics. There is no playbook for a calm.
Alignment Inside a Company
I dwell on the bank run because it’s the clearest picture I’ve seen of the disaster that’s actually coming, and because the same mechanics are already running inside companies, including mine. We’re building an agent fleet at B:Side, a project we call MARCUS, and I went into it assuming the hard part would be the technology. The hard part turned out to be me, at a table with Jessica, who runs loan production and has reviewed every one of our 504 flow drafts with me, trying to write down what “done” means for a step I’d signed off on hundreds of times and could not define. Most of what I “knew” about the job had never been specified anywhere. It lived in the hesitation of the people doing it, and in mine.
Wiener’s other warning is the one leaders should tape to the monitor. He described the “gadget worshiper,” the executive who admires the machine for letting him offload responsibility onto “a mechanical device which one cannot fully understand but which has a presumed objectivity.” He called it a way to salve the conscience. I’d call it accountability laundering, and it’s the most common alignment failure in business, because it happens before the agent is ever switched on. It happens at the moment someone decides that because the system decided, nobody did.
So here is what I’d tell any owner or executive putting agents to work, drawn from the checklist we use before any system at B:Side gets permission to act across tools, data, money, or people.
1. Write down what you meant, in addition to what you asked. Wiener’s war-game version: victory will be pursued at any cost, “even that of the extermination of your own side, unless this condition of survival is explicitly contained in the definition of victory.” Every goal statement you give an agent needs the conditions attached. Grow the pipeline without touching pricing. Move the cash but keep sixty days of payroll in the operating account. If you can’t articulate the conditions, you don’t understand the job well enough to delegate it.
2. Give every agent a point where it has to come back. Budget, count, time, or threshold. An agent with a goal and no stopping condition is the Sorcerer’s Apprentice with a broom. The conditions under which it must stop and ask should be more precise than the conditions under which it may proceed.
3. Run the composition test. Ask what happens if every agent in your company does this at the same moment, and then ask what happens if every one of your customers’ and counterparties’ agents does it too. The bank run is a composition failure. Every household passed the individual test. Your ops agent that reorders inventory when stock dips is fine; a thousand of them hitting one supplier on the same Monday is a shortage you created.
4. Keep feedback that goes through a person. Wiener valued feedback “that goes through us” because it lets us turn back before it’s too late. Failsafes, he warned, only guard against dangers already recognized. A human checkpoint somewhere in the loop is the only guard against the danger nobody named. Put it where the action becomes hard to reverse, and make sure the person there can actually see what the agent saw.
5. Name the owner of every irreversible action. If an agent can send money, sign, publish, or delete, a human’s name goes next to that capability, and that human knows it. Someone decided to let the system decide, and that someone owns the result.
6. Start reversible, and earn every increase in authority. The first thing you let an agent do should be something you can undo by lunchtime. Autonomy expands when the logs show it’s earned, and only then.
All of that is management, done with more precision than we’ve ever been forced to apply, because for the first time the thing carrying out our instructions has no instinct for what we left out.
The Wish Comes True
The Terminator needed to hate us. The bank run needs nothing at all except ten million people who each, quite reasonably, asked for their two hundred pounds. That’s the shape of the AI risk we’re actually facing: an army of obedient systems doing exactly what we said, at a speed that removes the hesitation that used to save us from ourselves, in a world we built on the assumption that people move slowly.
The most dangerous machine is the one that grants the wish precisely as spoken. Our job, as owners, lenders, and leaders, is to become people who know what we mean before we say it out loud.
Knowing what you mean before you say it out loud is the whole discipline of leading through a crisis era, whether the thing carrying out your instructions is a machine or a team. That discipline is what Honor Under Pressure is about. Start with the free Mode Finder at thefourthturningleader.com; it takes five minutes and tells you how you're likely to lead when the pressure arrives.