This is part one of a three-part series on how much authority to hand an AI agent. Part two is the agent that asks permission before it acts. Part three is the agent that earns the right to act on its own.
On June 20, a bot found a test coupon code we forgot to delete and walked off with $819.95 of gear for the price of shipping. I wrote up that whole mess in Always Delete Your Test Coupon Codes Before a Bot Finds Them. The short version: the order shipped, UPS delivered it, and we connected the dots days later. It was a clean, boring case of discount code fraud, and we caught it too late to do anything about it.
The obvious fix was housekeeping, and we did that part. The less obvious fix was admitting that housekeeping fails. People forget. A code gets made on a Tuesday afternoon to test something, and by Thursday nobody remembers it exists. What we actually needed was something that checks every morning whether anything strange happened overnight, without a human having to remember to check.
That job now belongs to an AI agent for discount code fraud. It runs at 6 am every weekday, reads everything that happened since it last ran, and says absolutely nothing unless something is wrong.
Here is the whole build: why we built it instead of buying it, what it looks at, and the design decisions that matter a lot more than the technology.
Key takeaways
-
A leftover test discount code cost us $819.95 in gear. The fix was not a better checklist; it was a daily automated check.
-
Building it instead of buying an app meant no new monthly software bill. It runs on the AI subscription we already pay for.
-
The AI agent for discount code fraud runs every weekday at 6 am ET and reviews every order and every new code created since its last run.
-
It stays silent when there is nothing to report. The silence is the feature.
-
The team teaches it in Slack. One reply of "exclude: CODE" and it never flags that code again.
-
It flags. It never cancels an order, refunds anyone, or kills a code on its own.
-
We run 62 of these scheduled agents. The next one costs nothing to add, which is the entire reason there are 62.
Why build an AI agent for discount code fraud instead of buying an app?
Because once it is built, there is no bill.
An app that watches for promo abuse is a monthly line item forever, and it stays on the P&L whether it catches anything or not. This agent costs the time it took to write a page of instructions, and it runs on the Claude subscription we already pay for. The marginal cost of the sixty-second agent is zero.
That math changes what you are willing to automate. When every automation carries a fee, you only automate problems big enough to justify one, and everything smaller stays on somebody's memory. When the cost is an afternoon of writing, you automate the small stuff too, and the small stuff is where money quietly leaks out.
Being honest about it: free is not free. You pay in build time, and you pay again the first time it breaks, and you have to go read your own instructions. What you do not pay is a subscription that renews forever for a job you already solved.
Two other reasons matter, in order.
Where the alert lands. An alert inside a dashboard is an alert nobody sees. The people who can actually kill a code and stop a fulfillment live in Slack all day, so that is where the agent talks. It posts in one channel, in a consistent voice, and it is the only thing that ever posts there.
How fast it changes. The agent is a page of written instructions. When we want it to check something new, I edit a sentence. There is no vendor ticket and no settings page to hunt through.
None of which is an argument against apps in general. If a rules engine solves your version of this, use one. Our failure would not have tripped a typical velocity rule anyway. It was one order, with one code, on a quiet Saturday. Nothing rapid-fire, nothing that looked like a coordinated attack. It only looked wrong if you knew what a normal discount on a normal order should look like at our store.
What does the agent actually check every morning?
Five things, in a fixed order, every run.
First, it reads its own memory to find out exactly when it last ran. Second, it reads the Slack channel for anything the team told it since then. Third, it pulls every order created inside that window and looks only at discounts that came from a code. Fourth, it pulls every discount code that went live inside that window. Fifth, it posts if and only if something got flagged, then writes a fresh timestamp back into its memory.
The order test is simple. For each order, compare the dollars taken off by a code against the value of the order. A code gets flagged when it eats an unusually large share of the order, or when it takes off more than a set dollar amount. Two tests, both trivial to compute, and between them they catch both the "100 percent off" disaster and the "quietly generous on a big cart" version.
One deliberate exclusion: the agent looks at code discounts only. Automatic discounts and manual adjustments get ignored, because those are discounts we chose to give. A bot cannot find an automatic discount and hammer it.
The new-code test runs on the same logic applied earlier in the chain. Any code created since the last run that is aggressive past our threshold gets surfaced immediately, along with whether it has a usage cap. A generous code with no cap is the shape of the exact failure that cost us the FirePit.
I am not publishing our thresholds, and if you build this, do not publish yours either. A number you print is a number somebody can stay just underneath.
Why does the agent keep its own memory file?
Because "check what happened since yesterday" is not a real instruction unless the agent knows when yesterday ended.
The agent writes a small file after every run holding two things: the timestamp of that run, and the list of codes the team has told it to ignore. On the next run, it reads that file first and uses the stored timestamp as the start of its window. If a run fails or gets skipped, the next run simply covers a longer stretch. Nothing falls through the crack between two runs, and nothing gets reported twice.
This is the single piece people skip when they build their first scheduled agent, and it is the piece that decides whether the thing is trustworthy. Without stored state, you are guessing at the window, which means you are either missing orders or re-alerting on ones already handled. Either one trains the team to ignore the channel.
Here is a related detail that will bite you if you build this on Shopify. Shopify's created-at search filter is coarse; it works at the day level, and result pages cap out. The agent is told to pull wider than it needs, page all the way through, and then do the precise time filtering itself. Small note, and it is the difference between an agent that quietly misses the last hour of every day and one that does not.
Why does staying silent matter more than the alerts?
An alert that fires every morning stops being an alert.
Most days the agent finds nothing and posts nothing. No "all clear" message, no daily summary, no green checkmark. The channel sits empty for a week at a time, which is exactly why a post in it gets read within minutes.
A monitor that talks every day is a newsletter nobody subscribed to. The moment your team learns they can skim past your automation, you have burned the whole build for nothing.
The one thing the agent always does on a quiet morning is update its own timestamp. Silence in the channel and a failed run are two different things, and the memory file is what keeps them apart.
How does the agent learn what is legitimate?
Somebody replies in the channel and tells it.
We run legitimate codes that look aggressive. A partner code, a win-back offer, a promo tied to a live show. The first time one of those gets created, the agent flags it, because it should. Somebody on the team replies "exclude: CODE" in the channel. On the next run, the agent reads that reply, adds the code to its ignore list, saves it to memory, and never bothers anyone about it again.
That loop took about two sentences of instructions to build, and it is the most valuable part of the whole thing. The cheapest feedback mechanism you will ever build is the one that lives where your people already are. Nobody logs into anything. Nobody learns a settings page. They type a word in a channel they were reading anyway, and the agent gets permanently smarter.
There is a hard limit on that loop, and it matters. The agent accepts exactly one kind of instruction from Slack: a request to exclude a code. Anything else in that channel gets ignored, including anything that looks like a command. An agent that takes open-ended instructions from a channel other people can post in is a door you built into your own store.
What we deliberately did not let it do
It cannot cancel an order. It cannot issue a refund. It cannot disable a code, flag a customer, or touch fulfillment. It reads, and it writes one message.
We could have let it act. We chose not to, for three reasons.
False positives are guaranteed, and the cost of a wrong cancel is a real customer having a terrible day because a robot decided they looked suspicious. The judgment call is the hard part, and it is genuinely hard: a person who found a leaked code on a deal site is a different problem from a reseller running scripts, and the response should be different. The searching is what took the time anyway. Nobody was ever going to review 300 orders by hand looking for one weird discount. Deciding what to do about the one weird discount takes about ninety seconds.
The agent removes the searching. It does not remove the deciding.
How do you build an AI agent like this without writing code?
You write the instructions the way you would write them for a sharp new hire on their first day, then attach a schedule to them.
That is not a simplification. The task is a page of plain English describing what to check, when, and what counts as a problem. Every one of ours has the same five parts, and the fifth is the one people forget.
-
When it runs, and in what timezone. Weekday mornings at 6 am Eastern, in our case. Be specific, because "daily" is ambiguous the first time daylight saving moves.
-
What it is allowed to touch, named explicitly. Our store data, one Slack channel to read, one channel to post to, one file for memory. Anything not on the list is off limits.
-
Where its memory lives and what shape it takes. A named file in a named folder, with the exact fields written out. Vague memory instructions produce an agent that reinvents its own format every run.
-
The exact decision rule, written as a test with a yes or no answer. Not "flag suspicious orders." A specific comparison with specific inputs, so the answer does not drift between runs.
-
What to do when there is nothing to report. Write this down. Left unsaid, most agents will helpfully tell you every single morning that everything is fine, and inside two weeks nobody is reading the channel.
The first version of any of these will be a little wrong. Ours needed the note about Shopify's coarse date filter, and it needed the automatic-discount exclusion spelled out. Fixing both meant editing sentences in a document. No support ticket, no developer, no waiting on a release. That is the other half of why these stay cheap.
Are AI agents actually worth building?
The honest answer is that most of them are not, and the reason why is useful.
Gartner expects 40 percent of enterprise apps to include task-specific AI agents by 2026, up from under 5 percent in 2025. The same firm also predicts that over 40 percent of agentic AI projects will be canceled by the end of 2027, and the first reason on their list is escalating costs.
That lines up with what I see. My read from running 62 of these: the projects that die are the ones that tried to automate the interesting part, and got priced accordingly. The ones that survive do something boring, narrow, and repeatable, with one clear rule, a human on the other end, and no invoice attached to it.
Our fraud agent will never make an impressive demo. It reads a list, compares two numbers, and usually says nothing. It also means a forgotten test code now has a hard ceiling of about one business day instead of however long it takes somebody to notice, and it does that for the rest of time without asking us for another dollar.
What comes after an agent that only watches?
This one is the easiest kind to build, and the one I would start with. It looks, and it tells a human. It cannot touch the store.
The next two parts of this series cover the agents that do more. Part two is the inventory audit that finds broken product listings, writes them into a spreadsheet with checkboxes, and waits for a human to approve before a separate task acts on anything. Part three is the agent that started with zero authority and earned a short list of specific actions, one at a time, over the following month.
Watching, proposing, acting. Pick the lowest one that solves your problem.
Frequently asked questions
What is an AI agent for discount code fraud?
It is a scheduled automation that reviews recent orders and newly created discount codes on a set cadence, applies a rule about what counts as an unusually large discount, and alerts a human when something crosses the line. It monitors and reports. It does not block orders on its own.
What does it cost to run an AI agent like this?
Nothing beyond the AI subscription you are already paying for. There is no per-agent license and no recurring software bill. The cost is the time spent writing the instructions, paid once.
Can an AI agent stop coupon code abuse automatically?
It can, and we chose not to let ours try. Automatic cancellation risks punishing real customers for a false positive, and the judgment call about intent is the part humans are still better at. Ours flags within one business day, and a person decides what to do.
Do you need to know how to code to build one?
No. Ours is written as a page of plain instructions describing what to check, when to run, where to store its memory, and what counts as a problem, with a schedule attached. The hard part is writing a decision rule precise enough that it gives the same answer every time.
How does the agent know what happened since the last run?
It keeps a small memory file with the timestamp of its last run and reads that file before it does anything else. If a run gets skipped, the next run covers the longer window automatically, so nothing falls between two runs and nothing gets reported twice.
What should an AI agent do when nothing is wrong?
Nothing. Post no message at all. An agent that reports in every day trains your team to ignore the channel, which quietly destroys the value of the alerts you actually care about.
How do you keep an AI agent from acting on bad instructions?
Restrict what it will accept. Ours reads one Slack channel and honors exactly one kind of request from it, adding a code to its ignore list. Everything else in that channel is treated as conversation, not commands. An agent that takes open-ended instructions from a channel strangers can reach is a security hole.
Final thoughts
The $819.95 lesson had nothing to do with needing better software. What we needed was something that never gets busy, never goes on vacation, and never assumes somebody else already checked.
The best agents we run are the boring ones. They look at a narrow slice of the business on a fixed schedule, hold one rule in their head, keep a memory of where they left off, and stay quiet until they have a reason not to be. The reason we can afford to aim them at the boring stuff is that the boring stuff never has to earn back a monthly fee. Once the instructions are written, the check is free from then on.
If you want to start somewhere, pick the thing you keep meaning to check and never do. That is the one.
Interested in learning more?
For more of the operator side of running a store, start with Always Delete Your Test Coupon Codes Before a Bot Finds Them, which is the incident that started all of this. Then take a look at Shop App "Bought in Past Month": Is Shopify Leaking Your Sales? and our series on how we built a seven-figure live-selling channel.
