The Black Box Problem With AI Amazon PPC (and the Fences I Use)

Last month an AI changed bids in my Amazon ad account 36 times without asking me. None of those changes surprised me.

That isn’t blind trust. It’s the opposite. I know exactly what it’s allowed to touch on its own, how far it can move anything, and how to put any of it back.

That’s the real black box problem with AI in Amazon PPC. Not that a model makes decisions. That’s the useful part. The problem is not knowing which decisions it can make on its own, how big they can be, and whether you can see them and undo them.

AI is genuinely good at this

I’m not going to pretend otherwise to sound careful. A model reading ninety days of performance across thousands of keywords is doing something a person can’t. It doesn’t get bored at keyword four thousand. It doesn’t have a favorite campaign, or an ad group it stopped opening because the numbers there feel bad. It finds the converting search term buried three levels down, and it finds it again tomorrow, without deciding it has already looked.

I sell my own products on Amazon and I use the AI every day. On the products where I’ve watched it be right, I let it act.

Its failure mode is confidence, not error

People picture AI risk as a calculator that’s occasionally off: a strange number you’d catch. That isn’t how it fails.

A model produces the most plausible answer available to it. When the inputs are good, plausible and correct are the same thing. When the inputs are stale or quietly mislabeled somewhere upstream, they come apart, and the answer is still plausible. It still cites specific numbers. It still reads like careful reasoning.

A wrong AI recommendation doesn’t look like a wrong answer. It looks like a right one.

I’ve watched our own engine produce a bid recommendation that was well argued, arithmetically correct, and computed against the wrong keyword’s numbers. The model wasn’t the broken thing. What fed it was. Better models catch more of this, and I upgrade every time one ships, but a model can’t recover facts the pipeline never kept.

Three failures where the model was never the problem

All three came out of my own account, and none would have shown up on a dashboard. Each time, the software carried on reporting normally after the thing it reported on had stopped being true.

A counter that reads zero looks exactly like nothing happening. Our own monitoring once reported zero held changes on every run for two days while the system was holding changes constantly. Everything was green. Nothing was being measured.

Silence got read as an answer. Amazon’s API doesn’t always reply. A careless system treats the empty space as information: “no suggested bid returned” quietly becomes “Amazon disagrees with your bid.” We shipped exactly that bug, and caught it because we count how often each decision path fires.

A success response isn’t proof anything happened. You switch something off, the API says success, and the spend keeps arriving, because “off” landed one level away from the thing doing the spending. The confirmation was real. It confirmed something else.

Accuracy is the wrong question. Exposure is the right one.

“Our AI is right about ninety-five percent of the time” sounds like an answer. It isn’t the right question.

A model that’s wrong once in twenty is fine when it makes twenty decisions and a person reads all of them. The same model is a liability with ten thousand chances to act every night and nobody required to look. The accuracy didn’t change. The exposure did.

The useful question is how many wrong changes can reach your account before someone notices. That one has numbers. How many changes go out without a person seeing them? How big can each one be? What can never go out on its own? Here are mine.

The fences: what my AI may change on its own

• It’s off until I switch it on, one AI manager at a time. Out of the box, every change waits in a queue for my approval.
• When it’s on, it can move a bid up to 15% up or 25% down by itself. Anything bigger waits for me.
• Some changes always wait, whatever the setting: pausing a keyword, adding a negative, adding new keywords or targets, changing a placement adjustment, changing bids on an automatic-targeting group, and rebalancing a campaign. Sponsored Display changes never go out on their own, on any plan.
• If I undo a change, it stays undone. The AI won’t make the same move again for 30 days.
• Every change writes a row (what changed, from what to what, and the numbers it acted on) and has an undo button.

That’s three boring properties: bounded, reversible, inspectable. None of them is AI. They’re arithmetic, a log and an undo button, and they’re the reason it’s reasonable to let the AI act at all.

The guardrails aren’t a limit on the product. They’re what make the autonomy worth having.

Rules make the obvious calls before the AI sees them

Most decisions in an ad account aren’t judgment calls. A keyword that hasn’t had traffic in two weeks gets a small nudge. A bid that changed a few days ago waits for data at its new price. A keyword that has spent more than a sale can pay for, with no sale, gets cut, and if it keeps going it’s put up for a pause. Rules make those calls, the same way every time, before the model is asked anything.

In the 30 days to November 8, my account had 2,016 bid decisions. Rules made 1,124 of them. The model got the other 892, the ones that needed judgment, and in 708 of those it chose to leave the bid alone.

That last number matters more than it looks. Leaving a bid alone is a decision, and it’s the one a model makes most.

What I let it do, and when I say no

1,025 changes went out of my account in those 30 days. 36 went out on their own, from the two AI managers I’ve switched to auto-apply: my cutting board gel and my hardwax oil, where I’d tested it and it worked well. I approved the other 989. Most were small nudges on keywords that weren’t getting traffic.

I said no 26 times, and 18 of those were cuts. That isn’t the AI being wrong in a normal month. This fall a wholesaler started winning the Buy Box on one of my best sellers and sales slumped. I’m buying back sales velocity, so I’m turning down cuts the numbers would justify on their own.

The AI sees the data. I see the business. The fence is what lets both be right.

An explanation is not an audit record

When your ACoS goes sideways over a quarter, someone has to answer four questions: what changed, when, who changed it, and on what evidence.

A model will happily explain itself, fluently, at any length. That explanation is written after the fact by the same machinery that made the decision, and it’s built to read well, not to be complete. Ask twice and you can get two reasonable answers for one change.

Compare that with a row: rule “High ACoS 30d” fired; it saw 47.2% ACoS, 31 clicks, $84.10 spend, 2 orders; the bid moved from $1.42 to $1.21, a 15% cut inside a $0.35 floor and a $2.00 ceiling; applied; reversible. Nothing in that row was written to persuade anybody. An explanation you can’t reproduce is a story. A row you can reproduce is a record.

Percentage-of-spend pricing is the incentive to watch

This one needs three things at once: the vendor earns more when your ad spend rises, the vendor’s software can raise your ad spend, and nobody is required to look before it does. Any one alone is ordinary. Put all three together and you have software that raises its own revenue and reports the result to the person paying for it. It doesn’t need a villain. The incentive works on its own.

Our pricing is flat and never a percentage of your ad spend. Growth is $69 a month and includes the AI recommendations, with auto-apply when you want it. Professional is $129 and adds the rule engine, which can also run unattended.

Seven questions to ask any AI Amazon PPC vendor

1. List everything your AI can change without asking me, and the size limit on each.
2. Show me the last change it made to my account, and the numbers it made it on.
3. What happens when Amazon’s API doesn’t answer? Does your system know “no” from “no reply”?
4. Can I set a hard ceiling it can’t exceed, and is it applied before the model or after?
5. How do I undo last night?
6. Have you checked your AI against what actually happened? What was the number, and when did you last re-run it?
7. If your own monitoring broke, what would my dashboard look like?

If the answer to the last one is “that wouldn’t happen,” you’ve learned what you need. It happens. It has happened to me.

Where I land

The AI acts. Inside limits I can list, switched on where I’ve watched it be right, with every change written down and one click from undone. That’s not caution about AI. It’s why I can let it work.

If you try it, don’t leave it switched off. Approve its changes for a couple of weeks, read why it made each one, then switch on auto-apply for the products where it earned it. That’s what I did.

Want to see your account first? The free 30-second PPC audit grades it in a minute, and the pricing page has every number on it. No call required.