Your AI doesn't need to be smarter to work in analytics. It needs a harness.

Brad Peters

I was half right.

AI on its own still doesn't work in analytics. AI inside a model harness does. The harness is everything around the model that decides what it looks at, what it checks, in what order, and when it stops.

Most of what gets sold as AI analytics right now is a pitch for a smarter model. I think that pitch has it backwards. Here is what I got wrong, what we tried, and what I learned.

We let people chat with their data. Then we read the logs.

When the models got good enough, we did the obvious thing first. We connected AI to real analytical sources and let business users ask whatever they wanted. The industry calls it conversational analytics.

Technically, it worked.

Then we read the logs. Business users didn't know what to ask. They started asking really crazy things.

That was the first lesson, and it stung. An open text box is not a product. It hands the hardest part of the job, knowing which question matters, straight back to the person who came to you for help.

The second failure you have probably seen yourself, if you have piloted a generic tool on unit-level P&L data. Answers that sound confident and are wrong.

Language models hallucinate. They also struggle with basic math, which matters a lot when the question is labor cost per occupied room across 80 properties.

Then there is the failure nobody warns you about. The pilot that works and doesn't matter.

A senior executive at a large national retailer told me about an AI proof of concept his company ran on public data. His verdict:

"My team can do that... while it is faster, it's not incremental."Senior executive, national retail chain

That is the CFO problem in one sentence. Faster is not a business case. Incremental is.

The model was never the problem

Here is what I had wrong all those years. I thought the limit was the AI. It wasn't. The limit was that nobody had told the AI how the business actually runs.

Left open, an AI agent doesn't know what matters, so every metric looks equally interesting. It doesn't know what's already explained, so it rediscovers the same seasonal dip every week. It doesn't know your definitions. Comp stores. Balanced stores. What counts as a bad month.

And it has no reason to stop. Nothing tells it when an answer is good enough, or wrong.

Developers have a name for the fix. They call it a harness. Their analogy: the model is the engine, and the harness is the car built around it. Steering, brakes, a route. The same model produces very different results depending on the harness around it, which is why the word is suddenly everywhere in engineering circles.

For an operator, I put it more simply. The model supplies the horsepower. The harness supplies the judgment. And the judgment has to come from you.

A harness built for operations tells the AI five things:

• What to screen for at every location, every cycle.
• Which thresholds separate normal variance from a real problem.
• What to drill into next when something trips.
• How to interpret what it finds, in your business's own terms.
• What it is not allowed to do. Wander off, invent a metric, or answer a question nobody asked.

A harness makes the AI smaller. On purpose.

Tell an operations leader you want to put their judgment inside an AI model, and the first reaction is usually some version of The Terminator. A machine that learned how you think and now runs your stores without you.

I understand the fear. I also think it has the picture backwards.

A harness does not make the AI bigger. It makes the AI smaller. On purpose.

Engineers are landing in the same place. Vercel's team found its internal data agent got better after they removed 80% of its tools. More options had made it worse.

Here is how I describe ours to every operator I talk to. It's a tight harness. Lots of AI is used, but we do not let AI just sort of run off on its own.

That constraint is where the trust comes from. Every location gets read the same way, against the same thresholds. Every conclusion points back to a specific screen, probe, and rule, which makes it the opposite of black-box AI. And when an operator disagrees with a conclusion, the rule changes, and the fix applies everywhere at once.

Our AI is not smarter than a generic tool. It is more specifically yours.

That is the whole argument.

Nobody can write down what they know. So we record it.

What feels obvious from inside the business is invisible to a model until someone captures it.

At the largest pawn chain in the United States, I kept asking questions like "What is an out of balance store? What does that actually mean?" None of it had ever been formally captured. The CEO told me that if you asked three different people what the death spiral was, their name for a specific way a store goes wrong, you would get three different answers.

Everyone knew. Nobody had written it.

So we don't ask people to write it. We record it. The way I put it to operators: if I took a tape recorder and recorded everything you thought as you looked at your BI reports, we stick that into the system so it could do that on your behalf.

That is close to literal. Our team does the capture. The customer does not configure anything.

At the pawn chain, I spent a week in Houston walking through their Power BI reports with district managers, regional managers, and the COO, across 11 stores. We came home with seven hours of recordings.

We transcribed them. Then we used the transcripts to codify the playbook: screens, thresholds, drill paths, interpretation rules. We got the facts right first, checking outputs against screenshots of the customer's own BI reports, so the numbers matched before anyone was asked to trust them.

Then comes the step that matters most. We show operators a straw man.

If you ask people to spec out what they want, you won't get a clear answer. Once they see something, they can tell you exactly why it's wrong.

That reaction is the judgment you're encoding.

What the harness does at every store

It walks the data the way your best operator would, and it stays inside the lines. The harness screens every location and only digs where it has a reason to.

• Screen every location on a schedule, weekly or every two weeks.
• Spawn an investigation only where a screen trips.
• Probe with 15 to 30 defined analyses per store per period, each with rules for how to interpret it.
• Test 10 to 20 variables at once with decision-tree analysis against a healthier prior period.
• Roll up findings from store to district to region, scored critical, severe, normal or healthy.

Each one is an AI investigation with a defined route, not an open-ended search. Think of it as an intelligent agent walking your data, but not AI let loose.

The report arrives on its own. Nobody logs in, writes a prompt, or builds a query.

And people stay in charge. It's very much human in the loop. Operators review the output, and their corrections feed back into the harness.

What I'd tell your CFO

You are not buying a smarter model. You are buying your own judgment, running at scale.

The model is not the asset. Every vendor has access to roughly the same models. The harness is the asset, because it holds judgment no competitor has. It came from your operators, and it runs on the existing BI stack you already paid for.

So the benchmark is not a faster junior analyst. It is your best operator, reading every location, every cycle.

It also survives turnover. The COO at that pawn chain has been in the industry for 27 years, and the CEO told me his biggest single business risk was losing him. When your veteran retires, the rules stay.

For your analytics team, this is not a replacement story. The harness takes the volume work off people who are keeping their head above water. You don't lose them. You 10x them.

This is what AI performance management looks like when it is built right. Scoped. Specific. Yours.

The Terminator was never the risk. The risk is an AI that doesn't know your business, answering questions nobody asked, with nobody checking.

Frequently asked questions

What is a model harness?

A model harness is the structure around an AI model that controls what it looks at, what it checks, in what order, and how it interprets results. In operations, the harness holds encoded business judgment, so the model follows a defined investigation workflow instead of improvising.

Why is a limited-scope AI more trustworthy?

Because every conclusion traces back to a defined screen, probe, and rule. When the model cannot wander outside those rules, outputs are consistent across locations, and errors can be found and corrected once for everywhere. That traceability is what separates real diagnostic analytics from a confident guess.

Whose judgment goes into a harness?

The people who run the business: the COO, VPs of operations, regional directors, and long-tenured general managers. Scoop's team records them as they review their own reports and KPI dashboards, then maps what they say into rules.

Does a harness replace our BI tools?

No. The harness is a diagnostic and action layer that sits on top of your existing warehouse and BI tools and applies your operators' interpretation to the data already there. Nothing is migrated or ripped out.