Clean Data First Is the Advice That Kills AI ROI

"Clean your data before you deploy AI" is the most responsible-sounding advice in analytics. For a multi-location business, it is also some of the most expensive. The cleanup never finishes, and every quarter spent on it is a quarter where most of your locations still don't get a real look from anyone who knows what they're looking at.

I know that advice well, because I spent years on the other side of it.

I used to run what is now Oracle's business intelligence and data warehousing platform. Back then, analytics cost "a million bucks even when you breathed," and a lot of that money went to one thing: getting the data ready. Label every metric. Get the data dictionary right. Get sign-off from the committee. Then, maybe, show someone an answer.

At Birst we worked hard to shrink that. We did a pretty good job of it. I still watched about half of all analytics projects fail, and it was rarely because the data wasn't clean enough. It was because the people implementing them didn't understand the business problem. That's the same reason most BI projects fail today.

My co-founder Neil Shepherd put the economics in one line:

*"The economical question isn't just capability. It's whether you can actually get the data in good shape within the window of owning these companies and seeing ROI."*Neil Shepherd, co-founder, Scoop

For PE-backed operators that window is literal. The typical private equity hold now runs about six years, and the clock doesn't pause while the data team normalizes feeds. Meanwhile, district managers keep running stores on reports nobody has time to read closely. That's the last mile of business intelligence, and it's where the value leaks out.

So let me be clear about what I'm arguing:

• Data quality matters Scoop can't work without enough data, and I won't pretend otherwise.
• Clean data first is a sequencing mistake. It puts an endless project in front of the work that pays.
• The better order start with what you have, agree on a few definitions, and let the work tell you what to fix.

Why does "clean data first" never finish in a multi-location business?

Because a business with 200 doors creates new mess faster than any central team can clean it up. Here's a normal year:

• New stores open with a slightly different point-of-sale setup.
• An acquisition brings in its own chart of accounts.
• A vendor changes a feed format without telling anyone.
• A franchisee codes labor hours differently than corporate does.

I hear the same thing from well-run operators all the time. Even the most prepared ones tell me it took many rounds to get their data clean, and some feeds never fully settle. That isn't a failing data team. That's what data looks like when a lot of people in a lot of buildings create it.

I say this to prospects constantly: it would be great if you already had a perfect data warehouse, but no one ever does. The research backs that up. When Harvard Business Review researchers had 75 executives check 100 recent records from their own departments, 47% of newly created records had at least one critical error. That's the starting line for almost everyone.

And while the data team asks for "one more quarter to normalize the feeds," your locations keep drifting. Standards slip one store at a time, and nobody catches it until the P&L does. That's the drift problem in retail, and no warehouse project fixes it. It's also why so many pilots look great and then stall: production data is always dirtier than your pilot.

Clean enough never arrives. The end of the hold period does.

If dirty data isn't the real problem, what is?

Definitions. In my experience, the numbers that send a manager the wrong way are usually accurate. They're just compared or added up the wrong way.

RAND researchers who studied why AI projects fail ended up in a similar place. Lack of data is on their list of five root causes. But the first cause they name is misunderstanding the problem: optimizing for the wrong metric, or building something that doesn't fit how the business actually runs. A perfect warehouse fixes neither.

The old BI answer was to define everything. You don't need everything. You need agreed definitions for the handful of measures your leaders actually decide on. That's the gap between a reporting program and real operational analytics. Here are three definition problems I see trip up operators. They're illustrations, not any one customer's numbers:

• A partial month compared with a full one
• Transactions read as visits
• Inventory added up like sales

The Monday collapse that wasn't

Picture a district manager opening the flash on the 12th of the month. All 12 of her stores are down about 60% against last month.

Nothing fell apart. On the 12th of a 30-day month, month-to-date sales are roughly 40% of a full month, and the report put that partial period next to a whole one. The agreed rule is simple:

• Compare the same number of elapsed days.
• Or use same-store sales logic so like is compared with like.
• Never show a partial period next to a full one without saying so.

Any retail analytics program that skips this produces a false alarm every month. Managers learn to ignore the alarms, and that's worse than having none.

Fewer transactions, same guests

Transactions down 8% reads like a traffic problem. It might be. It might also be:

• The same guests putting more on one ticket.
• A change to online ordering or loyalty that merged orders that used to be separate.
• A point-of-sale update that splits or combines checks differently.

Each answer sends the district manager somewhere different. Get the definition wrong and she spends a week fixing traffic at a store whose guests never left. Good AI investigation tests those explanations against each other before it names a cause.

The inventory you counted seven times

Sales add up across a week. Inventory on hand doesn't. Sales are a flow. Inventory is a stock, a quantity held on a date. Add up seven daily on-hand counts and your report shows seven times the inventory actually sitting in the back room. The agreed rule:

• Use the end-of-period count, or an average across the period.
• Capture it on a schedule with intelligent data snapshotting, so you're comparing the same moment every cycle.

Get a handful of definitions right and most of the remaining mess stops mattering to the decision.

What does starting with the data you have actually look like?

You connect to what's already there, agree on the few definitions that matter, capture how your best operator reads the numbers, and run it. Then you let the first reports be wrong in useful ways.

I'll tell you what actually happens, because it isn't what the slide says. The first report comes out and people look at it and say, "That's garbage." We got a definition wrong. Two systems didn't line up. A threshold was off. So we go back to the stores, listen to the managers, and come back. The next version gets way better.

That's the process working. Every "that's not right" is a definition getting fixed that would otherwise have sat in a data backlog for a year.

Here's how we run AI performance management on the data a business already has:

1. Connect what exists. Read-only access, or a scheduled extract, from the BI tools and warehouse you already run. Scoop is a diagnostic layer on top of BI. It doesn't replace your existing BI stack, and it doesn't build a new central data repository.
2. Agree the definitions. Start with the handful of measures your leaders actually make decisions on.
3. Capture the playbook. Our team sits with your best operators. The question I ask every COO: "If I took a tape recorder and recorded everything you thought as you looked at your BI reports, could we stick that into the system so it could do that on your behalf?"
4. Run it every cycle. Scoop reviews every location against that playbook. An engine calculates the numbers from your data, the AI interprets them within your rules, and our team reviews the cycle before anything goes out. Each manager gets a briefing with the findings, what to check, and the recommended actions. That's how Scoop works.

There's a byproduct nobody expects. Things get written down that the business never documented because it was too painful. The definitions you argued about for years finally have one answer.

What can't this approach do?

I'll be straight about the limits. Scoop needs enough data to work with, and it can't prove a cause the data doesn't show.

• If a store's labor feed is missing three weeks, the briefing says the feed is missing. It doesn't guess around the gap.
• If the data rules out staffing and pricing but can't see a road closure, the briefing marks that as an open question for the district manager.
• Every conclusion comes with the evidence behind it, so a manager can see why a store was flagged.

I'd rather a briefing say "we don't know, go check" than make something up. Knowing what the data establishes, and what still needs a person to verify, is how an investigation tells you why instead of guessing.

How does the data get cleaner if you don't clean it first?

The investigations tell your data team what to fix first. Data quality still improves. The effort just goes to the problems that changed a decision.

• Moves up: a feed a finding depended on. If a conclusion about labor at 14 stores rested on a shaky timeclock export, that export goes to the top of the list.
• Moves down: a known issue that never changed a decision. The duplicate product descriptions can wait.
• Comes back corrected: the open questions managers check in the field. Their answers fix definitions and add context the data never had.

That's data governance driven by decisions instead of a backlog. Your data team stops trying to clean everything and works on the few things that mattered this cycle. Every pass through the investigation workflow leaves the data a little better than it found it.

Clean data becomes an output of the work, not a gate in front of it.

What should a COO ask the data team this quarter?

Stop asking "Is the data clean yet?" Ask whether it's good enough for the decisions you make every cycle. Three questions for your next review:

• Which five to ten measures drive our location-level decisions, and do we all define them the same way?
• Which of our known data problems has actually changed a decision in the last six months?
• What could every location get a real look at next cycle, using the data we already have?

If the answers point to something you can start now, start. A Scoop implementation begins right there: connect the existing data, capture the playbook, run a working pilot. That's what I mean by AI performance management: your best operator's read on every location, every cycle, built on the data you already have.

Your data will never be perfect. Your hold period ends anyway. Start with what you have.

Frequently asked questions

Doesn't AI need clean data to work?

AI needs data that's good enough for the decision at hand. It doesn't need a perfect warehouse.

• Agreed definitions for the key measures matter most.
• Gaps should be reported as gaps, not hidden.
• The data has to fit the decision you're making, not every question someone might ask later.

What if our data really is too messy?

Then the first investigations show you exactly where it's too messy to support a decision. That's useful on its own.

• Some feeds will be good enough on day one.
• Others get marked as open questions until they're fixed.
• You fix them in the order that matters to the business. That holds for hotel portfolios and franchise systems alike.

Is Scoop a data cleanup or migration project?

No. Scoop reads from the systems you already run and sits on top of them.

• It uses read-only access or a scheduled extract.
• Nothing is ripped out or migrated, and you keep your BI stack.
• It doesn't create a new central data repository.

How is this different from building better dashboards?

A dashboard waits for someone to read it. A briefing tells each manager what changed, why, and what to check.

• It arrives every cycle without anyone asking for it.
• It covers every location, not just the ones someone had time to look at.
• It gets past the limitations of KPI dashboards by giving you the interpretation, not just the chart.