It demos well (that's the problem)

Brad Peters

From confident AI pilot to expensive dead end

You ran a pilot. You asked the AI a real question about your business, and it gave you a clean, well-formatted answer in seconds. It looked right. It probably was right, that time.

Here is the uncomfortable part: a wrong answer from one of these tools looks exactly the same as a right one.

I've spent my career building analytics products, and I've built this kind of tool myself. This is a structural quirk of how these tools fail, and it's worth understanding before you wire one into a decision that matters. It's also the reason we built Scoop the way we did. I'll get to that at the end.

Why AI analytics fails differently than a broken dashboard

Because it doesn't look broken. When a dashboard breaks, you can see it: a chart throws an error, a filter returns nothing, a number goes blank.

The failure announces itself. When a large language model gets an analytics question wrong, nothing announces anything.

The answer comes back fluent, formatted, and confident, and it's wrong.

In a dashboard, wrong looks wrong. In AI-generated prose, wrong reads fluent.

A traditional BI tool fails loudly. An AI-generated answer fails silently, and silent failures are the ones that reach a board deck. The researchers behind the leading academic benchmarks say it plainly: strong scores on public datasets make these models look good at turning questions into SQL, and on real enterprise data they are not.

How often does the best pipeline actually get it right?

When researchers stopped testing on tidy academic databases and started testing on real enterprise warehouses, accuracy collapsed.

The demo runs on the easy data. Your business runs on the hard data.

Benchmarks that tell the same story:

• BEAVER (MIT, 2024): the first text-to-SQL benchmark built from private enterprise data warehouses. The best agentic pipelines, running GPT-5.2, scored about 10.8% accuracy. That is roughly one question in nine answered correctly. Even when researchers handed the models oracle hints for every subtask, accuracy only reached 30.1%.
• Spider 2.0 (2025): on real enterprise SQL workflows, the best agent framework solved 17% to 21% of tasks when the benchmark launched. The same class of model scored 91% on the older academic Spider benchmark.
• Spider 2.0-Cryptic (2026): leaderboard scores on Spider 2.0's Snowflake track have since climbed past 90%, so researchers at TU Darmstadt and SAP ran a cleaner test. Same data. Same questions. They only renamed the columns to cryptic, SAP-style abbreviations, the kind where a delivery date is stored as BSTLDAT. A standard agent dropped from 41.6% of questions solved to 10.9%. An agent that could look up what each column means recovered to 43.6%.
• dbt Labs (2026): frontier models writing raw SQL against the original schema scored 64.5%. Even after three helper models were added to simplify the joins, they topped out at 84% to 90%. Routed through a governed semantic layer, the same models reached 98% to 100%. dbt's own summary: "Text-to-SQL will cheerfully give you a wrong number."

What a confident wrong answer actually looks like

It looks like a number you wouldn't think to question. The failures below aren't exotic.

They are the ordinary ones, the kind that ship to an executive because nothing about them looks off.

None of them throws an error. None of them raises a flag. Each one just returns a plausible, formatted, wrong result.

1. The filter drop

You ask, "How many customers churned last quarter?" Without proper filter extraction, the model quietly drops the churn condition and returns the total customer count instead. The number is large, clean, and completely wrong. It doesn't error. It answers a different question than the one you asked, and it doesn't tell you.

The retail version is a comp sales question. A senior executive at a large national retailer described the hidden rules to me this way: "Every business, the data means something else. No, you only use comp stores for this. You only use this for that." A model that was never told the comp-store rule doesn't stop and ask. It counts every store and reports the result.

2. The snapshot sum

You have 90 daily snapshots of $10M in inventory. Ask for total inventory value and a naive query sums all 90 rows: $900M. The answer is 90 times too high. It is still a number. It is still formatted like every other number in the report. Nobody catches it unless they already know to look for double-counting across snapshot dates.

3. The partial period

A month-to-date figure gets compared against a full prior month. Twelve days of the current month stack up against 30 days of the last one, and the chart reads as a 50% collapse. Finance panics over a number that is a data artifact, not a business event. The comparison is apples to half an orange, and the AI presented it as fact.

Three failures, one shared trait:

• None of them errors.
• None of them flags.
• They just ship.

Why the pilot looked so good

Because a pilot is built to succeed.

You picked clean questions on data you understand, and you could eyeball the answers to sanity-check them.

That's a reasonable way to kick the tires. It's also the exact condition under which these tools perform best, and nothing like the messy, high-stakes, hard-to-verify questions you actually want answered.

I learned this the hard way. When we built our own version of these tools, AI was running into the same sort of problems that people were. It didn't really understand the context of the data.

The demo hides the failure mode for three reasons:

Simple schemas

Pilot data is usually a handful of clean tables. Real warehouses have thousands of columns, cryptic names, and tribal join logic the model has never seen.

The same retail executive was blunt about his own stack: "I have supply chain systems, and there's merchandising systems, and they don't talk." Production data is always dirtier than your pilot let on.

Familiar questions

You test what you already know the answer to, so you catch the misses. The questions that matter are the ones you cannot check by hand.

Verifiable results

In a demo you can reconcile the number. In production, at volume, nobody reconciles every answer, which is the moment the silent failures start shipping.

The fix is not to stop using AI

It's to make sure the AI never does the arithmetic.

The models are genuinely good at understanding what you're asking. They're unreliable at silently computing the answer against a schema they half-understand. So you split those two jobs.

Let the model interpret. Do not let it calculate.

Engineers call this a semantic layer. I put it more simply: the model reads the question, and something that can't improvise does the math.

The architecture that holds up looks like this:

• The model reads the question and writes down what you want as a structured spec, not raw SQL guessed against the schema.
• A deterministic engine turns that spec into SQL you can inspect, using governed definitions of your metrics.
• Every number traces back to something a human can read, check, and audit.

The AI assists. It does not calculate. That single boundary is what separates a tool that demos well from a tool you can trust when the answer is headed somewhere that matters. It is the same reason a deterministic, rule-bound approach beats raw generation for anything that has to reconcile: the arithmetic lives in a layer the model is not allowed to improvise.

How Scoop provides an AI performance management layer

It's the reason the failure modes above can't ship the same way.

We built Scoop as an AI performance management layer that sits on top of your existing BI stack: your warehouse, your Power BI, your Tableau. It doesn't replace any of it. What it adds is the part those tools leave to you: interpretation you can trust, on numbers you can trace.

Here is how the boundary shows up in the product:

The model interprets. It does not do the math.

We use the model to interpret the question. Then we hand the math to an engine that can't guess.

Real data, real calculations, traceable to source. No number is improvised against a schema the model half-understands.

We keep the model on a tight harness. Lots of AI is used, but it doesn't get to improvise the numbers. The same question gets the same computation every cycle, because you can't trust an answer you can't reproduce.

Every answer is inspectable

Our three-layer architecture keeps the business logic in a governed layer, not in whatever SQL a model guessed this time.

You can read the logic behind a conclusion, check it, and audit it. A filter drop, a snapshot double-count, a partial-period mismatch: each one is caught in the layer, not shipped in the prose.

It runs on your stack, not instead of it

Pairing Scoop with your existing BI stack means no migration and no copies of your data. We're not asking you to rip and replace.

We read what you already have and add the interpretation and action layer on top.

The output is a report, not a query you have to trust blind

Scoop runs across every location, every cycle, and delivers what changed, why it changed, and what to do about it, with the evidence behind every conclusion.

You're not eyeballing a chatbot answer and hoping. You're reading a diagnosis you can check.

Frequently asked questions

Does this mean AI analytics tools are useless?

No. The models are strong at interpreting a question and weak at silently computing the answer against a complex schema. The problem is asking one system to do both. A tool that lets the model interpret and hands the arithmetic to a deterministic engine avoids the failure mode entirely. Scoop is built on that split. The issue is architecture, not whether to use AI.

Why did our pilot work fine if accuracy is really that low?

Pilots use clean data, familiar questions, and results you can eyeball. That is the best case for these tools. Real warehouses have thousands of columns and questions you cannot verify by hand, which is exactly where the benchmarks show accuracy dropping to 10% to 20%. The pilot did not lie, it just tested the easy case.

How would I even catch a wrong answer if it doesn't error?

Usually you cannot, not at volume, and that is the point. A filter drop, a snapshot double-count, or a partial-period comparison all return clean, formatted numbers. The reliable way to catch them is structural: force every number to trace back to inspectable SQL and governed definitions, so a human can audit the logic instead of trusting the prose. That traceability is what Scoop's layer is built to guarantee.

How is Scoop different from asking Power BI Copilot or ChatGPT the same question?

Those tools generate and run the query in one step, so you inherit the benchmark accuracy numbers. Scoop keeps interpretation and calculation separate. The model figures out what you are asking, a deterministic engine does the math against governed metric definitions, and Scoop sits on top of your existing BI rather than replacing it. You get an answer you can trace, not a fluent guess.

Isn't a 90% accurate model good enough with a human reviewing?

Only until volume exceeds what one reviewer can genuinely check. A reviewer works at ten answers a week and quietly skims at a thousand. And even 90% means one in ten board-grade answers is wrong, with nothing marking which one. Human review is a real control at small scale and a false comfort at production scale. The fix is structural, not more reviewers.

What's the single most useful question to ask a vendor?

Ask whether the model generates the SQL directly, or emits a specification that a deterministic engine compiles into inspectable SQL. If the model writes and runs the query itself, you inherit the benchmark accuracy numbers. If the arithmetic lives in a governed layer the model cannot improvise, you have the architecture that holds up. Scoop is in the second category by design.