"We know that new user retention is going down, but we're trying to figure out the why and how to address it."
That one sentence. Quietly devastating. Not panicked — just honest. They could see the number moving in the wrong direction. They had the data. They had the tools. They just didn't have the answer.
And the more I thought about it, the more I realized that sentence could come from almost any product or growth team in any industry right now. It's not a data problem. It's a gap problem — the gap between what your metrics show and what your metrics mean.
The "We Know What, But Not Why" Problem
Here's what I've noticed across hundreds of conversations with analytics and operations teams: most companies today are drowning in "what" and starving for "why."
They know their churn rate. They know their conversion rate. They know their month-over-month retention curve. What they don't know — what they genuinely struggle to answer — is what's driving those numbers. And that gap is costing them more than they realize.
This growth lead's situation was pretty typical for a fast-growing freemium app. Their analytics stack was actually reasonably solid. They were running event tracking through a popular product analytics platform. They had customer lifecycle data flowing in from multiple sources. They had a cancellation survey for subscribers who churned. They were tracking user properties, event history, cohort behavior.
What they didn't have was a clear way to connect all of that to a coherent answer about why users were leaving — or more importantly, who was likely to leave next.
Three Problems I Hear Every Single Week
1. Analytics is still a specialist sport
One of the first things this growth lead mentioned was that they wanted to "enable analytics across the organization a little bit more and let it be more of a democracy, rather than just the people that know how to pull it with their tools."
That framing — analytics democracy — is something I hear constantly, and yet it's still so far from reality for most teams. The honest truth is that most analytics platforms require a level of technical fluency that rules out the majority of the people who actually need the insights. So questions pile up in a queue. The people who can pull the data spend their time running reports instead of doing analysis. And the people who need the answers stop asking because the turnaround is too slow.
The ask isn't for everyone to become a data scientist. It's just for the question-to-answer loop to feel less like filing a support ticket.
2. You can't model what you haven't defined
This came up in a way I found genuinely interesting. We started talking about churn prediction, and it quickly became clear that they had two completely different notions of churn: subscribers who canceled a paid plan, and free users who simply... stopped coming back.
Neither type had a clean boolean field in the data. There was no "churned: true/false" column. Instead there were event triggers, cancellation timestamps, and behavioral signals — all useful, just not unified into a single definition the system could learn from.
What made the conversation productive was working through why that definition matters before you can do anything meaningful with ML. Predictive modeling needs a target. You have to decide what you're predicting before you can look for the patterns that predict it. And that decision — what counts as churn for your business — isn't something any platform can make for you. It's a business judgment that has to come first.
The practical implication is that a lot of teams think they're "not ready for ML" when actually they're one well-defined column away from unlocking significant predictive power.
3. The tool-cobbling trap
At one point, the growth lead mentioned they'd been using ChatGPT to analyze open-ended cancellation survey responses — essentially asking an LLM to surface themes from unstructured feedback. Which is clever! It's exactly the kind of creative workaround smart people build when the right tool doesn't exist or hasn't been configured.
But it also pointed to something I see all the time: teams that have assembled a patchwork of solutions, each doing one piece of the job, none of them talking to each other. A product analytics platform for behavioral data. A CRM for customer records. A separate tool for email campaigns. A spreadsheet for churn tracking. A general-purpose AI tool for ad-hoc text analysis.
Each piece works in isolation. The synthesis — the moment where you get a single, coherent answer to "who is churning and why" — lives nowhere.
The Bigger Pattern Here
What struck me about this conversation wasn't the specific technical situation. It was how normal it was. This is a well-run company with smart people and real data. They're not behind the curve. This is just where most growth teams live right now.
The reason I find this so important: the cost of that "we know the number, we just don't know why" gap isn't measured in data quality. It's measured in decisions that don't get made, interventions that come too late, resources allocated on intuition instead of evidence. A team that can't answer "why is new user retention declining" in real time isn't just slower. They're genuinely flying with instruments that only tell them their altitude, not whether they're ascending or descending.
And the scary part is that all the raw material for the answer is already there. The behavioral data exists. The cancellation events exist. The user properties exist. The cohort history exists. The gap isn't the data — it's the layer that turns that data into an investigation, rather than just a report.
The Shift That Happened During This Conversation
About halfway through, the conversation moved from "what's possible in theory" to "here's how we'd actually set this up." And that's where I saw the shift.
When they realized they could define multiple churn definitions — subscription churn, passive churn, whatever business logic made sense — and model each one independently, without rebuilding anything from scratch, something changed. When we talked through how Scoop can take a cohort export from their analytics platform, add a calculated column defining the churn condition, and use that as the foundation for an ML model — no data warehouse, no SQL, no separate data science team — the conversation stopped being exploratory and became practical.
By the end, the next steps were concrete: build a cohort of recently lapsed users, export it with user properties attached, add a simple boolean to classify them, and bring that into a guided POC. That's it. A week's worth of light work, and suddenly you have the foundation for a real churn prediction model.
The gap between "we know the number is down" and "here's what's driving it and who to target" was smaller than they thought. It almost always is.
What I Keep Thinking About
There's a version of this story where analytics finally becomes what it was always supposed to be for most organizations: something anyone on the team can use to get real answers, not just charts.
The companies that get there fastest aren't the ones with the most sophisticated data infrastructure. They're the ones that close the gap between question and investigation — and treat the "why" as just as answerable as the "what."
If you're running a freemium product, a subscription business, or any kind of consumer app and you're in the "we see the number moving, we just don't know why" place right now, that gap is worth looking at closely. It's usually not as wide as it feels.
