The Metrics That Lie: Why Your Product Dashboard Is Giving You False Confidence

Every B2B SaaS product team I’ve worked with has a dashboard. Most have several. They track daily active users, session duration, feature adoption rates, NPS scores. They review them weekly, report them monthly, and present them quarterly.

Here’s the uncomfortable truth: most of those metrics are measuring activity, not value. They tell you what users are doing, not whether they’re getting what they came for. And when a metric measures the wrong thing, it doesn’t just waste time — it gives you false confidence while your UX leaks quietly compound underneath.

I’ve watched product teams celebrate rising engagement while their renewal rates dropped. They had the data. They just had the wrong data.

Here are 4 metrics that lie, why they lie, and what to track instead.

1. Daily Active Users

DAU is the most popular vanity metric in B2B SaaS. Investors ask for it. Boards nod at it. Teams celebrate when it goes up.

Here’s what DAU actually measures: how many people opened your product today. It doesn’t measure whether they accomplished anything, whether they found what they needed, or whether they’ll come back tomorrow.

I’ve seen products where DAU was climbing because users were forced to log in to check status updates that should have been emailed to them. They weren’t engaged. They were doing data entry. The product had become a notification centre, not a value engine.

I’ve also seen the opposite: products where DAU looked flat or declining because power users had learned to batch their work. They were getting more value in fewer sessions. The metric punished efficiency.

Track instead: Task success rate per session. Pick the 3 highest-value actions in your product and measure what percentage of sessions result in at least one of them being completed. That’s engagement that matters.

2. Session Duration

Longer sessions feel good. They look good on a dashboard. They imply deep engagement, right?

Sometimes. But often, long sessions mean confusion. The user is lost. They can’t find what they need. They’re clicking around, scanning, re-reading, trying to remember where that button was.

I audited a product where the average session was 22 minutes. The team was proud of it. Then we watched recordings and found that users spent the first 8 minutes hunting for the feature they needed. Every single session. The remaining 14 minutes included at least 3 minutes of “wait, where do I go next?” pauses.

The “engagement” they were celebrating was actually friction wearing a different hat.

Track instead: Time-to-task and abandonment rate per task. How long does it take a user to complete the thing they came to do? How many start it and never finish? If time-to-task drops and abandonment stays flat, your UX is improving. If session duration drops and task completion goes up, you’re winning.

3. Feature Adoption Rate

Feature adoption is the metric that creates the most wasteful product decisions I’ve seen.

The logic sounds right: “We built this feature. We want people to use it. Let’s measure how many do.” The problem is that this metric incentivises teams to push users toward features that may or may not solve their actual problem.

I’ve watched products add onboarding modals for features nobody needed, run campaigns to drive adoption of tools that duplicated existing workflows, and measure success by how many users clicked a button they’d never click again.

Sometimes low adoption means the feature isn’t useful. Sometimes it means it’s working exactly as intended — a set-and-forget setting that does its job silently. Feature adoption as a north star metric will push you to chase clicks over outcomes every time.

Track instead: Feature retention rate. Of the users who tried a feature, what percentage come back to use it again within 7 days? That separates novelty from genuine value. If adoption is high but retention is low, the feature is a one-time trick, not a durable part of your product.

4. NPS Score

Net Promoter Score is a measure of sentiment, not behaviour. It tells you what users say about your product, not what they do with it.

The gap between those two things is wider than most teams realise. I’ve worked with products where NPS was consistently 60+ while churn was consistently climbing. Users loved recommending the product to others. They just didn’t stick around themselves.

Why? Because they admired the product conceptually but struggled with it operationally. The vision was compelling. The daily experience was frustrating. NPS captured the vision. Churn captured the reality.

I’ve also seen the reverse: products with middling NPS (30–40 range) that had excellent retention. Users didn’t love the product — they needed it. It was ugly, clunky, and opinionated, but it solved a painful problem reliably. Those users wouldn’t recommend it to a friend, but they’d never stop paying for it.

Track instead: Behavioural retention curves. Plot how many users who complete a specific task on day 1 are still completing it on day 7, day 30, and day 90. That curve predicts revenue better than any survey question.

Why These Metrics Survive

These four metrics persist because they’re easy to collect, easy to present, and easy to compare against benchmarks. They work well in board meetings. They fail completely at telling you whether your product is actually working for the people who pay for it.

The metrics that matter are harder to define, harder to instrument, and harder to benchmark. Task success rates vary by product. Retention curves vary by business model. There are no industry averages to point to.

But that’s the point. Your product is not an industry average. Your users have a specific job to do, and the only metric that matters is whether they’re doing it successfully and coming back to do it again.

Every team I’ve worked with that switched from activity metrics to outcome metrics found leaks they didn’t know existed. Not because the data was hidden — because they were looking at the wrong dashboard.

Where to Start

Pick one metric from this list that your team currently reports on. For the next month, also track the replacement metric I suggested alongside it. Compare what each one tells you about your product’s health.

If the two metrics tell the same story, you’re probably fine. If they tell different stories, you’ve just found a leak you didn’t know you had.

Not sure what to track or how to define your key tasks? A clear-eyed audit of your product flows will surface exactly where your metrics are lying to you.

→ flowaudit.site — free automated scan, 90 seconds

→ poplab.io/services — human audit, fix sprints, embedded design partner

This is based on patterns I’ve observed across 15 years of product design work. The metrics names are real. The behaviours behind them are real. You’ve seen them in your own dashboards.