Measuring the wrong things

Companies are spending billions of dollars on AI. Boards approve the budgets, employees get licenses, usage dashboards light up green — and yet, when it comes time to explain what any of it did for the bottom line, most executives come up short. This disconnect between capability and proof isn't a sign that the technology is overhyped. AI systems are demonstrably more capable than anything that came before them. The problem lies in how organizations are measuring and managing the transformation around that capability.

Most companies track adoption: how many people logged in, how many queries were run, how many hours were reportedly saved. These numbers are easy to collect and easy to put on a slide. But they don't reliably predict whether AI investment will show up in EBITDA. A company can have soaring usage numbers and flat margins. Proving ROI requires a different kind of measurement — one that connects daily AI activity to the financial outcomes leadership actually cares about.

‍

The Productivity J-Curve: why value lags investment

Economist Erik Brynjolfsson has described this dynamic through what he calls the productivity J-curve. When a powerful general-purpose technology enters an organization, buying it isn't enough. Companies have to rethink business processes, restructure organizational capital, and retrain their workforce to apply the technology well. These are intangible investments — they rarely appear on a balance sheet, but they're often larger than the direct cost of the technology itself.

While those investments are underway, measured productivity tends to fall before it rises. Employees spend time figuring out new workflows instead of producing output in the old ones. That's the downward slope of the J-curve. The upward slope comes later, once the reorganization is complete and the technology is actually embedded in how work gets done.

This creates a real risk for leadership teams: EBITDA and revenue are lagging indicators. By the time they move, whatever caused the movement happened months earlier. Waiting for the P&L to confirm that an AI initiative is working means discovering problems only after they've already cost the company time and money. Leaders need indicators that surface earlier in the curve, not after it.

‍

Why adoption metrics alone are misleading

Faced with pressure to show progress, many executives default to the simplest available proxy: usage. Token counts, seat licenses, query volume. It's understandable — these numbers exist automatically inside most enterprise AI deployments. But they're a crude and blunt instrument for assessing whether AI is creating value.

Research across companies deploying AI at scale reveals a consistent power-law distribution. A small group of users — typically 3 to 5 percent — captures outsized gains, seeing 2x, 5x, even 10x productivity improvements on the tasks they apply AI to.

The rest of the organization sits on the flat part of the curve. In one company, the median user was capturing roughly 1 percent of the potential value available to them, largely because they were using AI for low-stakes tasks like polishing email language rather than anything tied to core business objectives.

High adoption numbers can obscure this reality entirely. A department might show 90 percent weekly active usage while still capturing almost none of the available value, because usage and value are not the same measurement. Counting activity without accounting for what that activity is worth tells leadership very little about whether their investment is paying off.

‍

The missing ingredient: quality-adjusted measurement

Closing that distance requires moving past the question of whether people are using AI, and toward whether they're using it on high-value tasks, and doing so well. This is the logic behind a quality-adjusted quantity, or QQ, score: a way of weighting usage by the business value of the task it's applied to and the sophistication with which it's being done.

Building this kind of metric starts with granularity. Organizations need to enumerate the tasks their teams perform, map those tasks against what AI is genuinely good at, and then track whether real usage lines up with that opportunity map. From there, a set of leading indicators becomes available: time saved per employee per week, monetary value generated per use case, and trends in the complexity and sophistication of AI usage over time. These indicators move earlier than EBITDA and give leadership a way to steer before the trough of the J-curve turns into a permanent plateau.

‍

Task-level wins don't automatically roll up to EBITDA

Even well-designed task-level metrics have a limit. A single task can improve dramatically — a support ticket resolved faster, a document drafted more efficiently — without the surrounding workflow, or the P&L, moving at all. Task-level improvement is a necessary building block, not a finished result.

To close that distance, organizations need workflow-level KPIs running alongside task-level ones: customer satisfaction scores, end-to-end cycle time, cost-to-serve. If customs clearance times aren't shrinking, or if the variance in service delivery isn't tightening, then task-level gains are likely getting absorbed somewhere else in the process rather than reaching the bottom line. Task-level measurement is where transformation efforts should start, but it can't be where they stop.

‍

Why initiatives stall even with good metrics

Even organizations with solid measurement infrastructure often see initiatives stall out. Part of the reason is organizational: every company operates on what researchers call an organizational truce, an existing equilibrium acceptable to all the internal stakeholders involved in it. AI adoption disrupts that truce by making previously settled ways of working look inefficient or replaceable, and it's easy for someone with the standing to slow a project down to find a convincing reason to do so.

Part of the reason is also operational. Pilots tend to succeed because they're narrow — a simplified version of a real business problem, engineered to work cleanly. Scaling requires handling the long tail of exceptions, edge cases, and messy data that the pilot conveniently avoided, and that requires infrastructure investment most organizations underestimate.

Without leadership willing to challenge the truce and fund the unglamorous work of scaling, bottom-up AI enthusiasm plateaus well below its potential financial impact.

‍

Closing the distance: from measurement to management

The companies pulling ahead — often described as a top 5 to 10 percent tier — aren't succeeding because they have better AI. They're succeeding because they pair measurement with management action: reorganizing tasks around what AI can do well, identifying internal power users and propagating their methods, and evolving their metrics as the technology itself evolves.

The path from AI spend to EBITDA impact runs through a connected chain: granular task-to-opportunity mapping, quality-adjusted usage tracking, workflow-level KPI monitoring, and organizational willingness to restructure around what the data shows. None of this is a matter of waiting to see how AI transformation unfolds. It's a design problem, and the companies that treat it that way are the ones setting the outcome rather than reacting to it.

‍

Where to go from here

Adoption metrics will tell you that people are using AI. They won't tell you whether that usage is worth anything. Only a connected chain of task-level, quality-adjusted, and workflow-level metrics can credibly link AI investment to EBITDA outcomes — and building that chain is the core of what Workhelix's measurement platform is built to do. For a closer look at how this framework applies to your organization, get in touch for a demo.

‍