TBM 50/53: The Curse of "Success" Metrics

In workshop after workshop I have noticed a pattern.

Teams are able to map the beliefs and assumptions underpinning their product. But when it comes to deciding on actual metrics, they struggle.

I've wondered about this phenomenon for a while now.

Here is what makes it hard:

• Actual metrics become a basis for prioritization

• Actual metrics translate into team goals

• Team goals translate into individual job goals

• Goals matter in many ways (including team structure)

• People worry about picking the wrong metric

• People worry that managers will question their reasoning

The net effect is that they often opt for running a feature factory. Or they only focus on goals where the measurement problem is easy/definitive. Not because they lack the skills or experience—I’ve seen this happen to very talented teams. Rather, because they lack confidence when cause/effect is less certain. Metrics are too wrapped up in performance, optics, and scrutiny to be useful.

Teams have no problem coming up with options for measurement for other companies. When asked for a "success metric" for an active project, they often can come up with something. And that makes sense, because projects come and come. But for strategic frameworks and more persistent metrics...they struggle for the reasons mentioned above.

How do I (try to) help them?

First, I remind them that uncertainty is where we find opportunity. If your business model is 100% certain, there isn't a lot of upside. With a promising product we are always making decisions under conditions of uncertainty.

Second, I remind them that metrics aren't set in stone. We have to start somewhere. You know what you know. If we learn later that there isn't a solid relationship between two variables...well, we can try something else. In fact, this is the crux of experimentation. With time, we will increase our confidence levels. The key here is making it very clear what we are trying to measure, not just how. The what will frequently remain constant. The how may change.

Third, I try to differentiate measuring to learn vs. measuring to control and manage.

This is the crux. Consider the term "success metric". We've already jinxed the effort. We are already afraid of failing and learning.

So ask yourself…how can you make it safe to use measurement to learn? How can you make it OK to start somewhere with what you know, and gradually increase certainty? If you can flip that switch, you’ll be setting up your business for real success.