Innovation Metrics: Data-Driven Decision Making

In our previous blog post, we discussed the need for exploration projects to track four main key performance indicators (KPIs): risk and uncertainty, expected profitability, time spent, and cost. In this article, we will zoom in on calculating risk and uncertainty at the hypothesis level to help you make data influenced decisions to inform your next business idea.

By breaking down an idea into smaller chunks you can understand and test risk at a more granular level. We call this the hypotheses underlying your idea. In other words, the most important things that need to be true for your idea to work. If you don’t have recent evidence to support or refute a hypothesis, you need to run experiments to gather this evidence.

Keep track of your progress by using the Innovation Metrics Tracking Sheet. Below are the detailed explanations of each of the fields.

Hypothesize

Log all the experiments you have conducted to either support or refute a specific hypothesis. For each experiment you capture the experiment type, what you measured, the success criteria, how much time each experiment took, and what it cost. Read more about how to fill out a Test Card.

Test

Log what you have learned from the evidence to support or refute a specific hypothesis. You specifically capture all the evidence gathered, the number of data points, the strength of the evidence, and how confident you are that your insights are true. Read more about how to fill out a Learning Card.

Learn
What insights were you able to glean from the evidence you have collected? Determine if the evidence gathered from your experiment was able to

support a hypothesis (√)
refute a hypothesis (×)
gain new insight (!)
unclear (?)

How confident are you about the strength of the evidence you have gathered? The confidence level indicates how confident you are that the evidence is strong enough to support the insight (0 = not confident at all, to 1 = very confident). The more experiments you have conducted for the same hypothesis, the more confident you can be about its validity. This is especially the case when you have conducted call-to-action experiments that produce stronger evidence.

Action

Based on your insights and your confidence level, you can decide on one of four actions to take:

Persevere: typically this is when you have collected insights to support a hypothesis and feel fairly confident about the strength of evidence.

Test again: this is when you have collected insights to support a hypothesis but are not that confident about the strength of evidence. Either because the test you ran has fairly weak evidence strength or because the evidence from the test was fairly weak.

Pivot: typically this is when you have collected enough evidence to refute a specific hypothesis connected to a building block of your business model. For example, customer segments, key partners, or channels. This means taking the insights you have gained, going back to the drawing board and redesigning your business model accordingly.

Shelve: this is not an action to take lightly and something you do if you have collected several insights that have consistently refuted your hypotheses. Especially the hypotheses that are the most important to the success of your business idea.

Learn how to make data influenced decisions to systemically de-risk your business idea in our upcoming Testing Business Ideas Virtual Masterclass. Join us today!