The Key to Great Business Ideas? Metrics and Experiments with Small Groups

Businesses succeed with great ideas. But great ideas are hard to come by, at least great ideas that can achieve expected results. It turns out that the majority of business ideas simply never pan out or actually result in a negative impact rather than a positive one.

Data scientists from Stitch Fix recently pulled together some statistics and research about why business ideas frequently fail and how organizations can improve.

Research, particularly done by Stefan Thomke at Harvard Business School, found many examples of companies with poor track records when introducing new business ideas:

• Microsoft said that only about one-third of their new business ideas produce a positive impact.
• Google said that only about 4 percent of their ideas resulted in positive impact.
• Booking.com had success only about 10 percent of the time.

Identifying good ideas from bad ones early on is very difficult. The Stitch Fix authors wrote that “the problem is that many businesses behave as complex systems which cannot be understood by studying its components in isolation. Customers, competitors, partners, market force—each can adjust in response to the intervention in ways that are not observable from simple models of the components.”

A first step in improving odds of getting a winning business idea is to set up metrics that can measure the performance before and after the business idea is applied. That seems simple enough, but it turns out that many businesses never make an attempt to do it. Without metrics it’s hard to judge what the effect of the idea actually was.

How to be more successful with business ideas? The solution proposed by Thomke focuses on frequent experimentation. Try rolling out the idea to only a small subset of all customers. If the inital test proves successful, expand its reach, otherwise tweek the idea still on a small scale to improve results, or simply kill the idea and move on to something else. Thomke also suggests running lots of small limited rollouts to test many different ideas.

Thomke wrote that “the lesson is not merely that business experimentation can lead to better ways of doing things. It can also give companies the confidence to overturn wrongheaded conventional wisdom and the faulty business intuition that even seasoned executives can display. And smarter decision making ultimately leads to improved performance.”