How to Evaluate Startup Ideas — Part 1

After graduating from Y Combinator Startup School and building several startups, with the recent reaching over 5M users, I’ve been eager to share lessons learned when evaluating startup ideas.

According to a TED talk by Bill Gross, founder of Idealab, timing accounts for 42% of a startup’s success. He looked at five factors (funding, business model, team, idea, timing) that he believed are important, to get the one factor that accounts for a startup’s success. After analyzing the five factors on hundreds of successful startups, funding and idea only accounted for 14%-28% while timing hit the jackpot.


Part of why many great ideas fail is because the market wasn’t ready. Or in other words, it took too much time and money to educate the market to believe they needed what was being offered. A rule of thumb I go by is that you can’t change or manipulate the market. Not as a small fish at least.

If timing is the biggest factor for success, how can we evaluate when is good timing?

One of my favorite metrics for evaluating the readiness of a market is called the Knowledge Spectrum as seen below.


The spectrum represents all the knowledge in a certain field or market. The far left, represents no knowledge at all, while the far-right represents all knowledge or expertise. As you can assume, nearly anyone knows all there is to know in a specific field, but at most times there is too little knowledge.

The ‘current’ pointer represents where you assume the average user is current in regards to his knowledge, and the ‘target’ pointer represents the knowledge required to want your product solution. The knowledge gap represents the amount of education needed to get the user from his current knowledge state to the target.

Too little knowledge signals it would be very expensive to convert users to your product idea, therefore it’s more likely to fail. Too much knowledge might signal it’s already too crowded of a market. Therefore, your startup’s idea has to fall somewhere in the middle where a knowledge gap exists but is bridgeable within a short time frame of education (like a landing page). You need to assume with high confidence that your average user is currently placed close to the target (your solution).

Your startup’s idea has to fall somewhere in the middle where a knowledge gap exists, but it’s bridgeable within a short time frame of education

There are many examples of successful startups that had good timing on their side (Facebook), or that were too late in the game (Windows Mobile). So let’s give some interesting examples of startups that were too early, to emphasize the importance of market education. WebVan provided online grocery shopping in the late 1990s. Around the-dot com bubble, consumers were still not educated in using the internet for on-demand services(knowledge gap was too wide). WebVan burned through almost $400M in venture funding, only to crash and burn, yet ordering groceries online is commonplace today.

Another good example is Dodgeball which offered social location sharing via text messages as early as 2003. The company was acquired but eventually shut down. Checking into locations is now common behavior on Facebook and similar platforms.

The Knowledge Spectrum has a lot of depth to it and can be used in other areas as well. For example, you can use it to improve conversion rates within your landing page. By understanding where your average user’s current knowledge is, you can set the target (the landing page content) such that the knowledge gap is small. If you provide too much knowledge or set the target too high, conversion rates will significantly decrease since it’ll take too much energy from your users to understand what you’re selling, therefore they’ll most likely churn.

There are so many factors to take in mind when evaluating a startup idea. But one factor that is hardly measurable happens to be a very important one — luck (or bad luck). For example, think of all the startups that launched right before the great recession, or those that launched right before a hype (like cryptocurrencies). The metrics we use to measure an idea, are simply there to help asses the risk — the probability of failure or success.

In my next articles, I’ll show more popular metrics and things to consider when evaluating startup ideas, so stay tuned!