Investors sitting through Incubator or Accelerator demo days have three metrics to judge fledgling startups 1) great looking product demos, 2) compelling PowerPoint slides, and 3) a world-class team.
We think we can do better.
We now have the tools, technology and data to take incubators and accelerators to the next level. Teams can prove their competence and validate their ideas by showing investorsevidencethat theres a repeatable and scalable business model. And we can offer investors metrics to play Moneyball with the Investment Readiness Level.
Heres how:
Weve spent the last 3 years building a methodology, classes, an accelerator and software tools and weve tested them on ~500 startups teams.
Hypothesis Heres What We Thought
Experiments Heres What We Did
Data Heres What We Learned
Insights and Action Heres What We Are Going to Do Next
We focus on evidence and trajectory across the business model.Flashy demo days are great theater, but its not clear theres a correlation between giving a great PowerPoint presentation and a two minute demo and building a successful business model. Rather than a product demo we believe in a Learning Demo. Weve found that Lessons Learned day showing what the teamslearnedalong with the metrics that matter is a better fit than a Demo Day.
Lessons Learned dayallows us to directly assess the ability of the team to learn, pivot and move forward. Based on the lessons learned we generate anInvestment Readiness Levelmetric that we can use aspartof our go or no-go decision for funding.
Some background.
NASA and theTechnology Readiness Level(TRL)
In the 1970s/80s NASA needed a common way to describe the maturity and state of flight readiness of their technology projects. They invented a 9-step description of how ready a technology project was. They then mapped those 9-levels to a thermometer.
Whats important to note is that the TRL is imperfect. Its subjective. Its incomplete. But its a major leap over what was being used before. Before there was no common language to compare projects.
The TRL solved a huge problem it was a simple and visual way to share a common understanding of technology status. TheU.S. Air Force, then theArmyand then the entireU.S. Department of Defensealong with theEuropean Space Agency(ESA) all have adopted the TRL to manage their complex projects. As simple as it is, the TRL is used to manage funding and go/no decisions for complex programs worldwide.
We propose we can do the same for new ventures provide a simple and visual way to share a common understanding of startup readiness status. We call this the Investment Readiness Level .
The Investment Readiness Level (IRL)
The collective wisdom of venture investors (including angel investors, and venture capitalists) over the past decades has been mostly subjective. Investment decisions made on the basis of awesome presentation, the demo blew us away, or great team is used to measure startups. These are 20thcenturyrelics of the lack of data available from each team and the lack of comparative data across a cohort and portfolio.Those days are over.
Hypotheses testing and data collection
Weve instrumented our startups in our Lean LaunchPad classes and the NSF I-Corps incubator usingLaunchPad Centralto collect a continuous stream of data across all the teams. Over 10 weeks each team gets out and talks to 100 customers. And they aretesting hypotheses across all 9 boxes in the business model canvas.We collect this data into aLeaderboard(shown in the figure below) giving the incubator/accelerator manager a single dashboard to see the collective progress of the cohort. Metrics visible at a glance are number of customer interviews in the current week as well as aggregate interviews, hypotheses to test, invalidated hypotheses, mentor and instructor engagements. This data gives a feel for the evidence and trajectory of the cohort as a whole and a top-level of view of each teams progress.
Next, we have each team update theirBusiness Model Canvasweekly based on the 10+ customer interviews theyve completed.
The canvas updates are driven by the 10+ customer interviews a week each team is doing. Teams document each and every customer interaction in aDiscovery Narrative.These interactions provide feedback and validate or invalidate each hypothesis.
Underlying the canvas is anActivity Mapwhich shows the hypotheses tested and which have been validated or invalidated.
All this data is rolled into aScorecard, essentially aKanban boardwhich allows the teams to visualize the work to do, the work in progress and the work done for all nine business model canvas components.
Finally the software rolls all the data into anInvestment Readiness Levelscore.
MoneyBall
At first glance this process seems ludicrous. Startup success is all about the team. Or the founder, or the product, or the market no metrics can measure those intangibles.Baseball used to believe that as well. Until 2002 when the Oakland As baseball team took advantage of analytical metrics of player performance to field a team that competed successfully against much richer competitors.
Statistical analysis demonstrated thaton-base percentageandslugging percentagewere better indicators of offensive success, and the As became convinced that these qualities were cheaper to obtain on the open market than more historically valued qualities such as speed and contact. These observations often flew in the face of conventional baseball wisdom and the beliefs of many baseball scouts and executives.
By re-evaluating the strategies that produce wins on the field, the2002 Oakland Asspent $41 million in salary, and were competitive with theNew York Yankees, who spent $125 million.
Our contention is that the Lean Startup + Evidence based Entrepreneurship + LaunchPad Central Software now allows incubators and accelerators to have a robust and consistentdata set across teams. While it doesnt eliminate great investor judgement, pattern recognitions skills and mentoring it does provide them the option to play Moneyball.
if you cant see the video above clickhere
Last SeptemberAndy Sack,Jerry Engeland I taught our first stealth class for incubator/accelerator managers who wanted to learn how to play Moneyball.
Were offering one again this Januaryhere.
Lessons learned
Its not clear theres a correlation between a great PowerPoint presentation and two minute demo and building a successful business
We now have the tools and technology to take incubators and accelerators to the next step
We focus on evidence and trajectory across the business model
The data gathered can generate anInvestment Readiness Level score for each team
the Lean Startup + Evidence based Entrepreneurship + LaunchPad Central Software now allows incubators and accelerators to play Moneyball