Growth Experimentation

Growth experimentation is the disciplined, high-velocity process of generating hypotheses, running tests, and analyzing results across every lever of growth -- acquisition, activation, retention, monetization, and referral. It is the execution engine that turns product-market fit into scalable growth. Sean Ellis frames it as "a scientific approach to figuring out how to grow the business." The most important insight from a decade of growth work: the single biggest predictor of growth in any company is the number of tests they run per week. Not test quality. Not test cleverness. Volume. Jeff Bezos: "Our success at Amazon is a function of how many experiments we run per day per week per month."

The Growth Hacking Process (Ellis Framework):

1. Analyze -- Understand the current state. Quantitative: where are the biggest drop-offs in the growth engine? Qualitative: why are users behaving this way? Survey, interview, observe.
2. Ideate -- Generate a large backlog of test ideas. Anyone on the team can submit ideas. Weekly brainstorming sessions with each goal owner. Each idea is documented as an experiment with a clear hypothesis.
3. Prioritize -- Use ICE scoring to rank ideas:
   • I (Impact): If this works, how big is the impact on the metric? (1-10)
   • C (Confidence): How likely is it to work based on evidence? (1-10)
   • E (Ease): How easy is it to run the test and get a result? (1-10)
     Don't overthink the scores -- they're directional, not precise. "Sometimes people overthink it and start debating what the ice score should be. It's just for directionally steering you."
4. Test -- Run 3-5 tests per week (scale with team size). Use tools like Optimizely, VWO, or Unbounce for web experiments. For deeper product changes, build minimum viable tests.
5. Analyze results -- Share all results broadly -- wins and losses. Every test leads to learning. "Our analyst is the person who's generated by far the most ideas on our team because he sees the results before anybody else."

Velocity Over Quality:

Twitter's growth stalled at the end of 2010 when they were running 1-2 tests per month. A new VP of product increased the testing rate to 10 tests per week. Growth recovered quickly.

Ellis's own experience at GrowthHackers: "We were probably only running a test every other week even though when I asked the team 'oh yeah we run lots of tests.' When we actually tried to count, it wasn't that many." Once they targeted 3 tests per week and held themselves accountable, they broke out of 3 months of flat growth.

The math: if you run 1 test per month, you get 12 chances per year to find a breakthrough. If you run 5 tests per week, you get 260. The probability of finding the one test that changes the trajectory is dramatically higher at 260 shots than 12.

The Battleship Analogy:

Growth experimentation is like playing battleship. You don't know where the wins are hiding. The more shots you take, the more likely you are to find them. A team that obsesses over the "perfect test" and runs one experiment per month will almost always be outperformed by a team that runs 20 mediocre-but-fast experiments in the same period.

The Weekly Growth Meeting:

The heartbeat of the growth process. Structure:

1. Review results from last week's tests (15 min)
2. Update key metrics and goal progress (10 min)
3. Nominate tests for this week using ICE scores (20 min)
4. Assign ownership and resources (10 min)
5. Quick discussion of any blockers (5 min)

Rules:

• Every goal has an owner
• The owner nominates tests for their goal; anyone can nominate tests for any goal
• Balance tests across key objectives (don't put all 5 tests on one goal)
• Never discuss ICE scores in the meeting -- those are for individual prioritization
• Target 3-5 launched tests per week

The Double-Down Test:

When a test wins, the next question is: how far can we push this? If changing the headline produced a 20% improvement, what about changing the entire page layout? If a new onboarding sequence improved activation by 50%, what about adding a personalized element? Winners get a second round of deeper testing. Most teams celebrate a win and move on. The best teams double down.

1. Obsessing over one perfect test: Spending weeks designing, debating, and refining a single test. --> Opportunity cost of all the tests you didn't run. --> Launch the scrappy version this week. If it wins, then invest in the polished version.

2. Testing only acquisition channels: Confining experiments to ad platforms and landing pages because the product team won't cooperate. --> This is marketing, not growth hacking. --> Build the cross-functional alignment needed to test across the full growth engine.

3. Not sharing results: Running tests but keeping results in a spreadsheet that only the growth team sees. --> The rest of the company doesn't learn, doesn't contribute ideas, and doesn't understand why growth decisions are made. --> Share all results (wins and failures) with the full team weekly.

4. Treating testing as a project instead of a habit: Running a "testing sprint" for a month and then going back to normal. --> Growth hacking is not a campaign -- it's a permanent process. --> The weekly growth meeting never stops. Testing cadence never drops below 3/week.

5. Ignoring failed tests: Only celebrating wins and burying failures. --> Failed tests are data. They tell you what doesn't work, which narrows the search space for what does. --> Document and share every failure with the insight it produced.

• North Star Metric (prerequisite): Every experiment should move a metric that ladders up to the NSM. Without a clear NSM, experiments lack direction and results lack meaning.
• New User Activation (prerequisite): Activation is the highest-leverage testing area. Most early experiments should focus on activation before expanding to other levers.
• Product-Market Fit: Experimentation is premature without PMF. Testing growth tactics on a product nobody loves is like "optimizing the deck chairs on the Titanic."