The Power of Experimentation Frameworks

When I joined SelectQuote, product decisions were often based on assumptions, HiPPO opinions, or 'best practices' from other companies. While these can be useful starting points, they're not a substitute for evidence from your actual users.

I championed the adoption of Optimizely and introduced a comprehensive experimentation framework that fundamentally changed how we approached product development. The key wasn't just the tool—it was the cultural shift toward hypothesis-driven thinking.

Every experiment started with a clear hypothesis: 'We believe that [change] will result in [outcome] because [reason].' This forced teams to articulate their assumptions and define success criteria upfront. No more endless debates about whose opinion was right—we'd let the data decide.

We also implemented Jira Product Discovery to manage our hypothesis backlog and prioritize tests based on potential impact and effort. This created visibility across the organization and helped stakeholders understand the scientific rigor behind our approach.

The results spoke for themselves. Over 25+ experiments, we achieved consistent conversion lifts of 4-6% on tested experiences. More importantly, we also learned which ideas didn't work—equally valuable information that saved us from shipping poor experiences.

One of our most successful experiments involved simplifying our form flow. We hypothesized that reducing the number of fields on the first page would increase form starts. It worked—but we also discovered through follow-up tests that certain fields, when removed, actually hurt downstream conversion. This nuanced understanding only comes from continuous testing.

The lesson? Build a culture where it's safe to be wrong, where learning is valued as much as winning, and where evidence trumps opinions. That's when teams start doing their best work.