James & Paul interview Hannah Tickle, an experienced experimentation leader who's currently the Experimentation Team Lead at Trainline.

The episode discusses the role of experimentation in ecommerce, the importance of ownership of within organisations, and the impact of hypotheses, testing speed, and data analysis.

Hannah shares insights on building a culture of experimentation and the benefits of using a custom testing stack. She also touches on the challenges of incorporating user feedback and the importance of understanding the limitations of qualitative research.

Takeaways:

• True experimentation is essential for ecommerce businesses to improve strategic planning and prioritise product development.
• Experimentation should be centralised to ensure accessibility and consistency.
• Strong hypotheses are data-backed and focus on testing concepts rather than specific things.
• Testing should be fast, light, and big, with a focus on impactful changes rather than small tweaks.
• Understanding and analysing data is crucial for effective experimentation, and it's important to consider both quantitative and qualitative feedback.
• Building a culture of experimentation requires consistency, responsiveness, and support from senior leadership.