How Cogniteer Built an Experimentation Engine From Scratch

Summary In this episode of The Experimentation Edge, host Ashley Stirrup talks with Fabian Hans, founder and behavioral psychologist at Cogniteer, a consultancy that helps enterprises build in-house experimentation programs and raise both test velocity and win rate. Drawing on fifteen years in conversion rate optimization, Fabian explains why mass-producing the same A/B tests across clients quietly kills learning, why most ecommerce drop-offs are structural rather than your fault, and how matching the interface to how people actually buy, new versus returning, B2C versus B2B, can move conversion far more than another button. It is a practical, psychology-grounded conversation for product managers, engineers, data scientists, and growth leaders who want their experimentation programs to compound understanding, not just volume. Chapters 00:00 Introduction 01:25 From agency mass production to in house deep dives 04:05 Why some products resist selling online 06:35 The drop offs every ecommerce shop shares 07:45 The 50% win rate test Cogniteer reused 09:25 Why alignment beats developer resources 12:45 Two teams, two goals, one broken checkout 17:05 Selling water dispensers without a product catalog 22:15 Designing every experiment to lose 26:15 Personalizing buyers and where AI takes experimentation Takeaways - Deep dives beat mass produced tests, because understanding one business's users uncovers bigger levers than reusing the same test across many clients. - Many ecommerce drop offs are structural, since the basket and product page leak in roughly 80% of shops because it is ecommerce, not because of your product. - Product to channel fit decides what sells online, so books and fashion judge well on a screen while perfume and washing machines need cues the interface cannot fully provide. - The real bottleneck is alignment, not developer resources, so agree on the problem and its hierarchy before anyone builds a variation. - Match the interface to how people actually buy, because new buyers need information, returning buyers want speed, and B2B buyers often want a solution and an offer instead of a product catalog. Connect with the Guest Fabian Hans LinkedIn: https://www.linkedin.com/in/fabianhans-cogniteer/ About the guest: https://www.growthbook.io/podcast/guests/fabian-hans Episode page: https://www.growthbook.io/podcast/episode/1-29 Company Website: https://www.cogniteer.de/ Sponsor GrowthBook is the warehouse-native platform for experimentation, feature flags, and product analytics trusted by AI-native product teams at 3,000+ companies worldwide. Go to http://growthbook.io