Skip to content
Skip to main content
header search
NeuralForge
Menu
sticky search

Three Tips for Product Experimentation

LX Berlin is a community devoted to sharing skills and knowledge around learning experience with the ed-tech industry. Powered by Babbel, LX Berlin has organized a series of events since its start in 2019, offering insights into various aspects of learning experience. For its 8th edition, LX Berlin will combine an exciting new format and a partnership with the Fountain Institute, an online school for mid-career product designers. 

LX Berlin will team up with Jeff Humble and Hannah Baker from The Fountain Institute (TFI) to offer an impactful workshop on product experiments especially useful for UX designers, product designers, and product managers on March 3rd, 2022. 

But if you can’t wait to start learning, you can do it right now! We’ve asked our experts from the upcoming LX Berlin event, Jeff Humble, co-founder of TFI, and Anna Stutter Garcia, one of Babbel’s Senior LX Designer, and put together a list of 3 essential tips for product experimentation. 

1. First Thing’s First

Is product experimentation always the most optimal method to deal with a customer behavior question? Short answer: “no”. Jeff warns against unnecessary interventions:  “experiments are all about generating behavioral data on your customers. Sometimes, that data already exists, and all you have to do is gather it. If data already exists, research might be a more productive use of your time.”

How do you know when to experiment? Jennifer Dorman, Lead of LX Design in Babbel, explains: “in many respects, experimentation is about gaining confidence that a solution or intervention is going to solve a user problem or fulfill an unmet need. Sometimes that confidence can be gained by, first, ensuring that you deeply understand the user’s problem. This can be achieved through primary (talking with users) or secondary research (reviewing prior research studies and other academic resources). Second, concept-testing early ideas through user interviews or unmoderated concept tests can de-risk the solution or highlight the need for iteration before the time is spent to build it. Sometimes that solution can be sufficiently de-risked to justify a full launch.”

2. It’s Better To Be Wrong Fast

Naturally, you might hope to find evidence for your predictions in the results of a product experiment. But is that really what it’s all about? What defines success in product experimentation? According to Jeff Humble, “success in product experimentation should always be about the learning. If your hypothesis doesn’t make a bold claim, you might find yourself with lots of unsurprising results that don’t teach you anything.” Anna Stutter Garcia adds, “sometimes we quickly jump into designing experiments without thinking the question and predictions through, finding ourselves with muddled results that give us no direction for our next steps and no clear learnings.”

In other words, the worst result is no result. Don’t be afraid to fail. As Jeff explains, “experiments are like little bets we place on customer behavior. It’s better to make a big bet and prove yourself wrong quickly than to test a bunch of ideas that are safe. This is the basis of the lean mindset where your goal is to ‘be wrong, as quickly as possible,’ a mindset meant to help you innovate quickly without a lot of budget.”

3. Be Careful What You Wish For

So not everything needs testing, and hypotheses should be impactful. But what else could go wrong? According to Jeff, it’s vital to get outside of your own head: “one thing to watch out for when designing your hypothesis is confirmation bias. If we put on our scientific hat, we want to test our assumptions, not validate our ideas. That’s really hard to do unless you get everyone to agree on the assumptions first. That’s why we created the Experiment Cards: so you can pull out the assumptions with your team and test those first, rather than testing by ‘validating your ideas’ which can lead to confirmation bias.”

Now that these guidelines have given you some idea about product experimentation, make sure to join the LX Berlin session on March 3rd for a deep dive on the topic and feel free to bring in specific questions for a Q&A with our experts. In the meantime, check out these articles on product experimentation on The Fountain Institute blog!

Photo by Kelly Sikkema on Unsplash

More exciting content