How to Innovate with Quick Experiments

Product Discovery Group works with product teams as a mentor and Product Discovery Coach to build confidence in their ideas more quickly using less engineering effort. They engage clients and their customers in idea development, user testing, user research, and developing rapid prototypes.

I see a lot of building. Build, build, build. And in that process, people are really falling into the product discovery valley of death. You have great ideas, but you don't test them so there's the risk of people not seeing value and not adopting your product. You may have interviewed folks, but you haven't taken the solutions back to those folks to really see if they would use them.

Avoiding the Product Discovery Valley of Death

You can innovate and avoid the valley of death by testing solutions with quick experiments. This includes two key elements: getting the right team, and getting the best ideas from the team.  

When I say the right team, you really need a cross-functional team. This is important. As you get more folks on the team earlier in the process and you solicit their ideas, you're going to get a much richer set of ideas to start from. That might be in the problem space. That might be in the solution space, but you need this cross-functional team. I have clients that have a data scientist on every cross-functional team. I have clients that have doctors on every cross-functional team. So sometimes you need subject matter experts, sometimes data experts, or sometimes people from customer support.  

To get the best ideas from the team I want people to draw their ideas. All of the team should draw, not just the designer. The Sprint book is a great resource for this type of activity. But you don't need to do these things in five days. You can just take these exercises and apply them every sprint, every discovery cycle.

Innovation Challenges

The most common problem I see is that folks create cross-functional teams, but they don't drive engineer participation. And so what they're losing is what's just now possible. Engineers can hear the problem and they can apply a new solution they've been thinking about or reading about or one that might just come out of what their current workstream is focusing on. It's really important to make sure that you don't lose touch with your engineers. Keep them involved.

A lot of the most commonly used companies today had either all or more than half engineer founders. Amazon took some just now possible technology; secure websites and widespread internet access to launch in the 90s. My company Power Reviews, grew to 1,200 clients in 20 countries, all powered by iFrames and Ajax. Tesla really started with this concept of a long-lasting battery.  

Quick Experiments

Let’s use an example of a backlog prioritisation experiment. I want you to pretend that you work for Zoom and that you have thought of a bunch of ideas for improving breakout rooms. I can tell you that the various frameworks for prioritisation aren't that useful because you're often making up numbers for what's the impact, what's the confidence, what's the effort. If you do some discovery about the ideas, do some user testing, you can learn about the priority with respect to your customer's value. Do they want this thing? That's really what we're interested in. So let's go talk to users!

One experiment is that we could start sending push messages to users to gauge their interest. I call these micro prototypes. An examples could be:

“Zoom now offers advanced features to manage the success and health of breakout rooms.”  

I can gauge whether somebody's interested in breakout room improvements right away. Would they swipe to learn? Would they dismiss? Have them talk about it in front of you in a moderated interview.

Alternatively we could show different messages to quickly validate other ideas.

“Zoom now offers the ability to listen into a breakout room without having to enter it.”

“A new feature, see who has video turned on or off in breakout rooms without having to enter them.”

“New feature, for breakout rooms Zoom shows you which rooms are the quietest and which rooms have the most people talking.”

Again, you may not be the product manager for Zoom breakout rooms or Zoom itself but what I'm encouraging you to do is to think of the ideas in your backlog and quick ways to explain them to users and to get their interest.  

The results of the quick experiments help you to decide what to work on. I did this with a client who worked with educational technology. We went into a classroom and we showed a variety of ideas and the team was able to hear the excitement and understand the response as to which of these ideas was most valuable.  

Quick Experiment Challenges

People test too much, creating end-to-end experiences. You should not test an entire experience. I want you to pick a piece of that experience to innovate. If you have an e-commerce flow just test add to cart? We can be quicker if we can focus. You can certainly test each of these other aspects, but to be quick, you want to test them one at a time, not in these mega testing sessions.

I also want you to start with a vertical slice. So what is really valuable about that add to cart area? Can you launch something? Can you deliver a piece of value quickly?

Learning from experiments

Experiments help you avoid confirmation bias. When you offer no choices to users, effectively you're saying, hey, choose this thing. When you offer choices, you're being open-minded which means you might end up selecting ideas that came from different sources and people on the team.

Another issue that you will face is “blah” feedback. “I like this”. “That's interesting”. This is all kind of meh feedback, this is not what you want. People are being nice to you. I want to hear, “ooh”. I want to hear, “oh, wow”. I want to hear, “is this available?”. I want that emotional reaction.

The benefits of quick experiments

I worked with an entrepreneur who spent $200,000 building a website that nobody went to. He had found an outsourced company to build his dream idea. But the issue with the dream idea is that he wasn't able to market it and didn't quite understand what people would see value in it. It was valuable to him, but he hadn't done the research with others.  

And so when he came to me, we started from scratch. We built some prototypes to test what people would see on this website. And I'll tell you what this website does. Imagine that you have a home project and you need a plumber or an electrician. While you have to go find somebody, you might be doing it online. A lot of people ask their friends or their family. The premise of this site was that you would ask your neighbours. And so how can you facilitate the asking of neighbours? Will you have to find them? What neighbourhood are you in? And we tested the interest in this idea and we tested how people could find out about it. And what we learned led us to create a very bare-bones, no-code website.

We started with fake prototypes and quick experiments. We then moved to the next set of quick experiments which was to use a Squarespace site with Google spreadsheets and to drive traffic to that site from Nextdoor and Facebook groups. As you can imagine Squarespace plus Google Sheets is much cheaper than a $200,000 site custom. When this site got too overloaded and we started to actually drive traffic we went on to a site using Webflow.

Then he also decided to pivot his idea into a home service subscription service. So for plumber and electrician, whatever you needed, what about a monthly fee to have that taken care of? And so through these quick experiments, this entrepreneur was able to pivot and then grow his understanding of his market. And this was in a fraction of the time it took to build this other site that no traffic was going to.  

So again, quick experiments can help you innovate to get to the right idea. And so this is really what we're looking for. Instead of building all the time, it's learning, building, and then measuring.