When (and How) to NOT Listen to Your Users
We’re obviously big proponents of understanding your audience and users through research and empathy. It’s one of our 12 competencies of UX design because it helps you get out of your own head and understand what motivates users to adopt your product, service or solution.
But one of the keys to leveraging user input effectively is knowing how to not listen to them.
Steve Jobs famously turned the old maxim “the customer is always right” on its head with this quote:
“Some people say, ‘Give the customers what they want.’ But that's not my approach. Our job is to figure out what they're going to want before they do. I think Henry Ford once said, “If I'd asked customers what they wanted, they would have told me, 'A faster horse!'" People don't know what they want until you show it to them. That's why I never rely on market research. Our task is to read things that are not yet on the page.”
That’s not to say that user research is ineffective or a waste of time. It’s just that humans are unreliable when reporting their past behavior and inaccurate when predicting their future needs.
To get the full picture, you need to step into the lives of your users and see how they behave in the present. You need to be there alongside them as they navigate their daily tasks and see their problems through their eyes.
That’s why the Nielsen Norman Group’s first rule of usability testing since 2001 has been “Don’t listen to users.”
If you design your product based solely on what users tell you they want, you’ll end up with a product full of features that lack a cohesive vision and don’t deliver delightful and meaningful experiences and results.
Here are a few reasons that listening to users, rather than observing their true behavior, can lead you astray.
It’s not enough to know what users want.
User feedback most commonly comes from complaints about existing features or requests for new features. While it can be helpful to know what users want from your product, it’s not enough. You need to understand the “why.”
Focusing simply on the “what” will lead you and your product team on a neverending chase for feature requests that may or may not solve the customer’s underlying problem or need. To truly make the best use of feature requests, whether they arise in user testing, come from your sales team’s conversations with customers, or find their way to you through the leadership team, you need to understand the reason for the request.
Why do people think they need this feature? What perceived value are they hoping to get out of it? What problem do they think the feature will solve? Is this a common problem, or one that only applies to a small subset of special users or circumstances?
How to Dig Deeper
When you get a feature request, first ask “Why?” and then find ways to uncover answers to these deeper questions. Look for indications from user metrics, identify patterns of behavior, and conduct usability sessions to watch how users react to existing or proposed features in a real environment. From there, you can determine if the feature request makes sense for your product and you may even uncover new ideas that can solve the underlying problem even better than the initial request.
It’s not enough to listen to what users say.
In almost every usability testing we’ve conducted for our clients there has been a moment when the user has said one thing while doing the exact opposite.
The fact is that humans are not good at accurately representing their experiences through words. That’s because we all have our blindspots - the unconscious habits, motives, and thought patterns that drive our behaviors.
If you ask a user how they would get to a certain piece of information on a website, they might tell you they would follow your intended architecture for that task perfectly. But when you leave them to figure it out on their own you might find they wander from one point to another in search of the information.
We tend to follow roads when they’re the only way to get to what we need, but we quickly carve out informal paths, or ant trails, that get us there faster. Identifying the paths your users have carved out will help you build better roads for them in the future.
How to Dig Deeper
It’s far more important to look for what your users do than to listen to what they say. User interviews are a good tool for understanding motivation and intention, but they’re terrible at uncovering actual behavior. For that, you need to watch users in their natural use of the product. Conduct usability tests, examine behavioral metrics and screen recordings, and uncover any other data that can show you how they actually interact with your product.
It’s not enough to learn from your active users.
The tendency in product development is to listen to active customers and users and react exclusively to their needs and desires. The assumption is that you should address the needs of your existing customers in order to ensure their continued loyalty.
The problem with this approach is that you can often miss valuable insights from people who choose not to use your product.
Obviously, you need to know your target audience and design for their needs. No product can be everything to everyone. But the people who are not choosing to adopt your product are sometimes just as important as your fans and promoters. They can often tell you more about how you’re not realizing your full potential.
Users who are unfamiliar with your product, or those who have an unfavorable view of it, can be a gold mine of insights to optimize your product or service.
How to Dig Deeper
Conduct a competitive analysis and test other products to look for evidence that can tell you why users are not adopting your product. Why did some try your product but never come back? Why are others sticking with the competition and unwilling, or unable, to switch to you? What do they expect your product to do that it isn’t doing? Does that fit into your own vision for the product?
These questions can lead you to ideas for new features and even give you a new direction for your product to evolve. They can also be a great way to highlight the problems that your loyal users have learned to avoid. This can be especially helpful with users of internal business tools who often don’t have a choice in purchasing those products. They’re the most likely to find workarounds that hide the true problems in your product and prevent you from growing your solution.
It’s not enough to get feedback about your product.
The key to getting the right answers is knowing the right questions to ask. If you limit your user feedback to only what exists today in your product, then you’ll only get feedback about your current product. What about all the things your product could be in the future?
When you focus on gathering feedback about your existing product, you limit your understanding in a few important ways.
First, most people are going to lie to you about how they really feel. This usually isn’t an intentionally misleading lie, it’s just that most people are polite and don’t want to be seen as criticizing somebody else’s work or being cruel about their feedback. They’re much more likely to be honest when not talking directly to you as the product owner or designer. This makes anonymous discussion boards, review sites, and third-party researchers extremely valuable mechanisms for collecting feedback about your product.
Second, and more importantly, your users live full lives outside of your product. If you only ask them about their lives while they’re in your product, you’ll get a very limited view of their lived experience. They won’t think to tell you about the times they don’t use your product or when they intend to go into it but never do because life gets in the way.
How to Dig Deeper
It takes time and resources to uncover the context within which your product fits in the lives of your users. It’s not enough to ask them. You need to conduct diary studies and employ ethnographic methods to truly understand why they do what they do and how your product can make their lives better. Without that, you’ll continue to make small iterations on a tiny sliver of their lives without making the kind of exponential and meaningful impact that defines the most successful products.
Let us help you figure out when (and how) to NOT listen to your users.
We’ve conducted hundreds of hours of user research across dozens of product lines and business types. That kind of experience is the only way to truly know which user feedback is trivial and which is full of insights that can lead you to a clearer product vision and drive more value to your customers.
Let us put that experience to work for you and help you discover the ideas that will make your product more meaningful and delightful.