How To Listen To What Sally Is NOT Saying
In part 1, "How to Improve Product Conversations with Your Customers," you learned how to conduct qualitative research and learn how to improve customer conversations based what Sally Shopper is saying. So, you’re doing qualitative research. Good for you! You’re finished now, right? Wrong. Why?
Not telling the truth is not the same thing as saying that Sally is lying. There are many reasons why Sally may not or cannot tell you how she feels. Here are a few common reasons:
The Dunning-Kruger effect is a bias people have to estimate themselves much higher than is realistic. Studies confirm that most people have a tendency overestimate their humor, logic, and their own ability.
"Across the four studies, the authors found that participants scoring in the bottom quartile on tests of humor, grammar, and logic grossly overestimated their test performance and ability. Although test scores put them in the 12th percentile, they estimated themselves to be in the 62nd."
Once people have created an identity, they are unlikely to accept anything that does not agree with their idea of who they are. If Sally believes herself more intelligent than most others, it's not likely that she would confess that she couldn’t figure out your website.
Not saying can also be a byproduct of culture. The Japanese, for example, are masters of the not saying game. They will avoid a "no" at all costs and hand out unqualified "yesses" left and right. Many a Westerner has left the negotiation table in Japan scratching their head wondering, what just happened?
Maybe Sally is just polite. Some people were raised not to speak ill of others. Or maybe she loves your sales members but hates your product and is worried that if she complains, they might be at risk.
Whatever the reasons, most people will not or cannot tell you the whole truth. So it's hard to get all of the answers just by asking questions. You'll miss opportunities to increase sales if you don't improve conversations by using your product to listen to what Sally isn't saying.
Figuring out what Sally is not saying is where observational research and quantitative research come to the rescue. These two methods are easier to understand because they do precisely what they sound like they do.
- Observe: to watch.
Quantify: to make into a number. (Just the cold hard facts here.)
Observational research sees Sally’s interactions either directly or with recorded videos such as mouse trackers.
Alternatively, quantitative research measures completed tasks. There are several ways to measure completed tasks. You could, for example, measure how many people visited the page, or how many people clicked through to a goal page.
Observational and quantitative or analytic research experiments sit behind the scene, creepily watching how Sally the interacts with your product. It's like the Norman Bates of research. But rather than think Psycho it's best to understand how to help Sally reach her goals better and faster. Where does Sally get stuck in the process of trying to complete a task? Does she repeatedly try to use your product in a way it wasn’t intended to be handled?
Quantitative and observational research are both great ways for Sally to tell you what she's thinking without ever saying a word. So which one is better?
Well, you have to use both. Here's why. Analytic research is excellent at identifying which successful actions have taken place, but not at unsuccessful actions. Imagine Sally has visited your website and there is a big image at the top of the page. Observational data can reveal that Sally is clicking the image which is not set up as a link. She thinks you're asking her to take action, but you are just creating a beautiful picture. Observational data is one of the best ways to see where Sally is failing. Alternatively, quantitative data is perfect for seeing how Sally has succeeded.
Observational data is one of the best ways to see where Sally is failing to accomplish her goal. Alternatively, quantitative data is perfect for seeing how Sally has succeeded.
Observational research is easy to conduct with the right tools. Quantitative research requires analysis and a product understanding to get the most value out of the data you've collected.
Watch What's Happening
To gather useful research data watch Sally for hesitations when she's trying to complete a task. See if there are any attempts to click unclickable objects, or to perform a task in an indirect way.
How to Conduct Observational Research:
Heatmapsfor click, scrolls, and mouse movements (Hotjar)
- Focus groups (moderated or remote)
- Eye tracking
- Beta testing
- Learnability testing
Measure What's Happening
To collect good quantitative data that will lead to valuable results, you'll need a good understanding of data analytics and the software used to collect the data. You'll soon learn that "segmentation" is your best friend.
For example, segmented data may show you that women are 75% of your customers. If so, that information is valuable in determining future product improvements and communication with customers. So when you study the numbers, always look for ways to improve your ability to segment the data. You can read more about segmentation here.
How to Conduct Quantitative Research:
- Google Analytics (Widely-used for technology and free)
- US Census Bureau (Census data including people, businesses, and trade)
Why Not Only Use Those Observe And Quantify?
As powerful as observational and quantitative research are, they don’t tell you how Sally is feeling. They never give her a chance to tell you her side of the story, or what matters most to her. To learn more about qualitative research, read part 1, How to Improve Product Conversations with Your Customers.
Let Pivot help you with a digital marketing strategy to create better conversations with your prospects.
For a complete guide to improving your results with your Western audience, download the “East Meets West - Improves Sales with Western Customers” free ebook.