Companies: Beware of biased consumers

5/24/2010 - Marketers frequently rely upon surveys to gain insight into consumers' opinions and preferences. These results have important implications, ranging from the decision to add new product features to multi-million dollar rebranding efforts.

Gathering consumer insights is critical for any successful marketing campaign; yet, new research from the Kellogg School at Northwestern University reveals a potentially pervasive form of bias if surveys are not executed effectively.

In “Answering the Unasked Question: Response Substitution in Consumer Surveys,” Kellogg marketing professors David Gal and Derek Rucker have discovered a hidden bias when it comes to survey responses. In a series of three experiments, the researchers found that consumers have a tendency to misrepresent their opinions on surveys because they want to express opinions that were not asked, a phenomenon they have coined “response substitution.”

“Customers express their identity through the products and brands that they buy and own,” said Rucker, an associate professor of marketing. “We suggest consumers have a need to express themselves through their attitudes as well; because the need for expression can be great, when consumers cannot share their opinions they might just express them on an unrelated question, creating a bias.”

For example, imagine a customer who dined at a restaurant with terrible service and management. Later on, he is asked to fill out a survey about the quality of the food. The new study suggests that some respondents, such as this fictional customer, would use this opportunity to express their dislike toward the organization, even if they perceive that establishment to possess other redeeming qualities (such as excellent cuisine).

“Since the survey does not ask about the service at the restaurant, the only means for this unhappy customer to express his general disapproval of the service is by being negative about the food,” said Gal, an assistant professor of marketing. “Basically, respondents have the potential to project their negative attitudes onto the wrong aspect of the business so they can share their opinions. This bias can be a major source of misdirection for a company.”

To test their theory, the researchers devised three experiments. In one, participants were asked to evaluate the quality of candy produced by a company with immoral business practices. The Kellogg researchers discovered that individuals who viewed a company’s moral behavior as very important to their purchasing decisions rated the product quality of the candy poorly compared to respondents who did not view the company’s actions as very important to their purchasing decisions. Participants who felt strongly about the company’s actions were not only evaluating the candy’s quality but were using the quality rating as a means to express their disapproval toward the company’s practices.

Fortunately, there are ways managers and marketers can intervene. The authors suggest explicitly providing a place for people to express their opinions on the survey can help to minimize sources of survey bias.

“One simple remedy,” said Rucker, “is to inform respondents upfront that they will have an opportunity to express any other thoughts they might have in an open-ended format. That way, respondents can share any other opinions not intended for the survey.” Indeed, the researchers found that such a simple intervention significantly reduced response substitution.

Rucker and Gal also suggest questions be worded properly to limit misinterpretation and to improve comprehension.

The study will appear in a forthcoming issue of the Journal of Marketing Research.