Few would disagree that fundamental economic change is upon us. Business models are crumbling daily. From the auto industry to the banking industry, it is clear that old ways of doing things are no longer working. The market research industry is just as vulnerable to this shift, yet, like the auto industry before it, it is hardly aware of how deeply its business model is threatened.

The Long Disruption

The market research industry is built for the 20^th Century mass production model, which is rapidly disappearing. The “mass audience” is gone and a fragmented diverse populace has taken its place. This new “audience” defies the easy aggregation of summary statistics on which market research so often relies.  Chris Anderson of Wired figured this out long ago with his book The Long Tail.

The Economic Disruption of The Long Tail -- Anderson, Wired Magazine

He argued that technology lowered the cost of providing services to ever-smaller niches of people, making it possible to sell profitably goods and services that were once too specialized.

This technological shift also means the end of “appointment television.” Digital video recorders allow individuals to time shift their programming to suite them, and not the program executives at television networks.

The Birth (And Death) of Market Research

What does this all have to do with market research? Full-service market research firms are built for the blockbuster era, not for the time of the long tail.

Market research was heavily influenced by the school of “applied sociology,” lead by Paul Lazarsfeld. While at Columbia, Lazarsfeld pioneered many statistical techniques we use today, including the cross tabulation (Babbie and Benaquisto 2002) and the Lazarsfeld-Stanton Analyzer, a machine that records audience reaction to programming in real time (Mattlerart 1996).

The Lazarsfeld-Stanton Analyzer summarizing "the public"

CNN used a variant of this machine for the recent State of The Union address, showing real-time reactions from Democrats in blue, Republicans in red, and Independents in yellow.

The Lazarsfeld brand of insight is based on a fundamental assumption: that the “average” means something. An entry-level statistics course will teach you that average is dragged up or down by extreme values, and the long tail is nothing if not a collection of many extreme values. There is nothing meaningful about knowing that the “average American” rented 30 digital movies a month if, in fact, there were many thousands of Americans who rented none and a many tiny segments that rented somewhere between zero and 40 movies. The “average” is meaningless in this example, yet this ham-fisted approach to summarizing “the public” is what the market research industry is built upon.

Design Research for The Long Tail

Market researchers may argue that with proper segmentation, you can understand every niche within the long tail. This may be true, but to truly understand the diversity between people, your task is not simply to “summarize” the audience, but to delve deeply into the dynamics of what makes them different.

This is why design research is a better fit for today’s long-tail economic model. Context matters. Design research is all about understanding the context because it is rooted in qualitative methodologies, and ethnography in particular. Designers solve contextual problems. The award-winning Braille watch, for example, allows its users to check the time surreptitiously and quickly, something that is both polite and useful. A Lazarsfeld approach would not uncover the social subtleties of checking one’s watch, and certainly could not uncover the specific needs of the blind.

The Braille Watch by David Chavez

Dan Formosa details this limitation of market research in his insightful article in Interactions magazine. He argues that market research should focus on consumer response — after a product is designed. Design research, on the other hand, is about evaluating a product as it is being developed. I would go further; design research is about knowing what to build as well as evaluating the prototype.

Design research uncovers how long-tail niches develop and what differentiates them. It is not the equivalent to “market segments” because it provides specific direction on how to apply research findings. What are the dynamics of renting a movie? What motivates the “heavy renter”? What is it about her television or home that supports heavy renting? You cannot know the answer to these questions by simply providing a laundry list of demographic characteristics and psychographic survey results. You must know the context in which the long tail emerges.

Some may say that good quality market research would not make these kinds of mistakes. And they are right. Highly skilled social scientists are method-agnostic; they choose the right method for the right research question. However, full-service market research firms have become the GM of the industry — they keep building Hummers instead of Priuses. Focus groups don’t uncover contextual nuances, but they’re cheap and profitable. Surveys don’t get to the heart of why a product doesn’t work. Design research, using an ethnographic approach, provides “thick description” of the entire phenomenon of renting movies.

This is where market research cannot go. And this is where market research will fail, unless it rejects the “build another Hummer” mentality.

References

Babbie, E. and L. Benaquisto (2002). Fundamentals of Social Research. Scarborough, Thomson Nelson.

Mattlerart, A. (1996). The Invention of Communication. Minneapolis, University of Minnesota Press.