On or around October 7, BlackBerry users in Europe started noticing problems with their email and BlackBerry Messenger (BBM) service. By October 12, the outage had spread to North America. Not all of BlackBerry’s 70 million customers were affected, but a significant portion certainly were.

"How did the recent BlackBerry outage make you feel?"

Copernicus saw an opportunity to understand a little more about the BlackBerry user and their attachment to their smartphone. We decided to strike while the iron was hot! We immediately launched an online survey to get users’ fresh impressions of the BlackBerry outage.

This is the first of several analytical papers we will offer, based on the data collected.

BlackBerry Outage Survey Report by Copernicus Consulting Group is licensed under a Creative Commons Attribution-NoDerivs 3.0 Unported License.
Based on a work at copernicusconsulting.net.

Few would disagree that fundamental economic change is upon us. Business models are crumbling daily. From the auto industry to the banking industry, it is clear that old ways of doing things are no longer working. The market research industry is just as vulnerable to this shift, yet, like the auto industry before it, it is hardly aware of how deeply its business model is threatened.

The Long Disruption

The market research industry is built for the 20^th Century mass production model, which is rapidly disappearing. The “mass audience” is gone and a fragmented diverse populace has taken its place. This new “audience” defies the easy aggregation of summary statistics on which market research so often relies.  Chris Anderson of Wired figured this out long ago with his book The Long Tail.

The Economic Disruption of The Long Tail -- Anderson, Wired Magazine

He argued that technology lowered the cost of providing services to ever-smaller niches of people, making it possible to sell profitably goods and services that were once too specialized.

This technological shift also means the end of “appointment television.” Digital video recorders allow individuals to time shift their programming to suite them, and not the program executives at television networks.

The Birth (And Death) of Market Research

What does this all have to do with market research? Full-service market research firms are built for the blockbuster era, not for the time of the long tail.

Market research was heavily influenced by the school of “applied sociology,” lead by Paul Lazarsfeld. While at Columbia, Lazarsfeld pioneered many statistical techniques we use today, including the cross tabulation (Babbie and Benaquisto 2002) and the Lazarsfeld-Stanton Analyzer, a machine that records audience reaction to programming in real time (Mattlerart 1996).

The Lazarsfeld-Stanton Analyzer summarizing "the public"

CNN used a variant of this machine for the recent State of The Union address, showing real-time reactions from Democrats in blue, Republicans in red, and Independents in yellow.

The Lazarsfeld brand of insight is based on a fundamental assumption: that the “average” means something. An entry-level statistics course will teach you that average is dragged up or down by extreme values, and the long tail is nothing if not a collection of many extreme values. There is nothing meaningful about knowing that the “average American” rented 30 digital movies a month if, in fact, there were many thousands of Americans who rented none and a many tiny segments that rented somewhere between zero and 40 movies. The “average” is meaningless in this example, yet this ham-fisted approach to summarizing “the public” is what the market research industry is built upon.

Design Research for The Long Tail

Market researchers may argue that with proper segmentation, you can understand every niche within the long tail. This may be true, but to truly understand the diversity between people, your task is not simply to “summarize” the audience, but to delve deeply into the dynamics of what makes them different.

This is why design research is a better fit for today’s long-tail economic model. Context matters. Design research is all about understanding the context because it is rooted in qualitative methodologies, and ethnography in particular. Designers solve contextual problems. The award-winning Braille watch, for example, allows its users to check the time surreptitiously and quickly, something that is both polite and useful. A Lazarsfeld approach would not uncover the social subtleties of checking one’s watch, and certainly could not uncover the specific needs of the blind.

The Braille Watch by David Chavez

Dan Formosa details this limitation of market research in his insightful article in Interactions magazine. He argues that market research should focus on consumer response — after a product is designed. Design research, on the other hand, is about evaluating a product as it is being developed. I would go further; design research is about knowing what to build as well as evaluating the prototype.

Design research uncovers how long-tail niches develop and what differentiates them. It is not the equivalent to “market segments” because it provides specific direction on how to apply research findings. What are the dynamics of renting a movie? What motivates the “heavy renter”? What is it about her television or home that supports heavy renting? You cannot know the answer to these questions by simply providing a laundry list of demographic characteristics and psychographic survey results. You must know the context in which the long tail emerges.

Some may say that good quality market research would not make these kinds of mistakes. And they are right. Highly skilled social scientists are method-agnostic; they choose the right method for the right research question. However, full-service market research firms have become the GM of the industry — they keep building Hummers instead of Priuses. Focus groups don’t uncover contextual nuances, but they’re cheap and profitable. Surveys don’t get to the heart of why a product doesn’t work. Design research, using an ethnographic approach, provides “thick description” of the entire phenomenon of renting movies.

This is where market research cannot go. And this is where market research will fail, unless it rejects the “build another Hummer” mentality.

References

Babbie, E. and L. Benaquisto (2002). Fundamentals of Social Research. Scarborough, Thomson Nelson.

Mattlerart, A. (1996). The Invention of Communication. Minneapolis, University of Minnesota Press.

Those of you out there who’ve tried it know: recruiting research participants is HARD. Here are a few insights from the research to help you with better recuitment.

1. Personalized contact with respondents, followed by pre-contact and aggressive follow-up phone calls *: Don’t count on a form letter, email or random tweet to do the job. Capitalize on your personal relationship with that person. If you don’t have a personal relationship, ensure that you use the person’s name, and for God’s sake, spell it correctly!

   Once you’ve made initial contact, you are not done. Not by a long shot. Make sure you speak to the person (you can do this through IM or email if you’d like) to give them more information. They’re now interested. Don’t stop! One more step!

   Follow up 1 week after initial contact. Assuage any fears they may have. Answer any questions honestly. And above all, be available for more information.

2. External researchers with social capital are best**: University-based researchers have been shown to have the best participation rates, but you don’t have to be a professor.� Researcher Sister Marie Augusta Neal of Emmanuel College achieved a near perfect response rate because of her close ties to the respondents and their communities. The lesson here is, if you hire a consultant, make sure they’re trusted. Even better if they personally know the people to be recruited.
3. Monetary incentives have no effect, unless money is offered �no strings attached�***: Little known fact: the best way to use a monetary incentive is to offer it, up front, with absolutely no strings attached. The “free” money makes people feel more indebted socially. Evidence of this effect can be found in the book Freakonomics. Researchers found that daycare centres that levied late penalties on tardy parents actually had more of a late-pickup problem than those that levied no fine. Why? Because the parents reduced their relationship to the daycare as a mere transaction. Use the “gift economy” approach and ensure a feeling of indebtedness. My personal favourite is a coupon for a single iTunes song at $.99. It is cheap but appears to have great value. Offer it, up front, and then ask for participation

*� Cook, C., F. Heath, and R. Thompson. 2000. “A Meta-analysis of Response Rates in Web or Internet-based Surveys.” Educational and Psychological Measurement 60:821-836.

** Rogelberg, S., A. Luong, M. Sederburg, and D. Cristol. 2000. “Employee Attitude Surveys: Examining the Attitudes of Noncompliant Employees.” Journal of Applied Psychology 85:284-293.

***Hager, M., S. Wilson, T. Pollak, and P. Rooney. 2003. “Response Rates for Mail Surveys of Nonprofit Organizations: A Review and Empirical Test.” Nonprofit and Voluntary Sector Quarterly 32:252-267. Singer, E. (2006) Introduction: Nonresponse Bias in Household Surveys. Public Opinion Quarterly, 70, 637-645