Lies are an important source of design insight. Design research ought to embrace lies as potential sources of creative inspiration. Lies are indicators of a gap between what we are and what we think we ought to be. Well-designed products soften and assuage the effects of this gap.

Photo courtesy of: www.flickr.com/photos/lwr/213108466

The other day, one interviewee asked me, near the end of the interview, what “is this all about.” At first I was confused, having explained the study we were conducting and what specifically we were interested in finding out. Yet she pressed me further, wanting to know the “mystery” behind the study.

I reflected and realized that she was expecting a great “reveal” of the “real” purpose behind the study. She was expecting me to pull back the curtain and tell her what I was actually interested in.

If you have ever participated in a university psychology study, this story will sound familiar to you. You had likely been recruited as an undergraduate, incented to participate with the promise of a few percentage points tacked onto your final grade.

Or perhaps you attended a focus group, which had a one-way mirror at one end of the room that you were directed to “try to ignore.” Or maybe you have answered a telephone survey that had a mysterious combination of questions the meaning of which you could not decipher.

Perhaps you, like my participant, have been conditioned to believe that “unbiased” or “scientific” social research involves trickery or outright deception. Proponents of this approach may argue that in order to get “the truth,” researchers must mask their true intentions, lest participants lie. This kind of research seeks to sanitize the results, to make them somehow untainted by “bias.”

What underlies this idea of deception and lying in social research? There is an assumption that The Truth is something that lives within the minds of your participants and your job as a social researcher is to pry that nugget out of their minds. Your job is to eliminate any “bias” that would filter this truth.

This is the same assumption ethnographers make when they believe a year’s fieldwork is essential. The classic anthropological model is a one-year field assignment. But ethnographers who hold this view are actually similar to market researchers who assume participants may “lie.” They are hoping to establish “rapport,” so that participants will eventually “drop their guard” and show the ethnographer their “true” or “authentic self.”

But if you assume that the truth is something we create, in tandem with our participants, authenticity or truthfulness become irrelevant concepts. Instead, a researcher can assume participants do indeed lie, but that lying is an interesting data point. The savvy, design ethnographer can ask, “Now why did she lie about cleaning her oven weekly, when she clearly hasn’t cleaned it in months?”

This is not a clean stove

These kinds of questions can lead to interpretive gold.

Perhaps oven cleaning is considered “proper,” and women are encouraged to act “properly” by cleaning their ovens regularly. Perhaps a better-designed stove looks “clean” on its exterior, whether it is actually clean inside or not. Perhaps a better-designed stove provides women with mechanical “excuses” of why it should NOT be cleaned regularly, thereby absolving its owner of any shame.

Both design solutions assuage the guilt and the shame of having not lived up to a perceived norm.

Indeed, the Swiffer provides exactly this kind of “cover.” Hardwood floors are supposedly “different” than regular floors and require a “different” kind of broom and mop. It just so happens that the Swiffer is faster, more ergonomic, and less messy than a regular broom. It requires less effort, yet Swiffer uses are told they are doing the “proper” thing by using this “special” kind of broom.

When you are hunting for design solutions, become a lie detector. Do not question the veracity of a participant’s statement, but go deeper. Why did he say that he “tries to not be the dad on the cell phone”? What ideal is not living up to?

Design interventions based on lies could promise to be the most user-accepted designs.

Roger Martin has a great post on Harvard Business Review that summarizes how ethnographic research differs from quantitative surveys.

Martin writes:

Qualitative, and especially observational or ethnographic, research enables us to delve much more deeply into the relationship between our firm and its product/service and the customer. Because we aren’t obsessed about adding all the responses together for ‘rigorous quantitative analysis’, we can let the customer use his own voice/words/vocabulary.

This sounds a lot like the notion of “verstehen,” which refers to the deep understanding that comes from interpretive, qualitative research.

Quantitative research has its place; how else could we measure improvement if not through counting instances or events? Yet we often forget that quantitative data is primarily designed to summarize findings quickly. This is why it’s so popular but also why it’s inadequate to describe many experiences.

I like to us a football game metaphor to describe the real difference between qualitative and quantitative research. Let’s say that the Steelers beat the Patriots 49-15. What would you know about that game? Simply that the Steelers had won.

Would you really know where the turning point in the game came? Would you know about the significance of a mid-game interception? Or perhaps the critical sacking of the Patriots’ quarterback? No, you’d know nothing of the ebb and flow of the game, critical mistakes and successes, or even how the Patriots might feel about their loss. They might actually feel vindicated if their defensive line held tough against the Steelers for 3 out of 4 quarters.

Statistics are a great way of quickly conveying how a group of events, people, or things are similar and different. Mode, median and mean measure “central tendency,” and standard deviation and inter-quartile range tell you “dispersion.” With these two types of measures, you can tell me how similar people are when they choose orange juice, how different they are when they rent cars or attend movies. But you cannot tell me what “more pulp,” means to people, why a “subcompact” car turns off some people, or what people perceive the word “blockbuster” to actually mean.

In short, ethnographic research can clarify all of these deep, nuanced details that quantitative data skates over or takes for granted. Do you want to know how many people attended a “summer blockbuster?” Then by all means, count them. But if you want to know what kind of movie people believe a “blockbuster” to be, then you need to do in-depth ethnographic work.

Few would disagree that fundamental economic change is upon us. Business models are crumbling daily. From the auto industry to the banking industry, it is clear that old ways of doing things are no longer working. The market research industry is just as vulnerable to this shift, yet, like the auto industry before it, it is hardly aware of how deeply its business model is threatened.

The Long Disruption

The market research industry is built for the 20^th Century mass production model, which is rapidly disappearing. The “mass audience” is gone and a fragmented diverse populace has taken its place. This new “audience” defies the easy aggregation of summary statistics on which market research so often relies.  Chris Anderson of Wired figured this out long ago with his book The Long Tail.

The Economic Disruption of The Long Tail -- Anderson, Wired Magazine

He argued that technology lowered the cost of providing services to ever-smaller niches of people, making it possible to sell profitably goods and services that were once too specialized.

This technological shift also means the end of “appointment television.” Digital video recorders allow individuals to time shift their programming to suite them, and not the program executives at television networks.

The Birth (And Death) of Market Research

What does this all have to do with market research? Full-service market research firms are built for the blockbuster era, not for the time of the long tail.

Market research was heavily influenced by the school of “applied sociology,” lead by Paul Lazarsfeld. While at Columbia, Lazarsfeld pioneered many statistical techniques we use today, including the cross tabulation (Babbie and Benaquisto 2002) and the Lazarsfeld-Stanton Analyzer, a machine that records audience reaction to programming in real time (Mattlerart 1996).

The Lazarsfeld-Stanton Analyzer summarizing "the public"

CNN used a variant of this machine for the recent State of The Union address, showing real-time reactions from Democrats in blue, Republicans in red, and Independents in yellow.

The Lazarsfeld brand of insight is based on a fundamental assumption: that the “average” means something. An entry-level statistics course will teach you that average is dragged up or down by extreme values, and the long tail is nothing if not a collection of many extreme values. There is nothing meaningful about knowing that the “average American” rented 30 digital movies a month if, in fact, there were many thousands of Americans who rented none and a many tiny segments that rented somewhere between zero and 40 movies. The “average” is meaningless in this example, yet this ham-fisted approach to summarizing “the public” is what the market research industry is built upon.

Design Research for The Long Tail

Market researchers may argue that with proper segmentation, you can understand every niche within the long tail. This may be true, but to truly understand the diversity between people, your task is not simply to “summarize” the audience, but to delve deeply into the dynamics of what makes them different.

This is why design research is a better fit for today’s long-tail economic model. Context matters. Design research is all about understanding the context because it is rooted in qualitative methodologies, and ethnography in particular. Designers solve contextual problems. The award-winning Braille watch, for example, allows its users to check the time surreptitiously and quickly, something that is both polite and useful. A Lazarsfeld approach would not uncover the social subtleties of checking one’s watch, and certainly could not uncover the specific needs of the blind.

The Braille Watch by David Chavez

Dan Formosa details this limitation of market research in his insightful article in Interactions magazine. He argues that market research should focus on consumer response — after a product is designed. Design research, on the other hand, is about evaluating a product as it is being developed. I would go further; design research is about knowing what to build as well as evaluating the prototype.

Design research uncovers how long-tail niches develop and what differentiates them. It is not the equivalent to “market segments” because it provides specific direction on how to apply research findings. What are the dynamics of renting a movie? What motivates the “heavy renter”? What is it about her television or home that supports heavy renting? You cannot know the answer to these questions by simply providing a laundry list of demographic characteristics and psychographic survey results. You must know the context in which the long tail emerges.

Some may say that good quality market research would not make these kinds of mistakes. And they are right. Highly skilled social scientists are method-agnostic; they choose the right method for the right research question. However, full-service market research firms have become the GM of the industry — they keep building Hummers instead of Priuses. Focus groups don’t uncover contextual nuances, but they’re cheap and profitable. Surveys don’t get to the heart of why a product doesn’t work. Design research, using an ethnographic approach, provides “thick description” of the entire phenomenon of renting movies.

This is where market research cannot go. And this is where market research will fail, unless it rejects the “build another Hummer” mentality.

References

Babbie, E. and L. Benaquisto (2002). Fundamentals of Social Research. Scarborough, Thomson Nelson.

Mattlerart, A. (1996). The Invention of Communication. Minneapolis, University of Minnesota Press.

“But how many people did you talk to?” If you’ve ever done qualitative research, you’ve heard that question at least once. And the first time? You were flummoxed. In 3 short minutes, you can be assured that will never happen again.

Folks, qualitative research does not worry about numbers of people; it worries about deep understanding. Weber called this “verstehen.” (Come to think of it, most German people call it that too. Coincidence?). Geertz called it “thick description.” It’s about knowing — really knowing — the phenomenon you’re researching. You’ve lived, breathed, and slept this thing, this social occurrence, this…this…part of everyday life. You know it inside and out.

Courtesy of daniel_blue on Flickr

You know when it’s typical, when it’s unusual, what kinds of people� do this thing, and how. You know why someone would never do this thing, and when they would but just lie about it. In short, you’ve transcended merely noticing this phenomenon. Now, you’re ready to give a 1-hour lecture on it, complete with illustrative examples.

Now if that thing is, say, kitchen use, then stand back! You’re not an Iron Chef, you are a Platinum Chef! You have spent hours inside kitchens of all shapes and sizes. You know how people love them, how they hate them, when they’re ashamed of them and when (very rarely) they destroy them. You can tell casual observers it is “simplistic” to think of how many people have gas stoves. No, you tell them, it’s not about how many people, it’s about WHY they have gas stoves! It’s about what happens when you finally buy a gas stove! It’s about….so much more than how many.

Welcome to the world of verstehen. When you have verstehen, you can perhaps count how many people have gas stoves. Sure, you could determine that more men than women have them. Maybe you could find out that more of them were built between 1970 and 80 than 1990 and 2000. But what good is that number? What does it even mean?

When you’re designing, you must know what the gas stove means. You must know what it means to transform your kitchen into one that can and should host a gas stove. You must know why a person would be “ashamed” to have a gas stove (are they ashamed of their new wealth? do they come from a long line of safety-conscious firefighters?). You must know more than “how many.”

So the next time someone asks you, “how many people did you talk to?”, you can answer them with an hour-long treatise about why that doesn’t matter. You can tell them you are going to blow them away with the thick description of what this thing means to people. You are going to tell them you know more about this thing than anyone who ever lived, and then, dammit, you’re gonna design something so fantastic, so amazing that they too will be screaming in German. You have verstehen!

See my discussion about sampling methods in qual and quant research for more insight into the reasons why “how many” is irrelevant in qualitative research.

DT has a great post over at Design Sojourn that discusses Six Sigma methodology and how it relates to design. He cites Tim Brown at IDEO who argues that Six Sigma is essentially Newtonian, while design thinking is quantum. In his own design work, DT expressed doubts about using Six Sigma:

After studying the Six Sigma process, I point blank said: �There was no way any of my designers are going to be judged on the quality and success of a design based on how many sketches or iterations we did before we deliver it.�

Both Brown and DT cite Sara Beckman, who recently discussed the topic in the New York Times. Beckman reviews how Six Sigma focuses on incremental improvements, while design and design thinking focuses on big changes. For those of you who aren’t familiar with Six Sigma, it’s a method pioneered by Motorola, which aims to reduce the number of errors to 3 in one million. The “six sigma” refers to six standard deviations. The number of errors should be at the extreme end of the normal curve, or between + or – 3 standard deviations, represented by the Greek symbol sigma.

I argue that design is more complementary to the “interpretivist” paradigm of qualitative research while Six Sigma is positivist. Interpretivists don’t believe the world is a static place. They see reality as being continuously created by you, me and other social actors. There is no such thing as “The Truth” in interpretivist approaches, just different versions of the truth. Typical methods of interpretivists are ethnography, in-depth interviewing and discourse analysis. Positivist research, on the other hand, assumes that reality is static. Positivists believe that “The Truth,” is out there to be discovered. Typical methods would include quantitative surveys.

Designers should focus on interpretivist methods, therefore. They should uncover different versions of the truth using observation and interviewing, as well as deep reflection on symbols and their meanings. Surveys and other quantitative methods are more Six Sigma in that they can measure improvement over time. Designers ought to consider measuring improvement, but starting with qualitative approaches is best.

New research finds that there are seven key factors that promote social capital. In his book, Unanticipated Gains, Mario Luis Small did an ethnography of New York daycare centres. What he finds may surprise you: daycare centres are great “brokers” for social capital. I describe his findings on the Social Capital Value Add blog:

Small argues that actors get involved in networks in particular ways that are structured by the organizations themselves. What are the effects of organizational involvement on social capital? And how can organizations nurture the development of social capital?

Read the entire post.

The New York Times reports that the American Medical Association is considering doing away with the venerable symbol of the physician: the lab coat. There’s a very good reason to get rid of lab coats: they’re dirty. But the symbol of the lab coat is far more important. The New York Times reports the empirical flaw in wearing lab coats:

The group�s Council on Science and Public Health is looking at the role clothing plays in transmitting bacteria and other microbes and is expected to announce its findings next year.

This empirical finding shouldn’t be surprsing. We also know, for example, that male physician’s ties are wearable petri dishes. The verdict ought to be clear, therefore that we should get rid of lab coats. Not so fast, say physicians.

Getting rid of the lab coat is getting rid of one of the most important symbols of a physician’s identity. Dr. Richard Cohen told the New York Times how important that lab coat is:

�When a patient shares intimacies with you and you examine them in a manner that no one else does, you�d better look like a physician � not a guy who works at Starbuck�s.�

Here is the lesson for designers: empirical “fact” is not the whole story. What role any particular symbol plays in social life is just as critical. What’s fascinating about this story is that physicians are now trained in “evidence-based medicine,” meaning they are trained to diagnose and treat based on more “rigourous” science (I have my doubts about that rigour, but that’s another blog post).

Yet here is a clearly “scientific” reality about the danger of treating patients while wearing a bacteria-infested lab coat and/or tie, and physicians continue to wear them. For all their protestations of “evidence,” physicians too are social beings, embedded in a social world. They too must convey an identity, even if the symbols used for doing so compromise their ability to complete their stated vocational mission.

The symbol is powerful. Designers who base their decisions on so-called “evidence” ought to pay attention to other kinds of evidence, such as the enduring patterns of social interactions. We should pay attention to any enduring patterns of social behaviour but *especially* those which fly in the face of supposed “logic.”

Data-driven social experience is an entirely new manner of social interaction, one that obscures our emotional connections to people. Data makes social relationships visible, knowable, and countable in unprecedented ways. But it does not — and cannot — convey the emotional experience of social interaction. I’ve already discussed how digital technologies transform text and time. Now I want to explore how “data” transforms social experience.

Take the notion of the “social network.” Most people (especially those that read blogs!) think these synonymous with Web sites like Facebook. Truth be told, social network analysis has existed for almost a century. We’ve all heard the term “six degrees of separation,” but most of us don’t know that was coined by none other that Stanley Milgram, of the “shock experiments” fame, when he tracked letters mailed around the world.

Social networks are exceedingly difficult to know from a quantitative perspective. We all live inside social networks but we have a very hard time knowing how these networks are constructed. We may know, for example, that our friend Jeff is friends with another one, Sarah, but we don’t know if Sarah knows Jeff’s partner Sam. Social network analysis is a set of methods designed to learn exactly that.

Now imagine your social network, as it is represented on Facebook (what, you’re not on Facebook?). Below is an image from Visual Complexity that renders a social network visibly but also very easily, simply by mining the data inherent in Facebook’s structure:

from Visual Complexity

Note how we instantly and easily know how institutions are connected, and through which people. Previously, researchers would have to conduct extensive and expensive surveys to get these data. Now these data are easily calculated and visualized by anyone with access to a social network online.

Some people are talking about this visualization as a piece of intellectual property. Alex Iskold on Mashable, for example, asks “Who owns the social map?” I go further and ask, “What does it mean that our social world is mappable?”

Our social world is now infiltrated by masses of data. These data inform us about the structure of our interactions with others in ways that we could not recall correctly if asked. Suddenly we can now see our social world reflected back to us, punctuated by institutions, and social structures. When we see our social network through the eyes of data, we see the names of organizations, or the institutional affiliation of the people. We do not “see” the emotional experience that created our connections in the first place.

Suddenly, we may think we really are not that close with Jeff, because his partner Sam is really not friends with anyone I know. I can also see that Sarah and I have very few friends in common, which may lead me to think I don’t have much of a future friendship with her.

Those data crowd out the qualitative, embodied experience of the laughs I shared with Jeff and Sam at their cottage last summer. Those data obscure the fact that Sarah and I shared 3 long months as call centre employees together, a time that bonded us forever. A data-filled social world is one that masks the visceral, emotional experiences of face-to-face interaction.

Digital social life is revealed to us in fragmented, mashed up ways. Such ways were impossible before the freely available data on social networks, data that is now so ubiquitous, we don’t even see it.

Hugh Dubberly and Paul Pangaro had a great post on Interactions magazine about designing for conversations. They propose to use how a conversation actually works to make interactions better. They rely heavily on Claude Shannon’s conversation model to help guide the conceptual model of interaction designs.

In Shannon�s model an information source selects a message from a known set of possible messages, for example, a dot or a dash, a letter of the alphabet, or a word or phrase from a list. Human communication often relies on context to limit the expected set of messages.

I applaud Dubberly and Pangaro’s attempts to use rigourous theory to support interaction design. But I’d have to agree with Peter Jones as he wrote in the comment section, that other philosophically informed communication theories are more robust when it comes to designing for conversation. Peter specifically mentions Winograd and Flores’s “conversation for action model” which relies on Habermas’s contention that you are acting when you communicate.

I’ll add to Peter’s critique. Garfinkel’s ethnomethodological approach gave way to “conversation analysis,” which posits that speakers use “indexical expressions” (or phrases that are fraught with meaning but are meaningful to the participants through unspoken means). Where in Dubberly and Pangaro’s article is the discussion of such expressions?

Where also is the notion of turn taking? Turn taking is a very significant component of a conversation. Try to have a trans-atlantic mobile phone conversation and you’ll see how important smooth turn taking is to meaningful conversation.

I would exhort interaction designers to continue to read and integrate theory into their mental models. But I would also discourage them from taking the short route; theories are debated for a reason. Interaction design ought to be a robust digital representation of those debates, and include all aspects.

I’m currently forming research questions for an ethnography of Toronto-based technology and design workers. I am working through this question: is Twitter a “place”?

In her 2000 book Virtual Ethnography, Christine Hine argues that there are two analytic strategies to see “cyberspace.” First, one can view it as a “place,” where social norms emerge. Or second, one can view it as a cultural artifact. The second view allows us to see the designers behind the technology. Think of it as a hermeneutics of a technology, which allows us to see what assumptions its designers about their users (this is an approach that will make sense to interaction designers).

I believe Twitter to be a place, but one that is heavily influenced by its architects and its users. In other words, its design sets the stage for certain kinds of interactions, just as prisons, malls, and casinos do. The architecture of Twitter, which includes its dozens API-driven applications as well as its simple, Web-based interface, is constantly evolving by its network of users, API application designers, and the company of Twitter itself.

This approach suggests that Twitter has “interpretive flexibility,” which is how technology theorists argue that design is determinant; users decide how a technology will actually be used, within the confines of the material form of that technology.

Do you believe Twitter is a “place”? What kind of place? Or is Twitter a technology or technological artifact?