“But how many people did you talk to?” If you’ve ever done qualitative research, you’ve heard that question at least once. And the first time? You were flummoxed. In 3 short minutes, you can be assured that will never happen again.

Folks, qualitative research does not worry about numbers of people; it worries about deep understanding. Weber called this “verstehen.” (Come to think of it, most German people call it that too. Coincidence?). Geertz called it “thick description.” It’s about knowing — really knowing — the phenomenon you’re researching. You’ve lived, breathed, and slept this thing, this social occurrence, this…this…part of everyday life. You know it inside and out.

Courtesy of daniel_blue on Flickr

You know when it’s typical, when it’s unusual, what kinds of people  do this thing, and how. You know why someone would never do this thing, and when they would but just lie about it. In short, you’ve transcended merely noticing this phenomenon. Now, you’re ready to give a 1-hour lecture on it, complete with illustrative examples.

Now if that thing is, say, kitchen use, then stand back! You’re not an Iron Chef, you are a Platinum Chef! You have spent hours inside kitchens of all shapes and sizes. You know how people love them, how they hate them, when they’re ashamed of them and when (very rarely) they destroy them. You can tell casual observers it is “simplistic” to think of how many people have gas stoves. No, you tell them, it’s not about how many people, it’s about WHY they have gas stoves! It’s about what happens when you finally buy a gas stove! It’s about….so much more than how many.

Welcome to the world of verstehen. When you have verstehen, you can perhaps count how many people have gas stoves. Sure, you could determine that more men than women have them. Maybe you could find out that more of them were built between 1970 and 80 than 1990 and 2000. But what good is that number? What does it even mean?

When you’re designing, you must know what the gas stove means. You must know what it means to transform your kitchen into one that can and should host a gas stove. You must know why a person would be “ashamed” to have a gas stove (are they ashamed of their new wealth? do they come from a long line of safety-conscious firefighters?). You must know more than “how many.”

So the next time someone asks you, “how many people did you talk to?”, you can answer them with an hour-long treatise about why that doesn’t matter. You can tell them you are going to blow them away with the thick description of what this thing means to people. You are going to tell them you know more about this thing than anyone who ever lived, and then, dammit, you’re gonna design something so fantastic, so amazing that they too will be screaming in German. You have verstehen!

See my discussion about sampling methods in qual and quant research for more insight into the reasons why “how many” is irrelevant in qualitative research.

Social media “gurus” abound these days. Which ones are worth listening to and which ones are bullshitters?

Philosopher Harry Frankfurt exposed bullshitters in his famous essay “On Bullshit.” The liar knows what the truth is and cares very much about concealing it. The bullshitter, on the other hand, doesn’t care what the truth is and has no compunction in stretching it.

The same goes for social media “gurus.” Those that care what about rigourous examination of the social may be wrong, but at least they take great pains to analyze the phenomenon. Those that don’t care about systematic, theoretically informed social inquiry are interested only in stretching or shaping their own agendas.

How can you tell the difference?

Here are a few signs you’re dealing with a social media bullshitter.

1. They skate over the tension between structure and agency: The tension between structure and agency is an age-old sociological debate. Social media bullshitters somehow miss this very important point. They often argue that implementing social media or social business design will somehow evaporate decades or even centuries of organizational structures. If your social media guru tells you that adding social media and stirring will create equality, harmony, and profits, begin to question them. If, on the other hand, they tell you that your organization does not live in a vacuum, and that your social media will be integrated in people’s existing lives with their existing economic, technological, and ethnically grounded experience, then they may be onto something.
2. They use the same social research methods every time: A classically trained sociologist is trained in both qualitative and quantitative methods. They are designers in the sense that they have expertise, which they draw upon selectively, according to the research question. Social media bullshitters, on the other hand, likely have a common stock of tools that they use repeatedly, regardless of the nuance of the research question. If their answer is always, “do a focus group,” or always, “do a survey,” then question them.
3. They see no paradoxes. Ever: Sociologists are constantly grappling with paradoxes. Weber’s famous paradoxical finding was that bureaucracies are both efficient and inefficient. They work wonders building and managing railroads, for example, but they result in horrible catastrophes like the Challenger disaster. Weber explained this paradox by arguing that rationality, or the rule of rules, is an “iron cage,” that keeps us safe but enslaved. If your social media guru claims there will be no paradox, nuance, or ambiguity, question them.
4. They don’t know what social capital really is: Social capital is not something one can measure in terms of bank balances. It was the creation of French sociologist Pierre Bourdieu (come to think of it, the bullshitters wouldn’t know that either). Social capital is something one develops by being in a particular social location. I may go to an exclusive boarding school. My social capital is my network of well-off friends. Social capital is a particularly important concept when thinking about social media. Bourdieu noted that those in lower economic classes explicitly reject items they consider “above their station.” This means that luxury or “top of the line” is not always your best approach.

The bottom line is this: social media bullshitters have no knowledge of social theory or methodology. Trust a person who provides no easy answer, who carefully selects their research method, and who understands complex concepts.

Do you have more signs of being a social media bullshitter? Please share them here!

The technology consulting firm Gartner is predicting that social scientists will be very much in demand by businesses. Eweek summarizes Gartner’s outline of four types of roles for social scientists:

Web User Experience roles that include UI designers, virtual-assistant designers and interaction directors.
Behavior Analysis roles that include Web psychologists, community designers, and Web/social network miners.
Information Specialist roles that include information anthropologists who are expected to play historical Web fact finding and assisting in legal analysis, intellectual property management and where the quality of information is at risk.
Digital Lifestyle Experts roles that include helping senior management understand whats going on and stay aware, and building personal brands and managing online personas for desired online effect

Gartner’s Vice President Kathy Harris appears to have faith in social scientists’ ability to be creative:

Creative, artistic and clever people will develop the early iterations of these new jobs. This will enable businesses and government to take early advantage of new capabilities and develop them into mainstream skills.

I’m in full agreement that social science trains people in the right kinds of skills for the digital age. I was disappointed however to find that sociology had failed to capture Ms. Harris’s specific attention.Interestingly, the report mentions anthropologists and psychologists specifically, but not sociologists.

Sociologists have recently complained that they have not been given a place at Obama’s table. I argue that it’s because they’ve done a poor job of publicizing the great skills they have. Just last night I sat down with two other sociologists, one is a specialist in the sociology of science and the other in the socio-legal implications of changing family forms. Aren’t these the very people we need to help us understand the effects of genetic engineering? Or the potential outcomes of changing same-sex marriage laws?

I personally will continue to proclaim my training as a sociologist, and will convince business people that the “soft stuff” is a differentiator. I will also try to nudge my colleagues into the world of design, where their training in empathy and critical thought is welcome.

I examine the tools and processes I use to do my own work. Like many agency workers, I often bill by the hour. Check out my research on this phenomenon:

This paper is about time regimes that are typical in interactive agencies, as well as law firms, some construction companies and some management consultancies: the so-called “billable hour.” In this paper I ask how such a system is constructed, what tools are used to maintain it, and, most importantly, how do Web workers resist it?

More…

I invite comments to this discussion on my agencytime research blog. I am presenting this research at a conference in Edinburgh and wish to have insights from those who work under such time regimes.

So-called “design thinking” is the new It-Girl of management theory. It purports to provide new ways for managers and companies to provide innovative, creative solutions to old problems. But design thinking alone will not solve these problems because a lack of creativity was never the issue.

The real issue is one of power.

Design is attractive to management because it is a de-politicized version of the well known socio-cultural critique of managerial practices. Design thinking is so popular because it raises only questions of “creativity” or “innovation” without ever questioning the legitimacy of managerial practice. Instead, design thinking aspires only to “better” management technique by investigating “contextual problems” or the truly innocuous “pain points.”

The inconvenient truth is that the science of management fails because it treats people as either mere inputs into the production process or as faceless “consumers” who have no real stake in outcomes. Design thinking allows for these truths to remain unaddressed, thereby avoiding any discussion of power itself. Workers are cast as something to be organized or “incented.” Consumers are to have their “needs met.” And neither group is granted a meaningful stake in the creative process.

Within this frame, design techniques attempt to solve managers’ typically tone-deaf executions of creativity without ever naming the root cause of workers’ and consumers’ dissatisfaction, which is their lack of meaningful participation in the design process. Managers’ ability to control both the organization of work and the availability of consumer goods is the true problem, not an inability to think “creatively.”

Managers have control over the working conditions under which creativity is supposed to happen, as well as the the distribution of the fruits of such labour. One significant reason workers’ creativity does not flow easily from studio or factory to consumers is because of management’s need to control costs and secure profits. Were it not for the profit motive, workers would be free to radically innovate continually and consumers would have unrestricted access to such new and innovative goods. But because profit stands as the pre-eminent benchmark of business success, both workers and consumers are thwarted in their pursuit of supplying and demanding innovative goods.

In other words, there is no shortgage of creative solutions to “unmet needs,” only a shortage of profitable ways to provide them.

Hence the inevitable ineffectiveness of design thinking, if applied in isolation to the problem of creativity. Designers must consider what role power plays in an organization’s inability to create innovative products. But more importantly, designers must be prepared to identify and name power and its sources (e.g., the pursuit of profit at the expense of innovation).They must not simply use ethnographic techniques to uncover “unmet needs”.

This is perhaps where designers will feel most out of their depth. It is a long leap from solving contextual problems to providing an analysis of inequality. All the more reason then, for designers to study the socio-cultural theory that underlies ethnography and other qualitative research methods.

In particular, designers should study feminist writers such as Canadian sociologist Dorothy Smith. Smith founded the method she calls “institutional ethnography,” which takes the standpoint of its participants and not that of the organization. This method frequently yields lived experiences that differ from the “official record” because it assumes that users of a technology, a product or a social policy lack meaningful access to those who record such records.

Ethnographic approaches are a good starting point for designers to cultivate empathy and hone observational skills. But it is in issues of power that rememdies to innovation bottlenecks will be broken.

The New York Times is reporting that WalMart’s new fangled milk jug is getting mixed reviews.

What’s not to like? Plenty, as it turns out.

The jugs have no real spout, and their unorthodox shape makes consumers feel like novices at the simple task of pouring a glass of milk.

The design of the milk jug is so bad that WalMart has taken to doing in-store demonstrations of “how to pour” with this new jug.

This jug is a design flop! Right?

Well not so fast. It seems that the designers of the milk jug created it for a specific purpose: to save money. The new jugs are stackable, saving shipping costs and space. The company saves up to 70% of labour costs using these new jugs. The milk arrives at the store fresher, sometimes even the same day. This jug is a great design! Right?

The truth is somewhere in the middle. If business requirements trump user needs, this product is a winner! It saves time, energy, and most of all, money. It’s easier to ship, easier to manage, and much more efficient.

But if user needs trump business requirements, then this jug is a total flop. No  one knows how to use it. They spill it. Their children can’t pour it themselves, forcing parents to spend more time to use the jug. They feel stupid when they can’t pour it correctly.  Talk about crying over spilled milk! WalMart’s new milk jug off-loads all its design failings onto its users, keeping all the benefits of the new design for itself.

WalMart is famous for putting its business needs ahead of its workers and its communities. Off-loading the negative effects of this milk jug onto its consumers? That’s another in a long line of WalMart putting itself and its shareholders first.

Great design aligns business and user. There are trade-offs in every phase of product design. But not knowing what your users before making a design change makes it impossible to do this. The verdict? Not a total flop, but clearly a business-driven design. Truly great design balances the user’s needs with the business’s needs.

[digg=http://digg.com/design/What_Designers_Can_Learn_From_Facebook_s_Beacon_the_collision_of_fronts]

The blogosphere (and even the regular old newspaper-sphere) is alight with stories of Facebook’s online advertising flop, Beacon. What can designers learn from this flop? It’s not about privacy; it’s about the presentation of self. People have different “selves” for different places — virtual or otherwise — and designs must be consistent with these variety of selves.

Boing Boing’s Cory Doctorow posted an interesting story on InformationWeek that predicted the decline of Facebook because of its own success. He predicts that the more people that are one Facebook, the more confusing it is. Your “creepy coworkers,” your boss, and your friends you met at Burning Man are all in the same “place,” making it confusing, embarrassing and difficult for everyone.

What Doctorow is really describing is sociologist Erving Goffman’s notion of “the front.” Using the theatre as a metaphor Goffman argued that we actually “perform” multiple selves. Each place we go has a “front” that we learn to incorporate. A front has a wardrobe, a setting, a decor, make-up, a script and stage direction. We have a “front stage self” that we perform for everyone to see, a “back stage self” for only our closest intimates to see, and a “core self,” which is deeply private.

A doctor, for example, has a front that includes an office, a lab coat, a stethoscope and medical jargon. This is her “front stage” self. But when she’s talking to her best friend, she may use a “back stage self,” being less formal, not wearing a lab coat, or using less formal language. Her “core” self is secretly wishing she were a full-time marathoner, but she tells no one that.

Facebook’s Beacon didn’t work because it forces people to use multiple fronts AT THE SAME TIME. If I tag a recipe from Epicurious.com, but I broadcast that fact to friends that perceive me to be a party girl, I have a collision of fronts. If my boss demands to be my friend, I have a collision of fronts. If I rent The Notebook on Netflix, and my friends think I am a Goth, I have a collision of fronts.

Facebook’s Beacon forces its users to combine multiple selves. Goffman considers the collision of fronts to be a source of embarrassment or shame. Take, for example, the hilarious “Meeting in a Swimming Pool” gag on Just for Laughs. Swimmers have their swimming front (including a bathing suit, casual demeanour) and forced into a meeting, with its serious demeanour and fully clothed attendants. This is embarrassing.

Facebook has done the same thing by forcing its users to expose their selves to different fronts simultaneously. It is embarrassing, even shameful.

What Designers Can Learn From Facebook’s Beacon

• Discover your users’ fronts: If you are designing a product or a virtual place, ask your potential users what they consider the character of this “place” to be. Is is a formal place? Is it a casual atmosphere? What kinds of “props” are expected here? What would be an embarrassing topic of conversation or incident?
• Design using the theatre metaphor: Make the product consistent with that place, as if you were writing a play. Ensure that what you design is part of a script that users understand or expect.
• Pay attention to embarrassment: If your users mention shame or embarrassment in any way, gently press them about it. Discover the character of the “collision of fronts” that is the source of that embarrassment, and, above all, avoid forcing users to feel embarrassment.

[digg=http://digg.com/design/What_Designers_Can_Learn_From_Facebook_s_Beacon_the_collision_of_fronts]

Update: The New York Times is reporting that Facebook’s lawyers have not succeeded in having documents about its founder Zuckerman removed from an online magazine. These documents are “embarrassing.”

Update (12/19/07): Mashable is reporting that FB is now allowing people to “group” their friends, but they haven’t quite mastered the collision of fronts problem.

Many designers are self-taught, intuitive consumers of research who can translate insights into great designs. But few are trained in the arcane art of research itself. For that reason, many designers don’t know the finer differences between qual and quant research and end up using their respective results inappropriately.

Quantitative research is based on the assumption that random events are predictable, and if you compare your results to pure random results, you can discern distinctive, meaningful patterns about the social world.

Random events are relatively MORE predictable if you have more of them. Imagine if you flipped a coin 20 times. How many heads would you get? Now if you flipped it 20,000 times? You’re more likely to get an even 50/50 split — which is what most people would predict. If you got a 65/35 split with 2o flips, okay, could happen. But with 20,000 flips? No way. Something else is going on.

Translate that to design research by looking at gender, for example. Let’s say you have 20 people, 10 men and 10 women. 65% of the women choose one design, while only 35% of the men do. Is this a meaningful pattern? Impossible to say — you only have 20 people. Now if you had 200 people (100 men and 100 women) and 65 of the women chose one design, chances are you have a meaningful pattern.

This is why sample size matters in quantitative research. But, little known fact, sample size is COMPLETELY IRRELEVANT in qualitative research. Why?

Qualitative research assumes that people have meaningful experiences that can be interpreted. Notice how there’s nothing in there about “prediction” or “randomness.” People have experiences. Researchers discern what these experiences signify. That’s it. Sample size is not only irrelevant, it actually gets in the way of important insight.

Consider the case study, for example. Few people would say case studies are useless. We can learn a great deal about a single design case, where it went wrong and where it went right. The problem comes when you try to predict future events based on this single event.

If you abandon the need for prediction, then sample size never matters. You can always derive insight about design problems from even a single case. Designers that attempt to predict “success” of a single design change, for example, should test that change, repeatedly, with a probability sample.

I often have people ask me how to go about a design research project. Here’s a handy step-by-step guide..

1. The Research Question: this isn’t the same as a research topic. Research questions are answerable in a finite amount of time and yield specific, actionable answers. Some good examples:
   • “What do senior citizens find frustrating about taking their prescribed drugs?”
   • “How do high school students typically study for their driver’s licenses?”
   • “What would working mothers find valuable in shopping for groceries?

   Some bad examples:

   • “What’s an interesting new way to deliver online medical records?”
   • “How can we improve the driving experience?”
   • “What’s wrong with our marketing strategy?”
2. Determine the Method: This is likely the hardest part because every method presents a potential drawback. In general, if you don’t know much about the topic, use a qualitative method (and no, that does not mean “just do a focus group.”) If you know quite a bit about your topic but want to measure change, improvement or any other knowable quantity, choose quantitative research. That does mean “do a survey” sometimes — but not always.
3. Write and Test the “Questions”: I put that in quotes because sometimes it’s not exactly a set of questions. Maybe it’s a task flow and a set of observations the researcher must make. Sometimes it’s a semi-structured interview. Sometimes it’s a quant survey. Make sure you test these “questions,” even in a quick and dirty way with co-workers.
4. Recruit Respondents: Remember your research question? That should tell you whom you need to recruit. If you can’t figure it out, then you need to revise your research question. Be specific but not too narrow in your choices. The more requirements you impose, the smaller your potential base. If you’re doing a qualitative research project, keep interviewing until you start getting the same answers. This usually starts around 8 to 10 respondents. You’re not interested in “statistical significance” but the experiences of the people you’re talking to. For quant studies, statistical significance does apply and you should strive to have a minimum of 40 respondents. Remember though that the higher the number, the lower your margin of error.Design researchers can also rely on professional recruiters to get people for you. Good professional recruiters should get you the right people for a reasonable fee.
5. Prepare the “Stimulus”: If you’re testing a new office chair, make sure your prototype is ready. If you’re interested in something that doesn’t yet exist, consider using photographs to elicit ideas and reactions from respondents. If you’re testing a “concept” make sure that what you’re testing reflects what you really want to know. For example, a picture of a new office chair may not do you any good if what you really want to know is how comfortable the chair actually is.
6. Set Up The Research Space: This is an under-emphasized by oh-so-important aspect of research. Ethnographic research requires you to select the natural environment of your subjects, for example, and you must ensure you have been granted access to that space. If you’re interviewing, decide what kind of place might be conducive to good answers. Noisy restaurants or malls are unlikely to get you personal information, for example. This matters for quant research as well, as there’s a big difference between an in-person, a telephone, and an online survey.
7. Set Up the Interviews: For qualitative research, this takes a fair bit of back-and-forth. It’s helpful to enlist the help of a junior staff member or an administrator. Keep your records straight!
8. Determine the Data Collection Method: If you’re interviewing people in their homes, a TV camera may not be a good choice. Small digital recorders are available for iPods now (I love mine). Digital photos are also useful, but less discrete. And, if you can spare the staff, have one person dedicated to note taking. This frees up the interviewer to really engage with the participate, establish rapport, and probe for opportunistic findings. For quant research, this question usually involves technical questions like, how am I going to import these data into a data analysis tool?
9. Collect the Data: Do your interviews, watch your tasks, ask your questions, whatever. Remember to take notes throughout. These “field notes” are sometimes the most valuable you can have.
10. Answer Your Question: Remember your research question? This is where it comes in handy. You now know exactly what to do when you’re looking through your photos, or your notes, transcripts or whatever.

Links:

Survey research

Survey design tips

Intro to qualitative methods

Social research methods