The interview guide is not as important a document as most people think. Most if not all qualitative research would be improved if researchers stopped focusing on the guide, and focused more on the research questions.

Many qualitative researchers have had this very same experience: the client wants to add too many questions focusing too narrowly on their product. They are afraid to have open-ended, general questions for fear they won’t get the insight they’re looking for. They insist on structuring the interview guide to only see a tiny slice of the customer experience.

The result is a narrow understanding of the overall customer experience, which fails to provide deep insight.

This week I was reminded again why it’s more important to spend more time on research questions than on the guide itself.

I have the good fortune of having very informed, sophisticated clients. Just this week, in two separate meetings with two separate clients, we had the very same conversation about the details of the interview guide. I’m lucky enough that my clients agree with me, that the interview guide itself is not the most important output — it’s the final report that matters.

But they, like many of us, have other stakeholders to whom they are accountable. We collaboratively discussed how to handle questions from these stakeholders when they want to add innumerable questions to the interview guide.

This is how I typically handle this issue.

• Help the client understand the difference between a response and analysis: Many clients have been trained by their research providers that customers are the source of insight. They are not; research analysis is where insight comes from. I often give clients the example of “No one ever asked for a Post-it note” to show that we cannot put the burden of thinking onto customers. We still have a job to do after they talk to us.

• Create a separate document of research questions: When your client insists they want to know why someone uses a competitor’s product, you can record that question in a “research goals” or “report outline” document. The client will learn that you intend fully to answer that question, but that the research participant isn’t responsible for answering it.

• Show the gaps in current customer knowledge: I often like to ask clients what they know about their customers. They usually start by saying “a lot” but when you delve deeper, it turns out they don’t know much beyond their immediate product space. What keeps her up at night? Would she drive a Volvo or a Hummer? Is she interested in book clubs? They often have no idea because their research has been narrow. I then show them that knowing if she likes book clubs will help them reach her better.

• Put the consumer in the centre: It’s all well and good to say to your clients, “Trust me.” But chances are, they’re going to need some more evidence. The best way to do this is to put the consumer at the centre of every project, and not the product. You can use past projects to show clients what you’ve managed to achieve with open-ended questions. You can demonstrate that narrow product answers is no substitute for deep understanding. But you can only do this if make the consumer’s own experience the main research question. Product fit into the consumer’s world (or perhaps they don’t). In everyday life, people don’t run around thinking about products. Deep insight comes from this starting point.

Truly relevant questions may appear as only tangentially related when crafting the guide. It’s our job as researchers to show how general questions are always valuable.

I recently spoke with JohnnyHolland’s Jill Christ and Andrew Harris about research for the design process. The result is an easy-listening podcast, ready for download onto your iPod! Radio Johnny also has a huge number of other, fantastic podcasts.

Jill provides some highlights:

Sam Ladner share her insights about Design Research, and what the Interaction Design community can learn from social research. Sam discusses the difference between quantitative and qualitative research, and why one might choose one method over another. Sam also discusses how to uncover your stakeholders true research questions, and how to offer deep insights, even when there doesn’t seem to be enough time. She covers various approaches to research, shares insights that sociologists have studied for hundreds of years, and how designers can use them when designing for social spaces.

Check it out on JohnnyHolland.org

Copernicus Consulting is pleased to announce that its founder and Principle, Dr. Sam Ladner, will be conducting research with Dr. Catherine Middleton at Ryerson University’s Ted Rogers School of Management. Dr. Ladner will lead the research on the social effects of mobile phones and work/life balance.

“I’m thrilled be working with Catherine on this project,” Dr. Ladner said. “It’s been my personal academic interest for a long time. I’m very excited to collaborate with her.” Dr. Middleton is the Canada Research Chair in Communication Technologies in the Information Society.

The project is in its early stages (see the placeholder mobileworklife.ca or @mobileworklife for more information). Currently, Dr. Middleton and Dr. Ladner are sketching out the specific research activities, but the general approach will be ethnographic in nature.

“From my previous research, I discovered that mobile devices have a distinct impact on how private lives are separated from work lives,” said Dr. Ladner. “I plan to uncover how and in what ways these tools live with us in our homes and in our offices.”

The project is funded by the Social Sciences and Humanities Research Council of Canada. The research team is currently interested in recruiting companies and individuals to participate.

Dr. Ladner will continue at the helm of Copernicus as she leads the mobile work life research effort. “We do so many mobile-related projects, that this was a perfect fit,” she explained.

The Essence of Interaction Design Research: A Call for Consistency

This post is reproduced from the original Interactions magazine article

It started with an innocent query to the IxDA listserv. Someone was sure they had read an article in Interactions magazine once but could not find it again: Wasn’t there something written sometime by someone about something like sample size in usability research? asked an expectant interaction designer. Woe is the hapless interaction designer who is unprepared for the firestorm that follows the dreaded “sample size” question. 106 replies later, and not only was the answer clearly left unanswered but worse, it left many scratching their heads in genuine confusion: what is the essence of interaction design research? Is it data-driven and “scientific”? Is it exploratory and qualitative? No consensus was reached. Again.

What's your sample size again?

This schizophrenia is both a blessing and a curse. One the one hand, an interaction designer has the freedom to assemble her research program like an artist assembling an installation: whatever inspires her can indeed find a place in the final result. Yet, such a lack of standards leads to a distinct lack of consistency and expertise. If interaction design research is whatever you want it to be, what is to stop other occupations “colonizing” what ought to be the purview of the interaction research? See, for example, Dan Formosa’s article in this year’s January-February issue of Interactions, lamenting the intrusion of market research into the design field. When there are no standards, there is freedom.

As Sartre said, we are “condemned to be free,” meaning when there are no pre-defined codes of conduct, then we must tragically, wonderfully, horribly, create ourselves. The confusion over the essence of interaction design research is us, thrashing about as we desperately create ourselves.

You are condemned to be free: courtesy of http://artoftheprank.com/author/mibanez/

In this article, I explain how this lack of standardization affects the practice of interaction design research. In particular, I note that the dreaded “sample size” debate is actually indicative of a larger issue of theoretical training. I call on interaction designers to embrace standardization – not blindly, but with eyes wide open – for the benefit of the interaction design research and for the profession itself.

The Long and Winding Road

Most people stumble into interaction design. Unlike a profession such as medicine, for example, interaction design has a distinctively ill-defined apprenticeship. The proliferation of interaction design job titles demonstrates this lack of definition. A lack of standardization is liberating to many but has the unintended consequence of undermining the interaction designer’s autonomy. To become an, accountant, professor or engineer, individuals must meet compulsory standards, pass examinations, and prove their mastery of the profession’s “canon” of knowledge in order to practice it. Jobs that require a “canon” are typically called a “profession” instead of a mere “occupation.”

Indeed, a “profession” is not simply a job requiring skill. A profession differs from an occupation in that its members exercise exclusive control over a specific body of knowledge (Friedman, 2000, Greenwood, 1957, Larson, 1977). A profession must therefore have a clearly defined certification process, which in turn allows its members to exercise a sort of monopoly over the work itself. If a doctor is fired from a hospital, she continues to be a doctor. No hospital administrator can remove her ability to write prescriptions, for example. Only her peers can remove or grant this ability. Her peers have decided she has met the minimum acceptable standards to write prescriptions and practice medicine; the hospital administrator’s opinion is irrelevant. The power of the professional, then, is inextricably bound up with her knowledge and training.

The “community of practice” is no substitute for a profession. It is merely the poor man’s version of a profession; it refers to the informal knowledge sharing sessions of Xerox technicians, who bully each other instead of fighting for higher wages or more autonomy (Seely Brown and Duguid, 1991).  As with copier repair, there is no body of knowledge that is collectively recognized as comprising “interaction design,” much less “interaction design research.” In their 2006 survey, the IA Institute found 48% of self-identified information architects had no formal training, and almost 3% of those surveyed “weren’t sure” (!) if they had formal education (Information Architecture Institute, 2006).  It is for this reason that there is much confusion about what interaction design research should really look like. No accountant questions how to gather data for creating a cash-flow statement. Certainly, there may be debate about the “right” method, and perhaps there are several schools of thought to which individual accountants tend to subscribe. But in the end, there is no debate that a cash flow statement has X, Y, and Z and if it has A, B, and C, then it is not a cash flow statement, but a balance sheet.

Interaction designers have no such luxury. What exactly constitutes an “interaction”? Where does interaction design end and aesthetic design begin? These questions may seem overly theoretical to some, and indeed, they are theoretical. But it is this very line of questioning that defines the professionalization process. What constitutes a dentist over a dental hygienist? Dentists and dental professors themselves defined that difference – for their own benefit (Adams, 2003). Practitioners of a discipline must delineate the theoretical confines of a discipline (and the requisite knowledge that must be mastered to claim expertise in that discipline) in order to claim occupational autonomy. Simply forming a “community of practice” and gathering for “shop talk” is not sufficient. Xerox technicians have not successfully created a monopoly of knowledge over photocopiers, neither have they created a strong lobby for occupational control.

The HCI Connection

This is not to say that interaction design is completely bereft of an intellectual tradition. The IA Institute’s industry survey did find that, of the information architects that were formally trained, 40% of them had training in Library Science and another 12% in Human Computer Interaction. This suggests there is, at least, a significant number of practitioners (at least those identifying as “information architects”) with similar training. The HCI and Library Science disciplines inculcate their students with a distinctively quantitative approach to research. The November 2009 annual meeting for the American Society for Information Science and Technology (ASIS&T) included a full-day pre-conference workshop “infometrics” and “scientometrics,” which trained participants on a multitude of quantitative methods. The ASIS&T also maintains several “special interest groups” or SIGs that are specifically targeted around metrics, measurement and quantitative methods. Not one SIG specializes in “design” or “qualitative” methods. The Computer Human Interaction (CHI) SIG in ASIS&T professes interest in “online users and their behavior,” and not the symbolic, interpretive or otherwise cultural aspects of the online experience.

One Small Question: What is reality?

Underneath this focus on metrics and “behavior” is a set of implicit: assumptions within the HCI/Information Science tradition. This assumption cuts to the heart of the “sample size” debate: what is the nature of the world and what is the best way to research it? Most researchers subscribe, at least in part, to two established schools of methodological thought: quantitative and qualitative. While they may never be “purely” quantitative or qualitative in their research approaches, researchers tend to subscribe to the overall tenets of their school. The archetypical or “ideal type” quantitative researcher may not actually exist, but describing her methodological approach elucidates unspoken assumptions many researchers may have.

The archetypical quantitative researcher first starts with the assumption that the world is a “real” place that exists independently of human beings (Bryman, 2006). In other words, quantitative research has an objectivist ontology, one which assumes reality is an objective thing that can be researched. Accordingly, the ideal-type quantitative researcher also assumes that the scientific method is the best way to discover this reality, and that a researcher does not affect or shape the outcomes of the research, if appropriate steps to avoid “bias” are taken. On the whole, this approach means looking for the most “typical” occurrence, one which has a necessarily statistical description (Alasuutari, 1995).

Table 1: Qualitative Versus Quantitative Research Paradigms

By contrast, the archetypical qualitative researcher assumes the world is not an objective reality but something that is constructed by us humans, every moment of everyday (Denzin and Lincoln, 2000). Such a researcher considers how humans “make sense” of the world as having primary importance, so his methods are typically aimed at uncovering or “unriddling” this sensemaking process (Alasuutari, 1995). Numerical representations of the “typical” occurrence are irrelevant in this view because there is no typical occurrence.

One can see how “scientific” approaches to interaction design research evolved, therefore, from the objectivist, positivist research paradigm. In this paradigm, it makes sense to count and to find the “average.” And of course in order to do so, one must count sufficient numbers to make it statistically valid. But if one adopts the assumption that there is no such thing as “typical,” that how we make sense of language, for example, tells us how to build Web sites, then it is a logical choice to reject “sample size” as important. The process of sensemaking is more important to the constructivist, interpretivist researcher.

The Design Connection

It is unclear how many self-identified “interaction designers” would reject, wholesale, the title or description of “information architect.” Herein lies the problem. To reject “information architecture” in favor of “interaction design,” is actually to reject the positivist tradition of information “science” in favor of “design.” This is a significant turn.

Design spans both art and science, making its ontological and epistemological position unclear. Design requires both the “logical character of the scientific approach and the intuitive and artistic dimensions of the creative effort.” It spans both deductive and adductive logic; it is the “the process of creation and decision-making” (Borja De Mozota, 2003). Interaction designers draw on both the “science” of decision-making but also the art of creativity.

Is it any wonder then, where our collective schizophrenia comes from? We are fraught with existential angst by the very label of the occupation. We are not entirely sure if we are information scientists or if we are artists. We create our own professional identities as a bricolage, choosing pieces that suit us and rejecting those that don’t. If there are no standards, there is freedom.

Freedom Ain’t Free: A Call To Action

Interaction designers may feel blessed to draw on the scientific tradition for one research project, and on the interpretivist tradition for another. This may feel liberating. But it has its cost.

Professionals command higher pay, status, and autonomy precisely because they have agreed to subscribe to a canon of collected knowledge. They accept that they must prove their familiarity with, say, contracts law even though they do not plan to use it and could easily do without it, thank you very much. Professionals do endure such “irrelevant” learning because they recognize the benefits of having their occupation controlled, even somewhat, by their peers. They enjoy greater freedom at work (Greenwood, 1957). They have higher salaries (Larson, 1977). They can even withstand the slings and arrows of globalization and maintain their professional autonomy (Faulconbridge and Muzio, 2008). Professionals know that by sacrificing a little, they get a lot.

So a call to action. Interaction designers: now is the time to define the theoretical boundaries of your knowledge. What exactly constitutes an “interaction” and how exactly might one “design” it? What is the difference between an interaction designer and an information architect? What, by extension constitutes interaction design research? And finally, for once and for all, does an interaction designer need to care about sample size?

These questions must be answered. We must answer them. I’m not suggesting that interaction designers drop everything and begin furiously debating in the pages of academic journals. Rather, I am suggesting that design educators begin instilling clear and defined canons of knowledge in their students, that practitioners begin adopting (gasp!) standards when hiring, and that collectively, we pursue a consensus.

I present two illustrative examples of professionalization: engineering and medicine. Engineers structured their occupation and thereby collect some benefit, but physicians gained exclusive rights over key aspects of their practice, making their professionalization process much more successful. David Noble traces the professionalization of the engineer in his fascinating history America By Design (Noble, 1979).

It was businessmen, not university-based researchers, which lead the drive to professionalized engineering, resulting in engineers becoming “company men” instead of independent practitioners. Engineers successfully controlled entry into the profession but oftentimes rely on engineering employers for a professional identity. By contrast, physicians professionalized their occupation as a group of independent practitioners. Indeed, it is physicians that all other professions look to emulate (Ritzer and Walczak, 1988). While there have been many recent changes that limit physician autonomy (O’Connor and Lanning, 1992), physicians continue to maintain a near monopoly over the legal ability to prescribe drugs (in the United States, nurse practitioners can prescribe some drugs).

The lesson from these two professions is first to ensure practitioners, not companies, drive professionalization. The Interaction Design Association and the Information Architecture Institute are great starts in this direction. But secondly, interaction designers must gain exclusive control over a certain body of knowledge. For example, interaction designers may seek to “own” accessibility-compliant Web site design. Interaction designers may end up with several schools of thought, which is perfectly acceptable (there are, after all, Jungian psychiatrists as well as Freudians). But at the very least, we will never waste another single pixel on the dreaded “sample size” question!

About the Author

Sam Ladner is a sociologist with an interest in the design of technology and its effect on organizations. She mixes private-sector consulting work with academic research and teaching. Using a range of methods including interviewing, observation and ethnography, she consults on digital product design, organizational change, and the social aspects of technological innovation. She holds a PhD in sociology from York University. She currently works for her own firm as consultant and principal with Copernicus Consulting Group and frequently partners with design firms.

References

ADAMS, T. (2003) Feminization of Professions: The Case of Women In Dentistry. Canadian Review of Sociology and Anthropology, 40.

ALASUUTARI, P. (1995) Researching Culture: Qualitative Methods and Cultural Studies, Thousand Oaks, Sage.

BORJA DE MOZOTA, B. (2003) Design Management: Using Design to Build Value and Corporate Innovation, New York, All Worth Press.

BRYMAN, A. (2006) Integrating Quantitative and Qualitative Research: How Is It Done? Qualitative Research, 6, 97-113.

DENZIN, N. & LINCOLN, Y. (2000) Introduction: The Discipline and Practice of Qualitative Research. IN DENZIN, N. & LINCOLN, Y. (Eds.) Handbook of Qualitative Research. 2nd ed. Thousand Oaks, Sage.

FAULCONBRIDGE, J. & MUZIO, D. (2008) Organizational professionalism in globalizing law firms. Work, Employment and Society, 22, 7-25.

FRIEDMAN, M. (2000) Autonomy, Social Disruption and Women. IN MACKENZIE, C. & STOLJAR, N. (Eds.) Relational Autonomy: Feminist Perspectives on Autonomy, Agency, and the Social Self. Oxford, Oxford University Press.

GREENWOOD, E. (1957) Attributes of a profession. Social Work, 2, 44-55.

INFORMATION ARCHITECTURE INSTITUTE (2006) Information Architecture For The World Wide Web Industry Survey. Seattle, Information Architecture Institute.

LARSON, M. (1977) The Rise of Professionalism, Berkeley, University of California Press.

NOBLE, D. F. (1979) America by design : science, technology, and the rise of corporate capitalism, New York, Oxford University Press.

O’CONNOR, S. & LANNING, J. (1992) The End of Autonomy? Reflections of the Postprofessional Physician. Health Care Management Review, 17, 63-73.

RITZER, G. & WALCZAK, D. (1988) Rationalization and the Deprofessionalization of Physicians. Social Forces, 67, 1-22.

SEELY BROWN, J. & DUGUID, P. (1991) Organizational Learning and Communities of Practice: toward a unified view of working, learning and innovation. Organization Science, 2, 40-57.

“© ACM, (2009). This is the author’s version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in PUBLICATION, XVII.2 – March / April, 2010,

I am currently writing a grant proposal for a research project on mobile phones. This is the (very) short version:

All too often, technology designers create systems that unwittingly expose social actors to socially awkward situations. Companies like Facebook struggle to satisfy their users’ needs to present different selves in different social contexts. The dreaded “My mom is on Facebook” problem is so pervasive it was recently lampooned on Saturday Night Live. Such problems persist because technology designers lack an actionable, sociologically informed understanding of how face-to-face social interaction intersects with and co-constitutes online social interaction. Off-line and online social interactions frequently occur between the same actors, sometimes simultaneously, yet we have little understanding of how online interaction affects, and is affected by off-line interaction. I propose to work with a mobile technology company to investigate how material social life intersects with digital social life, which is now increasingly by the use of Web-enabled smart phones.

Further down in the application, I zero in on my specific research questions:

Social actors often simultaneously present a “work” self through their smart phone, while presenting a “domestic self” to their family members surrounding them. These devices, like the BlackBerry, were originally created for business use (Aoki and Downes, 2003), as was the telephone itself (Flinchy 1997). Researchers have already found that mobile phones make the presentation of a consistent “self” tenuous and vulnerable to disruption in various public spaces (Fortunati, 2005). What are the social consequences of a business technology brought into the domestic context?

I will be adding to this proposal in the coming weeks, but in the meantime, I welcome (nay, beg for) comments from the technology design community. Please weigh in!

Few would disagree that fundamental economic change is upon us. Business models are crumbling daily. From the auto industry to the banking industry, it is clear that old ways of doing things are no longer working. The market research industry is just as vulnerable to this shift, yet, like the auto industry before it, it is hardly aware of how deeply its business model is threatened.

The Long Disruption

The market research industry is built for the 20^th Century mass production model, which is rapidly disappearing. The “mass audience” is gone and a fragmented diverse populace has taken its place. This new “audience” defies the easy aggregation of summary statistics on which market research so often relies.  Chris Anderson of Wired figured this out long ago with his book The Long Tail.

The Economic Disruption of The Long Tail -- Anderson, Wired Magazine

He argued that technology lowered the cost of providing services to ever-smaller niches of people, making it possible to sell profitably goods and services that were once too specialized.

This technological shift also means the end of “appointment television.” Digital video recorders allow individuals to time shift their programming to suite them, and not the program executives at television networks.

The Birth (And Death) of Market Research

What does this all have to do with market research? Full-service market research firms are built for the blockbuster era, not for the time of the long tail.

Market research was heavily influenced by the school of “applied sociology,” lead by Paul Lazarsfeld. While at Columbia, Lazarsfeld pioneered many statistical techniques we use today, including the cross tabulation (Babbie and Benaquisto 2002) and the Lazarsfeld-Stanton Analyzer, a machine that records audience reaction to programming in real time (Mattlerart 1996).

The Lazarsfeld-Stanton Analyzer summarizing "the public"

CNN used a variant of this machine for the recent State of The Union address, showing real-time reactions from Democrats in blue, Republicans in red, and Independents in yellow.

The Lazarsfeld brand of insight is based on a fundamental assumption: that the “average” means something. An entry-level statistics course will teach you that average is dragged up or down by extreme values, and the long tail is nothing if not a collection of many extreme values. There is nothing meaningful about knowing that the “average American” rented 30 digital movies a month if, in fact, there were many thousands of Americans who rented none and a many tiny segments that rented somewhere between zero and 40 movies. The “average” is meaningless in this example, yet this ham-fisted approach to summarizing “the public” is what the market research industry is built upon.

Design Research for The Long Tail

Market researchers may argue that with proper segmentation, you can understand every niche within the long tail. This may be true, but to truly understand the diversity between people, your task is not simply to “summarize” the audience, but to delve deeply into the dynamics of what makes them different.

This is why design research is a better fit for today’s long-tail economic model. Context matters. Design research is all about understanding the context because it is rooted in qualitative methodologies, and ethnography in particular. Designers solve contextual problems. The award-winning Braille watch, for example, allows its users to check the time surreptitiously and quickly, something that is both polite and useful. A Lazarsfeld approach would not uncover the social subtleties of checking one’s watch, and certainly could not uncover the specific needs of the blind.

The Braille Watch by David Chavez

Dan Formosa details this limitation of market research in his insightful article in Interactions magazine. He argues that market research should focus on consumer response — after a product is designed. Design research, on the other hand, is about evaluating a product as it is being developed. I would go further; design research is about knowing what to build as well as evaluating the prototype.

Design research uncovers how long-tail niches develop and what differentiates them. It is not the equivalent to “market segments” because it provides specific direction on how to apply research findings. What are the dynamics of renting a movie? What motivates the “heavy renter”? What is it about her television or home that supports heavy renting? You cannot know the answer to these questions by simply providing a laundry list of demographic characteristics and psychographic survey results. You must know the context in which the long tail emerges.

Some may say that good quality market research would not make these kinds of mistakes. And they are right. Highly skilled social scientists are method-agnostic; they choose the right method for the right research question. However, full-service market research firms have become the GM of the industry — they keep building Hummers instead of Priuses. Focus groups don’t uncover contextual nuances, but they’re cheap and profitable. Surveys don’t get to the heart of why a product doesn’t work. Design research, using an ethnographic approach, provides “thick description” of the entire phenomenon of renting movies.

This is where market research cannot go. And this is where market research will fail, unless it rejects the “build another Hummer” mentality.

References

Babbie, E. and L. Benaquisto (2002). Fundamentals of Social Research. Scarborough, Thomson Nelson.

Mattlerart, A. (1996). The Invention of Communication. Minneapolis, University of Minnesota Press.

“But how many people did you talk to?” If you’ve ever done qualitative research, you’ve heard that question at least once. And the first time? You were flummoxed. In 3 short minutes, you can be assured that will never happen again.

Folks, qualitative research does not worry about numbers of people; it worries about deep understanding. Weber called this “verstehen.” (Come to think of it, most German people call it that too. Coincidence?). Geertz called it “thick description.” It’s about knowing — really knowing — the phenomenon you’re researching. You’ve lived, breathed, and slept this thing, this social occurrence, this…this…part of everyday life. You know it inside and out.

Courtesy of daniel_blue on Flickr

You know when it’s typical, when it’s unusual, what kinds of people� do this thing, and how. You know why someone would never do this thing, and when they would but just lie about it. In short, you’ve transcended merely noticing this phenomenon. Now, you’re ready to give a 1-hour lecture on it, complete with illustrative examples.

Now if that thing is, say, kitchen use, then stand back! You’re not an Iron Chef, you are a Platinum Chef! You have spent hours inside kitchens of all shapes and sizes. You know how people love them, how they hate them, when they’re ashamed of them and when (very rarely) they destroy them. You can tell casual observers it is “simplistic” to think of how many people have gas stoves. No, you tell them, it’s not about how many people, it’s about WHY they have gas stoves! It’s about what happens when you finally buy a gas stove! It’s about….so much more than how many.

Welcome to the world of verstehen. When you have verstehen, you can perhaps count how many people have gas stoves. Sure, you could determine that more men than women have them. Maybe you could find out that more of them were built between 1970 and 80 than 1990 and 2000. But what good is that number? What does it even mean?

When you’re designing, you must know what the gas stove means. You must know what it means to transform your kitchen into one that can and should host a gas stove. You must know why a person would be “ashamed” to have a gas stove (are they ashamed of their new wealth? do they come from a long line of safety-conscious firefighters?). You must know more than “how many.”

So the next time someone asks you, “how many people did you talk to?”, you can answer them with an hour-long treatise about why that doesn’t matter. You can tell them you are going to blow them away with the thick description of what this thing means to people. You are going to tell them you know more about this thing than anyone who ever lived, and then, dammit, you’re gonna design something so fantastic, so amazing that they too will be screaming in German. You have verstehen!

See my discussion about sampling methods in qual and quant research for more insight into the reasons why “how many” is irrelevant in qualitative research.

Social media “gurus” abound these days. Which ones are worth listening to and which ones are bullshitters?

Philosopher Harry Frankfurt exposed bullshitters in his famous essay “On Bullshit.” The liar knows what the truth is and cares very much about concealing it. The bullshitter, on the other hand, doesn’t care what the truth is and has no compunction in stretching it.

The same goes for social media “gurus.” Those that care what about rigourous examination of the social may be wrong, but at least they take great pains to analyze the phenomenon. Those that don’t care about systematic, theoretically informed social inquiry are interested only in stretching or shaping their own agendas.

How can you tell the difference?

Here are a few signs you’re dealing with a social media bullshitter.

1. They skate over the tension between structure and agency: The tension between structure and agency is an age-old sociological debate. Social media bullshitters somehow miss this very important point. They often argue that implementing social media or social business design will somehow evaporate decades or even centuries of organizational structures. If your social media guru tells you that adding social media and stirring will create equality, harmony, and profits, begin to question them. If, on the other hand, they tell you that your organization does not live in a vacuum, and that your social media will be integrated in people’s existing lives with their existing economic, technological, and ethnically grounded experience, then they may be onto something.
2. They use the same social research methods every time: A classically trained sociologist is trained in both qualitative and quantitative methods. They are designers in the sense that they have expertise, which they draw upon selectively, according to the research question. Social media bullshitters, on the other hand, likely have a common stock of tools that they use repeatedly, regardless of the nuance of the research question. If their answer is always, “do a focus group,” or always, “do a survey,” then question them.
3. They see no paradoxes. Ever: Sociologists are constantly grappling with paradoxes. Weber’s famous paradoxical finding was that bureaucracies are both efficient and inefficient. They work wonders building and managing railroads, for example, but they result in horrible catastrophes like the Challenger disaster. Weber explained this paradox by arguing that rationality, or the rule of rules, is an “iron cage,” that keeps us safe but enslaved. If your social media guru claims there will be no paradox, nuance, or ambiguity, question them.
4. They don’t know what social capital really is: Social capital is not something one can measure in terms of bank balances. It was the creation of French sociologist Pierre Bourdieu (come to think of it, the bullshitters wouldn’t know that either). Social capital is something one develops by being in a particular social location. I may go to an exclusive boarding school. My social capital is my network of well-off friends. Social capital is a particularly important concept when thinking about social media. Bourdieu noted that those in lower economic classes explicitly reject items they consider “above their station.” This means that luxury or “top of the line” is not always your best approach.

The bottom line is this: social media bullshitters have no knowledge of social theory or methodology. Trust a person who provides no easy answer, who carefully selects their research method, and who understands complex concepts.

Do you have more signs of being a social media bullshitter? Please share them here!

DT has a great post over at Design Sojourn that discusses Six Sigma methodology and how it relates to design. He cites Tim Brown at IDEO who argues that Six Sigma is essentially Newtonian, while design thinking is quantum. In his own design work, DT expressed doubts about using Six Sigma:

After studying the Six Sigma process, I point blank said: �There was no way any of my designers are going to be judged on the quality and success of a design based on how many sketches or iterations we did before we deliver it.�

Both Brown and DT cite Sara Beckman, who recently discussed the topic in the New York Times. Beckman reviews how Six Sigma focuses on incremental improvements, while design and design thinking focuses on big changes. For those of you who aren’t familiar with Six Sigma, it’s a method pioneered by Motorola, which aims to reduce the number of errors to 3 in one million. The “six sigma” refers to six standard deviations. The number of errors should be at the extreme end of the normal curve, or between + or – 3 standard deviations, represented by the Greek symbol sigma.

I argue that design is more complementary to the “interpretivist” paradigm of qualitative research while Six Sigma is positivist. Interpretivists don’t believe the world is a static place. They see reality as being continuously created by you, me and other social actors. There is no such thing as “The Truth” in interpretivist approaches, just different versions of the truth. Typical methods of interpretivists are ethnography, in-depth interviewing and discourse analysis. Positivist research, on the other hand, assumes that reality is static. Positivists believe that “The Truth,” is out there to be discovered. Typical methods would include quantitative surveys.

Designers should focus on interpretivist methods, therefore. They should uncover different versions of the truth using observation and interviewing, as well as deep reflection on symbols and their meanings. Surveys and other quantitative methods are more Six Sigma in that they can measure improvement over time. Designers ought to consider measuring improvement, but starting with qualitative approaches is best.

The New York Times reports that the American Medical Association is considering doing away with the venerable symbol of the physician: the lab coat. There’s a very good reason to get rid of lab coats: they’re dirty. But the symbol of the lab coat is far more important. The New York Times reports the empirical flaw in wearing lab coats:

The group�s Council on Science and Public Health is looking at the role clothing plays in transmitting bacteria and other microbes and is expected to announce its findings next year.

This empirical finding shouldn’t be surprsing. We also know, for example, that male physician’s ties are wearable petri dishes. The verdict ought to be clear, therefore that we should get rid of lab coats. Not so fast, say physicians.

Getting rid of the lab coat is getting rid of one of the most important symbols of a physician’s identity. Dr. Richard Cohen told the New York Times how important that lab coat is:

�When a patient shares intimacies with you and you examine them in a manner that no one else does, you�d better look like a physician � not a guy who works at Starbuck�s.�

Here is the lesson for designers: empirical “fact” is not the whole story. What role any particular symbol plays in social life is just as critical. What’s fascinating about this story is that physicians are now trained in “evidence-based medicine,” meaning they are trained to diagnose and treat based on more “rigourous” science (I have my doubts about that rigour, but that’s another blog post).

Yet here is a clearly “scientific” reality about the danger of treating patients while wearing a bacteria-infested lab coat and/or tie, and physicians continue to wear them. For all their protestations of “evidence,” physicians too are social beings, embedded in a social world. They too must convey an identity, even if the symbols used for doing so compromise their ability to complete their stated vocational mission.

The symbol is powerful. Designers who base their decisions on so-called “evidence” ought to pay attention to other kinds of evidence, such as the enduring patterns of social interactions. We should pay attention to any enduring patterns of social behaviour but *especially* those which fly in the face of supposed “logic.”