Previous post:

Next post:

The Essence of Interaction Design Research: A Call for Consistency

by Sam Ladner on November 20, 2010 · 4 comments

in Blog, Research Methods, culture, design, home, interaction design, management, organizations, personas, qualitative research, quantitative research, sample size

The Essence of Interaction Design Research: A Call for Consistency

This post is reproduced from the original Interactions magazine article

It started with an innocent query to the IxDA listserv. Someone was sure they had read an article in Interactions magazine once but could not find it again: Wasn’t there something written sometime by someone about something like sample size in usability research? asked an expectant interaction designer. Woe is the hapless interaction designer who is unprepared for the firestorm that follows the dreaded “sample size” question. 106 replies later, and not only was the answer clearly left unanswered but worse, it left many scratching their heads in genuine confusion: what is the essence of interaction design research? Is it data-driven and “scientific”? Is it exploratory and qualitative? No consensus was reached. Again.

Wee dinosaurs

What's your sample size again?

This schizophrenia is both a blessing and a curse. One the one hand, an interaction designer has the freedom to assemble her research program like an artist assembling an installation: whatever inspires her can indeed find a place in the final result. Yet, such a lack of standards leads to a distinct lack of consistency and expertise. If interaction design research is whatever you want it to be, what is to stop other occupations “colonizing” what ought to be the purview of the interaction research? See, for example, Dan Formosa’s article in this year’s January-February issue of Interactions, lamenting the intrusion of market research into the design field. When there are no standards, there is freedom.

As Sartre said, we are “condemned to be free,” meaning when there are no pre-defined codes of conduct, then we must tragically, wonderfully, horribly, create ourselves. The confusion over the essence of interaction design research is us, thrashing about as we desperately create ourselves.

You are condemned to be free: courtesy of http://artoftheprank.com/author/mibanez/

In this article, I explain how this lack of standardization affects the practice of interaction design research. In particular, I note that the dreaded “sample size” debate is actually indicative of a larger issue of theoretical training. I call on interaction designers to embrace standardization – not blindly, but with eyes wide open – for the benefit of the interaction design research and for the profession itself.

The Long and Winding Road

Most people stumble into interaction design. Unlike a profession such as medicine, for example, interaction design has a distinctively ill-defined apprenticeship. The proliferation of interaction design job titles demonstrates this lack of definition. A lack of standardization is liberating to many but has the unintended consequence of undermining the interaction designer’s autonomy. To become an, accountant, professor or engineer, individuals must meet compulsory standards, pass examinations, and prove their mastery of the profession’s “canon” of knowledge in order to practice it. Jobs that require a “canon” are typically called a “profession” instead of a mere “occupation.”

Indeed, a “profession” is not simply a job requiring skill. A profession differs from an occupation in that its members exercise exclusive control over a specific body of knowledge (Friedman, 2000, Greenwood, 1957, Larson, 1977). A profession must therefore have a clearly defined certification process, which in turn allows its members to exercise a sort of monopoly over the work itself. If a doctor is fired from a hospital, she continues to be a doctor. No hospital administrator can remove her ability to write prescriptions, for example. Only her peers can remove or grant this ability. Her peers have decided she has met the minimum acceptable standards to write prescriptions and practice medicine; the hospital administrator’s opinion is irrelevant. The power of the professional, then, is inextricably bound up with her knowledge and training.

The “community of practice” is no substitute for a profession. It is merely the poor man’s version of a profession; it refers to the informal knowledge sharing sessions of Xerox technicians, who bully each other instead of fighting for higher wages or more autonomy (Seely Brown and Duguid, 1991).  As with copier repair, there is no body of knowledge that is collectively recognized as comprising “interaction design,” much less “interaction design research.” In their 2006 survey, the IA Institute found 48% of self-identified information architects had no formal training, and almost 3% of those surveyed “weren’t sure” (!) if they had formal education (Information Architecture Institute, 2006).  It is for this reason that there is much confusion about what interaction design research should really look like. No accountant questions how to gather data for creating a cash-flow statement. Certainly, there may be debate about the “right” method, and perhaps there are several schools of thought to which individual accountants tend to subscribe. But in the end, there is no debate that a cash flow statement has X, Y, and Z and if it has A, B, and C, then it is not a cash flow statement, but a balance sheet.

Interaction designers have no such luxury. What exactly constitutes an “interaction”? Where does interaction design end and aesthetic design begin? These questions may seem overly theoretical to some, and indeed, they are theoretical. But it is this very line of questioning that defines the professionalization process. What constitutes a dentist over a dental hygienist? Dentists and dental professors themselves defined that difference – for their own benefit (Adams, 2003). Practitioners of a discipline must delineate the theoretical confines of a discipline (and the requisite knowledge that must be mastered to claim expertise in that discipline) in order to claim occupational autonomy. Simply forming a “community of practice” and gathering for “shop talk” is not sufficient. Xerox technicians have not successfully created a monopoly of knowledge over photocopiers, neither have they created a strong lobby for occupational control.

The HCI Connection

This is not to say that interaction design is completely bereft of an intellectual tradition. The IA Institute’s industry survey did find that, of the information architects that were formally trained, 40% of them had training in Library Science and another 12% in Human Computer Interaction. This suggests there is, at least, a significant number of practitioners (at least those identifying as “information architects”) with similar training. The HCI and Library Science disciplines inculcate their students with a distinctively quantitative approach to research. The November 2009 annual meeting for the American Society for Information Science and Technology (ASIS&T) included a full-day pre-conference workshop “infometrics” and “scientometrics,” which trained participants on a multitude of quantitative methods. The ASIS&T also maintains several “special interest groups” or SIGs that are specifically targeted around metrics, measurement and quantitative methods. Not one SIG specializes in “design” or “qualitative” methods. The Computer Human Interaction (CHI) SIG in ASIS&T professes interest in “online users and their behavior,” and not the symbolic, interpretive or otherwise cultural aspects of the online experience.

One Small Question: What is reality?

Underneath this focus on metrics and “behavior” is a set of implicit: assumptions within the HCI/Information Science tradition. This assumption cuts to the heart of the “sample size” debate: what is the nature of the world and what is the best way to research it? Most researchers subscribe, at least in part, to two established schools of methodological thought: quantitative and qualitative. While they may never be “purely” quantitative or qualitative in their research approaches, researchers tend to subscribe to the overall tenets of their school. The archetypical or “ideal type” quantitative researcher may not actually exist, but describing her methodological approach elucidates unspoken assumptions many researchers may have.

The archetypical quantitative researcher first starts with the assumption that the world is a “real” place that exists independently of human beings (Bryman, 2006). In other words, quantitative research has an objectivist ontology, one which assumes reality is an objective thing that can be researched. Accordingly, the ideal-type quantitative researcher also assumes that the scientific method is the best way to discover this reality, and that a researcher does not affect or shape the outcomes of the research, if appropriate steps to avoid “bias” are taken. On the whole, this approach means looking for the most “typical” occurrence, one which has a necessarily statistical description (Alasuutari, 1995).

Table 1: Qualitative Versus Quantitative Research Paradigms

Quantitative Qualitative
Role of theory in research Deductive, testing of theory Inductive, generating theory
Ontological orientation Objectivism Constructionism
Epistemological orientation Natural science model; “positivism” Interpretivism

By contrast, the archetypical qualitative researcher assumes the world is not an objective reality but something that is constructed by us humans, every moment of everyday (Denzin and Lincoln, 2000). Such a researcher considers how humans “make sense” of the world as having primary importance, so his methods are typically aimed at uncovering or “unriddling” this sensemaking process (Alasuutari, 1995). Numerical representations of the “typical” occurrence are irrelevant in this view because there is no typical occurrence.

One can see how “scientific” approaches to interaction design research evolved, therefore, from the objectivist, positivist research paradigm. In this paradigm, it makes sense to count and to find the “average.” And of course in order to do so, one must count sufficient numbers to make it statistically valid. But if one adopts the assumption that there is no such thing as “typical,” that how we make sense of language, for example, tells us how to build Web sites, then it is a logical choice to reject “sample size” as important. The process of sensemaking is more important to the constructivist, interpretivist researcher.

The Design Connection

It is unclear how many self-identified “interaction designers” would reject, wholesale, the title or description of “information architect.” Herein lies the problem. To reject “information architecture” in favor of “interaction design,” is actually to reject the positivist tradition of information “science” in favor of “design.” This is a significant turn.

Design spans both art and science, making its ontological and epistemological position unclear. Design requires both the “logical character of the scientific approach and the intuitive and artistic dimensions of the creative effort.” It spans both deductive and adductive logic; it is the “the process of creation and decision-making” (Borja De Mozota, 2003). Interaction designers draw on both the “science” of decision-making but also the art of creativity.

Is it any wonder then, where our collective schizophrenia comes from? We are fraught with existential angst by the very label of the occupation. We are not entirely sure if we are information scientists or if we are artists. We create our own professional identities as a bricolage, choosing pieces that suit us and rejecting those that don’t. If there are no standards, there is freedom.

Freedom Ain’t Free: A Call To Action

Interaction designers may feel blessed to draw on the scientific tradition for one research project, and on the interpretivist tradition for another. This may feel liberating. But it has its cost.

Professionals command higher pay, status, and autonomy precisely because they have agreed to subscribe to a canon of collected knowledge. They accept that they must prove their familiarity with, say, contracts law even though they do not plan to use it and could easily do without it, thank you very much. Professionals do endure such “irrelevant” learning because they recognize the benefits of having their occupation controlled, even somewhat, by their peers. They enjoy greater freedom at work (Greenwood, 1957). They have higher salaries (Larson, 1977). They can even withstand the slings and arrows of globalization and maintain their professional autonomy (Faulconbridge and Muzio, 2008). Professionals know that by sacrificing a little, they get a lot.

So a call to action. Interaction designers: now is the time to define the theoretical boundaries of your knowledge. What exactly constitutes an “interaction” and how exactly might one “design” it? What is the difference between an interaction designer and an information architect? What, by extension constitutes interaction design research? And finally, for once and for all, does an interaction designer need to care about sample size?

These questions must be answered. We must answer them. I’m not suggesting that interaction designers drop everything and begin furiously debating in the pages of academic journals. Rather, I am suggesting that design educators begin instilling clear and defined canons of knowledge in their students, that practitioners begin adopting (gasp!) standards when hiring, and that collectively, we pursue a consensus.

I present two illustrative examples of professionalization: engineering and medicine. Engineers structured their occupation and thereby collect some benefit, but physicians gained exclusive rights over key aspects of their practice, making their professionalization process much more successful. David Noble traces the professionalization of the engineer in his fascinating history America By Design (Noble, 1979).

It was businessmen, not university-based researchers, which lead the drive to professionalized engineering, resulting in engineers becoming “company men” instead of independent practitioners. Engineers successfully controlled entry into the profession but oftentimes rely on engineering employers for a professional identity. By contrast, physicians professionalized their occupation as a group of independent practitioners. Indeed, it is physicians that all other professions look to emulate (Ritzer and Walczak, 1988). While there have been many recent changes that limit physician autonomy (O’Connor and Lanning, 1992), physicians continue to maintain a near monopoly over the legal ability to prescribe drugs (in the United States, nurse practitioners can prescribe some drugs).

The lesson from these two professions is first to ensure practitioners, not companies, drive professionalization. The Interaction Design Association and the Information Architecture Institute are great starts in this direction. But secondly, interaction designers must gain exclusive control over a certain body of knowledge. For example, interaction designers may seek to “own” accessibility-compliant Web site design. Interaction designers may end up with several schools of thought, which is perfectly acceptable (there are, after all, Jungian psychiatrists as well as Freudians). But at the very least, we will never waste another single pixel on the dreaded “sample size” question!

About the Author

Sam Ladner is a sociologist with an interest in the design of technology and its effect on organizations. She mixes private-sector consulting work with academic research and teaching. Using a range of methods including interviewing, observation and ethnography, she consults on digital product design, organizational change, and the social aspects of technological innovation. She holds a PhD in sociology from York University. She currently works for her own firm as consultant and principal with Copernicus Consulting Group and frequently partners with design firms.

References

ADAMS, T. (2003) Feminization of Professions: The Case of Women In Dentistry. Canadian Review of Sociology and Anthropology, 40.

ALASUUTARI, P. (1995) Researching Culture: Qualitative Methods and Cultural Studies, Thousand Oaks, Sage.

BORJA DE MOZOTA, B. (2003) Design Management: Using Design to Build Value and Corporate Innovation, New York, All Worth Press.

BRYMAN, A. (2006) Integrating Quantitative and Qualitative Research: How Is It Done? Qualitative Research, 6, 97-113.

DENZIN, N. & LINCOLN, Y. (2000) Introduction: The Discipline and Practice of Qualitative Research. IN DENZIN, N. & LINCOLN, Y. (Eds.) Handbook of Qualitative Research. 2nd ed. Thousand Oaks, Sage.

FAULCONBRIDGE, J. & MUZIO, D. (2008) Organizational professionalism in globalizing law firms. Work, Employment and Society, 22, 7-25.

FRIEDMAN, M. (2000) Autonomy, Social Disruption and Women. IN MACKENZIE, C. & STOLJAR, N. (Eds.) Relational Autonomy: Feminist Perspectives on Autonomy, Agency, and the Social Self. Oxford, Oxford University Press.

GREENWOOD, E. (1957) Attributes of a profession. Social Work, 2, 44-55.

INFORMATION ARCHITECTURE INSTITUTE (2006) Information Architecture For The World Wide Web Industry Survey. Seattle, Information Architecture Institute.

LARSON, M. (1977) The Rise of Professionalism, Berkeley, University of California Press.

NOBLE, D. F. (1979) America by design : science, technology, and the rise of corporate capitalism, New York, Oxford University Press.

O’CONNOR, S. & LANNING, J. (1992) The End of Autonomy? Reflections of the Postprofessional Physician. Health Care Management Review, 17, 63-73.

RITZER, G. & WALCZAK, D. (1988) Rationalization and the Deprofessionalization of Physicians. Social Forces, 67, 1-22.

SEELY BROWN, J. & DUGUID, P. (1991) Organizational Learning and Communities of Practice: toward a unified view of working, learning and innovation. Organization Science, 2, 40-57.

“© ACM, (2009). This is the author’s version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in PUBLICATION, XVII.2 – March / April, 2010,

Share:
  • Twitter
  • del.icio.us
  • Facebook
  • FriendFeed
  • StumbleUpon
  • Digg
  • LinkedIn
  • Technorati
  • email

Categories: Blog · Research Methods · culture · design · home · interaction design · management · organizations · personas · qualitative research · quantitative research · sample size

{ 4 comments… read them below or add one }

1 IKnowMore December 15, 2010 at 3:28 am

Table 1 is nonsense.

Firstly, scientific methodology is not deductive, but predominantly inductive. Deduction is a process of reasoning by which one takes certain propositions as hypotheses and through the application of certain rules arrives at other propositions. Inductive reasoning is a process of reasoning in which one takes one set of data and uses it as the basis to extrapolate about a large set of data, or to make predictions about data one might collect. Sometimes that extrapolation is statistical in character, but sometimes it is not. For example, in mathematics both deductive and inductive reasoning is used. One might use deduction to prove that a certain relation on a set is a partial order. Or one might use induction to show that there are an infinite n umber of primes. And then there is abductive reasoning.

As far as I can tell, you are confusing the accepted terminology.

Still, I might accuse you of a really horrible kind of reductionism, because in practice scientists proceed using all three modes or reasoning (if not more). All you accomplish is to reproduce and construct a stereotype that, I think, is used to excuse a rather vulgar sort of antipositivism, at the cost of, ironically, the facts of the matter.

Not only that, but you reduce scientific methodologies to various sorts of statistical procedures, as if that sort of quantitative reasoning, or averaging was the essence of science, and not just a methodological aspect of particular research programs. In particular, mathematics can be used in distinctly non-quantitative, and qualitative ways. It seems to presume that math and science is about counting things and not with describing the way things are, or even as they appear to be. But surprise surprise, if you look at the literature you will see that scientists do all kinds of interesting things with mathematics, e.g. qualitative formal models of points of view, of the sharing of information, of emotion, of analogy and metaphor, of endoperspectives, of meaning, indeterminacy, vagueness, linguistic pragmatics, semiosis, narrative, context dependency, situatedness, embodiment, and so on. I think that some self-described quals just suffer a massive failure of imagination (I don’t want to accuse them of laziness).

In short, your distinction between quals and quants is bunk, part of a large pattern (objectively observed and verified) of the social construction of a counter-factual cultural ontology.

If you do subscribe to an antipositivist program, denying that there is a real world out there and facts to be learned, that the world (including Pluto and some anonymous asteroids in the asteroid belt just beyond it) is social constructed by humans (rather than just the representations of that world), I sincerely wonder what you tell your clients you are doing, or what you think they are hiring you for.

Reply

2 Sam Ladner December 15, 2010 at 7:47 am

Hi Anon.,

Thanks for trolling! Ah but seriously, your comments are welcome. However, you haven’t read much in the way of methodology. I am not at all confusing terminology, but using it correctly.

What you seem to be pointing out is the gap between scientific practice and scientific reasoning. As Latour and Woolgar will tell you, science is a messy business, filled with all sorts of inductive reasoning. But this is the practice; the ideal-type reasoning of the scientific method is deductive.

Vulgar anti-positivism? I think not! This is, in fact, critical realism, which you would know to be entirely post-positivist! I do not deny there is a real world out there (why, you seem to assume that I do!). But I do deny that there is such a thing as Actualism, that is, that social events are predictable. Critical realists do believe that the world is full of things, not events, which are predictable quantitatively. Indeed, even physicists would agree with me.

In short, I’m surprised by your abusive tone. I don’t usually attract trolls, and I’m sure Anonymous is quite busy taking down PayPal or Mastercard. So I have my first real troll — and he’s ontologically confused!

Reply

3 Qualitative Research February 16, 2011 at 2:22 pm

great article. I became more and more excited to learn about qualitative research
best regards

Reply

4 Sam Ladner February 16, 2011 at 2:23 pm

That’s great! Exactly what we’d hoped!

Reply

Leave a Comment

You can use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Previous post:

Next post:

Blog
design
qualitative research
product design
Services
Research design
Ethnography
In-depth Interviewing
Usability testing
Projects
Consumer Electronics
High technology
Health Care