Weick, 8 Wiener, 9 Winograd, 10 and Woods 11
Vicente, 7 Weick, 8 Wiener, 9 Winograd, 10 and Woods 11
in shaping our vision of sensemaking and the impli- cations for collaborations between smart people and smart machines to solve “wicked problems.”
Wicked Problems
Nousala, Dubberly, and Pangaro wondered at our use of the term wicked problems in our conceptual model of a sociotechnical system (Figure 1 in the target article). We chose the label to represent the kinds of problems that sociotechnical systems must grapple with as a way to emphasize that these are ‘open systems,’ and that the problems they must solve are ‘complex.’ The systems are open to the extent that there are external disturbances that cannot be fully specified or anticipated, and the problems are com- plex in the sense that there are typically large num- bers of incommensurate variables—in some cases, an unspecifiable number of variables—that could poten- tially be relevant.
Additionally, the problems are such that there is typically no unambiguously correct solution or right answer. Thus, these problems are only solvable in the sense of satisficing—in other words, the solutions can only be evaluated pragmatically—are the conse- quences of decisions/actions deemed satisfactory with respect to the mission of the organization. The key is that there is no analytical basis for identifying any particular solution as optimal or absolutely right.
Note that satisficing is generally discussed relative to human limitations, or bounded rationality. The implication is that the choice of a less than optimal solution is due to humans’ internal constraints. How- ever, we are using the term satisficing here to reflect
a property of the problems. In this case, satisficing is not a compromise relative to a potentially optimal solution, but rather it indicates that it is impossible to specify an optimal solution. A solution either leads to satisfying or unsatisfying consequences, but there is no absolute basis for identifying a particular solution or set of solutions as absolutely better than another set. Further, in some cases there will be no a priori normative basis to predict whether a solution will
be satisfying or even to judge post hoc whether the choices made were good ones. Healthcare provides
a good example—in cases of diseases such as cancer, it will often be difficult to unambiguously judge
whether a patient recovered or died because of or in spite of the treatments. Were negative side effects from treatments avoidable or not? At what point do the negative side effects outweigh any benefits of extending life? Was the ultimate outcome worth the suffering?
Derek Miller’s commentary provides a clarifica- tion and amplification of our use of the term wicked problems. The problems of sociotechnical systems often involve conflicts of values. These values are often incommensurate in the sense that there is no common metric for integrating them into a unified cost-function or single figure of merit to unambig- uously determine the quality of a solution. Miller suggests some examples, including how much liberty we should sacrifice in pursuit of order or security. The critical thing here is that the right balance across competing values cannot be absolutely prescribed in- dependently from the context—including economic, social, political, and cultural considerations—and these contexts are changing, or non-stationary. A bal- ance that is stable in one context or at one time may not be stable in another context.
We would further claim that wicked problems are far more common than is generally thought. Picking a graduate school, buying a house, choosing
a mate, diagnosing and treating a cancer patient, and selecting a career path would all fit our concept of what a wicked problem is, due to the trade-off of incommensurate values they require, their non-sta- tionary dynamics, and the size and dimensionality of their possibility sets. This should be even more obvious in the case of larger sociotechnical organiza- tions that are dealing with problems with more global political, social, and cultural implications—managing
a national healthcare system, for example.
There may be better terms for characterizing the complex challenges of sociotechnical systems. For ex- ample, Simon and Newell use the term “ill-structured problems”:
“Problems are ill-structured when they are not well-structured. In some cases, for example, the essential variables are not numerical at all, but symbolic or verbal…. Second, there are many important situations in everyday life where the objective function, the goal, is vague and non- quantitative…. Third, there are many practical problems—it would be accurate to say ‘most practical problems’—for which computational
algorithms simply are not available.” 12
The essential motivation behind the use of wicked problems as a term is to counter a tendency for The essential motivation behind the use of wicked problems as a term is to counter a tendency for
human fulfillment is a chimera,” 13 we would argue
that the hope that sociotechnical problems are analyt- ically tractable or solvable in any absolute sense is a chimera. The best we can hope for is a dynamic sta- bility where the inevitable mistakes that the system makes are at least survivable—satisfactory—and perhaps that the system learns from each mistake, so that fitness generally improves with experience. This
perception-action loop is closed through a complex or wicked problem
leads to a second concern of Dubberly and Pangaro.
domain. Image © 2016 by John M. Flach and Kyle J. Behymer.
particularly reflected in the clash of values described
Control versus Stability
so well by Miller in his commentary. In the dynamic Dubberly and Pangaro also raised a concern about the
of sociotechnical systems, the stable solutions are not fact that there is no explicit goal in our model ( Figure
specified a priori as goals and/or cost-functions. Rather,
1 in the target paper) noting that a goal is “a key com- the stable solutions typically emerge as the result of
contests between competing values. This is more sim- of any explicit goal is intentional and in fact, we
ponent of any control system model.” 14 The omission
ilar to a prey-predator system than a servomechanism. think it is misleading to call this a “control system
That is, in the same way that stable population levels model.” Although this is a circular system, where
can emerge from competition in natural ecologies, the consequences of actions feed back as informa-
stable solutions can emerge in sociotechnical systems tion that can shape future actions, it is definitely
as a result of competition among diverse values. Lind- not a simple servomechanism. There is no explicit
blom has described this dynamic of sociotechnical goal, and thus there is no clear basis for computing
systems as “muddling through” or “incrementalism.” 16 an “error signal.” As noted in the previous section,
Solutions typically emerge as a result of incremental rather than a well-specified goal, performance in
adjustments in organizations that are highly resistant sociotechnical systems is often influenced by a host
to change. This resistance reflects a friction that may of diverse values that may or may not be shared by 17 often be essential for stability in a complex ecology.
the distributed agents involved in observing the In a self-organizing system, the stable points are state of the world, and acting upon that world.
not imposed from without—an explicit a priori goal— Unfortunately, our choice to use the labels “con-
rather, the stable points emerge from the internal dy- troller” and “observer” in the original figure contrib-
namics of the coupling. In the prey-predator example, utes to confusion about the kind of system we are
the critical question from a control theoretic perspec- describing. Thus, we have revised the figure to use the
tive is to understand the conditions that lead to stable alternative labels of “perception” and “action” ( Figure
populations, rather than the collapse of the ecosystem R1 ). We have also changed the labels on the compo-
resulting in extinction. In sociotechnical systems, nents to eliminate the label “control.” The original
stability will be a function of the capabilities of the choice of terms was intended to emphasize paral-
distributed organization relative to the demands of lels to concepts used in control theory to describe
the wicked problems. This leads naturally to another systems. However, we see control theory as a field
of Dubberly and Pangaro’s concerns. concerned primarily with understanding stability in closed-loop systems. Thus, our view of control theory may be somewhat inconsistent with the way many
Requisite Variety
social scientists understand it to be—a servomecha- As Dubberly and Pangaro note, there are two primary nism. In our view, control theory considers trade-offs
sources of variety in our model of a sociotechnical across many potential solutions to achieving stability,
system. One source is the variety associated with the where the simple servomechanism is but one of the
wicked problem. This determines the constraints asso- simpler, more primitive examples. 15 ciated with the possible outcomes that are, in essence,
So what kind of system do we envision in Figure the problem space—like the beach in Simon’s classic R1 ? We see this as a self-organizing system. This is
analogy. 18 The other source is the variety associated
From Autonomous to Sociotechnical Systems
A B C analogy. The key implication of Ashby’s Law of Requi-
site Variety 19 is that complete control—the ability of the ant to go anywhere on the beach—requires that the organization has at least as much variety as the problem space. In other words, if the organization is
more constrained than the problem space, there will Organiza on
Wicked
Problem
be some states that will not be reachable. To frame this in terms of the functional components of the
variety associated with a wicked problem (solid circles) and the variety
organization, lower variety in the organization of associated with an organization of agents (dashed circles). Image © 2016
by John M. Flach and Kyle J. Behymer.
agents implies that either there are important dis- criminations with respect to the problem state that
illustrates a bounded rationality in which the variety the perception component is incapable of making, or
of the problem exceeds the variety of the organiza- the problem requires some actions or moves that the
tion. In this case, full or optimal control is not pos- action component cannot make. In terms of Simon’s
sible. However, it still may be possible for the system analogy, for example, constraints on the locomotion
to achieve a stable equilibrium or to satisfice. In other capabilities of the ant might make it impossible for it
words, the system may not be able to go everywhere, to reach some areas of the beach.
but it may be able to avoid catastrophe most of the In the classical use of the term satisficing, there
time—for example, through the use of smart heu- is at least an implication that the requisite variety of
ristics. The second relation (B) illustrates a situation the unaided human is often less than the problem—
where the organization has more variety than the there are internal constraints, biases or limitations—
wicked problem. In this case, full control is possible. that require the human to settle for a less than
On the positive side, the excess variety may allow in- optimal solution. For example, Flach has suggested
creased flexibility and redundancy, increasing system that typically no single agent in an organization will
stability. On the negative side, the excess variety may satisfy the requisite variety demands for many soci-
be only so much “noise,” possibly making it difficult
for the system to discover or settle on even a satis- that it may be possible for a collection of diverse
otechnical problems. 20 However, Flach also suggests
fying solution.
agents—including human and autonomous systems— The third relation in Figure R2 (C) illustrates to satisfy the demands of Ashby’s law. However, there
what is probably the most common situation— a is another dimension of variety that connects to the
partial overlap between the variety associated with previous ideas of wicked problems and satisficing that
the problem and the variety associated with the Dubberly and Pangaro raise. This is the potential for
organization. In this context, it is easier to frame the there to be variety within the organization above and
discussion in terms of constraints, or factors that beyond that required to meet the demands of requi-
limit variety. Thus, the constraints associated with the site variety.
organization are not fully aligned with the constraints The difficulty of wicked problems may lie in part
of the wicked problem. This implies that the ratio- with the fact that there is greater variety in the orga-
nality of the organization will be bounded relative to nization than in the problem. One obvious source of
the problem and that there may be internal noise that this variety, suggested in Miller’s commentary, would
increases the difficulty of identifying satisfying strat-
be differences in the values among the various agents egies for achieving stability. The critical point is that in an organization. For example, even if a solution is
in shaping an organization to deal with wicked prob- satisfactory because the organization remains viable,
lems, one must often consider both 1) how to scale-up there may often not be consensus across the agents
the limited variety of individual agents to satisfy the about whether any particular solution is satisfying. In
requisite demands of the wicked problem, and 2) how solving sociotechnical problems, no matter the quality
to filter out the noise, or extract the signal from the of a solution, there will almost always be people
noise that reflects a diversity of opinions within the within the organization who are unsatisfied and who
organization. This leads naturally to the last of Dub- will second-guess decisions and actions taken.
berly and Pangaro’s concerns.
Figure R2 illustrates some of the potential re- lations between variety associated with a wicked problem (solid circle) and variety associated with the
The Quality of Communications
organization (dashed circle). The first relation (A) Dubberly and Pangaro note that our model “does
she ji The Journal of Design, Economics, and Innovation Volume 2, Number 2, Summer 2016 she ji The Journal of Design, Economics, and Innovation Volume 2, Number 2, Summer 2016
and ‘effective.’” 21 Then they pose the million-dollar tations are typically constructed using spatial analogs question: “How do we achieve it?” In the target paper,
or metaphors. The incorporation of the mouse and we finessed this by providing examples of systems
the desktop metaphor that transformed the com- where we feel that the communications were effective
puter from a scientific instrument to an all-purpose (Chef Watson) and ineffective (Clippy). We did this
personal information system is a classical example of partly because we don’t think there is a single, simple
how design can improve the quality of communica- prescription for quality communications. The design
tions between humans and computers. Shneiderman’s challenge will always be to align the constraints
concept for direct manipulation interfaces is another (variety) internal to the organization with the con-
example of how representations can facilitate coor- straints (variety) of the wicked problems to be solved.
dination between humans and computers to manage As suggested in the previous section, this will involve 24 complex problems.
aligning the internal constraints with the problem Another potentially powerful way to tune the in- constraints, which may require 1) diversifying the or-
ternal constraints of an organization to better match ganization to scale up the internal variety to meet the
the deep structure of the problem domain is through demands of Ashby’s Law, and 2) tuning the internal
narratives or stories that reflect the insights of ex- constraints to pull out the signal from the noise.
perts in dealing with critical events. Klein, Snowden, One reason that it is difficult to specify a priori
and others have found that narratives provide a useful rules or standards for quality communications is 25 vehicle for both mining expertise and for training.
that communication and language both appear to We hypothesize that the reason that analogs, met-
be dynamic, self-organizing processes. For example, aphors, and narratives facilitate communications Rączaszek-Leonardi writes:
is that they translate constraints of the problem “Language is thus claimed to be embodied, sit-
into terms that are more familiar with the agents, uated and distributed, and thus linguistic struc-
making the signals—the problem constraints— tures, instead of being independent ‘vessels of
more salient relative to the background noise. meaning’, are seen as immersed in a variety of
Another important consideration in the design dynamical events that give rise to them and are,
of organizations to facilitate communications is in turn, controlled by them … communication,
the formalization of lines of communication and instead of being meaning transfer between in-
authority. For example, a major source of discus- dividuals is rather seen as creating something-
sion in both economic and military systems is in-common, or communion, that is, formation of
the relative benefits and costs of centralization, temporal, adaptive wholes, in which individuals
or hierarchy, relative to more distributed organi-
22 coordinate in functional synergies.” 26 zations, or networks. The key again seems to
be discovering the appropriate balance between With respect to the role of effective communications
internal constraints and the demands of particular and collaboration, there is a kind of chicken and
problem domains. For example, we can image egg problem that is always associated with circular
situations where conversations should be inclu- dynamics. Each is both simultaneously being shaped
sive and collaborative, taking full advantage of by and shaping the other. Effective communication
the diversity among a network of collaborating is simultaneously a pre-requisite for, and a product
agents, as did the puzzle-solving exercises used in of, effective coordination. Thus, as designers, it may 27 the MIT study on group intelligence. However,
not be possible to predetermine the quality of com- we can also image situations where the addition munications. However, there may be ways that de-
of more hierarchical constraints will reduce the signers can influence or constrain communications in
internal variety—potential noise—in ways that will potentially productive ways. One way that designers
facilitate achieving stability. For example, Sage and can do this is to align the internal constraints with
Cuppan describe a federalist style of organization the problem constraints through the construction of
that can be very effective in helping various agen- representations. This is the central focus of Ecolog-
cies—like police departments, fire departments, ical Interface Design—construct graphical interfaces
and hospitals—to coordinate activities in response that represent the deep structure of problems in a 28 to regional disasters/emergencies. Thus, our
way that is compatible with users’ capabilities—for guess is that various organizational forms—hier- example, by designing interfaces that represent the
archies, heterarchies, networks, federalism—will
From Autonomous to Sociotechnical Systems
8 Karl E. Weick, Sensemaking in Organizations (Thousand Oaks, CA:
to meeting the requisite variety demands for Sage, 1995). achieving stability in different contexts. There is no 9 Norbert Wiener, Cybernetics: Or Control and Communication in the
Animal and the Machine (Cambridge, MA: MIT Press, 1948).
single one-size-fits-all solution.
10 Terry Winograd and Fernando Flores, Understanding Computers and Cognition: A New Foundation for Design (Boston, MA: Addi - son-Wesley, 1986).
Conclusion
11 David D. Woods and Erik Hollnagel, Joint Cognitive Systems: Patterns
Again, we thank the editors of She Ji and the authors
in Cognitive Systems Engineering (Boca Raton, FL: 2006).
of the commentaries for enriching the discussion of
12 Herbert A. Simon and Allen Newell, “Heuristic Problem Solving: The Next Advance in Operations Research,” Operations Research
design thinking relative to improving the quality of
6, no. 1 (1958): 5.
collaborations. As a result of the feedback from the
13 Derek B. Miller, “Value-Pluralism and the Collaboration Impera-
commentaries, we would like to reframe the discus-
tive in Sociotechnical Systems,” commentary to Kyle Behymer
sion from “Designing for Effective Collaboration” to
and John Flach, “From Autonomous Systems to Sociotechnical Systems: Designing Effective Collaborations,” She Ji: The Journal of
“Enabling Effective Collaboration.” This is to empha-
Design, Economics, and Innovation 2, no. 2 (2016): 115.
size the self-organization dynamic of sociotechnical
14 Hugh Dubberly and Paul Pangaro, “Distinguishing Between
systems. Thus, we are skeptical that it is possible for
Control and Collaboration—and Communication and Conver-
designers to fully determine the quality of commu-
sation,” commentary to Kyle Behymer and John Flach, “From
nications or collaborations through Autonomous Systems to Sociotechnical Systems: Designing a priori design
Effective Collaborations,” She Ji: The Journal of Design, Economics
decisions. Rather we think that the quality of both
and Innovation 2, no. 2 (2016): 117.
will be an emergent property of the system dynamic.
15 For example, see John M. Flach and Fred A. Voorhorst, What
However, we think that focusing design thinking on
Matters?: Putting Common Sense to Work (Dayton: Wright State
the sociotechnical dynamic—communication and
University Library, 2016).
collaboration—rather than exclusively on the techno-
16 Lindblom, “Science of ‘Muddling Through’,” 88; Lindblom, “Still Muddling,” 517.
logical artifacts—autonomous systems—is critical to
17 Nordal Åkerman, ed., The Necessity of Friction: Nineteen Essays on a
enabling quality collaborations to emerge. As Dub-
Vital Force, reprint (Boulder: Westview, 1998).
berly and Pangaro note, this is not a new or novel in-
18 Simon,
sight. Clearly, this was a critical insight that led to the
19 Ashby W. Ross, An Introduction to Cybernetics (London: Chapman &
development of personal computers and later devel-
Hall, 1956).
opments such as smart phones. Yet, despite this his-
20 John M. Flach, “Supporting Self-Designing Organizations,” com-
tory, this is a lesson that we have to keep reminding
mentary to Donald A. Norman and Pieter Jan Stappers, “DesignX:
ourselves of in order not to become seduced by the
The Design of Complex Sociotechnical Systems,” She Ji: The Journal of Design, Economics and Innovation 2, no. 1 (2016): 95–99.
growing powers of information technologies and the
21 Dubberly and Pangaro, “Distinguishing Between Control and
illusion that with enough computational power it will
Collaboration,” 117.
be possible to completely tame the wickedness that is
22 Joanna Raczaszek-Leonardi, “Multiple Systems and Multiple Time
inherent in a complex world.
Scales of Language Dynamics: Coping with Complexity,” Cybernet- ics & Human Knowing 21, no. 1-2 (2014): 37–38.
1 Sidney Dekker, Drift into Failure: From Hunting Broken Components 23 Kevin B. Bennett and John M. Flach, Display and Interface Design: to Understanding Complex System (Farnham, UK: Ashgate, 2011).
Subtle Science, Exact Art (Boca Raton: CRC Press, 2011). 2 Charles E. Lindblom, “The Science of ‘Muddling Through’,” Public
24 Ben Shneiderman, Designing the User Interface (Reading, MA: Administration Review 19, no. 2 (1959): 79–88; Charles E. Lindblom,
Addison Wesley, 1992).
“Still Muddling, Not Yet Through,” Public Administration Review 39, 25 Gary A. Klein, “Using Knowledge Engineering to Preserve Corpo- no. 6 (1979): 517–26.
rate Memory,” in The Psychology of Expertise: Cognitive Research 3 Donald A. Norman and Stephen W. Draper, eds., User Centered
and Empirical AI, ed. Robert R. Hoffman (New York: Springer-Verlag, System Design: New Perspectives on Human-Computer Interaction
1992), 170–87; David J. Snowden, “Narrative Patterns, The Perils (Hillsdale: Lawrence Erlbaum Associates, 1986).
and Possibilities of Using Story in Organizations,” Knowledge 4 James G. March, Decisions and Organizations (Oxford: Basil Black-
Management 4, no. 10 (2001): 1–14.
well, 1989). 26 John Arquilla and David Ronfeldt, In Athena’s Camp: Preparing for 5 Jens Rasmussen, Information Processing and Human-Machine
Interaction: An Approach to Cognitive Engineering (New York: Elsevier 27 Anita Williams Woolley et al., “Evidence for a Collective Intelli- Science Ltd., 1986).
gence Factor in the Performance of Human Groups,” Science 330, 6 Herbert A. Simon,
no. 6004 (2010): 686–88.
MA: MIT Press, 1996). 28 Andrew P. Sage and Christopher D. Cuppan, “On the Systems Engi- 7 Kim J. Vicente, Cognitive Work Analysis: Toward Safe, Productive,
neering and Management of Systems of Systems and Federations and Healthy Computer-Based Work (Mahwah: Lawrence Erlbaum
of Systems,” Information, Knowledge Systems Management 2, no. 4 Associates, 1999).
she ji The Journal of Design, Economics, and Innovation Volume 2, Number 2, Summer 2016
Jordan Beck , School of Informatics and Computing, Indiana University, USA Erik Stolterman , School of Informatics and Computing, Indiana University, USA
Examining Practical, Everyday Theory Use in Design Research
Abstract This paper discusses how theories (as objects) are used in articles
Keywords
published in Design Studies. While theory and theory construction have
Design research
been given time and attention in the literature, less is known about how
Design theory Research methodology
researchers put theories to work in their written texts—about “practical, everyday” theory use. In the present paper, we examine 32 articles and syn- thesize six models of “theory use” based on our examination.
Received October 4, 2015 Accepted January 14, 2016
Emails
Jordan Beck (corresponding author) joebeck@indiana.edu
Erik Stolterman estolter@indiana.edu
Copyright © 2016, Tongji University and Tongji University Press. Publishing services by Elsevier B.V. This is an open access article under the CC BY-NC-ND license ( http://creativecommons.org/licenses/by-nc-nd/4.0/ ). The peer review process is the responsibility of Tongji University and Tongji University Press.
http://www.journals.elsevier.com/she-ji-the-journal-of-design-economics-and-innovation http://dx.doi.org/10.1016/j.sheji.2016.01.010
Examining Practical, Everyday Theory Use in Design Research
1 Ken Friedman, “Theory
Theory is integral to any academic discipline. It takes researchers beyond obser-
Construction in Design Re- search: Criteria: Approaches,
vation and interpretation into the realm of sharable knowledge. Theory provides
and Methods,” Design Studies
us with the means to structure knowledge, to evaluate and assess it, to construct
24, no. 6 (2003): 507–22, DOI:
it, and to share it. In the everyday practice of research, theory can be seen as a
http://doi.org/10.1016/S0142- 694X(03)00039-5 ; Terence
concrete tool used for practical purposes. For instance, theory can be used as an
Love, “Philosophy of Design: A
explanatory tool, or as a predictive one. But these are not the only ways in which
Meta-Theoretical Structure for
theory functions as a tool. The use of theory is multifaceted and complex. 1 How-
Design Theory,” Design Studies 21, no. 3 (2000): 293–313, DOI:
ever, in this article our aim is not to demarcate or define theory. Instead, our aim is
http://doi.org/10.1016/S0142-
to investigate how researchers present their use of theory in written texts.
694X(99)00012-5 ; Karl E. Weick,
The reason for our investigation originates in a parallel study 2 which engages
“Theory Construction as Disci- plined Imagination,” Academy
with the question of what constitutes a design theory. We have examined publica-
of Management Review 14, no. 4
tions from many different domains of design research, and we have found that the
(1989): 516–31, DOI: http://doi.
way theory is mentioned, written about, and used can be difficult to understand. This
org/10.5465/AMR.1989.4308376 ; Karl E. Weick, “What Theory Is
difficulty pointed us towards the important two-part question of (1) how researchers
Not, Theorizing Is,” Administra-
in the design field use theory, and (2) how that use manifests in their writings.
tive Science Quarterly 40, no.
Our interest is primarily in the everyday practice of theory use. By “everyday
3 (1995): 385, DOI: http://doi. org/10.2307/2393789 .
practice,” we mean to distinguish between what we might call “revolutionary contributions” to the field and contributions that might be considered normal or
2 Jordan Beck and Erik Stolter-
routine. In order to accomplish this, we have examined a selection of articles from
Theories of Design That Do Not
a single calendar year in one of the most prestigious journals in the design research
Scientize Design?” in Proceed-
community—Design Studies.
ings, from the 11th European Academy of Design Conference:
We propose six models that capture the different ways researchers use theory
The Value of Design Research
in their publications. We suggest that these models support a deeper understanding
(Boulogne: Paris Descartes Uni-
of the structure of publications in Design Studies. In addition, and perhaps more im-
versity, 2015), accessed August
portantly, these models facilitate interesting and useful questions about the state of
3, 2016, http://www.academia. edu/11241766/Can_there_be_sci-
theory use in design research in general, such as: why is theory used more in some
HQWLÀFBWKHRULHVBRIBGHVLJQBWKDWB
ways than others? How do authors employ theory in different ways in the same
do_not_scientize_design .
texts? To what extent does current theory use in design research tell us something
3 Friedman, “Theory Construc-
about design theory, as opposed to other kinds of theory? What is the current state of
tion,” 513.
theory use in design research?
4 John Ziman, Real Science:
We intend this paper to make two primary contributions to the field: First it
What It Is, and What It Means
describes the current state of theory use in design research; and second, it explores
(Cambridge, UK: Cambridge University Press, 2002), 128.
the implications of this existing state when it comes to design research in general.
A deeper understanding of how theory is used in the design research community
5 Simon Blackburn, The Oxford Dictionary of Philosophy, 2nd
can potentially better position its constituents to be more intentional in their
ed. (Oxford: Oxford University
theory use.
Press, 2005), 363. 6 John S. Gero, “Design
Prototypes: A Knowledge
Theory as an Object
Representation Schema for Design,” AI Magazine 11, no. 4
Theory can be understood in many different ways. For instance, it has been de-
(1990): 26; John S. Gero and Udo
scribed as a model, 3 likened to a map, 4 and defined as a way of looking at some
Kannengiesser, “The Situated
phenomena “with explanatory or predictive implications.” 5 A simple distinction
Function-Behaviour-Structure Framework,” Design Studies
that we have found useful for our purposes is that theory can be understood either
25, no. 4 (2004): 373–91,
as an object (i.e., as a kind of knowledge entity) or as a process (i.e., as theorizing).
DOI: http://doi.org/10.1016/j.
When a researcher develops an explanation of how or why some phenom-
destud.2003.10.010 ; John S. Gero and Udo Kannengiesser, “The
enon occurs, they are engaging in theorizing—in a process. The explanation itself
Function-Behaviour-Structure
becomes a theoretical object. For instance, the FBS framework, 6 CK theory, 7 dis-
Ontology of Design,” in An An-
tributed cognition, 8 and strong concepts, 9 could all be interpreted as theoretical
thology of Theories and Models of Design, ed. Amaresh Chakrabarti
objects. A parallel distinction can be made between designing and a design: when a
and Lucienne T. M. Blessing
designer designs, they are engaged in design as a process, while the outcome of that
(London: Springer-Verlag, 2004),
process becomes the design(ed) object.
263–83, http://link.springer.com/ chapter/10.1007/978-1-4471-
Both aspects are of course of great importance. So when we consider our re-
6338-1_13 .
search question, “How is theory used in written texts in design research,” we have
she ji The Journal of Design, Economics, and Innovation Volume 2, Number 2, Summer 2016 she ji The Journal of Design, Economics, and Innovation Volume 2, Number 2, Summer 2016
7 Armand Hatchuel and Benoît
studying one would be quite different than the other. For our purposes, we made Weil, “A New Approach of
Innovative Design: An Introduc-
the decision to frame theory as an object.
tion to CK Theory,” in DS 31:
Framing theories as objects can serve many different ends. For instance, it
Proceedings of ICED 03, The 14th
would be possible to study theories (as objects) in design to explore their constitu- International Conference on En-
gineering Design, Stockholm, ed.
tion and structure, or to study their internal consistency, or their scope or depth.
A Folkeson, K Gralen, M Norell,
It would also be possible to compare and contrast theories with the purpose of
and U Sellgren (2003), 109–10;
gaining insight into the field where they are used. These ways of investigating
Armand Hatchuel and Benoît Weil, “C-K Design Theory:
theories can lead to detailed analyses of their intrinsic status. However, such ap-
An Advanced Formulation,”
proaches would require the theories under investigation to be opened up, and their
Research in Engineering Design
inner workings exposed. We have taken a simpler approach: We are only interested
19, no. 4 (2009): 181–92, DOI: http://doi.org/10.1007/s00163-
in how theories are used as knowledge objects, and we are not interested (in this
008-0043-4 .
context) in the internal constitution of the object itself.
8 James Hollan, Edwin Hutchins,
Distinguishing a theoretical knowledge object in a publication is an important
and David Kirsh, “Distributed
issue to address. For our purposes, we err on the side of inclusivity, since previous
Cognition: Toward a New
work has shown that theories can be referred to in research publications as every-
Foundation for Human-Comput- er Interaction Research,” ACM
thing—from “theory” or “model” to “perspective,” “rationale,” “idea,” “process,”
Transactions on Computer-Hu-
and “phenomenon.” 10 We are interested in identifying what could reasonably be
man Interaction (TOCHI) 7, no.
construed as theoretical knowledge objects based on our own understanding of
2 (2000): 174–96, DOI: http://doi. org/10.1145/353485.353487 .
theory and our understanding of how other researchers in the design field under-
stand and talk about theory. 11 As a consequence, the way we frame theory could
9 Kristina Höök and Jonas Löwgren, “Strong Concepts:
be seen in alignment with a broad, inclusive definition of theory “as [an] abstracted
Intermediate-Level Knowledge
knowledge [object] that tells us something about fundamental entities at the core
in Interaction Design Research,”
of a discipline.” 12
ACM Transactions on Comput- er-Human Interaction (TOCHI)
The way we frame theories suggests that the use of a theory can at least be un-
19, no. 3 (2012): 23; Jonas
derstood in three ways: as a tool, as a reference, and as a knowledge contribution.
Löwgren, “Annotated Portfolios
In the following section, we will briefly discuss each of these aspects of theory as an
and Other Forms of Intermedi- ate-Level Knowledge,” interac-
object. There are of course other ways to understand theories as objects, but these
tions 20, no. 1 (2013): 30–34.
are the ones we have chosen to focus on in our analysis.
10 Jo E. Hannay, Dag I. K. Sjøberg, and Tore Dybå, “A
Theory as a Tool
Systematic Review of Theory
Framing theory as a tool implies that a user uses a theory for a particular purpose.
Use in Software Engineering
13 Experiments,” Theory has been described and defined by many researchers in terms of its utility. IEEE Transactions
on Software Engineering 33, no.
It has been framed as a tool for explaining, describing, or predicting phenomena.
2 (2007): 87–107, DOI: http://doi.
It has been described in design research as a tool for “binding together” our knowl-
org/10.1109/TSE.2007.12 .
edge of design practice and as a tool for “providing an understanding” of design
11 Steven R. Haynes and John
writ large. 14 Tools have functions that are in most cases related to a particular need
M. Carroll, “The Range and
or purpose. For instance, when a designer needs to communicate an idea quickly Role of Theory in Information
Systems Design Research: From
and cheaply to a stakeholder, she uses a sketch. Similarly, when researchers need
Concepts to Construction,” in
to explain how or why some phenomenon occurs, they use theory—since theory
2010 International Conference
provides a means of structuring knowledge in precise ways to share with other on Information Systems (ICIS
2010) Proceedings, Paper 136,
researchers. 15 There are of course situations when particular tools are not well
accessed August 3, 2016, http://
suited for the task at hand. Sketching is not necessarily an ideal tool for “selling”
aisel.aisnet.org/cgi/viewcontent.
a design to a client. Similarly, theory is not necessarily an ideal tool for explaining cgi?article=1139&context=i-
cis2010_submissions .
reality in the same way that an analogy or metaphor might be. Finally, the tool metaphor lends itself to the idea of appropriation. Tools that were designed for one 12 Erik Stolterman and Mikeal
Wiberg, “Concept-Driven
purpose often end up serving others. Pencil and paper, as tools for sketching, have
Interaction Design Research,”
structural properties conducive to low-fidelity 3D prototyping. Theory, a tool for
Human–Computer Interaction 25,
explaining, predicting, or describing, has structural properties conducive to other no. 2 (2010): 95–118. purposes as well.
13 Yvonne Rogers, “HCI Theory: Classical, Modern,
and Contemporary,” Theory as a Reference Synthesis
Lectures on Human-Centered
Common definitions of reference include: the action of mentioning or alluding
Informatics 5, no. 2 (2012): 1–129,
Examining Practical, Everyday Theory Use in Design Research
DOI: http://doi.org/10.2200/
to something or the use of information to ascertain something. When theory is
S00418ED1V01Y201205HCI014 ; Ben Shneiderman, Catherine
used as a reference, it most often appears in the introductory and/or background
Plaisant, Maxine S. Cohen, and
sections of a given written text. With this type of theory use, researchers often
Steven M. Jacobs, Designing
reference frameworks and models instead of referencing theory per se. Other inves-
the User Interface: Strategies for Effective Human-Computer
tigations of theory use in written texts have found that authors refer to theories as
Interaction, 5th ed. (Boston:
everything, from “theory,” “model,” “perspective,” “rationale,” “idea,” “process,”
Prentice Hall, 2009).
to “phenomenon.” 16 In many cases, the reference remains vague. For instance, the
14 Amaresh Chakrabarti
reference might amount to little more than authors’ names, or a paper title and
and Lucienne T.M. Blessing,
the publication year. The names and the publication year “stand in” for an explicit
“Theories and Models of Design: A Summary of Findings,” in
citation of a particular theory or theories. Theory as a reference can perform sev-
An Anthology of Theories and
eral functions that may appear in concert or individually in a given research paper.
Models of Design, ed. Amaresh
For instance, a theory can be used to establish the basis for a research project, to
Chakrabarti and Lucienne T.M. Blessing (London: Springer-Ver-
situate a text within a lineage, to establish the knowledge base of the writer, or to
lag, 2014), 15, accessed August
show connection to a community or school of thought. We will discuss some of
3, 2016, http://link.springer.
these in greater detail when we present our models of theory use.
com/chapter/10.1007/978- 1-4471-6338-1_1 ; Friedman, “Theory Construction,” 511;
Theory as a Knowledge Contribution
“Understanding Designing:
When theory is used as a tool or as a reference, it usually means that the theory in
What Are Designers Doing When They Design (John Gero),”
question already exists. Characterizing theory as a knowledge contribution suggests
YouTube video, 1:01:30, from the
that it had not existed prior to the researcher’s articulation of it. The researchers
ASP Course “Design Methods”
brought it into being by formulating their results such that they would be under-
coordinated by Gaetano Cascini,
stood as a theory, for instance, as a set of constructs and their definitions, as well as a set of propositions about how the constructs relate to one another. by “Alta Scuola Politecnica,” 17 This
October 8, 2013, https://www.
formulation of a theory becomes a knowledge object. How this object is to be
youtube.com/watch?v=TRiET-
“used” by others is partly a consequence of the authors’ claims about it, and partly
j2ujvQ .
a consequence of the readers’ interpretation. For example, the author might claim
15 Hannay, Sjøberg, and Dybå,
that the knowledge contribution relates to existing theory, or the research problem
“A Systematic Review of Theory Use”; Ziman, Real Science.
at hand, or to something else entirely. But it would also be possible for a reader to see applications beyond the authors’ claims. For example, the mathematicians who
16 Hannay, Sjøberg, and Dybå, “A Systematic Review of Theory
developed tensor calculus did so “without any premonition of its later use in phys-
Use,” 87–107.
ics,” 18 in particular in the General Theory of Relativity. In our analysis, we have
17 Karl E. Weick, “Theory
primarily focused on what the resulting theory is claimed to contribute and not on
Construction as Disciplined
reader interpretation.
Imagination.”
With the notions of theory as a tool, a reference, and a knowledge contribution in
18 Ziman, Real Science, 139.
mind, we turn our attention to describing our approach, including the conceptual-
19 George Psathas, Conver-
ization, design, and implementation of the current research project.
sation Analysis: The Study of Talk-in-Interaction, Qualitative Research Methods Series, vol.
Approach
35 (Thousand Oaks, CA: Sage
Publications, 1995); Harvey
Prior to assembling a corpus of texts, we established that we would engage in a
round of “unmotivated looking” in 19 Structures of Social Action: in order to see how the texts could be said to use
Sacks, “Notes on Methodology,”
Studies in Conversation Analysis,
theory without establishing or imposing strict definitions of “theory” or of “use.”
ed. J. Maxwell Atkinson and
One of the challenges we faced at the outset of our work was that of defining
John Heritage (New York: Press
“theory use.” How might we capture what it means to “use” theory? We found that
Syndicate of the University of Cambridge, 1984), 21–27.
framing theory as an object pointed us in the direction of an answer. When theory
[Editor’s note] This reference
is seen as an object—or a thing—then it becomes possible to see a theory as a tool.
note is added in the light of
When theory is seen as a tool then we can define “use” in terms of how it functions
Danah Henriksen’s comments.
in a text. What purpose does theory fulfill in a given written text?
20 Barney G. Glaser and Anselm
These initial questions and decisions were not guided by existing “theoretical
L. Strauss, The Discovery of frameworks,” but rather by a “general perspective… [and a] general problem area.” Grounded Theory: Strategies for 20 Qualitative Research (Piscataway,
These initial decisions could therefore be seen to be in alignment with the initial
NJ: AldineTransaction, 2009), 45.
decisions of a grounded theory approach.
There are many different publication venues one might look at in order to
she ji The Journal of Design, Economics, and Innovation Volume 2, Number 2, Summer 2016 she ji The Journal of Design, Economics, and Innovation Volume 2, Number 2, Summer 2016
21 Friedman, “Theory Con-
year’s worth of articles (n=32) published in Design Studies, which is one of the top struction,” 507–22; Terence Love,
“Philosophy of Design,” 293–313;
journals in the field. It is one of the oldest design research journals, and it aims to
Terence Love, “Constructing
provide an interdisciplinary forum for inquiry into design activities. It has been
a Coherent Cross-Disciplinary
a key source of material for scholars seeking answers to questions about how to Body of Theory about Designing
21 define, construct, and evaluate theory with a particular focus on design theory. and Designs: Some Philosoph- We ical Issues,” Design Studies
do not position our analysis as a response to or critique of this existing content.
23, no. 3 (2002): 345–61, DOI:
Rather, we see our work as complementary: a contribution to our collective knowl-
http://doi.org/10.1016/S0142- 694X(01)00043-6 .
edge of theory in design research. In order to determine which function theory performs in the works included
22 Wayne C. Booth, Gregory G. Colomb, and Joseph M. Williams,
in our corpus, we adopted an emergent coding approach. We separately reviewed
The Craft of Research, 2nd ed.
the corpus, and recorded preliminary observations about how theory functioned in
(Chicago: University of Chicago
each text. Then we reconvened to analyze and compare our observations.
Press, 2003); Kate L. Turabian, A Manual for Writers of Research
This stage yielded four groupings, which we named originating, positioning,
Papers, Theses, and Dissertations:
shaping, and shaped. Originating papers were those whose research questions orig-
Chicago Style for Students and
inated from theory, or whose question was about a specific theory or theories. Po-
Researchers, 8th ed. (Chicago: University of Chicago Press,
sitioning papers were those that positioned their work in relation to existing work
that may or may not have included theory per se. Shaping papers were those that used theory in a manner similar to positioning papers, with the difference being that theory actually shaped—or re-shaped—the original research question. Finally, shaped papers were papers that used theory as a filter through with to pass prelimi- nary research findings. We saw findings as shaped by theory.
We adopted these four groupings as a coding scheme, and independently applied them to the texts in the corpus. However, we found that this preliminary coding scheme did not yield an acceptable level of reliability. So we continued discussing the similarities and differences in our application, and we iterated upon the coding scheme accordingly.
In parallel with this iteration, we developed models to represent the relation- ships and interactions between theory and the other core elements in our selected papers. In attempting to establish what could constitute the core elements of our selected papers, we made the decision to model as few elements as possible: fewer elements reduce the complexity of the models but also lead to coarser descriptions. And while we relied on grounded theory techniques to generate kinds of theory use, we developed our list of core elements of a research publication based on two
widely used reference texts on research and academic writing. 22 We ended up with
the following core elements: question, examination, findings, and theory. Questions identify the needs or interests that the researcher deems worthy of understanding, explaining, predicting, or describing. For example, our framing question identifies an interest in understanding how theory is used in design research. Examination captures the approach taken to answer the question—that is, it includes all forms of analytical or empirical work done by the researcher to investigate the question at hand. Findings refer to the outcome of this examination. Theory is the fourth core element in each of our models.
It is obvious that by using so few elements to capture complex objects such as research papers, there is a concomitant loss of precision. However, it is not our intent to precisely capture on a detailed level the intrinsic structure of each article. Instead, our purpose is to find a level of analysis that makes it possible to extract patterns, similarities and differences across articles in a manageable way. We be- lieve that our level of analysis has made this possible.
After we developed the original set of four models, we critiqued and iterated on them by independently applying them to the papers in our corpus. Our cri- tique focused primarily on the match between the models and the papers. Did the models accurately capture the essential structure of the core elements in each
Examining Practical, Everyday Theory Use in Design Research
a methodological and/or analytical tool. We developed additional models to account for these types of theory use. In the interest of completeness, we also developed a model to account for papers that do not show any use of theory. However, this type of paper did not appear in our corpus, which is itself interesting.
Our critique and iteration finally led to six models:
0) No theory,
1) Theory as the object of study,
2) Theory as a contextualizing tool,
3) Theory as a shaping tool,
4) Theory as a methodological tool, and
5) Theory as an analytical tool. In the following section, we will explain each of these models and point to relevant
examples from our corpus. For each model, we include textual excerpts from our corpus in order to illustrate the type of content that pointed towards a particular model. We want to enable readers to evaluate the texts on similar terms as we did.