Data collection

Data collection

Data collection in qualitative research involves a variety of techniques: in-depth interviewing, document analysis, and unstructured observations. Though these techniques are often referred to by a single term—participant observation—this is in fact misleading. Furthermore, in a number of cases it is incorrect to associate qualitative social science with participant observation. Many qualitative studies, for example the early Chicago studies, rely on a single data-collection method, either document analysis or interviewing.

Researchers have frequently asserted that a particular method was superior to all others. Thomas and Znaniecki made such a claim regarding life histories: “We are safe in saying that personal life- records, as complete as possible, constitute the perfect type of sociological material” (quoted in Madge, 1962:61; emphasis in original). Similar assertions were made in a debate between Becker and Geer (1957; 1958), on the one hand, and Trow (1957) on the other (exchange reprinted in Filstead, 1970), the former arguing for the virtues of participant observation and the latter for interviewing.

60 A handbook of qualitative methodologies

Such declarations, we suggest, are presumptuous; Bulmer (1984: xv) has rightly pointed to the unproductiveness of engaging in a “best method” debate in an absolute sense. In fact, in Becker and Geer’s rejoinder to Trow they agree with him on the point that the problem under investigation dictates the method to use. This point is illustrated further in a typology constructed by Zelditch (1970) and refined by Denzin (1970a:30–1), in which techniques of data collection are correlated with types of information (Table 2.1).

Table 2.1 Data collection and information types: methods of obtaining information

The typology suggests that participant observation is best suited for case studies and life histories (“incidents and histories” in Table 2.1), and least suited for overviews of entire populations (“frequency distribution”). For the study of organizations (“institutionalized norms and statuses”) participant observation is deemed an adequate, but inefficient data-collection method. Interviewing, according to Zelditch, seems a viable data-collection device for all three types of studies, but less appropriate for surveys of large groups or populations (“frequency distributions”) than for the study of cases (“incidents”) and organizations (“institutionalized norms and statuses”). What Zelditch calls enumerations refers to survey research methodology and is considered most appropriate for the study of the distribution of characteristics in a population. There are, of course, limitations to such a typology, and they become clear when one attempts to place specific qualitative communication studies in the cells of the table. Participant observation, for example, was the primary data-collection method employed by Gans (1979) in his study of news organizations; his results were more than “adequate.” Interviewing,

Social science inquiry 61

on the other hand, has been employed in a wide variety of studies of media organizations and institutional procedures, but, we suggest, it is far from always “efficient” or the “best form.” As Deutscher (1973) demonstrated, there is often a discrepancy between reports of attitudes gained through interviews and observations of the behavior related to those attitudes. Once again, the “how” of research (methodology) should

be deliberated carefully in each particular case with reference to “what” and “why” (the subject matter and purpose of inquiry). In order to examine the explanatory value of specific methods, we next consider more closely participant observation, which is frequently identified as the ideal method for qualitative research. Second, we discuss the relevance of employing multiple methods, what is commonly known as “triangulation.”

An often quoted definition of participant-observation has been offered by Becker and Geer (1957:28):

By participation observation we mean that method in which the observer participates in the daily life of the people under study, either openly in the role of researcher or covertly in some disguised role, observing things that happen, listening to what is said, and questioning people, over some length of time.

The primary purpose of participant-observation research, accordingly, is to describe in fundamental terms various events, situations, and actions that occur in a particular social setting. This is done through the development of case studies of social phenomena, normally employing a combination of data-collection techniques. Other definitions further stress the multiple methods of participant observation:

it is probably misleading to regard participant observation as a single method…it refers to a characteristic blend or combination of methods and techniques that is employed in studying certain types of subject matter: primitive societies, deviant subcultures, complex organizations…social movements and informal groups…[it] involves some amount of genuinely social interaction in the field with the subject of study, some direct observation of relevant events, some formal and a great deal of informal interviewing, some systematic counting, some collection of documents and artifacts, and openendness in the direction the study takes.

(McCall and Simmons, 1969:1)

62 A handbook of qualitative methodologies

One problem with this latter definition is that it includes nearly every form of data-collection and interpretation under the heading of participant observation. Hence it becomes difficult to discriminate and, most important, to compare and assess the findings that different data-collection methods generate. Participant observation is best suited, in comparison with survey or experimental designs, for interpretive inquiry into social interaction from the perspective of the people involved.

In previous research, the multiple method approach is best known under the term triangulation, and has been advocated most vocally by Webb and colleagues and, later, by Denzin:

If no single measurement class is perfect, neither is any scientifically useless…for the most fertile research for validity comes from a combined series of different measures, each with its idiosyncratic weakness, each pointed to a single hypothesis.

(Webb et al., 1966:174)

Triangulation, or the use of multiple methods, is a plan of action that will raise sociologists above the personalistic biases that stem from single methodologies.

(Denzin, 1970b:27)

One of the assumptions of a multiple method strategy is that such an approach provides for more valid results than a single research strategy. Or, as Jick (1979:604) puts it, the basic assumption of all triangulation is “that the weaknesses in each single method will be compensated by the counter-balancing strengths of another.”

Various forms of triangulation have been proposed (for example, Brewer and Hunter, 1989). One of the most elaborate developments of the technique includes four types: triangulation of the data, the investigator, the theory, and the method (Denzin, 1970b). Data triangulation refers to the dimensions of time, space, and analytical level in which information is obtained. Investigator triangulation involves the more standard approach of using several analysts or coders, often as part of a multidisciplinary team of scientists. Theoretical triangulation suggests application of concepts and perspectives from diverse theories and disciplines. Finally, methodological triangulation constitutes a research strategy in

Social science inquiry 63

which different methods are employed for data gathering and analysis around a single object of study.

Some calls for triangulation may be rooted in a scientifically naive notion that multiple methods can reveal a single, “true” reality beyond frameworks of theory and interpretation. Phillips (1973:91), for one, has raised the question of whether triangulation actually increases rather than reduces biases inherent in particular data-collection methods. In spite of such reservations, triangulation may become a constructive force in the development of methodology as well as theory. For one thing, it can stimulate inventive uses of familiar research methods, and thus may help to uncover unexpected dimensions of the area of inquiry. For another, given appropriate theoretical and meta- theoretical reflection on the status of each set of data and findings, it may at times allow for more confidence in the conclusions of qualitative studies. Perhaps most important, triangulation can assist in constructing

a more encompassing perspective on specific analyses, what anthropologists call “holistic work” or “thick description” (Jick, 1979:608–9; also Geertz, 1973).

What should be noted, finally, is that triangulation does not absolve qualitative researchers of interpretive work. Indeed, when findings derived from different methods conflict or fail to corroborate each other (as well as when they support each other), this signals not the end of the study, but the beginning of a phase of theoretical analysis examining the nature of agreements and disagreements. To repeat, further empirical but also theoretical work is needed to specify the explanatory value of different methods of data collection. This may become a priority of qualitative research, along with the development of systematic analytical procedures.

Participant observation, often including triangulation, has been applied also to processes of mass communication. In particular, studies of media organizations have been a proving ground for this data- collection method. Exemplary works include Gans’ (1979) study, Deciding What’s News and Tuchman’s (1978) Making News. At the audience end of the process, Lull (1980), as already noted, has conducted pioneering work on television audiences based on participant observation, which has inspired a new generation of audience researchers committed to ethnographies of media use in the natural setting of the home (see Lull, 1988b). However, it is striking that in these media studies, as in much research from other disciplines, there is little or no indication of how the collected data were analysed.

64 A handbook of qualitative methodologies