Directory UMM :Data Elmu:jurnal:I:Information and Management:Vol37.Issue6.2000:
Information & Management 38 (2000) 59±72
Citation classics from the information systems literature
Kent A. Walstroma,*, Lori N.K. Leonardb,1
a
Applied Computer Science Department, Campus Box 5150, Illinois State University, Normal, IL 61790-5150, USA,
b
School of Accounting, University of Tulsa, 600 S. College Ave., Tulsa, OK 74104-3189, USA
Received 14 September 1999; accepted 3 January 2000
Abstract
An analysis of 118,364 references from 3752 articles published in top IS journals during the 10 years from 1986 to 1995 is
presented. Ninety-one `citation classics' were identi®ed by combining the results of this analysis with previous ®ndings.
Within this group of 91 publications, 13 `super classic' publications were identi®ed. These classics and super classics can be
categorized along any of several dimensions to gain a better understanding of the IS ®eld. These ®ndings can be used, among
other things, for developing reading lists and identifying the characteristics that make a publication `classic'. # 2000 Elsevier
Science B.V. All rights reserved.
Keywords: Diffusion of IS research; IS literature; Citation analysis
ISRL: IB0301; IB0301.01
1. Introduction
Whether we are attempting to understand the broad
foundations of a research discipline or the speci®c
implications of a narrow stream of research within a
discipline, we need to review what has already been
discovered so we can explore that which is yet
unknown. As the information systems (IS) ®eld grows
and matures there is an increase in the amount of new
literature published each year. Scanning the total
volume thus becomes a daunting task and doctoral
students interested in the cross-disciplinary study of
IS, ask `can you suggest reading material for me in the
IS ®eld?' Moreover, practitioners may want to read the
*
Corresponding author. Tel.: 1-309-438-8338;
fax: 1-309-438-5113.
E-mail address: [email protected] (K.A. Walstrom).
1
Tel.: 1-918-631-2787.
`best' that the academic ®eld has to offer. The purpose
of this study was to identify the most cited IS publications from the past 10 years and use them to
identify the `classic' publications in the ®eld.
2. Literature review
The `must read' publications list of 10 experts in the
IS ®eld would certainly not be the same, either in
content or order. The differences in these lists may be a
result of several factors, including: the large number of
articles (several thousand to choose from), the diversity of backgrounds of the experts, and the research
and teaching interests currently being pursued by
them. Examining several Ph.D. seminar reading lists,
revealed marked differences in the material required
of IS Ph.D. students in an introductory course. Selecting just one of these might result in important works
0378-7206/00/$ ± see front matter # 2000 Elsevier Science B.V. All rights reserved.
PII: S 0 3 7 8 - 7 2 0 6 ( 0 0 ) 0 0 0 5 4 - 9
60
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
being missed, though they are considered important by
a greater number of experts. The continual demand for
IS professionals and knowledge workers in the marketplace makes the demand for IS faculty members
either constant or increasing. Government and university pressure provides a demand for IS literate
researchers, since IS technology is applicable to all
®elds of inquiry.
To gain an understanding of any large body of
knowledge requires considerable time and effort.
Reading the best the ®eld has to offer maximizes
the use of time resources. People now realize that it
is no longer possible to read all the available literature
and ®ltering strategies have, therefore, emerged. Over
the years, literacy experts have published their ®ltering strategies in the form of `lists of great books' in an
attempt to help mankind identify which works merit
investigation [1,2,19,20,24]. The thoughts of these
experts are summed in the following quote:
``No single person has a right to say which are
the hundred or thousand best books. . . . But
numerous lists of great books have been drawn
up by various persons at one time or other, and
they provide what is on the whole the best guide
through the bewildering shelves of any large
library.''
Van Doren in [25].
A similar filtering strategy can be used in understanding academic literature. In the academic world, one of
these lists is provided at the end of every refereed
journal article or conference proceedings produced.
This reference list can be used to identify sources on
the topic of interest in the article. Collected together
over time and over a variety of publication outlets,
these lists provide a composite list of the best sources
available in the field of IS. The Proceedings of the
First ICIS meeting in 1980 reveals an attempt to
identify the literature which had to be included in a
doctoral level program of study [10].
2.1. The early years
Dickson [17] identi®ed ®ve early IS books that
included readings: Sanders [29]; Coleman and Riley
[6]; Sanders [28] Davis and Everest [11]; and Dock
et al. [18]. A review of these books provides a parsimonious look at articles that were then considered by
these authors to be important. Table 1 shows the
articles that appeared in more than one of these books.
One article, Ackoff 1967, appeared in all the ®ve
books. Two articles, Dearden 1972 and Kriebel
1972, appeared in three of the ®ve books. Another
20 articles appeared in two of the books.
2.1.1. Citations
Classics can be identi®ed subjectively by asking
individuals, but the views of several experts yield a
more representative selection of publications. According to de Solla Price [16], a `classic' is any publication
cited four or more times in a year. Citation analysis has
also been identi®ed as another means of determining
`classic' publications [5,21,27]. Citation analysis
allows for the collection of perceptions of hundreds,
or even thousands, of `experts', simultaneously and
over time. `Citation classics' are extraordinary,
because of the large number of citations they receive.
While, 30 years ago considerably fewer articles were
written and printed than today, this number still provides a good starting place. Especially since we are not
interested in all available literature, only that small
portion regarded as IS literature.
3. The middle years: 1972±1985
Researchers produced citation ®ndings about the IS
literature for the period from 1972 to 1985. Culnan, in
two separate studies [7,8], reported her ®ndings of a
co-citation analysis of the IS discipline. The appendix
from these studies is helpful in identifying works that
met the `classic' criteria in the IS discipline from 1972
until mid-1985. Table 2 summarizes the ®ndings from
Culnan's ®rst co-citation analysis from 1972 to mid1982. The number of citations per year was determined by dividing the total number of citations
identi®ed from the study by 10.5. During this ten
and a half year period, seven publications were found
to meet the criteria of four or more citations a year:
1.
2.
3.
4.
5.
6.
7.
Churchman, 1971.
Mason and Mitroff, 1973.
Churchman, 1968.
Mason, 1969.
Ackoff, 1967.
Lucas, 1975.
Churchman and Schainblatt, 1965.
Table 1
Articles appearing in more than one book of readings (listed by frequency)
Sanders,
1970
[29]
Coleman and
Riley, 1973
[6]
Sanders,
1974
[28]
Davis and
Everest,
1976 [11]
Dock,etal.,
1977 [18]
Ackoff, R.L., 1967, `Management Misinfornation Systems,' Management Science, December,
pp. B-147-B-156.
Dearden, 1972, `MIS is a Mirage,' Harvard Business Review, January±February, pp. 90±99.
Kriebel, 1972, `The Future of MIS,' Business Automation, June, pp. 18±19.
Adams, W., 1972, `New Role for Top Management in Computer Applications,' Financial Executive,
April, pp. 54±56.
Allen, B.R., 1972, `Computer Security: Part I.,' Data Management, January, pp. 18±24.
Allen, B.R., 1972, `Computer Security: Part II.,' Data Management, February, pp. 25±30.
Andrus, 1971, `Approaches to Information Evaluation,' MSU Business Topics, Summer, pp. 40±45.
Argyris, C., 1971, `MIS: The Challenge to Rationality,' Management Science, February.
Beishon, R.J., 1970, `Information Flow and Managers' Decisions,' Managerial Accounting, June.
Deckard, N.S., 1970, `Capturing the Development Costs of an Integrated MIS?' Journal of Systems
Management, January.
Dickson, G.W., 1968, `Management Information±Decision Systems,' Business Horizons, December.
Dickson, G.W. and Simmons, J.K., 1970, `The Behavioral Side of MIS: Some Aspects,'
Business Horizons, August.
Ernst, M., 1971, `Management, The Computer, and Society,' Computers and Automation, September,
pp. 8±11.
Fredricks, W.A., 1971, `A Manager's Perspective of Management Information Systems,'
MSU Business Topics, Spring.
Gibson, C.F. and Nolan, R.L., 1974, `Managing the Four Stages of EDP Growth,' Harvard
Business Review, January±February.
Holmes, R.W., 1970, `Twelve Areas to Investigate for Better MIS,' Financial Executive, July.
Hoos, I.R., 1971, `Information Systems and Public Planning,' Management Science, June,
pp. B658±B671.
Kanter, J., 1972, `Impact of Computers on the Business Organization,' Data Management,
April, pp. 20±23.
Nanus, B., 1972, `Managing the Fifth Information Revolution,' Business Horizons, April.
Nichols, G.E., `On The Nature of Management Information,' Management Accounting, April 1969,
pp. 9±13.
Nicholson, C.H., 1970, `MIS In Perspective,' Chemical Engineering Progress, January.
Tolliver, E.M., 1970, `Myths of Automated Management Systems,'
Journal of Systems Management, August.
Zani, 1970, `Blueprint for MIS,' Harvard Business Review, November±December.
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Article
X
X
X
61
62
Publication
# Citations
Citations
per year
Churchman, C.W., 1971, The Design of Inquiring Systems, Basic Books: New York
Mason, R.O. and Mitroff, I.I., 1973, `A Program for Research on Management Information Systems,' Management Science, January (19:5),
pp. 475±487.
Churchman, C.W., 1968, The Systems Approach, Delacorte Press: New York
Mason, R.O., 1969, `A Dialectical Approach to Strategic Planning,' Management Science, April (15:8), pp. B403±B414.
Ackoff, R.L., 1967, `Management Misinformation Systems,' Management Science, (14:4), pp. 147±156.
Lucas, H.C., 1975, Why Information Systems Fail, Columbia University Press: New York.
Churchman, C.W. and Schainblatt, A.H., 1965, `The Researcher and the Manager: A Dialectic of Implementation,' Management Science, (11:4),
pp. B69±B87.
Anthony, R.N., 1965, Planning and Control Systems: A Framework for Analysis, Harvard University, Graduate School of Business Administration,
Division of Research: Boston, MA.
Davis, G.B., 1974, Management Information Systems: Conceptual Foundations, Structure and Development, McGraw-Hill: New York.
Gorry, G.A. and Scott-Morton, M., 1971, `A Framework for Management Information Systems,' Sloan Management Review, (13:1), pp. 55±57.
Swanson, E.B., 1974, `Management Information Systems: Appreciation and Involvement,' Management Science, October (2:2), pp. 178±183.
Ackoff, R.L., 1970, A Concept of Corporate Planning, Wiley±Interscience: New York
Ackoff, R.L., 1974, Redesigning the Future: A Systems Approach to Social Problems, Wiley: New York.
Ackoff, R.L., 1962, Scientific Method: Optimising Applied Research Decisions, Wiley: New York.
King, W.R. and Cleland, D.I., 1975, `The Design of Management Information Systems: An Information Analysis Approach,' Management Science,
November (22:3), pp. 286±297.
Mitroff, I.I., 1971, `A Communication Model of Dialectical Inquiring Systems: A Strategy for Strategic Planning,' Management Science,
June (17:10), pp. B634±B648.
103
63
9.8
6
63
53
52
50
47
6
5
5
4.8
4.5
41
3.9
35
34
34
34
34
32
25
3.3
3.2
3.2
3.2
3.2
3
2.4
24
2.3
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Table 2
Culnan's co-citation analysis findings for 1972 to mid-1982 (listed by frequency)
24
2.3
22
21
20
17
16
2.1
2
1.9
1.6
1.5
15
13
1.4
1.2
13
13
1.2
1.2
12
1.1
12
11
11
10
10
1.1
1
1
1
1
10
10
1
1
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Mitroff, I.I., 1979, The Subjective Side of Science: A Philosophical Inquiry into the Psychology of Apollo Moon Scientists, Elsevier Scientific
Publishing Company: New York.
Dearden, J., 1972, `MIS is a Mirage,' Harvard Business Review, January±February, pp. 90±99.
McKenny, J.L. and Keen, P.G.W., 1974, `How Manager's Minds Work,' Harvard Business Review, May±June (52:3), pp. 79±90.
Nolan, R.W., 1973, `Managing the Computer Resource: A Stage Hypothesis,' Communications of the ACM, (16:7), pp. 399±405.
Hiltz, S.R. and Turoff, M., 1978The Network Nation: Human Communication via Computer.
Keen, P.G.W. and Scott-Morton, M.S., 1978, Decision Support Systems: An Organizational Perspective, Addison-Wesley Publishing Company:
Reading, MA.
Ackoff, R.L., 1960, `Unsuccessful Case Studies and Why,' Operations Research, March±April (8:2), pp. 259±262.
Chervany, N.L. and Dickson, G.W., 1974, `An Experimental Evaluation of Information Overload in a Production Environment,' Management
Science, June (20:10), pp. 1335±1344.
Nolan, R.W., 1973, `Computer Data Bases: The Future is Now,' Harvard Business Review, September±October (51:5), pp. 98±114.
Mitroff, I.I. and Featheringham, T.R., 1974, `On Systematic Problem Solving and the Error of the Third Kind,' Behavioral Science, (19),
pp. 383±393.
Mitroff, I.I., Betz, F., Pondy, L.R. and Sagasti, F., 1974, `On Managing Science in the Systems Age: Two Schemas for the Study of Science as a
Whole Systems Phenomenon,' Interfaces, (4:3), pp. 46±58.
Bennett, J.L., 1972, `The User Interface in Interactive Systems,' Annual Review of Information Science and Technology, (7), pp. 159±196.
Whisler, T.L., 1970, The Impact of the Computer on Organizations, Praeger: New York.
Danzinger, J.N., 1977, `Computers, Local Governments, and the Litany of EDP,' Public Administration Review, (37:1), pp. 28±37.
Lucas, H.C., 1975, `Performance and the Use of an Information System,' Management Science, April (21:8), pp. 908±919.
Mitroff, I.I. and Emshoff, J.R., 1979, `On Strategic Assumption-Making: A Dialectical Approach to Policy and Planning,' Academy of
Management Journal, January (4:1), pp. 1±12.
King, W.R. and Cleland, D.I., 1971, `Manager-Analyst Teamwork in MIS,' Business Horizons, April, pp. 59±68.
King, W.R., 1964, `Performance Evaluation in Marketing Systems,' Management Science, July (10:4), pp. 659±666.
63
64
Table 3
Culnan's co-citation analysis findings for 1980 to mid-1985 (listed by frequency)
# Citations
Citations
per year
Keen, P.G.W. and Scott Morton, M.S., 1978, Decision Support Systems: An Organizational Perspective, Addison-Wesley: Reading, MA
Mason, R.O. and Mitroff, I.I., 1973, `A Program for Research on Management Information Systems,' Management Science, January (19:5),
pp. 475±487.
Lucas, H.C., 1975, Why Information Systems Fail, Columbia University Press: New York
Zmud, R.W., 1979, `Individual Differences and MIS Success: A Review of the Empirical Literature,' Managment Science, February (25:2),
pp. 966±979.
Anthony, R.N., 1965, Planning and Control Systems: A Framework for AnalysisHarvard University Press: Boston, MA
Dickson, G.W., Senn, J.A., and Chervany, N.L., 1977, `Research in Management Information Systems; The Minnesota Experiments,'
Management Science, May (23:9), pp. 913±923.
Davis, G.B., 1974, Management Information Systems: Conceptual Foundations, Structure and Development, McGraw-Hill: New York
Benbasat, I. and Schroeder, R.G., 1978, `An Experimental Investigation of Some MIS Design Variables, MIS Quarterly, June (2:2), pp. 43±54.
Alter, S., 1980, Decision Support Systems: Current Practice and Continuing Challenges, Addison-Wesley: Reading, MA
Zmud, R.W., 1978, `An Empirical Investigation of the Dimesionality of the Concept of Information,' Decision Sciences, April (9:2), pp. 187±196.
Benbasat, I.W. and Dexter, A.S., 1979, `Value and Events Approaches to Accounting: An Experimental Evaluation,' Accounting Review,
October (54:4), pp. 735±749
Gorry, G.A. and Scott Morton, M.S., 1971, `A Framework for Management Information Systems,' Sloan Management Review, Fall (12:1),
pp. 55±70.
Ackoff, R.L., 1967, `Management Misinformation Systems,' Management Science, December (14:4), pp. B147±156
McKenney, J.L. and Keen, P.G.W., 1974, `How Managers' Minds Work,' Harvard Business Review, May±June (52:3), pp.79±90.
Kling, R., 1980, `Social Analyses in Computing,' Computing Surveys, March (12:1), pp 61±108.
Sprague, R.H. and Carlson, E.D., 1982, Building Effective Decision Support Systems, Prentice Hall: Englewood Cliffs: NJ
Ginzberg, M.J., 1978, `Steps Toward More Effective Implementation of MS and MIS,' Interfaces, May (8:3), pp. 57±63.
King, W.R., 1978, `Strategic Planning for Management Information Systems,' MIS Quarterly, March (2:1), pp. 27±37
Sprague, R.H., 1980, `A Framework for the Development of Decision Support Systems,' MIS Quarterly, December (4:4), pp. 1±26.
Swanson, E.B., 1974, `Management Information Systems: Appreciation and Involvement,'Management Science, October (20:2), pp. 178±188.
Ein Dor, P. and Segev, E., 1978, `Organizational Context and the Success of Management Information Systems,' Management Science,
June (24:10), pp. 1067±1077.
Rockart, J.F., 1979, `Chief Executives Define their Own Data Needs,' Harvard Business Review, March±April (57:2), pp. 81±91.
Mason, R.O. and Mitroff, I.I., 1981, Challenging Strategic Planning Assumptions, John Wiley: New York
Nolan, R.L., 1979, `Managing the Crises in Data Processing,' Harvard Business Review, March±April (57-2), pp. 115±126.
Keen, P.G.W., 1983, `Information Systems and Organizational Change,' Communications of the ACM, January (24:1), pp. 24±33.
Ginzberg, M.J., 1975, A Process Approach to Management Science Implementation, Unpublished Ph.D. dissertation, Massachusetts
Institute of Technology, Cambridge, MA
81
68
14.7
12.4
60
42
10.9
7.6
38
36
6.9
6.5
30
27
27
27
24
5.5
4.9
4.9
4.9
4.4
23
4.2
23
20
20
19
16
16
16
16
13
4.2
3.6
3.6
3.5
2.9
2.9
2.9
2.9
2.4
12
11
11
11
10
2.2
2
2
2
1.8
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Publication
65
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Table 3 summarizes the ®ndings from Culnan's
second co-citation analysis of the IS ®eld from
1980 to mid-1985. The number of citations per year
was determined by dividing the total number of citations identi®ed from the study by 5.5. During this ®ve
and a half year period, 13 publications were found to
meet the criteria of four or more citations a year:
1. Keen and Scott Morton, 1978.
2. Mason and Mitroff, 1973.
3. Lucas, 1975.
4. Zmud, 1979.
5. Anthony, 1965.
6. Dickson, Senn, and Chervany, 1977.
7. Davis, 1974.
8. Benbasat and Schroeder, 1978.
9. Alter, 1980.
10. Zmud, 1978.
11. Benbasat and Dexter, 1979.
12. Gorry and Scott Morton, 1971.
13. Ackoff, 1967.
attempt to compensate for this loss, the number of
citations for both studies were added together and
divided by 16. While this method is not precise, the
information gained from the analysis is worth the risk
of introducing distortion. Using this, three more publications meet the criteria for being classic:
1. Anthony, R.N., 1965.
2. Davis, G.B., 1974.
3. Keen, P.G.W. and Scott Morton, M.S., 1978.
4. Methodology
From the two previous lists of 20 total publications,
three publications appear on both:
1. Ackoff, R.L., 1967.
2. Lucas, H.C., 1975.
3. Mason, R.O. and Mitroff, I.I., 1973.
These three publications demonstrate their ability to
remain classics over a fourteen and a half year time
span. (In fact, one publication, Ackoff, 1967, also
appeared in all readings of the ®ve books). Since these
two studies do not provide citation data for each year,
but only a summary total, some information may be
lost between the ®ndings of the two studies. In an
Over 10 years have elapsed since the time frame
covered by the Culnan studies. The current study picks
up where this study ended: the 10 year period from
1986 to 1995. With the plethora of journals available,
the options were numerous. The top IS journals were
selected for analysis. Herein lies a dif®culty. In some
®elds, such as Accounting and Economics, there
appears to be consensus. In other ®elds (often younger
disciplines), there will be a variety of answers from
different people. There may be some agreement about
the top one or two. However, because of the number of
journals considered IS related there are likely to be 25
or 30 major journals. Over the years, empirical evidence has been collected to provide a broader picture
of the journal forums: Table 4 shows a summary of the
top journals.
Based on these studies nine journals were selected
for this study. Citation data was collected from each
article published in each of these journals for the time
period under investigation, 1986±1995.
Table 4
Top information systems journals rankings
Journal
Hardgrave and
Walstrom, 1997 [23]
Walstrom, et al.,
1995 [30]
Gillenson and Stutz,
1991 [22]
MIS Quarterly
Information Systems Research
Management Science
Communications of the ACM
Journal of MIS
Decision Sciences
Decision Support Systems
Information & Management
Organization Science
1
2
3
4
5
6
10
20
8
1
3
4
2
7
8
11
12
±
2
±
1
3
5
4
±
12
±
66
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
The Social Science Citation Index (SSCI) was used
to collect as much of the data as possible. The SSCI
contained all the citations from Communications of
the ACM, Decision Sciences, MIS Quarterly, and
Management Science. It also contained citation information for some years for Decision Support Systems,
Information & Management, Information Systems
Research, and Organization Science. The remaining
citations from these journals and all the citations from
the Journal of Management Information Systems were
collected manually by the authors. Once all the citations were collected, they were ordered to determine
which publications appeared most often.
5. Results
Table 5 shows that 3752 articles were found in the
nine journals during the years from 1986 to 1995
inclusive. In addition, 118,364 references were
attached to the 3752 articles. These 118,364 references were used for further analysis. An increase in
the number of articles published from 323 in 1986 to
378 in 1995 is noted, with a high of 433 in 1993. The
number of references more than doubled from 7406 in
1986 to 15,762 in 1995, with a high of 16,955 in 1992.
The number of references per article nearly doubled
from 23 in 1986 to 42 in 1995, with the latter being the
high.
The SAS FREQ procedure was used to identify any
publication among the 118,364 citations which
Table 5
Summary of the number of articles and references in the top is
journals from 1986±1995
Year
Number of
articles
Number of
references
Average number of
references per article
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
Total
323
320
364
338
368
393
422
433
413
378
3752
7406
7832
8619
8834
10632
12546
16955
15769
14,009
15762
118364
23
25
24
26
29
32
40
36
34
42
31.5
appeared four or more times in any one year from
1986 to 1995. This list of publications was then added
to the previous list of publications identi®ed in both
the Culnan studies. A total of 278 publications were
identi®ed for further investigation. A total citation
count for each of these 278 publications was performed for the time period 1986±1995. Any publication which averaged at least four citations per year
over the 10 year period, or at least, 40 total citations
were identi®ed as a highly visible work in the IS ®eld
of study. Table 6 shows that from the 278 publications,
61 were cited an average of four or more times per year
from 1986 to 1995. Seven publications were cited over
100 times: ®ve books and two journal articles.
Not all the articles have an equal probability of
meeting the cutoff of 40 citations. Works published
after 1985 have less likelihood of meeting the cutoff
criteria, because they have fewer years in which to
reach 40 citations. This makes the achievement of
articles such as DeSanctis and Gallupe (1987) and
Brancheau and Wetherbe [3] even more remarkable,
because they were available for citation 3 years less. In
fact, even without considering the lag between manuscript acceptance and actual publication (which is
often well over a year) a work published one year
is unlikely to be cited until at least the next year. Ten of
the 61 works in Table 6 were published in 1986 or
later.
To compensate for this inequity, an additional analysis was run to identify publications which met the
average cutoff of four or more citations per year, but
not the total cutoff of 40 citations. Table 7 shows an
additional 15 publications (published after 1985)
which averaged four or more citations since the year
of their publication.
Combining the articles from Tables 6 and 7 results
in 76 total publications for further consideration: 29
books, 47 journal articles. The books appear more
often towards the top of the list and the journal articles
appear more frequently towards the bottom. Table 8
shows the journals which published these 47 articles.
Nearly half of the articles were published in either MIS
Quarterly or Management Science. Nearly 25 percent
more were published in Communications of the ACM
and Harvard Business Review. Among the nine other
journals each containing one article were two newer
journals, Information Systems Research and Organization Science.
Table 6
Publications averaging four or more citations per year from 1986 to 1995 (listed by frequency)
Citations
per year
Porter, M.E., 1980, Competitive Strategy: Techniques for Analyzing Industries and Competitors, Free Press, New York.
Sprague, R.H. and Carlson, E.D., 1982, Building Effective Decision Support Systems, Prentice-Hall, Englewood Cliffs, NJ.
Thompson, J.D., 1967, Organizations in Action: Social Science Bases of Administrative Theory, McGraw-Hill: New York.
Keen, P.G.W. and Scott Morton, M.S., 1978, Decision Support Systems: an Organizational Perspective, Addison-Wesley Pub. Co., Reading, MA.
DeSanctis, G. and Gallupe, R.B., 1987, `A Foundation for the Study of Group Decision Support Systems,' Management Science, May (33:5), pp. 589±609.
Nunnally, J.C., 1967, 1978, Psychometric Theory, McGraw-Hill, New York.a
Ives, B., Olson, M.H. and Baroudi, J.J., 1983, `The Measurement of User Information Satisfaction,' Communications of the ACM, October (26:10), pp. 785±793.
Ives, B. and Olson, M.H., 1984, `User Involvement and MIS Success: A Review of Research,' Management Science, May (30:5), pp. 586±603.
Weick, K.E., 1969, 1979, The Social Psychology of Organizing, Addison-Wesley Publishing Company: Reading, MA.a
Keeney, R.L. and Raiffa, H., 1976, Decisions with Multiple Objectives: Preferences and Value Tradeoffs, Wiley: New York.
Williamson, O.E., 1975, Markets and Hierarchies, Analysis and Antitrust Implications: A Study in Economics of Internal Organizations, Free Press: New York.
McFarlan, F.W., 1984, `Information Technology Changes the Way You Compete,' Harvard Business Review, May±June (62:3), pp. 98±103.
Porter, M.E. and Millar, V.E., 1985, `How Information Gives You Competitive Advantage,' Harvard Business Review, July±August (63), pp. 149±160.
Porter, M.E., 1985, Competitive Advantage: Creating and Sustaining Superior Performance, Free Press: New York.
Cyert, R.M. and March, J.G., 1963, A Behavioral Theory of the Firm, Prentice-Hall: Englewood Cliffs, NJ.
March, J.G. and Simon, H.A., 1958, Organizations, Wiley: New York.
Davis, G.B., 1974, 1985, Management Information Systems: Conceptual Foundations, Structure and Development, McGraw-Hill: New York.a
Pfeffer, J., 1978, The External Control of Organizations: A Resource Dependence Perspective, Harper & Row: New York.
Huber, G.P., 1984, `Issues in the Design of Group Decision Support Systems,' MIS Quarterly, September (8:3), pp. 195±204.
Ives, B. and Learmonth, G. P., 1984, `The Information System as a Competitive Weapon,' Communications of the ACM, December (27:12), pp. 1193±1201.
Rogers, E.M., 1962, 1983, Diffusion of Innovations, Free Press: New York.a
Brancheau, J.C. and Wetherbe, J.C., 1987, `Key Issues in Information Systems Management,' MIS Quarterly, March (11:1), pp. 23±45.
King, W.R., 1978, `Strategic Planning for Management Information Systems,' MIS Quarterly, March (2:1), pp. 27±37.
Bailey, J.E. and Pearson, S.W., 1983, `Development of a Tool for Measuring and Analyzing Computer User Satisfaction,' Management Science,
May (29:5), pp. 530±545.
Mintzberg, H., Raisinghani, D. and Theoret, A., 1976, `The Structure of `Unstructured' Decision Processes,' Administrative Science Quarterly,
June (21:2), pp. 246±275.
Galbraith, J.R., 1973, Designing Complex Organizations, Addison-Wesley Publishing Company: Reading, MA.
Markus, M.L., 1983, `Power, Politics, and MIS Implementation,' Communications of the ACM, June (26:6), pp. 430±444.
Zmud, R.W., 1979, `Individual Differences and MIS Success: A Review of the Empirical Literature,' Management Science, February (25:2), pp. 966±979.
Gorry, G.A. and Scott Morton, M.S., 1971, `A Framework for Management Information Systems,' Sloan Management Review, (13:1), pp. 55±70.
Lawrence, P.R. and Lorsch, J.W., 1967, Organization and Environment: Managing Differentiation and Integration, Division of Research,
Graduate School of Business Administration, Harvard University: Boston, MA.
Newell, A. and Simon, H.A., 1972, Human Problem Solving, Prentice-Hall: Englewood Cliffs, NJ.
Daft, R.L. and Lengel, R.H., 1986, `Organizational Information Requirements, Media Richness and Structural Design,' Management Science,
May (32:5), pp. 554±571.
Cash, Jr., J.I. and Konsynski, B., 1985, `IS Redraws Competitive Boundaries,' Harvard Business Review, March±April (64:2), pp. 134±142.
Simon, H. A., 1960, The New Science of Management Decision, Harper, New York.
Swanson, E.B., 1974, `Management Information Systems: Appreciation and Involvement,' Management Science, October (20:2), pp. 178±188.
Williamson, O.E., 1985, The Economic Institutions of Capitalism: Firms, Markets, Relational Contracting, Free Press: New York.
141
134
124
119
116
108
104
98
95
91
88
86
86
84
81
79
77
76
71
70
68
67
63
62
14.1
13.4
12.4
11.9
11.6
10.8
10.4
9.8
9.5
9.1
8.8
8.6
8.6
8.4
8.1
7.9
7.7
7.6
7.1
7.0
6.8
6.7
6.3
6.2
62
6.2
61
61
61
60
59
6.1
6.1
6.1
6.0
5.9
59
56
5.9
5.6
55
55
54
53
5.5
5.5
5.4
5.3
67
Total
86±95
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Publication
68
Table 6 (Continued )
Publication
a
Two dates listed indicating both editions of the same book were counted together.
Citations
per year
52
52
51
51
5.2
5.2
5.1
5.1
49
49
49
49
49
48
48
47
47
46
45
45
45
44
4.9
4.9
4.9
4.9
4.9
4.8
4.8
4.7
4.7
4.6
4.5
4.5
4.5
4.4
44
4.4
44
4.4
43
4.3
42
42
41
41
4.2
4.2
4.1
4.1
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Burns, T. and Stalker, G.M., 1961, 1966, The Management of Innovation,Tavistock Publications: London.a
Dickson, G.W., Leitheiser, R.L. and Wetherbe, J.C., 1984, `Key Information Systems Issues for the 1980's,' MIS Quarterly, September (8:3), pp. 135±159.
Chandler, A.D., 1962, Strategy and Structure: Chapters in the History of the Industrial Enterprise, M.I.T. Press: Cambridge, MA.
Dennis, A.R., George, J.F., Jessup, L.M., Nunamaker, J.F. and Vogel, D.R., 1988, `Information Technology to Support Electronic Meetings,'
MIS Quarterly, December (12:4), pp. 591±624.
Huber, G.P., 1984, `The Nature and Design of Post-Industrial Organizations,' Management Science, August (30:8), pp. 928±951.
McGrath, J.E., 1984, Groups: Interaction and Performance, Prentice-Hall, Englewood Cliffs, NJ.
Nelson, R.R., 1982, An Evolutionary Theory of Economic Change, Belknap Press of Harvard University Press: Cambridge, MA.
Parsons, 1983, `Information Technology: A New Competitive Weapon,' Sloan Management Review, Fall (25:1), pp. 3±14.
Sprague, R.H., 1980, `A Framework for the Development of Decision Support Systems,' MIS Quarterly, December (4:4), pp. 1±26.
Galbraith, J.R., 1977, Organization Design, Addison-Wesley Publishing Company: Reading, MA.
Mintzberg, H., 1979, The Structuring of Organizations: A Synthesis of the Research, Prentice-Hall: Englewood Cliffs, NJ.
Bonczek, R.H., Holsapple, C.W. and Whinston, A.B., 1981, Foundations of Decision Support Systems, Academic Press: New York.
Rockart, J.F., 1979, `Chief Executives Define their Own Data Needs,' Harvard Business Review, March±April (57:2), pp. 81±91.
Rockart, J.F. and Flannery, L.S., 1983, `The Management of End User Computing,' Communications of the ACM, October (26:10), pp. 776±784.
Argyris, C. and Schon, D.A., 1978, Organizational Learning, Addison-Wesley: Reading, MA.
Benson, D.H., 1983, `A Field Study of End User Computing: Findings and Issues,' MIS Quarterly, December (7:4), pp. 35±45.
Johnston, H.R. and Carrico, S.R., 1988, `Developing Capabilities to Use Information Strategically,' MIS Quarterly, March (12:1), pp. 37±48.
Davis, F.D., 1989, `Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology,' MIS Quarterly, September (13:3),
pp. 319±340.
Kraemer, K.L. and King, J.L., 1988, `Computer-Based Systems for Cooperative Work and Group Decision Making,' ACM Computing Surveys,
June (20:2), pp. 115±146.
Nunamaker, J.F., Applegate, L.M. and Konsynski, B.R., 1987, `Facilitating Group Creativity: Experience with a Group Decision Support System,'
Journal of Management Information Systems, Spring (3:4), pp. 5±19.
Markus, M.L. and Robey, D., 1988, `Information Technology and Organizational Change: Causal Structure in Theory and Reseach,' Management Science,
May (34:5), pp. 583±598.
Kahneman, 1979, `Prospect Theory: An Analysis of Decision Making Under Risk,' Econometric, March (47:2), pp. 263±291.
Tversky, 1974, `Judgement Under Uncertainty: Heuristics and Biases,' Science, (185), pp. 1124±1131.
Eisenhardt, K.M., 1989, `Building Theories from Case Study Research,' Academy of Management Review, (14:4), pp. 532±550.
Nolan, R.L., 1979, `Managing the Crises in Data Processing,' Harvard Business Review, March±April (57:2), pp. 115±126.
Total
86±95
Table 7
Additional publications averaging four or more citations per year since publication (listed by frequency)
1986±1995
Citations
per year
DeLone, W. H. and McLean, E. R., 1992, `Information Systems Success: The Quest for the Dependent Variable,' Information Systems Research,
March (3:1), pp. 60±95.
Huber, G.P., 1990, `A Theory of the Effects of Advanced Information Technologies on Organizational Design, Intelligence, and Decision Making,'
Academy of Management Review, (15:1), pp. 47±71.
Nunamaker, J.F., Dennis, A.R., Valacich, J.S., Vogel, D.R. and George, J.F., 1991, `Electronic Meeting Systems to Support Group Work,'
Communications of the ACM, July (34:7), pp. 40±61.
Davis, F. D., Bagozzi, R. P. and Warshaw, P. R., 1989, `User Acceptance of Computer Technology: A Comparison of Two Theoretical Models,'
Management Science, August (35:8), pp. 982±1003.
Davenport, T.H., 1993, Process Innovation: Reengineering Work Through Information Technology, Harvard Business School Press: Boston.
Doll, W. J. and Torkzadeh, G., 1988, `The Measurement of End-User Computing Satisfaction,' MIS Quarterly, June (12:2), pp. 259±274.
Jarvenpaa, S.L., 1989, `The Effect of Task Demands and Graphical Format on Information Processing Strategies,' Management Science,
March (35:3), pp. 285±303.
DeSanctis, G. and Poole, M.S., 1994, `Capturing the Complexity in Advanced Technology Use: Adaptive Structuration Theory,'
Organization Science, May (5:2), pp. 121±147.
Hartwick, J. and Barki, H., 1994, `Explaining the Role of User Participation in Information System Use,' Management Science, April (40:4), pp. 440±465.
Gallupe, R. B. and DeSanctis, G., 1988, `Computer-Based Support for Group Problem-Finding: An Experimental Investigation,' MIS Quarterly,
June (2:2), pp. 277±296.
Zuboff, S., 1988, In the Age of the Smart Machine: The Future of Work and Power, Basic Books: New York.
Straub, D. W., 1989, `Validating Instruments in MIS Research,' MIS Quarterly, June (13:2), pp. 147±169.
Malone, T.W., Grant, K.R., Turbak, F.A., Brobst, S.A. and Cohen, M.D., 1987, `Intelligent Information-Sharing Systems,' Communications
of the ACM, May (30:5), pp. 390±402.
Siegel, J., 1986, `Group Processes in Computer-Mediated Communication,' Organizational Behavior and Human Decision Process, (37:2), pp. 157±187.
Nunamaker, J.F., 1989, `Experience at IBM,' Decision Support Systems, (5:2), pp. 183±196.
24
8
30
6
24
6
35
5.8
11
35
30
5.5
5
5
5
5
5
33
5
4.7
32
26
34
4.6
4.3
4.25
37
24
4.1
4
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Publication
69
70
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Table 8
Journal distribution of the most cited articles, 1986 to 1995
Journal
Number of articles
MIS Quarterly
Management Science
Communications of the ACM
Harvard Business Review
Academy of Management Review
Sloan Management Review
Nine other journals
12
11
6
5
2
2
Once each
As shown in Table 9, 97 authors were involved in
writing the 76 publications. One author, Nunamaker,
was involved in four of the publications. Five authors,
DeSanctis, Huber, Ives, Porter, and Simon were
involved with three of the publications. Sixteen
authors were involved with two of the publications
and 75 authors were involved with one of the publications.
Of the 76 publications, 17 were published between
1955 and 1975 and 14 more were published between
1976 and 1980. As might be expected, over half were
Table 9
Authors of the top cited articles, 1986±1995
Author
Number of articles
Nunamaker
DeSanctis
Huber
Ives
Porter
Simon
Davis, F
Dennis
Galbraith
Gallupe
George
Konsynski
March
Markus
Mintzberg
Olson
Rockart
Scott Morton
Sprague
Vogel
Wetherbe
Williamson
75 other authors
4
3
3
3
3
3
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
Once each
published in the 1980s: 20 were published between
1981 and 1985 and 20 more were published between
1986 and 1990. The last ®ve were published between
1991 and 1995.
6. Discussion
6.1. The traditional classics
From the early books of readings and Culnan's
studies, 19 individual publications were identi®ed
as classic. This list includes:
1. Ackoff, 1967.
2. Alter, 1980.
3. Anthony, 1965.
4. Benbasat and Dexter, 1979.
5. Benbasat and Schroeder, 1978.
6. Churchman, 1968.
7. Churchman, 1971.
8. Churchman and Schainblatt, 1965.
9. Dearden, 1972.
10. Dickson, Senn, and Chervany, 1977
11. Davis, G.B., 1974.
12. Gorry and Scott Morton, 1971.
13. Keen and Scott Morton, 1978
14. Kriebel, 1972.
15. Lucas, 1975.
16. Mason, 1969.
17. Mason and Mitroff, 1973.
18. Zmud, 1978.
19. Zmud, 1979.
7. The new classics
From 1986 to 1995, a new-breed of classic IS
publication was identi®ed. Sixty-one publications
meet the classic criteria. The other 15 works, if they
continue to be cited at the same rate they have been
since their publication, will be new classics. Of particular interest was the ratio of 47 journal articles and 29
books. Books typically consist of many more pages
than articles.
The list of traditional classics combined with the list
of new classics represents 91 individual publications.
Identi®ed over about a 25 year period, approximately
four works per year were represented.
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
7.1. Super classics
From the classic publications, some can be identi®ed as `super classic'. de Solla Price [14] refers to the
concept but fails to provide a de®nition. The inference
was that a super classic had to remain classic over time
versus the `well-known phenomenon of papers being
considered obsolescent after a decade'. Some articles
meet this criteria of super classics. The citing of
Ackoff, 1967 has fallen off over the years (averaging
two citations per year from 1986 to 1995), however, its
commanding presence in the early years and from
1972 to 1985 makes it the leading candidate for the
®rst super classic in the IS ®eld.
Four publications were present as classic in the
®ndings from Culnan's studies and from the analysis
of citations from 1986 to 1995: Davis, 1974 (and
Olson, M.H., 1985); Gorry and Scott Morton, 1971;
Keen and Scott Morton, 1978; and Zmud 1979. Therefore, a total of ®ve publications appear to meet the
criteria of super classic because of their sustained
citation visibility over time.
Other publications should perhaps be considered
super classic because of the number of citations over a
shorter period of time. This is also termed the `immediacy factor' or `bunching' discussed by de Solla
Price. For these publications, if the total been spread
out over the 24 year period from 1972 to 1995, they
would have averaged four citations per year, or a total
of 96 citations. These eight publications were:
1.
2.
3.
4.
5.
6.
7.
8.
Churchman, 1971.
DeSanctis and Gallupe, 1987.
Ives, Olson, and Baroudi, 1983.
Ives and Olson, 1984.
Nunnally, 1967, 1978.
Porter, 1980.
Sprague and Carlson, 1982.
Thompson, 1967.
A review of Table 7 reveals several publications, such
as DeLone and McLean (1992), that may sustain
citation visibility over a longer period of time. They
are worthy of note for future development.
7.2. Uses for the findings
Findings from this study can be used to develop
reading lists for doctoral students and by IS research-
71
ers to identify gaps in their knowledge. It can also be
used as a surrogate for the entire IS ®eld, allowing
researchers to pre-test their theories on a small sample
of the most visible works before examining a greater
quantity of literature. For instance, a follow up of the
points of work and reference studies conducted by
Culnan and Swanson [9] and Cheon et al. [4] could be
conducted using the ®nding of this study.
Third, these ®ndings can be used to update past
studies which have identi®ed content patterns in the IS
®eld or methodological evolution of the ®eld. These
type of studies could be follow ups to the intellectual
structure mapping of MIS and intellectual development studies conducted by Culnan. The underlying
disciplines impacting the IS ®eld can be analyzed by
examining the base discipline of these publications.
Invisible colleges (corresponding groups of active and
knowledgeable people in any particular research ®eld
corresponding [15]), can sometimes be identi®ed by
performing co-citation analysis using the 97 authors
who were involved with the 76 publications.
It also provides a means to examine how IS research
matches needs identi®ed by practitioners during the
same time period, in studies such as Neiderman et al.
[26] and Brancheau and Wetherbe [3]. Finally, further
study of these classic articles could reveal the characteristics that make an article or book, a `classic'
(similar to [12,13]).
8. Conclusions
With all the new material being published, perhaps
some of the older works are being pushed aside and
not included in the curriculum of Ph.D. students and
young faculty. Identifying these `classics' allows the
neophyte to ®nd and ®ll the gaps in his or her knowledge base. A greater understanding and appreciation
for the classic literature results in greater understanding and comprehension of the current literature.
A citation analysis was performed on the 118,364
references attached to the 3752 articles published in
nine top IS journals during the time period from 1986
to 1995. While citation analysis is not a perfect means
of identifying classic publications, it provides an
extensive method of determining highly visible works,
or citation classics. Ninety-one were identi®ed by
combining the results of this analysis with previous
72
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
®ndings and 13 super classic publications were identi®ed.
References
[1] M.J. Adler, C. Van Doren, How to Read a Book: The Classic
Guide to Intellectual Reading, Simon & Schuster, New York,
1972.
[2] M.J. Adler, P. Wolff, A General Introduction to the Great
Books and to a Liberal Education, Encyclopedia Britannica,
Chicago, 1959.
[3] J.C. Brancheau, J.C. Wetherbe, Key issues in information systems management, MIS Quarterly 11 (1), 1987, pp.
23±45.
[4] M.J. Cheon, C.C. Lee, V. Grover, Research in MIS Ð points
of work and reference: a replication and extension of the
Culnan and Swanson study. DATA BASE (1992) 21±29.
[5] D.E. Chubin, A.L. Porter, F.A. Rossini, `Citation classics'
analysis: an approach to characterizing interdisciplinary
research, Journal of the American Society for Information
Science 35 (6), 1984, pp. 156±172.
[6] R.J. Coleman, M.J. Riley, MIS: Management Dimensions,
Holden-Day, San Francisco, CA, 1973.
[7] M.J. Culnan, The intellectual development of management
information systems, 1972±1982: a co-citation analysis,
Management Science 32 (2), 1986, pp. 156±172.
[8] M.J. Culnan, Mapping the intellectual structure of MIS,
1980±1985: a co-citation analysis, MIS Quarterly 11 (3),
1987, pp. 341±353.
[9] M.J. Culnan, E.B. Swanson, Research in management
information systems, 1980±1984: points of work and
reference, MIS Quarterly 10 (3), 1986, pp. 289±301.
[10] G.B. Davis, The knowledge and skill requirements for the
doctorate in IS, in: Proceedings of the First International
Conference on Information Systems, 1980, pp. 174±186.
[11] G.B. Davis, G.C. Everest, Readings in Management Information Systems, McGraw-Hill, New York, 1976.
[12] M.S. Davis, `That's classic!' the phenomenology and rhetoric
of successful social theories, Philosophy of Social Science
(1986) 285±301.
[13] M.S. Davis, `That's interesting!' towards a phenomenology of
sociology and a sociology of phenomenology, Philosophy of
Social Science (1971) 309±344.
[14] D.J. de Solla Price, Little Science, Big Science, Columbia
University Press, New York, 1963.
[15] D.J. de Solla Price, Ethics of scientific publication, Science,
147 (1964) 655±657.
[16] D.J. de Solla Price, Networks of scientific papers, Science,
149 (1965) 510±515.
[17] G.W. Dickson, Management information systems: evolution
and status, Advances In Computers 20, 1981, pp. 1±37.
[18] V.T. Dock, V.P. Luchsinger, W.R. Cornette, MIS: A Managerial Perspective, Science Research Associates, Chicago, 1977.
[19] J. Erskine, The Delight of Great Books, The Bobbs-Merrill
Company, Indianapolis, IN, 1928.
[20] C. Fadiman, The Lifetime Reading Plan, 3rd Edition, Harper
& Row, New York, 1988.
[21] E. Garfield, Mapping the world of science: is citation analysis
a legitimate evaluation tool? In: D.N. Jackson, J.P. Rushton
(Eds.), Scientific Excellence: Origins and Assessment, Sage,
Newbury Park, CA, 1987.
[22] M.L. Gillenson, J.D. Stutz, Academic issues in IS: journals
and books, MIS Quarterly 15 (4), 1991, pp. 447±452.
[23] B.C. Hardgrave, K.A. Walstrom, Forums for MIS scholars,
Communications of the ACM 40 (11), 1997, pp. 119±124.
[24] C.L. Hind, One Hundred Best Books? G.P. Putnam's Sons,
New York, 1928.
[25] J. Lee, Have You Read 100 Great Books? Jasper Lee
Company, New York, 1946.
[26] F. Niederman, J.C. Brancheau, J.C. Wetherbe, Information
systems management issues for the 1990s, MIS Quarterly 15
(4), 1991, pp. 475±495.
[27] C. Oppenheim, S.P. Renn, Highly cited old papers and the
reasons why they continue to be cited, Journal of the American
Society for Information Science 29, 1978, pp. 225±231.
[28] D.H. Sanders, Computers and Management: In a Changing
Society, 2nd Edition, McGraw-Hill, New York, 1974.
[29] D.H. Sanders, Computers and Management, McGraw-Hill,
New York, 1970.
[30] K.A. Walstrom, B.C. Hardgrave, R.L. Wilson, Forums for
management information systems scholars, Communications
of the ACM 38 (3), 1995, pp. 93±107.
Kent A. Walstrom is an Associate
Professor of Information Systems at
Illinois State University. He received
his PhD from Oklahoma State University. Dr. Walstrom's research interests
include executive information systems,
the history of the information systems
field, electronic commerce, and the use
of information systems to support
leadership. Previous publications include
articles in Communication of the ACM,
Computers and Operations Research, Information & Management,
Journal of Computer Information Systems, among others.
Lori N.K. Leonard is an Assistant
Professor of Management Information
Systems at the University of Tulsa. Dr.
Leonard received her PhD from the
University of Arkansas and is an active
member of the Decision Sciences Institute. Her research interests include
electronic commerce, electronic data
interchange, ethics in computing, simulation, and data warehousing. Her publications have appeared in Journal of
Computer Information Systems, Proceedings of the National
Decision Sciences Institute, Proceedings of the Decision Sciences
Institute Southwest Region, and Proceedings of Americas Conference on Information Systems.
Citation classics from the information systems literature
Kent A. Walstroma,*, Lori N.K. Leonardb,1
a
Applied Computer Science Department, Campus Box 5150, Illinois State University, Normal, IL 61790-5150, USA,
b
School of Accounting, University of Tulsa, 600 S. College Ave., Tulsa, OK 74104-3189, USA
Received 14 September 1999; accepted 3 January 2000
Abstract
An analysis of 118,364 references from 3752 articles published in top IS journals during the 10 years from 1986 to 1995 is
presented. Ninety-one `citation classics' were identi®ed by combining the results of this analysis with previous ®ndings.
Within this group of 91 publications, 13 `super classic' publications were identi®ed. These classics and super classics can be
categorized along any of several dimensions to gain a better understanding of the IS ®eld. These ®ndings can be used, among
other things, for developing reading lists and identifying the characteristics that make a publication `classic'. # 2000 Elsevier
Science B.V. All rights reserved.
Keywords: Diffusion of IS research; IS literature; Citation analysis
ISRL: IB0301; IB0301.01
1. Introduction
Whether we are attempting to understand the broad
foundations of a research discipline or the speci®c
implications of a narrow stream of research within a
discipline, we need to review what has already been
discovered so we can explore that which is yet
unknown. As the information systems (IS) ®eld grows
and matures there is an increase in the amount of new
literature published each year. Scanning the total
volume thus becomes a daunting task and doctoral
students interested in the cross-disciplinary study of
IS, ask `can you suggest reading material for me in the
IS ®eld?' Moreover, practitioners may want to read the
*
Corresponding author. Tel.: 1-309-438-8338;
fax: 1-309-438-5113.
E-mail address: [email protected] (K.A. Walstrom).
1
Tel.: 1-918-631-2787.
`best' that the academic ®eld has to offer. The purpose
of this study was to identify the most cited IS publications from the past 10 years and use them to
identify the `classic' publications in the ®eld.
2. Literature review
The `must read' publications list of 10 experts in the
IS ®eld would certainly not be the same, either in
content or order. The differences in these lists may be a
result of several factors, including: the large number of
articles (several thousand to choose from), the diversity of backgrounds of the experts, and the research
and teaching interests currently being pursued by
them. Examining several Ph.D. seminar reading lists,
revealed marked differences in the material required
of IS Ph.D. students in an introductory course. Selecting just one of these might result in important works
0378-7206/00/$ ± see front matter # 2000 Elsevier Science B.V. All rights reserved.
PII: S 0 3 7 8 - 7 2 0 6 ( 0 0 ) 0 0 0 5 4 - 9
60
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
being missed, though they are considered important by
a greater number of experts. The continual demand for
IS professionals and knowledge workers in the marketplace makes the demand for IS faculty members
either constant or increasing. Government and university pressure provides a demand for IS literate
researchers, since IS technology is applicable to all
®elds of inquiry.
To gain an understanding of any large body of
knowledge requires considerable time and effort.
Reading the best the ®eld has to offer maximizes
the use of time resources. People now realize that it
is no longer possible to read all the available literature
and ®ltering strategies have, therefore, emerged. Over
the years, literacy experts have published their ®ltering strategies in the form of `lists of great books' in an
attempt to help mankind identify which works merit
investigation [1,2,19,20,24]. The thoughts of these
experts are summed in the following quote:
``No single person has a right to say which are
the hundred or thousand best books. . . . But
numerous lists of great books have been drawn
up by various persons at one time or other, and
they provide what is on the whole the best guide
through the bewildering shelves of any large
library.''
Van Doren in [25].
A similar filtering strategy can be used in understanding academic literature. In the academic world, one of
these lists is provided at the end of every refereed
journal article or conference proceedings produced.
This reference list can be used to identify sources on
the topic of interest in the article. Collected together
over time and over a variety of publication outlets,
these lists provide a composite list of the best sources
available in the field of IS. The Proceedings of the
First ICIS meeting in 1980 reveals an attempt to
identify the literature which had to be included in a
doctoral level program of study [10].
2.1. The early years
Dickson [17] identi®ed ®ve early IS books that
included readings: Sanders [29]; Coleman and Riley
[6]; Sanders [28] Davis and Everest [11]; and Dock
et al. [18]. A review of these books provides a parsimonious look at articles that were then considered by
these authors to be important. Table 1 shows the
articles that appeared in more than one of these books.
One article, Ackoff 1967, appeared in all the ®ve
books. Two articles, Dearden 1972 and Kriebel
1972, appeared in three of the ®ve books. Another
20 articles appeared in two of the books.
2.1.1. Citations
Classics can be identi®ed subjectively by asking
individuals, but the views of several experts yield a
more representative selection of publications. According to de Solla Price [16], a `classic' is any publication
cited four or more times in a year. Citation analysis has
also been identi®ed as another means of determining
`classic' publications [5,21,27]. Citation analysis
allows for the collection of perceptions of hundreds,
or even thousands, of `experts', simultaneously and
over time. `Citation classics' are extraordinary,
because of the large number of citations they receive.
While, 30 years ago considerably fewer articles were
written and printed than today, this number still provides a good starting place. Especially since we are not
interested in all available literature, only that small
portion regarded as IS literature.
3. The middle years: 1972±1985
Researchers produced citation ®ndings about the IS
literature for the period from 1972 to 1985. Culnan, in
two separate studies [7,8], reported her ®ndings of a
co-citation analysis of the IS discipline. The appendix
from these studies is helpful in identifying works that
met the `classic' criteria in the IS discipline from 1972
until mid-1985. Table 2 summarizes the ®ndings from
Culnan's ®rst co-citation analysis from 1972 to mid1982. The number of citations per year was determined by dividing the total number of citations
identi®ed from the study by 10.5. During this ten
and a half year period, seven publications were found
to meet the criteria of four or more citations a year:
1.
2.
3.
4.
5.
6.
7.
Churchman, 1971.
Mason and Mitroff, 1973.
Churchman, 1968.
Mason, 1969.
Ackoff, 1967.
Lucas, 1975.
Churchman and Schainblatt, 1965.
Table 1
Articles appearing in more than one book of readings (listed by frequency)
Sanders,
1970
[29]
Coleman and
Riley, 1973
[6]
Sanders,
1974
[28]
Davis and
Everest,
1976 [11]
Dock,etal.,
1977 [18]
Ackoff, R.L., 1967, `Management Misinfornation Systems,' Management Science, December,
pp. B-147-B-156.
Dearden, 1972, `MIS is a Mirage,' Harvard Business Review, January±February, pp. 90±99.
Kriebel, 1972, `The Future of MIS,' Business Automation, June, pp. 18±19.
Adams, W., 1972, `New Role for Top Management in Computer Applications,' Financial Executive,
April, pp. 54±56.
Allen, B.R., 1972, `Computer Security: Part I.,' Data Management, January, pp. 18±24.
Allen, B.R., 1972, `Computer Security: Part II.,' Data Management, February, pp. 25±30.
Andrus, 1971, `Approaches to Information Evaluation,' MSU Business Topics, Summer, pp. 40±45.
Argyris, C., 1971, `MIS: The Challenge to Rationality,' Management Science, February.
Beishon, R.J., 1970, `Information Flow and Managers' Decisions,' Managerial Accounting, June.
Deckard, N.S., 1970, `Capturing the Development Costs of an Integrated MIS?' Journal of Systems
Management, January.
Dickson, G.W., 1968, `Management Information±Decision Systems,' Business Horizons, December.
Dickson, G.W. and Simmons, J.K., 1970, `The Behavioral Side of MIS: Some Aspects,'
Business Horizons, August.
Ernst, M., 1971, `Management, The Computer, and Society,' Computers and Automation, September,
pp. 8±11.
Fredricks, W.A., 1971, `A Manager's Perspective of Management Information Systems,'
MSU Business Topics, Spring.
Gibson, C.F. and Nolan, R.L., 1974, `Managing the Four Stages of EDP Growth,' Harvard
Business Review, January±February.
Holmes, R.W., 1970, `Twelve Areas to Investigate for Better MIS,' Financial Executive, July.
Hoos, I.R., 1971, `Information Systems and Public Planning,' Management Science, June,
pp. B658±B671.
Kanter, J., 1972, `Impact of Computers on the Business Organization,' Data Management,
April, pp. 20±23.
Nanus, B., 1972, `Managing the Fifth Information Revolution,' Business Horizons, April.
Nichols, G.E., `On The Nature of Management Information,' Management Accounting, April 1969,
pp. 9±13.
Nicholson, C.H., 1970, `MIS In Perspective,' Chemical Engineering Progress, January.
Tolliver, E.M., 1970, `Myths of Automated Management Systems,'
Journal of Systems Management, August.
Zani, 1970, `Blueprint for MIS,' Harvard Business Review, November±December.
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
X
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Article
X
X
X
61
62
Publication
# Citations
Citations
per year
Churchman, C.W., 1971, The Design of Inquiring Systems, Basic Books: New York
Mason, R.O. and Mitroff, I.I., 1973, `A Program for Research on Management Information Systems,' Management Science, January (19:5),
pp. 475±487.
Churchman, C.W., 1968, The Systems Approach, Delacorte Press: New York
Mason, R.O., 1969, `A Dialectical Approach to Strategic Planning,' Management Science, April (15:8), pp. B403±B414.
Ackoff, R.L., 1967, `Management Misinformation Systems,' Management Science, (14:4), pp. 147±156.
Lucas, H.C., 1975, Why Information Systems Fail, Columbia University Press: New York.
Churchman, C.W. and Schainblatt, A.H., 1965, `The Researcher and the Manager: A Dialectic of Implementation,' Management Science, (11:4),
pp. B69±B87.
Anthony, R.N., 1965, Planning and Control Systems: A Framework for Analysis, Harvard University, Graduate School of Business Administration,
Division of Research: Boston, MA.
Davis, G.B., 1974, Management Information Systems: Conceptual Foundations, Structure and Development, McGraw-Hill: New York.
Gorry, G.A. and Scott-Morton, M., 1971, `A Framework for Management Information Systems,' Sloan Management Review, (13:1), pp. 55±57.
Swanson, E.B., 1974, `Management Information Systems: Appreciation and Involvement,' Management Science, October (2:2), pp. 178±183.
Ackoff, R.L., 1970, A Concept of Corporate Planning, Wiley±Interscience: New York
Ackoff, R.L., 1974, Redesigning the Future: A Systems Approach to Social Problems, Wiley: New York.
Ackoff, R.L., 1962, Scientific Method: Optimising Applied Research Decisions, Wiley: New York.
King, W.R. and Cleland, D.I., 1975, `The Design of Management Information Systems: An Information Analysis Approach,' Management Science,
November (22:3), pp. 286±297.
Mitroff, I.I., 1971, `A Communication Model of Dialectical Inquiring Systems: A Strategy for Strategic Planning,' Management Science,
June (17:10), pp. B634±B648.
103
63
9.8
6
63
53
52
50
47
6
5
5
4.8
4.5
41
3.9
35
34
34
34
34
32
25
3.3
3.2
3.2
3.2
3.2
3
2.4
24
2.3
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Table 2
Culnan's co-citation analysis findings for 1972 to mid-1982 (listed by frequency)
24
2.3
22
21
20
17
16
2.1
2
1.9
1.6
1.5
15
13
1.4
1.2
13
13
1.2
1.2
12
1.1
12
11
11
10
10
1.1
1
1
1
1
10
10
1
1
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Mitroff, I.I., 1979, The Subjective Side of Science: A Philosophical Inquiry into the Psychology of Apollo Moon Scientists, Elsevier Scientific
Publishing Company: New York.
Dearden, J., 1972, `MIS is a Mirage,' Harvard Business Review, January±February, pp. 90±99.
McKenny, J.L. and Keen, P.G.W., 1974, `How Manager's Minds Work,' Harvard Business Review, May±June (52:3), pp. 79±90.
Nolan, R.W., 1973, `Managing the Computer Resource: A Stage Hypothesis,' Communications of the ACM, (16:7), pp. 399±405.
Hiltz, S.R. and Turoff, M., 1978The Network Nation: Human Communication via Computer.
Keen, P.G.W. and Scott-Morton, M.S., 1978, Decision Support Systems: An Organizational Perspective, Addison-Wesley Publishing Company:
Reading, MA.
Ackoff, R.L., 1960, `Unsuccessful Case Studies and Why,' Operations Research, March±April (8:2), pp. 259±262.
Chervany, N.L. and Dickson, G.W., 1974, `An Experimental Evaluation of Information Overload in a Production Environment,' Management
Science, June (20:10), pp. 1335±1344.
Nolan, R.W., 1973, `Computer Data Bases: The Future is Now,' Harvard Business Review, September±October (51:5), pp. 98±114.
Mitroff, I.I. and Featheringham, T.R., 1974, `On Systematic Problem Solving and the Error of the Third Kind,' Behavioral Science, (19),
pp. 383±393.
Mitroff, I.I., Betz, F., Pondy, L.R. and Sagasti, F., 1974, `On Managing Science in the Systems Age: Two Schemas for the Study of Science as a
Whole Systems Phenomenon,' Interfaces, (4:3), pp. 46±58.
Bennett, J.L., 1972, `The User Interface in Interactive Systems,' Annual Review of Information Science and Technology, (7), pp. 159±196.
Whisler, T.L., 1970, The Impact of the Computer on Organizations, Praeger: New York.
Danzinger, J.N., 1977, `Computers, Local Governments, and the Litany of EDP,' Public Administration Review, (37:1), pp. 28±37.
Lucas, H.C., 1975, `Performance and the Use of an Information System,' Management Science, April (21:8), pp. 908±919.
Mitroff, I.I. and Emshoff, J.R., 1979, `On Strategic Assumption-Making: A Dialectical Approach to Policy and Planning,' Academy of
Management Journal, January (4:1), pp. 1±12.
King, W.R. and Cleland, D.I., 1971, `Manager-Analyst Teamwork in MIS,' Business Horizons, April, pp. 59±68.
King, W.R., 1964, `Performance Evaluation in Marketing Systems,' Management Science, July (10:4), pp. 659±666.
63
64
Table 3
Culnan's co-citation analysis findings for 1980 to mid-1985 (listed by frequency)
# Citations
Citations
per year
Keen, P.G.W. and Scott Morton, M.S., 1978, Decision Support Systems: An Organizational Perspective, Addison-Wesley: Reading, MA
Mason, R.O. and Mitroff, I.I., 1973, `A Program for Research on Management Information Systems,' Management Science, January (19:5),
pp. 475±487.
Lucas, H.C., 1975, Why Information Systems Fail, Columbia University Press: New York
Zmud, R.W., 1979, `Individual Differences and MIS Success: A Review of the Empirical Literature,' Managment Science, February (25:2),
pp. 966±979.
Anthony, R.N., 1965, Planning and Control Systems: A Framework for AnalysisHarvard University Press: Boston, MA
Dickson, G.W., Senn, J.A., and Chervany, N.L., 1977, `Research in Management Information Systems; The Minnesota Experiments,'
Management Science, May (23:9), pp. 913±923.
Davis, G.B., 1974, Management Information Systems: Conceptual Foundations, Structure and Development, McGraw-Hill: New York
Benbasat, I. and Schroeder, R.G., 1978, `An Experimental Investigation of Some MIS Design Variables, MIS Quarterly, June (2:2), pp. 43±54.
Alter, S., 1980, Decision Support Systems: Current Practice and Continuing Challenges, Addison-Wesley: Reading, MA
Zmud, R.W., 1978, `An Empirical Investigation of the Dimesionality of the Concept of Information,' Decision Sciences, April (9:2), pp. 187±196.
Benbasat, I.W. and Dexter, A.S., 1979, `Value and Events Approaches to Accounting: An Experimental Evaluation,' Accounting Review,
October (54:4), pp. 735±749
Gorry, G.A. and Scott Morton, M.S., 1971, `A Framework for Management Information Systems,' Sloan Management Review, Fall (12:1),
pp. 55±70.
Ackoff, R.L., 1967, `Management Misinformation Systems,' Management Science, December (14:4), pp. B147±156
McKenney, J.L. and Keen, P.G.W., 1974, `How Managers' Minds Work,' Harvard Business Review, May±June (52:3), pp.79±90.
Kling, R., 1980, `Social Analyses in Computing,' Computing Surveys, March (12:1), pp 61±108.
Sprague, R.H. and Carlson, E.D., 1982, Building Effective Decision Support Systems, Prentice Hall: Englewood Cliffs: NJ
Ginzberg, M.J., 1978, `Steps Toward More Effective Implementation of MS and MIS,' Interfaces, May (8:3), pp. 57±63.
King, W.R., 1978, `Strategic Planning for Management Information Systems,' MIS Quarterly, March (2:1), pp. 27±37
Sprague, R.H., 1980, `A Framework for the Development of Decision Support Systems,' MIS Quarterly, December (4:4), pp. 1±26.
Swanson, E.B., 1974, `Management Information Systems: Appreciation and Involvement,'Management Science, October (20:2), pp. 178±188.
Ein Dor, P. and Segev, E., 1978, `Organizational Context and the Success of Management Information Systems,' Management Science,
June (24:10), pp. 1067±1077.
Rockart, J.F., 1979, `Chief Executives Define their Own Data Needs,' Harvard Business Review, March±April (57:2), pp. 81±91.
Mason, R.O. and Mitroff, I.I., 1981, Challenging Strategic Planning Assumptions, John Wiley: New York
Nolan, R.L., 1979, `Managing the Crises in Data Processing,' Harvard Business Review, March±April (57-2), pp. 115±126.
Keen, P.G.W., 1983, `Information Systems and Organizational Change,' Communications of the ACM, January (24:1), pp. 24±33.
Ginzberg, M.J., 1975, A Process Approach to Management Science Implementation, Unpublished Ph.D. dissertation, Massachusetts
Institute of Technology, Cambridge, MA
81
68
14.7
12.4
60
42
10.9
7.6
38
36
6.9
6.5
30
27
27
27
24
5.5
4.9
4.9
4.9
4.4
23
4.2
23
20
20
19
16
16
16
16
13
4.2
3.6
3.6
3.5
2.9
2.9
2.9
2.9
2.4
12
11
11
11
10
2.2
2
2
2
1.8
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Publication
65
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Table 3 summarizes the ®ndings from Culnan's
second co-citation analysis of the IS ®eld from
1980 to mid-1985. The number of citations per year
was determined by dividing the total number of citations identi®ed from the study by 5.5. During this ®ve
and a half year period, 13 publications were found to
meet the criteria of four or more citations a year:
1. Keen and Scott Morton, 1978.
2. Mason and Mitroff, 1973.
3. Lucas, 1975.
4. Zmud, 1979.
5. Anthony, 1965.
6. Dickson, Senn, and Chervany, 1977.
7. Davis, 1974.
8. Benbasat and Schroeder, 1978.
9. Alter, 1980.
10. Zmud, 1978.
11. Benbasat and Dexter, 1979.
12. Gorry and Scott Morton, 1971.
13. Ackoff, 1967.
attempt to compensate for this loss, the number of
citations for both studies were added together and
divided by 16. While this method is not precise, the
information gained from the analysis is worth the risk
of introducing distortion. Using this, three more publications meet the criteria for being classic:
1. Anthony, R.N., 1965.
2. Davis, G.B., 1974.
3. Keen, P.G.W. and Scott Morton, M.S., 1978.
4. Methodology
From the two previous lists of 20 total publications,
three publications appear on both:
1. Ackoff, R.L., 1967.
2. Lucas, H.C., 1975.
3. Mason, R.O. and Mitroff, I.I., 1973.
These three publications demonstrate their ability to
remain classics over a fourteen and a half year time
span. (In fact, one publication, Ackoff, 1967, also
appeared in all readings of the ®ve books). Since these
two studies do not provide citation data for each year,
but only a summary total, some information may be
lost between the ®ndings of the two studies. In an
Over 10 years have elapsed since the time frame
covered by the Culnan studies. The current study picks
up where this study ended: the 10 year period from
1986 to 1995. With the plethora of journals available,
the options were numerous. The top IS journals were
selected for analysis. Herein lies a dif®culty. In some
®elds, such as Accounting and Economics, there
appears to be consensus. In other ®elds (often younger
disciplines), there will be a variety of answers from
different people. There may be some agreement about
the top one or two. However, because of the number of
journals considered IS related there are likely to be 25
or 30 major journals. Over the years, empirical evidence has been collected to provide a broader picture
of the journal forums: Table 4 shows a summary of the
top journals.
Based on these studies nine journals were selected
for this study. Citation data was collected from each
article published in each of these journals for the time
period under investigation, 1986±1995.
Table 4
Top information systems journals rankings
Journal
Hardgrave and
Walstrom, 1997 [23]
Walstrom, et al.,
1995 [30]
Gillenson and Stutz,
1991 [22]
MIS Quarterly
Information Systems Research
Management Science
Communications of the ACM
Journal of MIS
Decision Sciences
Decision Support Systems
Information & Management
Organization Science
1
2
3
4
5
6
10
20
8
1
3
4
2
7
8
11
12
±
2
±
1
3
5
4
±
12
±
66
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
The Social Science Citation Index (SSCI) was used
to collect as much of the data as possible. The SSCI
contained all the citations from Communications of
the ACM, Decision Sciences, MIS Quarterly, and
Management Science. It also contained citation information for some years for Decision Support Systems,
Information & Management, Information Systems
Research, and Organization Science. The remaining
citations from these journals and all the citations from
the Journal of Management Information Systems were
collected manually by the authors. Once all the citations were collected, they were ordered to determine
which publications appeared most often.
5. Results
Table 5 shows that 3752 articles were found in the
nine journals during the years from 1986 to 1995
inclusive. In addition, 118,364 references were
attached to the 3752 articles. These 118,364 references were used for further analysis. An increase in
the number of articles published from 323 in 1986 to
378 in 1995 is noted, with a high of 433 in 1993. The
number of references more than doubled from 7406 in
1986 to 15,762 in 1995, with a high of 16,955 in 1992.
The number of references per article nearly doubled
from 23 in 1986 to 42 in 1995, with the latter being the
high.
The SAS FREQ procedure was used to identify any
publication among the 118,364 citations which
Table 5
Summary of the number of articles and references in the top is
journals from 1986±1995
Year
Number of
articles
Number of
references
Average number of
references per article
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
Total
323
320
364
338
368
393
422
433
413
378
3752
7406
7832
8619
8834
10632
12546
16955
15769
14,009
15762
118364
23
25
24
26
29
32
40
36
34
42
31.5
appeared four or more times in any one year from
1986 to 1995. This list of publications was then added
to the previous list of publications identi®ed in both
the Culnan studies. A total of 278 publications were
identi®ed for further investigation. A total citation
count for each of these 278 publications was performed for the time period 1986±1995. Any publication which averaged at least four citations per year
over the 10 year period, or at least, 40 total citations
were identi®ed as a highly visible work in the IS ®eld
of study. Table 6 shows that from the 278 publications,
61 were cited an average of four or more times per year
from 1986 to 1995. Seven publications were cited over
100 times: ®ve books and two journal articles.
Not all the articles have an equal probability of
meeting the cutoff of 40 citations. Works published
after 1985 have less likelihood of meeting the cutoff
criteria, because they have fewer years in which to
reach 40 citations. This makes the achievement of
articles such as DeSanctis and Gallupe (1987) and
Brancheau and Wetherbe [3] even more remarkable,
because they were available for citation 3 years less. In
fact, even without considering the lag between manuscript acceptance and actual publication (which is
often well over a year) a work published one year
is unlikely to be cited until at least the next year. Ten of
the 61 works in Table 6 were published in 1986 or
later.
To compensate for this inequity, an additional analysis was run to identify publications which met the
average cutoff of four or more citations per year, but
not the total cutoff of 40 citations. Table 7 shows an
additional 15 publications (published after 1985)
which averaged four or more citations since the year
of their publication.
Combining the articles from Tables 6 and 7 results
in 76 total publications for further consideration: 29
books, 47 journal articles. The books appear more
often towards the top of the list and the journal articles
appear more frequently towards the bottom. Table 8
shows the journals which published these 47 articles.
Nearly half of the articles were published in either MIS
Quarterly or Management Science. Nearly 25 percent
more were published in Communications of the ACM
and Harvard Business Review. Among the nine other
journals each containing one article were two newer
journals, Information Systems Research and Organization Science.
Table 6
Publications averaging four or more citations per year from 1986 to 1995 (listed by frequency)
Citations
per year
Porter, M.E., 1980, Competitive Strategy: Techniques for Analyzing Industries and Competitors, Free Press, New York.
Sprague, R.H. and Carlson, E.D., 1982, Building Effective Decision Support Systems, Prentice-Hall, Englewood Cliffs, NJ.
Thompson, J.D., 1967, Organizations in Action: Social Science Bases of Administrative Theory, McGraw-Hill: New York.
Keen, P.G.W. and Scott Morton, M.S., 1978, Decision Support Systems: an Organizational Perspective, Addison-Wesley Pub. Co., Reading, MA.
DeSanctis, G. and Gallupe, R.B., 1987, `A Foundation for the Study of Group Decision Support Systems,' Management Science, May (33:5), pp. 589±609.
Nunnally, J.C., 1967, 1978, Psychometric Theory, McGraw-Hill, New York.a
Ives, B., Olson, M.H. and Baroudi, J.J., 1983, `The Measurement of User Information Satisfaction,' Communications of the ACM, October (26:10), pp. 785±793.
Ives, B. and Olson, M.H., 1984, `User Involvement and MIS Success: A Review of Research,' Management Science, May (30:5), pp. 586±603.
Weick, K.E., 1969, 1979, The Social Psychology of Organizing, Addison-Wesley Publishing Company: Reading, MA.a
Keeney, R.L. and Raiffa, H., 1976, Decisions with Multiple Objectives: Preferences and Value Tradeoffs, Wiley: New York.
Williamson, O.E., 1975, Markets and Hierarchies, Analysis and Antitrust Implications: A Study in Economics of Internal Organizations, Free Press: New York.
McFarlan, F.W., 1984, `Information Technology Changes the Way You Compete,' Harvard Business Review, May±June (62:3), pp. 98±103.
Porter, M.E. and Millar, V.E., 1985, `How Information Gives You Competitive Advantage,' Harvard Business Review, July±August (63), pp. 149±160.
Porter, M.E., 1985, Competitive Advantage: Creating and Sustaining Superior Performance, Free Press: New York.
Cyert, R.M. and March, J.G., 1963, A Behavioral Theory of the Firm, Prentice-Hall: Englewood Cliffs, NJ.
March, J.G. and Simon, H.A., 1958, Organizations, Wiley: New York.
Davis, G.B., 1974, 1985, Management Information Systems: Conceptual Foundations, Structure and Development, McGraw-Hill: New York.a
Pfeffer, J., 1978, The External Control of Organizations: A Resource Dependence Perspective, Harper & Row: New York.
Huber, G.P., 1984, `Issues in the Design of Group Decision Support Systems,' MIS Quarterly, September (8:3), pp. 195±204.
Ives, B. and Learmonth, G. P., 1984, `The Information System as a Competitive Weapon,' Communications of the ACM, December (27:12), pp. 1193±1201.
Rogers, E.M., 1962, 1983, Diffusion of Innovations, Free Press: New York.a
Brancheau, J.C. and Wetherbe, J.C., 1987, `Key Issues in Information Systems Management,' MIS Quarterly, March (11:1), pp. 23±45.
King, W.R., 1978, `Strategic Planning for Management Information Systems,' MIS Quarterly, March (2:1), pp. 27±37.
Bailey, J.E. and Pearson, S.W., 1983, `Development of a Tool for Measuring and Analyzing Computer User Satisfaction,' Management Science,
May (29:5), pp. 530±545.
Mintzberg, H., Raisinghani, D. and Theoret, A., 1976, `The Structure of `Unstructured' Decision Processes,' Administrative Science Quarterly,
June (21:2), pp. 246±275.
Galbraith, J.R., 1973, Designing Complex Organizations, Addison-Wesley Publishing Company: Reading, MA.
Markus, M.L., 1983, `Power, Politics, and MIS Implementation,' Communications of the ACM, June (26:6), pp. 430±444.
Zmud, R.W., 1979, `Individual Differences and MIS Success: A Review of the Empirical Literature,' Management Science, February (25:2), pp. 966±979.
Gorry, G.A. and Scott Morton, M.S., 1971, `A Framework for Management Information Systems,' Sloan Management Review, (13:1), pp. 55±70.
Lawrence, P.R. and Lorsch, J.W., 1967, Organization and Environment: Managing Differentiation and Integration, Division of Research,
Graduate School of Business Administration, Harvard University: Boston, MA.
Newell, A. and Simon, H.A., 1972, Human Problem Solving, Prentice-Hall: Englewood Cliffs, NJ.
Daft, R.L. and Lengel, R.H., 1986, `Organizational Information Requirements, Media Richness and Structural Design,' Management Science,
May (32:5), pp. 554±571.
Cash, Jr., J.I. and Konsynski, B., 1985, `IS Redraws Competitive Boundaries,' Harvard Business Review, March±April (64:2), pp. 134±142.
Simon, H. A., 1960, The New Science of Management Decision, Harper, New York.
Swanson, E.B., 1974, `Management Information Systems: Appreciation and Involvement,' Management Science, October (20:2), pp. 178±188.
Williamson, O.E., 1985, The Economic Institutions of Capitalism: Firms, Markets, Relational Contracting, Free Press: New York.
141
134
124
119
116
108
104
98
95
91
88
86
86
84
81
79
77
76
71
70
68
67
63
62
14.1
13.4
12.4
11.9
11.6
10.8
10.4
9.8
9.5
9.1
8.8
8.6
8.6
8.4
8.1
7.9
7.7
7.6
7.1
7.0
6.8
6.7
6.3
6.2
62
6.2
61
61
61
60
59
6.1
6.1
6.1
6.0
5.9
59
56
5.9
5.6
55
55
54
53
5.5
5.5
5.4
5.3
67
Total
86±95
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Publication
68
Table 6 (Continued )
Publication
a
Two dates listed indicating both editions of the same book were counted together.
Citations
per year
52
52
51
51
5.2
5.2
5.1
5.1
49
49
49
49
49
48
48
47
47
46
45
45
45
44
4.9
4.9
4.9
4.9
4.9
4.8
4.8
4.7
4.7
4.6
4.5
4.5
4.5
4.4
44
4.4
44
4.4
43
4.3
42
42
41
41
4.2
4.2
4.1
4.1
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Burns, T. and Stalker, G.M., 1961, 1966, The Management of Innovation,Tavistock Publications: London.a
Dickson, G.W., Leitheiser, R.L. and Wetherbe, J.C., 1984, `Key Information Systems Issues for the 1980's,' MIS Quarterly, September (8:3), pp. 135±159.
Chandler, A.D., 1962, Strategy and Structure: Chapters in the History of the Industrial Enterprise, M.I.T. Press: Cambridge, MA.
Dennis, A.R., George, J.F., Jessup, L.M., Nunamaker, J.F. and Vogel, D.R., 1988, `Information Technology to Support Electronic Meetings,'
MIS Quarterly, December (12:4), pp. 591±624.
Huber, G.P., 1984, `The Nature and Design of Post-Industrial Organizations,' Management Science, August (30:8), pp. 928±951.
McGrath, J.E., 1984, Groups: Interaction and Performance, Prentice-Hall, Englewood Cliffs, NJ.
Nelson, R.R., 1982, An Evolutionary Theory of Economic Change, Belknap Press of Harvard University Press: Cambridge, MA.
Parsons, 1983, `Information Technology: A New Competitive Weapon,' Sloan Management Review, Fall (25:1), pp. 3±14.
Sprague, R.H., 1980, `A Framework for the Development of Decision Support Systems,' MIS Quarterly, December (4:4), pp. 1±26.
Galbraith, J.R., 1977, Organization Design, Addison-Wesley Publishing Company: Reading, MA.
Mintzberg, H., 1979, The Structuring of Organizations: A Synthesis of the Research, Prentice-Hall: Englewood Cliffs, NJ.
Bonczek, R.H., Holsapple, C.W. and Whinston, A.B., 1981, Foundations of Decision Support Systems, Academic Press: New York.
Rockart, J.F., 1979, `Chief Executives Define their Own Data Needs,' Harvard Business Review, March±April (57:2), pp. 81±91.
Rockart, J.F. and Flannery, L.S., 1983, `The Management of End User Computing,' Communications of the ACM, October (26:10), pp. 776±784.
Argyris, C. and Schon, D.A., 1978, Organizational Learning, Addison-Wesley: Reading, MA.
Benson, D.H., 1983, `A Field Study of End User Computing: Findings and Issues,' MIS Quarterly, December (7:4), pp. 35±45.
Johnston, H.R. and Carrico, S.R., 1988, `Developing Capabilities to Use Information Strategically,' MIS Quarterly, March (12:1), pp. 37±48.
Davis, F.D., 1989, `Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology,' MIS Quarterly, September (13:3),
pp. 319±340.
Kraemer, K.L. and King, J.L., 1988, `Computer-Based Systems for Cooperative Work and Group Decision Making,' ACM Computing Surveys,
June (20:2), pp. 115±146.
Nunamaker, J.F., Applegate, L.M. and Konsynski, B.R., 1987, `Facilitating Group Creativity: Experience with a Group Decision Support System,'
Journal of Management Information Systems, Spring (3:4), pp. 5±19.
Markus, M.L. and Robey, D., 1988, `Information Technology and Organizational Change: Causal Structure in Theory and Reseach,' Management Science,
May (34:5), pp. 583±598.
Kahneman, 1979, `Prospect Theory: An Analysis of Decision Making Under Risk,' Econometric, March (47:2), pp. 263±291.
Tversky, 1974, `Judgement Under Uncertainty: Heuristics and Biases,' Science, (185), pp. 1124±1131.
Eisenhardt, K.M., 1989, `Building Theories from Case Study Research,' Academy of Management Review, (14:4), pp. 532±550.
Nolan, R.L., 1979, `Managing the Crises in Data Processing,' Harvard Business Review, March±April (57:2), pp. 115±126.
Total
86±95
Table 7
Additional publications averaging four or more citations per year since publication (listed by frequency)
1986±1995
Citations
per year
DeLone, W. H. and McLean, E. R., 1992, `Information Systems Success: The Quest for the Dependent Variable,' Information Systems Research,
March (3:1), pp. 60±95.
Huber, G.P., 1990, `A Theory of the Effects of Advanced Information Technologies on Organizational Design, Intelligence, and Decision Making,'
Academy of Management Review, (15:1), pp. 47±71.
Nunamaker, J.F., Dennis, A.R., Valacich, J.S., Vogel, D.R. and George, J.F., 1991, `Electronic Meeting Systems to Support Group Work,'
Communications of the ACM, July (34:7), pp. 40±61.
Davis, F. D., Bagozzi, R. P. and Warshaw, P. R., 1989, `User Acceptance of Computer Technology: A Comparison of Two Theoretical Models,'
Management Science, August (35:8), pp. 982±1003.
Davenport, T.H., 1993, Process Innovation: Reengineering Work Through Information Technology, Harvard Business School Press: Boston.
Doll, W. J. and Torkzadeh, G., 1988, `The Measurement of End-User Computing Satisfaction,' MIS Quarterly, June (12:2), pp. 259±274.
Jarvenpaa, S.L., 1989, `The Effect of Task Demands and Graphical Format on Information Processing Strategies,' Management Science,
March (35:3), pp. 285±303.
DeSanctis, G. and Poole, M.S., 1994, `Capturing the Complexity in Advanced Technology Use: Adaptive Structuration Theory,'
Organization Science, May (5:2), pp. 121±147.
Hartwick, J. and Barki, H., 1994, `Explaining the Role of User Participation in Information System Use,' Management Science, April (40:4), pp. 440±465.
Gallupe, R. B. and DeSanctis, G., 1988, `Computer-Based Support for Group Problem-Finding: An Experimental Investigation,' MIS Quarterly,
June (2:2), pp. 277±296.
Zuboff, S., 1988, In the Age of the Smart Machine: The Future of Work and Power, Basic Books: New York.
Straub, D. W., 1989, `Validating Instruments in MIS Research,' MIS Quarterly, June (13:2), pp. 147±169.
Malone, T.W., Grant, K.R., Turbak, F.A., Brobst, S.A. and Cohen, M.D., 1987, `Intelligent Information-Sharing Systems,' Communications
of the ACM, May (30:5), pp. 390±402.
Siegel, J., 1986, `Group Processes in Computer-Mediated Communication,' Organizational Behavior and Human Decision Process, (37:2), pp. 157±187.
Nunamaker, J.F., 1989, `Experience at IBM,' Decision Support Systems, (5:2), pp. 183±196.
24
8
30
6
24
6
35
5.8
11
35
30
5.5
5
5
5
5
5
33
5
4.7
32
26
34
4.6
4.3
4.25
37
24
4.1
4
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Publication
69
70
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
Table 8
Journal distribution of the most cited articles, 1986 to 1995
Journal
Number of articles
MIS Quarterly
Management Science
Communications of the ACM
Harvard Business Review
Academy of Management Review
Sloan Management Review
Nine other journals
12
11
6
5
2
2
Once each
As shown in Table 9, 97 authors were involved in
writing the 76 publications. One author, Nunamaker,
was involved in four of the publications. Five authors,
DeSanctis, Huber, Ives, Porter, and Simon were
involved with three of the publications. Sixteen
authors were involved with two of the publications
and 75 authors were involved with one of the publications.
Of the 76 publications, 17 were published between
1955 and 1975 and 14 more were published between
1976 and 1980. As might be expected, over half were
Table 9
Authors of the top cited articles, 1986±1995
Author
Number of articles
Nunamaker
DeSanctis
Huber
Ives
Porter
Simon
Davis, F
Dennis
Galbraith
Gallupe
George
Konsynski
March
Markus
Mintzberg
Olson
Rockart
Scott Morton
Sprague
Vogel
Wetherbe
Williamson
75 other authors
4
3
3
3
3
3
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
Once each
published in the 1980s: 20 were published between
1981 and 1985 and 20 more were published between
1986 and 1990. The last ®ve were published between
1991 and 1995.
6. Discussion
6.1. The traditional classics
From the early books of readings and Culnan's
studies, 19 individual publications were identi®ed
as classic. This list includes:
1. Ackoff, 1967.
2. Alter, 1980.
3. Anthony, 1965.
4. Benbasat and Dexter, 1979.
5. Benbasat and Schroeder, 1978.
6. Churchman, 1968.
7. Churchman, 1971.
8. Churchman and Schainblatt, 1965.
9. Dearden, 1972.
10. Dickson, Senn, and Chervany, 1977
11. Davis, G.B., 1974.
12. Gorry and Scott Morton, 1971.
13. Keen and Scott Morton, 1978
14. Kriebel, 1972.
15. Lucas, 1975.
16. Mason, 1969.
17. Mason and Mitroff, 1973.
18. Zmud, 1978.
19. Zmud, 1979.
7. The new classics
From 1986 to 1995, a new-breed of classic IS
publication was identi®ed. Sixty-one publications
meet the classic criteria. The other 15 works, if they
continue to be cited at the same rate they have been
since their publication, will be new classics. Of particular interest was the ratio of 47 journal articles and 29
books. Books typically consist of many more pages
than articles.
The list of traditional classics combined with the list
of new classics represents 91 individual publications.
Identi®ed over about a 25 year period, approximately
four works per year were represented.
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
7.1. Super classics
From the classic publications, some can be identi®ed as `super classic'. de Solla Price [14] refers to the
concept but fails to provide a de®nition. The inference
was that a super classic had to remain classic over time
versus the `well-known phenomenon of papers being
considered obsolescent after a decade'. Some articles
meet this criteria of super classics. The citing of
Ackoff, 1967 has fallen off over the years (averaging
two citations per year from 1986 to 1995), however, its
commanding presence in the early years and from
1972 to 1985 makes it the leading candidate for the
®rst super classic in the IS ®eld.
Four publications were present as classic in the
®ndings from Culnan's studies and from the analysis
of citations from 1986 to 1995: Davis, 1974 (and
Olson, M.H., 1985); Gorry and Scott Morton, 1971;
Keen and Scott Morton, 1978; and Zmud 1979. Therefore, a total of ®ve publications appear to meet the
criteria of super classic because of their sustained
citation visibility over time.
Other publications should perhaps be considered
super classic because of the number of citations over a
shorter period of time. This is also termed the `immediacy factor' or `bunching' discussed by de Solla
Price. For these publications, if the total been spread
out over the 24 year period from 1972 to 1995, they
would have averaged four citations per year, or a total
of 96 citations. These eight publications were:
1.
2.
3.
4.
5.
6.
7.
8.
Churchman, 1971.
DeSanctis and Gallupe, 1987.
Ives, Olson, and Baroudi, 1983.
Ives and Olson, 1984.
Nunnally, 1967, 1978.
Porter, 1980.
Sprague and Carlson, 1982.
Thompson, 1967.
A review of Table 7 reveals several publications, such
as DeLone and McLean (1992), that may sustain
citation visibility over a longer period of time. They
are worthy of note for future development.
7.2. Uses for the findings
Findings from this study can be used to develop
reading lists for doctoral students and by IS research-
71
ers to identify gaps in their knowledge. It can also be
used as a surrogate for the entire IS ®eld, allowing
researchers to pre-test their theories on a small sample
of the most visible works before examining a greater
quantity of literature. For instance, a follow up of the
points of work and reference studies conducted by
Culnan and Swanson [9] and Cheon et al. [4] could be
conducted using the ®nding of this study.
Third, these ®ndings can be used to update past
studies which have identi®ed content patterns in the IS
®eld or methodological evolution of the ®eld. These
type of studies could be follow ups to the intellectual
structure mapping of MIS and intellectual development studies conducted by Culnan. The underlying
disciplines impacting the IS ®eld can be analyzed by
examining the base discipline of these publications.
Invisible colleges (corresponding groups of active and
knowledgeable people in any particular research ®eld
corresponding [15]), can sometimes be identi®ed by
performing co-citation analysis using the 97 authors
who were involved with the 76 publications.
It also provides a means to examine how IS research
matches needs identi®ed by practitioners during the
same time period, in studies such as Neiderman et al.
[26] and Brancheau and Wetherbe [3]. Finally, further
study of these classic articles could reveal the characteristics that make an article or book, a `classic'
(similar to [12,13]).
8. Conclusions
With all the new material being published, perhaps
some of the older works are being pushed aside and
not included in the curriculum of Ph.D. students and
young faculty. Identifying these `classics' allows the
neophyte to ®nd and ®ll the gaps in his or her knowledge base. A greater understanding and appreciation
for the classic literature results in greater understanding and comprehension of the current literature.
A citation analysis was performed on the 118,364
references attached to the 3752 articles published in
nine top IS journals during the time period from 1986
to 1995. While citation analysis is not a perfect means
of identifying classic publications, it provides an
extensive method of determining highly visible works,
or citation classics. Ninety-one were identi®ed by
combining the results of this analysis with previous
72
K.A. Walstrom, L.N.K. Leonard / Information & Management 38 (2000) 59±72
®ndings and 13 super classic publications were identi®ed.
References
[1] M.J. Adler, C. Van Doren, How to Read a Book: The Classic
Guide to Intellectual Reading, Simon & Schuster, New York,
1972.
[2] M.J. Adler, P. Wolff, A General Introduction to the Great
Books and to a Liberal Education, Encyclopedia Britannica,
Chicago, 1959.
[3] J.C. Brancheau, J.C. Wetherbe, Key issues in information systems management, MIS Quarterly 11 (1), 1987, pp.
23±45.
[4] M.J. Cheon, C.C. Lee, V. Grover, Research in MIS Ð points
of work and reference: a replication and extension of the
Culnan and Swanson study. DATA BASE (1992) 21±29.
[5] D.E. Chubin, A.L. Porter, F.A. Rossini, `Citation classics'
analysis: an approach to characterizing interdisciplinary
research, Journal of the American Society for Information
Science 35 (6), 1984, pp. 156±172.
[6] R.J. Coleman, M.J. Riley, MIS: Management Dimensions,
Holden-Day, San Francisco, CA, 1973.
[7] M.J. Culnan, The intellectual development of management
information systems, 1972±1982: a co-citation analysis,
Management Science 32 (2), 1986, pp. 156±172.
[8] M.J. Culnan, Mapping the intellectual structure of MIS,
1980±1985: a co-citation analysis, MIS Quarterly 11 (3),
1987, pp. 341±353.
[9] M.J. Culnan, E.B. Swanson, Research in management
information systems, 1980±1984: points of work and
reference, MIS Quarterly 10 (3), 1986, pp. 289±301.
[10] G.B. Davis, The knowledge and skill requirements for the
doctorate in IS, in: Proceedings of the First International
Conference on Information Systems, 1980, pp. 174±186.
[11] G.B. Davis, G.C. Everest, Readings in Management Information Systems, McGraw-Hill, New York, 1976.
[12] M.S. Davis, `That's classic!' the phenomenology and rhetoric
of successful social theories, Philosophy of Social Science
(1986) 285±301.
[13] M.S. Davis, `That's interesting!' towards a phenomenology of
sociology and a sociology of phenomenology, Philosophy of
Social Science (1971) 309±344.
[14] D.J. de Solla Price, Little Science, Big Science, Columbia
University Press, New York, 1963.
[15] D.J. de Solla Price, Ethics of scientific publication, Science,
147 (1964) 655±657.
[16] D.J. de Solla Price, Networks of scientific papers, Science,
149 (1965) 510±515.
[17] G.W. Dickson, Management information systems: evolution
and status, Advances In Computers 20, 1981, pp. 1±37.
[18] V.T. Dock, V.P. Luchsinger, W.R. Cornette, MIS: A Managerial Perspective, Science Research Associates, Chicago, 1977.
[19] J. Erskine, The Delight of Great Books, The Bobbs-Merrill
Company, Indianapolis, IN, 1928.
[20] C. Fadiman, The Lifetime Reading Plan, 3rd Edition, Harper
& Row, New York, 1988.
[21] E. Garfield, Mapping the world of science: is citation analysis
a legitimate evaluation tool? In: D.N. Jackson, J.P. Rushton
(Eds.), Scientific Excellence: Origins and Assessment, Sage,
Newbury Park, CA, 1987.
[22] M.L. Gillenson, J.D. Stutz, Academic issues in IS: journals
and books, MIS Quarterly 15 (4), 1991, pp. 447±452.
[23] B.C. Hardgrave, K.A. Walstrom, Forums for MIS scholars,
Communications of the ACM 40 (11), 1997, pp. 119±124.
[24] C.L. Hind, One Hundred Best Books? G.P. Putnam's Sons,
New York, 1928.
[25] J. Lee, Have You Read 100 Great Books? Jasper Lee
Company, New York, 1946.
[26] F. Niederman, J.C. Brancheau, J.C. Wetherbe, Information
systems management issues for the 1990s, MIS Quarterly 15
(4), 1991, pp. 475±495.
[27] C. Oppenheim, S.P. Renn, Highly cited old papers and the
reasons why they continue to be cited, Journal of the American
Society for Information Science 29, 1978, pp. 225±231.
[28] D.H. Sanders, Computers and Management: In a Changing
Society, 2nd Edition, McGraw-Hill, New York, 1974.
[29] D.H. Sanders, Computers and Management, McGraw-Hill,
New York, 1970.
[30] K.A. Walstrom, B.C. Hardgrave, R.L. Wilson, Forums for
management information systems scholars, Communications
of the ACM 38 (3), 1995, pp. 93±107.
Kent A. Walstrom is an Associate
Professor of Information Systems at
Illinois State University. He received
his PhD from Oklahoma State University. Dr. Walstrom's research interests
include executive information systems,
the history of the information systems
field, electronic commerce, and the use
of information systems to support
leadership. Previous publications include
articles in Communication of the ACM,
Computers and Operations Research, Information & Management,
Journal of Computer Information Systems, among others.
Lori N.K. Leonard is an Assistant
Professor of Management Information
Systems at the University of Tulsa. Dr.
Leonard received her PhD from the
University of Arkansas and is an active
member of the Decision Sciences Institute. Her research interests include
electronic commerce, electronic data
interchange, ethics in computing, simulation, and data warehousing. Her publications have appeared in Journal of
Computer Information Systems, Proceedings of the National
Decision Sciences Institute, Proceedings of the Decision Sciences
Institute Southwest Region, and Proceedings of Americas Conference on Information Systems.