62 3. The Code Model Decoded
the conditions or seen the necessity of Shannon’s tightly constructed definitions. As Reddy notes, attempts to apply the theory outside its original scope “required a very clear
understanding, not so much of the mathematics of the theory, but rather of the conceptual foundations of the theory. By and large, these attempts were all accounted to be failures”
Reddy 1979 :303–304. Of particular concern are Shannon’s definition of ‘information’
and ‘message’, the associated requirements regarding the message, and his requirements regarding the transmitter, receiver, and the concept of code. These will be addressed in
turn.
3.2.3.1.1. Information
From Shannon’s perspective as an engineer and mathematician, the problem of communication was a matter of evaluating raw information, that is to say, information of
any sort, without concern for how it related to a context, or how it may be used or employed. As such, he required a narrowly circumscribed technical definition of
information, which was quite distinct from its common usage.
Undoubtedly, some of the confusion arises from the fact that Shannon never actually defines
information, rather, he defines how he will measure it:
If the number of messages in the set [of possible messages] is finite then this number or any monotonic function of this number can be regarded as a measure of the information
produced when one message is chosen from the set, all choices being equally likely. As was pointed out by Hartley the most natural choice is the logarithmic function. Although this
definition must be generalized considerably when we consider the influence of the statistics of the message and when we have a continuous range of messages, we will in all cases use an
essentially logarithmic measure.
Shannon 1948 :379,
1949 :3
Considering Shannon’s initial audience, his neglect in defining ‘information’ is not surprising. As he states in the introduction to his paper, Shannon depends upon both
Nyquist 1924
, 1928
and Hartley 1928
, both of whom precede him with a similar use of the term. As with Shannon, Nyquist and Hartley provide rather technical definitions
which sometimes “muddy the water” for those unfamiliar with their topic. Colin Cherry conveniently summarizes the issues:
In 1924, Nyquist in the United States and Küpfmüller in Germany simultaneously stated the law that, in order to transmit telegraph signals at a certain given rate, a definite bandwidth
is required, a law which was expressed more generally by Hartley in 1928. Hartley showed that in order to transmit a given “quantity of information,” a definite product bandwidth x
time is required. We may illustrate this law in the following way. Suppose we have a gram- ophone record of speech; this we may regard as a “message.” If played in normal speed, the
message might take 5 minutes for transmission and its bandwidth might range 100–5000 cycles per second. If the speed of the turntable were doubled, the time would be halved; but
also the pitch and hence the bandwidth would be doubled. However, Hartley went further and defined information as the successive selection of signs or words from a given list, rejecting
all “meaning” as a mere subjective factor it is the signs we transmit, or physical signals; we do not transmit their “meaning”. He showed that a message of N signs chosen from an
“alphabet” or code book of S signs has S
N
possibilities and that the “quantity of information” is most reasonably defined as the logarithm, that is, H=N log S.
Cherry 1966 :43–44
3. The Code Model Decoded 63
As stated previously, Shannon’s theory is a theory of probability. In considering what constitutes ‘information’, he approaches it from the perspective of probability. Strictly
speaking, he cannot know in any absolute sense whether or not the “shape” of an incoming signal is intentional. He can, however, evaluate the likelihood that the shape of
that incoming signal is attributable to chance, which in this context is described as electromagnetic noise. In this situation, the ability to successfully select signs, words, or
characters from the message or signal alphabets, is the primary concern. Any meaning associated with those units is irrelevant to the immediate problem.
This understanding of information is also the basis of Shannon’s stipulation on monotonic functions. A monotonic function is a function which has “the property either
of never increasing or of never decreasing as the independent variable increases” Mish
1983 :769. In common terms, the relationship between function and message must be
unwavering if the receiver is to attempt a reconstruction of the message from that function.
The following quotations provide some additional insight into this definition of information:
It doubtless seems queer, when one first meets it, that information is defined as the logarithm of the number of choices.
Weaver 1949b :101
Information is, we must steadily remember, a measure of one’s freedom of choice in selecting a message. The greater this freedom of choice, and hence the greater the information, the
greater is the uncertainty that the message actually selected is some particular one. Thus greater freedom of choice, greater uncertainty, greater information go hand in hand.
Weaver 1949b
:108–109 Information is defined as the ability to make nonrandom selections from some set of
alternatives. Reddy 1979
:303 The technical notion of ‘information,’ as a measure of uncertainty, should be sharply distin-
guished from the intuitive notion of information, as concerning significance or meaning. A randomly chosen and hence meaningless stream of letters will generate more information
than a stream of letters composing a meaningful text, since it is far more uncertain that is, it is far less predictable.
Chater 1994 :1685
Often the messages to be transmitted have meaning: they describe or are related to real or conceivable events. This, however, is not always the case. In transmitting music, for example,
the meaning, if there is any, is much subtler than in the case of a verbal message. In some situations the engineer is faced with transmitting a totally meaningless sequence of numbers or
letters. In any case, meaning is quite irrelevant to the problem of transmitting the information.
Gallager 1988 :569
Shannon did not attempt to address the nature of any meaningful intention which might have initiated the communication process, nor was he interested in how the struc-
ture or semantic aspects of any underlying message might relate to some imagined or real world. His intended audience of electrical and computational engineers seems to have
understood this position, but most others seem to have neglected this technical definition, thereby confusing the technical sense of ‘information’ with its more common usage.
64 3. The Code Model Decoded
Despite distinctions between the information theoretic notion of information and the common notion, which includes meaning, many theoreticians have attempted to apply the
theory to semantics Chater 1994
:1687. Even Shannon’s colleague Warren Weaver seems to have struggled with the necessary exclusion of semantics. The fact that these
efforts have met little success should offer no surprise. As Langacker notes, “One has no guarantee that a seemingly apt metaphor will actually prove appropriate and helpful when
pushed beyond the limited observations that initially inspired it”
1991 :507. Shannon
himself commented upon such misapplications during a 1989 interview. As the inter- viewer John Horgan records: “Shannon himself doubted whether certain applications of
his theory would come to much. ‘Somehow people think it can tell you things about meaning,’ he once said to me, ‘but it can’t and wasn’t intended to’”
Horgan 1996 :
207– 208. Everett Rogers similarly comments: “Shannon saw information theory as limited
to engineering communication and warned the scientific world against applying it more
broadly to all types of human communication. Nevertheless, communication scholars have not paid much attention to Shannon’s warning”
Rogers 1994 :428.
3.2.3.1.2. Message
