abundant in content to produce meaningful measures of intelligibility. Secondly, comprehension scores resulting from this type of test cannot be meaningfully compared with each other since every story is
different in content. Thirdly, participants are often uncomfortable with a “question and answer” type test, especially when administered by outsiders. Fourthly, only ten questions are asked about each text.
The notion that correct responses to such a limited set of stimuli represent “full comprehension” of the text is dubious. Finally, preparation of this type of test is complex and cumbersome, involving translation
of the questions into every dialect on which the text will be tested and insertion into every recording in the appropriate place.
8.3.2 Word recognition tests
Various types of test have been used for investigating intelligibility of isolated words. Schüppert and Gooskens 2010 designed a test in which each subject listened to a recording of fifty individual words in
the target dialect, in this case either Danish or Swedish. As they heard each word they were presented with four pictures. The subject had to match the word with the correct picture.
Tang and van Heuven 2009 designed a test in which each subject had to correctly categorise individual words such as “apple” or “duck” which they heard in a series of fifteen different dialects.
There were ten possible semantic categories to choose from such as body parts, vegetables, textiles, verbs of action and animals.
Word recognition tests have the advantage of being quick and easy to prepare and administer. A large number of subjects and different dialects can be tested. However, they do not test language in
context and could therefore be deemed “unnatural”. People are used to hearing language in connected speech with a specific context. When comparing their word recognition test with a type of sentence
intelligibility test see below, Tang and van Heuven 2009:727 concluded that “the sentence- intelligibility test yields more credible results than the word-intelligibility test … The results show that
sentence-intelligibility reflects traditional dialect taxonomy better than word-intelligibility does.”
8.3.3 Sentence completion tests
The “sentence-intelligibility” test referred to by Tang and van Heuven 2009 was in effect a “sentence completion” test based on a version of the SPIN Speech Perception In Noise test designed by Kalikow
et al., 1977 for testing comprehension of English. In Tang and van Heuven’s version, the subjects were played a series of incomplete sentences. Their task was to complete each sentence with a single word.
The test designers tried to ensure that this word always a noun was predictable from the content of the sentence. An example sentence is: “Our seats were in the second row.” A set of sixty sentences was
translated into all fifteen Chinese dialects to be tested. Tang and van Heuven used a “Latin square design” Box et al., 1978 to rotate sentences so that every listener listened to each sentence in a
different dialect, and so that all sentences in all dialects were listened to by each entire group of subjects. This type of test has several advantages. Firstly, it tests comprehension of language in context. If the
incomplete sentence is not completely understood, there is no way that the subject can provide the correct answer. Secondly, it tests a wide range of content. The sixty sentences covered a large number of
domains and contexts, far more than could a single story of the type used by Casad and others see 8.2.1 above. Finally, it is easy to score and the results for each dialect are directly comparable to each other.
There are also several disadvantages to such a test. Firstly, the subjects are not presented with natural discourse. Listening to incomplete sentences is artificial. Secondly, and perhaps more importantly, the
subjects could potentially achieve low scores despite a high level of comprehension simply because their “logic” or “way of thinking” suggested a different way of completing the sentences to the one the test
designers had envisaged.
8.3.4 Sentence translation tests
