Designing the sentences Methodology

People may be very poor at translation or they may simply “not like translation”. It was largely to address these criticisms that the “question and answer” method Casad, 1974 was developed. Almost all early intelligibility studies using Voegelin and Harris’ method such as Hickerson et al., 1952 attempted to test mutual intelligibility of indigenous languages in the Americas. Their subjects were required to translate recorded texts into English which was usually their L2 and sometimes their L3. In these cases, Wolff’s criticism is probably justified. Translating an unfamiliar dialect of an indigenous American language into a language which is entirely unrelated and typologically extremely different English is no small task, even for those who understand well the dialect on which they are being tested and who have high proficiency in English. The experience of the Sui survey, however, suggests that a translation or, as we prefer to call it, a L2 retelling method can work extremely well provided that certain conditions are met. Firstly, our subjects were required to retell the text in a language with which they were very familiar, their local dialect of Southwestern Mandarin. Secondly, Sui and Southwestern Mandarin are typologically very similar. Both are SVO, uninflected and largely monosyllabic, and both rely on a rich inventory of aspect particles. Southwestern Mandarin spoken in the Sui area is itself heavily influenced by Sui. As we administered the tests, our impression was that most people found the translation task relatively straightforward. Thirdly, the investigator conducting the tests was herself a native speaker of Southwestern Mandarin Guizhou dialect. The entire testing procedure was conducted in Southwestern Mandarin and, we feel, our subjects were far more comfortable with the “L2 retelling” approach than our subjects in previous surveys who were faced with a question and answer method. In rare cases where the subject was not confident retelling in the local Chinese dialect, they were permitted to retell in their L1, in which case it was translated and scored in the same way as Kluge’s 2010 method. Fourthly, we were sensitive to the subjects’ proficiency in Chinese dialect when we scored the tests. A wide leeway was given in interpreting their retellings so as not to penalise an inability to express something in Chinese see section 8.4.6 for an explanation of our scoring methodology. Finally, we asked every subject to provide a judgment of their own comprehension of the texts. In cases where their own judgment was vastly different from their measured intelligibility score and where the interviewer felt that a low Chinese proficiency had adversely affected their results, their scores were discounted see section 8.4.6.4. Of course, our methodology is far from perfect and has many flaws. These will become apparent as the chapter progresses. Overall, though, we are more confident with the results using this method than results collected during past surveys using a story question and answer method and a story retelling L1 method.

8.4 Methodology

8.4.1 Designing the sentences

Sentences were designed in Chinese and based upon those created by Lam et al., 2010 for an intelligibility survey of Lalo dialects in Dali prefecture, Yunnan. It was important that the content should be familiar to our subjects. Therefore we discussed the sentences with a mother tongue Sui speaker and adapted and supplemented them accordingly. In total we came up with 45 sentences, most of them containing content closely related to everyday life in a Sui village. For example, “He killed a pig and extracted oil from its fat; he got enough oil to last him one month.” appendix D, group A, sentence 1 One rarely comes across pig slaughtering and oil extraction in the city; however, almost every Sui household raises pigs and it is extremely common to extract pig’s oil for cooking. All 45 sentences were relatively long and consisted of multiple clauses. In this way, some context was provided which hopefully mitigated against subjects scoring poorly due to rapid switching of topic and semantic domain from sentence to sentence. We tried to ensure that the sentences were connected and complete and followed patterns of logic common among rural Sui speakers. We did our utmost to maximise the number of semantic domains, word types and grammatical structures covered by the 45 sentences. We included a wide range of kinship terms, numbers, colours, animals, plants, body parts, verbs of motion and so on. Our final set of 42 sentences see below contained an average of 309 different vocabulary items 3 , of which an average of 174 occurred only once in the entire set and 58 occurred only twice. Most of our sentences consisted of straightforward narrative discourse, for example, “His aunt fell ill, but after taking some herbal medicine she got better and was able to start working in the fields again.” appendix D, group A, sentence 2 However, we also composed a few more complex sentences such as, “If you want to go up onto the roof, there is a long staircase on the left-hand side of the house; you can go up from there.” appendix D, group A, sentence 14. This sentence contains a conditional clause and is phrased in the second person. It is therefore more challenging for the test subject. More challenging sentences such as this one always came towards the end of the test so that the test subjects had a chance to gain confidence by retelling more simple sentences first.

8.4.2 Translating and recording the sentences