Designing the sentences Methodology
People may be very poor at translation or they may simply “not like translation”. It was largely to address these criticisms that the “question and answer” method Casad, 1974 was developed.
Almost all early intelligibility studies using Voegelin and Harris’ method such as Hickerson et al., 1952 attempted to test mutual intelligibility of indigenous languages in the Americas. Their subjects
were required to translate recorded texts into English which was usually their L2 and sometimes their L3. In these cases, Wolff’s criticism is probably justified. Translating an unfamiliar dialect of an
indigenous American language into a language which is entirely unrelated and typologically extremely different English is no small task, even for those who understand well the dialect on which they are
being tested and who have high proficiency in English.
The experience of the Sui survey, however, suggests that a translation or, as we prefer to call it, a L2 retelling method can work extremely well provided that certain conditions are met. Firstly, our
subjects were required to retell the text in a language with which they were very familiar, their local dialect of Southwestern Mandarin. Secondly, Sui and Southwestern Mandarin are typologically very
similar. Both are SVO, uninflected and largely monosyllabic, and both rely on a rich inventory of aspect particles. Southwestern Mandarin spoken in the Sui area is itself heavily influenced by Sui. As we
administered the tests, our impression was that most people found the translation task relatively straightforward.
Thirdly, the investigator conducting the tests was herself a native speaker of Southwestern Mandarin Guizhou dialect. The entire testing procedure was conducted in Southwestern Mandarin and,
we feel, our subjects were far more comfortable with the “L2 retelling” approach than our subjects in previous surveys who were faced with a question and answer method. In rare cases where the subject
was not confident retelling in the local Chinese dialect, they were permitted to retell in their L1, in which case it was translated and scored in the same way as Kluge’s 2010 method.
Fourthly, we were sensitive to the subjects’ proficiency in Chinese dialect when we scored the tests. A wide leeway was given in interpreting their retellings so as not to penalise an inability to express
something in Chinese see section 8.4.6 for an explanation of our scoring methodology. Finally, we asked every subject to provide a judgment of their own comprehension of the texts. In
cases where their own judgment was vastly different from their measured intelligibility score and where the interviewer felt that a low Chinese proficiency had adversely affected their results, their scores were
discounted see section 8.4.6.4. Of course, our methodology is far from perfect and has many flaws. These will become apparent as
the chapter progresses. Overall, though, we are more confident with the results using this method than results collected during past surveys using a story question and answer method and a story retelling L1
method.