Ve racity Check

81 Ve racity Check

Veracity checks (Pine, 1995; Green & Stutzman, 1986) are designed to help detect insufficient effort responding (IER). This occurs when respondents may provide careless ratings or may lack motivation to accurately respond to the survey (Huang et al., 2012) which may be indicated when a JAQ has many items or when SME respondents are suspected not to be engaged in the process. Bogus task items were inserted into the

task statement section of each JAQ to detect IER. If a respondent endorses a bogus item as I >eing part of the job, the response may indicate they are not paying attention to the items (i.e., skimming or not reading the items), or that they are making similar

enc orsements for all items regardless of content. The bogus items in the present study wefe not endorsed by many SME respondents. This could be because they were too

obviously different from the actual tasks or most respondents were answering accurately. The JAQ for the position of sergeant contained two bogus task statements. The first bogus item, “Examine patient to obtain information on medical condition and sur *ical risk,” was endorsed on all scales by five respondents and on one scale and by four different respondents. The second bogus item, “Study animals in their natural

hal itats, assessing effects of the environment and industry on animals,” was endorsed on ont scale (consequence of error) by one respondent.

The JAQ for the position of lieutenant contained two bogus task statements. The firsjt, “Perform animal grooming duties such as washing, brushing, clipping, and trimming coats, cutting nails” was not endorsed by any respondents. The second, “Start eng ines, operate controls, and pilot airplanes to transport passengers, mail, or freight,”

As with the JAQs for sergeant and lieutenant, the JAQ for the position of captain contained two bogus task statements. The first, “Conduct research to extend mathematical knowledge in traditional areas, such as algebra and geometry” was

en( orsed by one participant on all five scales. The second, “Develop ways of altering soi s to suit different types of crops,” was not endorsed by any respondents.

Respondents who endorsed bogus items were identified, but the small number of respondents who endorsed bogus items did not allow for an analysis of the individual traits of those SME respondents that may have been related to the endorsement of such items (i.e., based on experience level, performance ratings, or demographics). The reliability of the job analysis ratings of tasks and KSAs on all five scales was examined usi lg coefficient alpha with and without respondents who endorsed bogus items. A list of the obtained reliability coefficients for the ratings on all scales and both the task

statements and KSAS for the JAQ for each job title with and without the removal of respondents endorsing any bogus item on any scale is listed in Table 14.

The reliability estimates were compared using Fisher’s r-to-Z transformation folllowed by a t-test for significant differences. Fisher’s r-to-Z transformation is a method of c onverting reliability coefficients to a standard score so that they may be compared to onej another (Ferguson, 1981). The formula for Fisher’s M o-Z transformation is:

zr = (l/2)[loge(l+r) - loge(l-r)]. The comparison of coefficient alphas between bogus responders and others

showed no significant differences for the job title of sergeant (z = 0 . 66 , p = .26), lieutenant (z = 0.16,/? = 0.44), or captain (z = 0.07,p = 0.47). Including respondents who

(th dependent variable in the present research). Since no significant differences were defected and the alphas were numerically identical, exclusion of bogus responders would have had no impact on study results. Therefore, responses from all SMEs were included in i he subsequent analysis.

Table 14 Coefficient Alpha fo r Each JAQ with and without Bogus Item Endorsers