The Diagnostic Behavioral Assessment for

Research in Autism Spectrum Disorders 8 (2014) 362–375

Contents lists available at ScienceDirect

Research in Autism Spectrum Disorders

Jo u rn al h om ep ag e: h ttp ://ees .elsevier .co m /RASD/d efau lt.as p

The Diagnostic Behavioral Assessment for autism spectrum disorder—Revised: A screening instrument for adults with intellectual disability suspected of autism spectrum disorders

a , 1 Tanja , Sappok * , Isabell Gaul a , 1 , b Thomas Bergmann a , Isabel Dziobek , Sven c Bo¨lte , Albert Diefenbacher a , Manuel Heinrich a

a Evangelisches Krankenhaus Ko¨nigin Elisabeth Herzberge, Abteilung fu¨r Psychiatrie, Psychotherapie und Psychosomatik, Herzbergstr. 79,

b 10365 Berlin, Germany Cluster of Excellence Languages of Emotion, Freie Universita¨t Berlin, Habelschwerter Allee 45, 14195 Berlin, Germany

c Department of Women’s and Children’s Health, Center of Neurodevelopmental Disorders, Karolinska Institutet, SE-171 77 Stockholm,

Sweden

Article history: Given the strong association between intellectual disability (ID) and autism spectrum

Received 1 November 2013 disorder (ASD), standardized instruments for the assessment of ASD in adults with ID are

Received in revised form 16 December 2013 desirable. The Diagnostic Behavioral Assessment for ASD – Revised (DiBAS-R) is a DSM-5/ICD-

Accepted 26 December 2013 10 based caregiver-report screening tool that consists of 19 Likert-scaled items. This study

evaluated the item-validities, item-difficulties, item-variances, part-whole corrected item

Keywords: total-correlations, reliability, and the factorial, diagnostic, and convergent/discriminant

Autism spectrum disorder validities of the DiBAS-R in a clinical, adult ID sample (N = 219). Factor analysis yielded two

Intellectual disability consistent dimensions; i.e., social interaction/communication and stereotypy/rigidity/ Diagnostics

Psychometric properties sensory abnormalities. The diagnostic validity was adequate, as reflected by an area under

Adults the curve of 0.89 and balanced sensitivity and specificity values of 81%. The DiBAS-R total

scores were significantly correlated with the Social Communication Questionnaire (r = 0.52),

the Scale for Pervasive Developmental Disorders in Mentally Retarded Persons (r = 0.50), and

the Autism-Checklist (r = 0.59), while no significant correlation with the Modified Overt

Aggression Scale was observed. The interrater reliability was excellent (ICC = 0.88). These

findings indicate that the DiBAS-R is a promising and psychometrically sound instrument

for ASD screening of adults with ID.

ß 2014 Elsevier Ltd. All rights reserved.

1. Introduction Individuals with intellectual disability (ID) exhibit an increased risk for autism spectrum disorder (ASD), which is

associated with high rates of comorbid mental health problems and challenging behaviors ( Matson & Shoemaker, 2009;

McCarthy et al., 2010; Sappok, Bergmann, Kaiser, & Diefenbacher, 2010; Sappok, Budczies, Dziobek, et al., 2013 ). ASD is a

neurodevelopmental disorder characterized by impairments in social communication and repetitive, stereotyped behaviors

* Corresponding author. Tel.: +49 30 5472 4950; fax: +49 30 5472 2943.

1 t.sappok@keh-berlin.de , tanja.sappok@t-online.de (T. These Sappok). authors shared first authorship.

E-mail addresses:

1750-9467/$ – see front matter ß 2014 Elsevier Ltd. All rights reserved.

http://dx.doi.org/10.1016/j.rasd.2013.12.016

T. Sappok et al. / Research in Autism Spectrum Disorders 8 (2014) 362–375 363 and interests (DSM-5; American Psychiatric Association, 2013 ). Approximately 25% of people with ID exhibit comorbid ASD

( Fombonne, 2009; Kim, 2011; Sappok et al., 2010 ). While ASD onset occurs in early childhood, many ASD patients with

comorbid ID remain undiagnosed and untreated until adulthood ( Malfa, Lassi, Bertelli, Salvini, & Placidi, 2004; Sappok,

Diefenbacher, Budczies, et al., 2013 ). Diagnosing mental disorders or ASD in patients with ID remains challenging for a number of reasons. Firstly, individuals with ID are less able to report their inner experiences due to diminished speech comprehension and expressive abilities ( Balboni, Coscarelli, Giunti, & Schalock, 2013 ). Secondly, due to diagnostic overshadowing ( Reiss &

Szyszko, 1983 ), problem behaviors or mental disorders may be attributed to the ID itself rather than an additional

comorbid diagnostic entity. Thirdly, diagnostic substitution, i.e., diagnosing ASD rather than ID may occur ( King & Bearman, 2009; Shattuck, 2006; Weintraub, 2011 ). Fourthly, neurological disorders, such as sensory or motor impairments and epilepsy, may further hamper diagnostic clarification ( Matson & Shoemaker, 2009 ). Finally, currently,

many adults with ID have been raised in long-term mental institutions and/or have lost contact with their relatives ( Haberfellner, Grausgruber, Grausgruber-Berner, Ortmair, & Scho¨ny, 2004 ); thus, their medical histories are fragmented,

which produces difficulties in the proper diagnostic classification of ASD. On the background of DSM-5’s emphasis on a

lifetime perspective for diagnosing ASD, a thorough medical history with detailed and reliable early childhood information get even more important.

For this reason, ASD diagnostics for this group are mainly based on assessments of current behavior. Individuals

with ID, especially those with comorbid ASD, may behave differently when interacting with unfamiliar people or when

in an unfamiliar environment, such as a clinical setting ( Gerber et al., 2011; Kumin, 1994 ). Therefore, it is mandatory

that information from the private living environment, including information from close caregivers, be sought in the diagnostic process. Diagnostic clarification allows for more appropriate treatment options in ID and comorbid ASD that

include non-drug strategies and lead to improved mental health and quality of life ( Gordon et al., 2011; van Bourgondien, Reichle, & Schopler, 2003 ). Standardized, evidence-based instruments can support clinicians and

researchers in this diagnostic process; e.g., the Autism Diagnostic Observation Schedule (ADOS; Lord et al., 1989 ), the

Pervasive Developmental Disorder in Mental Retardation Scale (PDD-MRS; Kraijer, 2006; Kraijer & Melchers, 2003 ), the

Social Communication Questionnaire (SCQ; Berument, Rutter, Lord, Pickles, & Bailey, 1999; Rutter, Bailey, & Lord, 2001 ), the Autism Spectrum Disorder-Diagnostic Scale for Adults (ASD-DA; Matson, Boisjoli, Gonza´lez, Smith, &

Wilkins, 2007; Matson, Wilkins, Boisjoli, & Smith, 2008 ), and the Autism-Checklist (ACL; Sappok, Heinrich, & Diefenbacher, 2013 ). The ADOS is a semi-structured observational instrument that assesses social communicative abilities ( Lord et al.,

1989 ), while the ADI-R is a semi-structured parental interview that evaluates social interaction, communication, and

restrictive, repetitive behaviors and interests from childhood to adulthood ( Lord, Rutter, & Le Couteur, 1994 ). These two

instruments are frequently used in combination for diagnosing autism in children and they have recently been validated

for adults with ID ( Sappok, Diefenbacher, Budczies, et al., 2013 ). However, both measures are time-consuming and can

only be applied to a limited number of adults with ID ( Sappok, Diefenbacher, Budczies, et al., 2013 ). With increasing

severity of ID and comorbidity of ASD, the feasibility of the ADOS was reduced to 68%, while the applicability of the ADI-R

was even reduced to 37%, presumably due to loss of contact to close relatives. The SCQ is a screening instrument that

consists of 40 binary items that are rated by parents or close caregivers ( Berument et al., 1999; Rutter et al., 2001 ). There

are lifetime and current versions of the SCQ. The SCQ-lifetime has been found to have particularly poor specificity values

in cases of moderate and severe ID ( Sappok, Diefenbacher, Gaul, & Bo¨lte, in press ). Raising the cut-off value of the SCQ-

current improves diagnostic validity for this special group of patients, but the specificity remains rather low. The ASD-DA

is another diagnostic instrument that consists of 31 items in which the rater endorses as 0 for no impairment/not different,

or 1 for some impairment/different. The raters are instructed to compare the target person to an individual with a similar

age living in the community. A three-factor model was computed for the scale, which showed good psychometric properties ( Matson et al., 2007; Matson et al., 2008 ). The PDD-MRS assesses behaviors during daily routines and was

conceptualized for individuals with ID between the ages of 2 to 70 years ( Kraijer, 2006; Kraijer & Melchers, 2003 ). Although this test is considered a screening instrument, its completion takes approximately half an hour, and the test

must be administered by a specialist; e.g., a psychologist or psychiatrist. The Autism-Checklist (ACL) is an ICD-10-based

screening instrument for physicians. This test evaluates characteristic social interaction, communication, and stereotypies. The completion of this test requires approximately 10 min. As the sensitivity and specificity values of this test are 91 and 68%, respectively, the ACL is a suitable measure for adults with ID and suspected ASD ( Sappok, Heinrich, et al., 2013 ). In conclusion, there is a need for an ASD screening instrument that can be easily administered by

close caregivers without specific knowledge of ASD. For this purpose, we developed a 20-item questionnaire, the Diagnostic Behavioral Assessment for ASD (DiBAS) that is derived from the ICD-10 and DSM-5 criteria for ASD. In a pilot

study, the DiBAS was applied to 91 patients with ID and suspected ASD ( Sappok, Gaul, et al., 2014 ). Item validity analysis

revealed 8 items that did not differentiate sufficiently between individuals with and without comorbid ASD, and despite

the appropriate sensitivity of 83%, the specificity was low (64%). Thus, an item-revision of the DiBAS was recommended to

further improve its diagnostic validity. The invalid items were replaced by another 8 ICD-10/DSM-5-based questions to

replenish the DiBAS-Revised (DiBAS-R).

The aim of the present study was to examine the reliability and the factorial, diagnostic, and convergent/discriminate

validities of the DiBAS-R in screening for ASD in adult patients with ID on the item and scale levels.

T. Sappok et al. / Research in Autism Spectrum Disorders 8 (2014) 362–375

2. Material and methods

2.1. Setting and design

The study was conducted at a department of psychiatry specialized in mental health care for adults with ID in Berlin,

Germany. This service consists of an in- and outpatient unit and offers assessment and treatment for adults with ID and

mental disorders and/or severe challenging behaviors. Given this setting, all participants of this study have had an additional

mental or behavioral problem on admission. ASD assessment including the DiBAS-R was applied after remission of the acute

mental illness in the in- or outpatient service. Diagnostic classification including ASD and severity of ID was conducted in

accordance with the diagnostic research criteria for mental disorders proposed by ICD-10. The DiBAS-R was completed by a

close caregiver, i.e., a parent or a staff member of a residential home in outpatients and a psychiatric nurse or special needs

caregiver in inpatients. In a random fashion, the DiBAS-R was handed out twice to assess interrater reliability by two ratings

from different raters on the same individual.

ASD diagnoses were assigned by a multidisciplinary team consensus conference according to the ICD-10 diagnostic

research criteria for autism or atypical autism [F84.0/F84.1]. If no information about the developmental history could be

obtained, atypical autism was diagnosed. As differentiation between autism (F84.0) and atypical autism (F84.1) almost

always depended on the availability of early childhood information (F84.10), we subsumed these diagnoses under the

broader term of ASD; however, participants clinically appeared to have severe forms of classical autism.

The multidisciplinary team consisted of at least two psychiatrists, a clinical psychologist, a special needs caregiver,

therapists, and nursing staff that was experienced in the fields of ID and ASD. The diagnoses were based on all available

information, including medical histories, psychiatric and physical examinations, structured video-based behavior analyses

across a variety of contexts, and various standardized measures such as the ACL ( Sappok, Heinrich, et al., 2013 ), the SCQ-

current ( Bo¨lte & Poustka, 2006; Rutter et al., 2001 ), the PDD-MRS ( Kraijer, 2006; Kraijer & Melchers, 2003 ), and, in cases of

diagnostic uncertainty, the ADOS ( Lord, Rutter, DiLavore, & Risi, 2001 ), the ADI-R ( Rutter, LeCouteur, & Lord, 2003 ), the

Scheme of Appraisal of Emotional Development (SAED; Dosen, 2005a, 2005b; Sappok, Budczies, Bo¨lte, et al., 2013 ), and the

Music-based Scale for Autism Diagnostics (MUSAD; Bergmann, Sappok, Diefenbacher, & Dziobek, 2012 ). Convergent validity

was assessed by correlation analysis with the SCQ, the ACL, and the PDD-MRS, while divergent validity was assessed by

correlation analysis with a non-ASD measure, the Modified Overt Aggression Scale (MOAS). The ACL was completed by the

primary psychiatrist, the SCQ-current was completed by an informant from the patient’s private living environment, and the

PDD-MRS, the ADOS, and the ADI-R were completed by a psychologist (H.K.) who was not involved in the study. Existing data

from diagnostic procedures were used, and these procedures were performed with the informed consent of the patients as a

part of routine patient care (Landeskrankenhausgesetz § 25.1, version 18.09.2011). The study was additionally approved by

the local ethics committee (06.10.2009) and was conducted according to the recommendations of the Declaration of Helsinki.

2.2. Sample The sample consisted of 219 adults with ID who were admitted to the in- or outpatient specialized psychiatric service

described above between 1/2012 and 7/2013. The mean age of the participants was 35.0 years (SD = 12.0). Overall, 77 (35%)

participants were diagnosed with additional ASD. More than half of all participants (n = 125; 57%) were males. The rate of

comorbidity between ID and ASD was higher in the males than in the females, x 2 (1, N = 219) = 6.70, p = 0.01, and 68.8% of the

individuals diagnosed with ASD were male. Thus, the frequency of male adults with ASD was higher than the expected

frequency (standardized Pearson residual = 2.6). The baseline characteristics of the study population are described in detail

in Table 1 . Level of ID was assessed with the Disability Assessment Schedule (DAS, Meins & Su¨ßmann, 1993 ) or standardized

intelligence tests, e.g., the Colored Progressive Matrices, the Kaufmann-Assessment-Batteries for Children, the Snijders-

Omen-Nonverbal Intelligence Test, and the Wechsler-Intelligence Test for Adults. The DAS has demonstrated convergent

validity with established measures of nonverbal IQ, such as the Colored Progressive Matrices (r = 0.75) and the Columbia

Mental Maturity Scale (r = 0.77; Holmes, Shah, & Wing, 1982; Meins and Su¨ßmann, 1993 ). In cases in which standardized ID

assessments were not available, intellectual functioning was categorized based on the daily living skills and social-

communication maturity using judgment of a clinical psychiatrist who was experienced in the field of ID. Sixty-eight (31%)

individuals were classified with mild ID, 83 (38%) with moderate ID and 68 (31%) with severe to profound ID. Combined, the

individuals with ID/ASD exhibited greater degrees of ID, x 2 (2, N = 219) = 17.92, p < 0.001. The frequency of individuals with

severe-to-profound ID was higher than the expected frequency in the ASD group (standardized Pearson residual = 4.0). The

proportions of participants with mild ID were lower than expected in the ASD group (standardized Pearson residual = 2.6) and higher in the ID-only group (standardized Pearson residual = 2.6). However, the frequency of moderate ID in the ASD

group did not differ meaningfully from the expected frequency (standardized Pearson residual = 0.9). Mood disorders (F3: n = 93, 43%), schizophrenia (F2: n = 56, 26%) and neurotic, stress-related and somatoform disorders

(F4: n = 37, 17%) were the most common psychiatric conditions in this sample. Non-ASD individuals were more likely to have

personality disorders compared to individuals with ASD, x 2 (1, N = 219) = 9.99, p < 0.01. No individual with ASD

(standardized Pearson residual = 3.2) and 17 (12%) individuals without ASD were diagnosed with personality disorder

(standardized Pearson residual = 3.2).

T. Sappok et al. / Research in Autism Spectrum Disorders 8 (2014) 362–375 365 Table 1

Sample characteristics.

Sample characteristics (N = 219) ASD/ID combined ID-only p a

General characteristics

N 77 142

Age 35.2 (12.0) 34.9 (12.1) 0.88 b Gender (male) 53 (68.8%) 72 (50.7%) 0.01 *

Severity of ID

Mild ID 14 (18.2%) 54 (38.0%) < 0.001 *** Moderate ID 26 (33.8%) 57 (40.1%)

Severe–profound ID 37 (48.1%) 31 (21.8%)

Psychiatric comorbidities

Mental disorders due to psychoactive substance use (F1x.x) 2 (2.6%) 6 (4.2%) 0.72 c Schizophrenia (F2x.x) 24 (31.2%) 32 (22.5%) 0.16 Mood disorders (F3x.x) 30 (39.0%) 63 (44.4%) 0.44 Neurotic, stress-related and somatoform disorders (F4x.x) 13 (16.9%) 24 (16.9%) 0.99

Personality disorders (F6x.x) 0 (0%) 17 (12.0%) < 0.01 ** Neurological comorbidities

Hearing disorder 3 (3.9%) 5 (3.5%) 0.99 c Visual disorder 7 (9.1%) 13 (9.2%) 0.99 Movement disorder 3 (3.9%) 16 (11.3%) 0.06 Epilepsy 19 (24.7%) 37 (26.1%) 0.82

Medication High potency antipsychotic 49 (63.6%) 82 (57.7%) 0.40 Low potency antipsychotic 28 (36.4%) 39 (27.5%) 0.17 Antidepressant 19 (24.7%) 50 (35.2%) 0.11 Anticonvulsant 22 (28.6%) 56 (39.4%) 0.11 Benzodiazepine 12 (15.6%) 17 (12.0%) 0.45

a p-value as result of x 2 -test if not indicated otherwise.

c t-Test for independent samples. Fisher’ exact test.

* p < 0.05.

** p < 0.01.

*** p < 0.001.

Epilepsy was the most common neurological disorder in this sample (n = 56, 26%); 20 (9%) participants suffered from a

disabling visual disorder, 8 (4%) had a severe hearing disorder, and 19 (9%) suffered from a relevant movement disorder. One

hundred thirty-one (60%) were taking high-potency antipsychotics such as olanzapine, risperidone, or aripiprazole, 67 (31%)

were taking low-potency antipsychotics such as promethazine, melperone, or pipamperone, 69 (32%) were taking antidepressants, 78 (36%) were taking anticonvulsants, and 29 (13%) were taking benzodiazepines. No significant group

differences were observed between the ASD and non-ASD groups with respect to neurological comorbidities or medication.

The sample characteristics of the adults ID alone and those with combined ID and ASD are summarized in Table 1 .

2.3. Measures

2.3.1. Diagnostic Behavioral Assessment for ASD – revised (DiBAS-R)

The DiBAS-R is a 20 item screening scale that was developed to assess autistic features in adults with ID and is completed

by professional caregivers or relatives. Each question is worded in plain language to allow rating by persons without any

specific knowledge of ASD. The questionnaire was written and administered in German; the items were translated for an

English-speaking audience (c.f. Table 2 ). The scale is easy to administer and self-explanatory and thus does not require any

preparatory training.

The items of the DiBAS-R are the result of an item selection procedure that was conducted in a clinical sample of adults

with ID (N = 91), which has been described in detail elsewhere ( Sappok, Gaul, et al., 2014 ). Briefly, 20 questions were

formulated according to the diagnostic criteria for ASD listed in the ICD-10 and DSM-5. Information from a review of the

literature focusing on symptoms that differentiate ASD form ID-only, an item analysis on other diagnostic measures for ASD,

e.g., the ADOS, the DiBAS, and the ACL, and experiences from clinical experts in the field of ID and ASD endorsed the item

deduction process. These items were evaluated in terms of diagnostic validity using the final diagnostic classification of the

multidisciplinary case conference as an external criterion ( Sappok, Gaul, et al., 2014 ). The resulting 12 discriminative items

were complemented by 8 additional items that assess typical ASD behaviors in adults with ID. The items are scored on the

following 4-point ordinal Likert scale: certainly true (3 points), often true (2 points), sometimes true (1 point), and never true (0

points). The final scores range from 0 to 60, and higher scores indicate greater ASD symptom loads.

2.3.2. Social Communication Questionnaire (SCQ)

The SCQ is an ASD screening scale that assesses autistic features in communication, interaction, and stereotyped behaviors using 40 binary items that were derived from the ADI–R ( Berument et al., 1999; Lord et al., 1994 ). The maximum

T. Sappok et al. / Research in Autism Spectrum Disorders 8 (2014) 362–375

Table 2

Item characteristics and factor loadings.

Subscale Pattern matrix Item characteristics M a

(N = 219) (N = 196)

SCI

SRS

Item

Item r it ID/ ID-

variance ASD only Social communication and interaction

factor

factor

difficulty

1. Does he smile back when smiled at? 0.71 0.22 0.43 1.18 0.72 1.9 1.0

2. Can you tell how he feels by his facial expression? 0.75 0.02 0.40 0.78 0.63 1.6 1.0

3. Does he have friendships with peers? 0.81 0.15 0.70 1.21 0.75 2.8 1.7

5. Does he show you things he likes or is interested in to share 0.88 0.04 0.53 1.21 0.77 2.2 1.3 enjoyment with you?

7. Does he comfort others if they are sad? 0.83 0.15 0.79 0.80 0.72 2.8 2.1

8. Does he involve you in activities that result in taking turns 0.87 0.05 0.66 1.03 0.74 2.5 1.7 and shared experiences?

9. Does he spontaneously join group activities? 0.79 0.00 0.64 1.17 0.65 2.4 1.6

11. Does he respond in a positive way when somebody else 0.74 0.04 0.47 0.78 0.69 1.9 1.2 approaches him?

12. Does he refer to himself in the first person, e.g., ‘‘I‘‘, and ‘‘me‘‘? 0.61 0.27 0.52 1.91 0.64 2.5 1.1

14. Does he look up and pay attention to you when you talk to him 0.63 0.17 0.32 1.01 0.59 1.5 0.7 without calling his name?

16. Does he nod to mean ‘yes’? 0.60 0.06 0.59 1.62 0.52 2.3 1.5

18. Does he talk to you just to be friendly? 0.62 0.08 0.71 1.22 0.53 2.5 1.9

Stereotypy, rigidity, and sensory abnormalities

4. Are there particular rituals that are important for him? –0.17 0.78 0.52 1.42 0.61 2.4 1.1

6. Does he repeat certain words in exactly the same way? 0.13 0.61 0.31 1.19 0.47 1.3 0.7

10. Does he have unusual hobbies or interests; e.g., flipping through books, 0.06 0.68 0.31 1.42 0.55 1.6 0.6 timetables, or electrical appliances?

13. Does he like to smell or tap at objects/walls? 0.23 0.67 0.18 0.93 0.56 1.2 0.2

15. Does he show challenging behavior when unpredictable changes occur? 0.15 0.72 0.51 1.12 0.66 2.3 1.2

17. Does he show odd movements of his body, such as rocking, finger flapping, 0.24 0.81 0.38 1.56 0.76 2.1 0.6

walking on tiptoes, or spinning around his body’s axis?

19. Does he show self-injuries behavior? 0.16 0.57 0.36 1.35 0.54 1.7 0.7 Note. Factor loadings printed in boldface are > 0.40 (Geomin rotation). SCI = Social Communication and Interaction; SRS = Stereotypy, Rigidity, and Sensory

Abnormalities; M = mean values for individuals with ID and ASD and ID-only; r it = part-whole corrected item total correlation calculated for each subscale

separately. a

All ps as result of group comparison based on Mann–Whitney U tests 0.001.

scores are 39 for verbal and 33 for nonverbal individuals because one item classifies general verbal abilities and can lead to

the exclusion of items that ask for individual characteristics in the use of language. To fulfill the diagnostic purpose, individuals should be at least 4 years old and should necessarily have achieved a developmental age of 2 years or more

( Rutter et al., 2001 ). In this study, the German version of the SCQ-current was applied ( Bo¨lte & Poustka, 2006 ). The diagnostic

validity of the SCQ has been assessed in toddlers, young children and adolescents (e.g., Allen, Silove, Williams, & Hutchins,

2007; Berument et al., 1999; Bo¨lte, Crecelius, & Poustka, 2000; Bo¨lte, Holtmann, & Poustka, 2008 , Bo¨lte, Poustka, & Constantino, 2008; Oosterling et al., 2009 ). A recent study of the diagnostic validity of this scale in adults with ID revealed a

high sensitivity but a low specificity in this population ( Sappok, Diefenbacher, Gaul, et al., in press ).

2.3.3. Pervasive Developmental Disorder in Mental Retardation Scale (PDD-MRS)

The PDD-MRS is an interview-based screening questionnaire that was specifically designed to assess ASD in individuals

with ID using 12 binary, partly weighted items. Total PDD-MRS scores can range from 0 to 19, and scores > 9 result in ASD

classification ( Kraijer & Bildt, 2005 ). A comprehensive study that was conducted to norm this instrument resulted in

promising overall sensitivity and specificity (both 92%; Kraijer & Bildt, 2005 ). Additionally, the PDD-MRS has produced good

results in terms of its diagnostic value over the complete ranges of ID and age covered in the present study ( Kraijer & Bildt,

2.3.4. Autism-Checklist (ACL)

The ACL is an expert-rated screener for ASD in adults with ID and was derived from the ICD-10 research criteria for autism

(F84.0) and atypical autism (F84.1). Each domain (social communication, interaction, and stereotyped behaviors) consists of

4 ordinal items that include 0 (no ASD), 0.5 (suspected for ASD) and 1 (ASD). The ACL has shown good psychometric properties

and an internal consistency (i.e., Cronbach’s alpha) of a = 0.81, a diagnostic validity (i.e., area under the curve) of 0.86, an

overall predictive value of 80.5%, a Cohen’s kappa of 0.60, and sensitivity and specificity values of 91% and 68%, respectively.

The ACL sum score was highly correlated with established screening measures such as the SCQ (Spearman’s rho (r s ) = 0.62)

and the PDD-MRS (r s = 0.49). The interrater reliability assessed in a sample of 53 individuals was good (Cohen’s kappa = 0.70;

r s = 0.55; Sappok, Heinrich, et al., 2013 ).

T. Sappok et al. / Research in Autism Spectrum Disorders 8 (2014) 362–375 367

2.3.5. Modified Overt Aggression Scale (MOAS)

The MOAS is a rating scale that assesses the intensity of aggression (verbal, toward objects, toward the self, and toward

others; Knoedler, 1989; Yudofsky, Silver, Jackson, Endicott, & Williams, 1986 ). Oliver, Crawford, Rao, Reece and Tyrer (2007)

assessed the interrater reliability of this instrument in a small sample of adults with ID. The analysis resulted in promising

estimation for MOAS overall score (ICC = 0.93).

2.4. Data analysis

Differences in demographic and clinical characteristics between the participants with combined ID and ASD and those

with ID alone were assessed using a x 2 -tests (or Fisher’ exact tests when appropriate) for categorical variables or t-tests for

independent samples in case of continuous variables (i.e., age). Standardized Pearson residuals were additionally calculated

when the x 2 -tests were significant to get an impression of the size and the direction of the difference of observed and

expected frequencies in single cells of a contingency table ( Agresti, 2002 ). In this case the term ‘expected’ refers to the

frequencies estimated on the basis of observed frequencies under the assumption that the two categorical variables are not

associated with each other ( Eid, Gollwitzer, & Schmitt, 2011 ). A standardized Pearson residual (absolute value) greater than 2

indicates a relevant discrepancy ( Agresti, 2002 ).

An exploratory factor analysis (EFA) was applied to the items of the DiBAS-R. In this newly developed scale, the factor

structure was not predetermined. Since the items have not been factor analyzed before, an exploratory procedure was used.

An oblique rotation method (Geomin) that allowed for factor correlations was utilized ( Fabrigar, Wegener, MacCallum, &

Strahan, 1999 ). Single items were treated as ordered-categorical variables; thus, a robust weighted least square-mean and

variance adjusted (WLSMV)-estimator was used ( Muthe´n, Du Toit, & Spisic, 1997 ). As recommended, more than one criterion

was considered when determining the number of the factors ( Costello & Osborne, 2005 ). In this study, eigenvalue criteria

(i.e., factors with eigenvalues greater than 1 were considered to likely be meaningful), scree plot evaluations, and the interpretability of the extracted factors (which was taken as the most important criterion) were utilized. Additionally, root

mean square errors of approximation (RMSEAs), standardized root mean square residuals (SRMRs), comparative fit indices

(CFIs) and chi-squared statistics are reported for evaluations of model fit. RMSEAs 0.06, SRMRs 0.08 and CFIs 0.95 were

used as cut offs for good model fit ( Hu & Bentler, 1999; Muthe´n, 1998–2004 ). The chi-squared statistics were required to be

non-significant to indicate good model fits. The entire sample (N = 219) was included in the EFA, which was performed with

MPlus 6.1 ( Muthe´n & Muthe´n, 1998–2010 ). The results of the factor analysis were used to group the items into subscales. The

reliabilities of the subscales and the reliability of the entire scale were calculated in terms of internal consistencies using

Cronbach’s alpha.

Cases with one or more missing values on any of the items were excluded from further analyses. The item analyses

consisted of calculating the item-validities, item-difficulties, item-variances, and the part-whole corrected item total- correlations (separately for each subscale; Kelava & Moosbrugger, 2012 ). All these measures are of highly descriptive value.

Item validity measures the discriminant power of each single item for differentiation of ASD and non-ASD behaviors.

Considering the likert-type scale, item validity was assessed by Mann–Whitney U tests.

Item-difficulties were calculated by summing all observed scores of a single item and dividing this sum through the score

which would result if all individuals reach the most symptomatic category which is certainly true (3 points; Kelava &

Moosbrugger, 2012 ). A higher value of item difficulty indicates that more individuals display a certain behavior assessed

with this item. Each item should show a variance above zero to be able to depict differences between individuals. The

association of each item with the whole scale was evaluated with part whole corrected item-total correlations. In general a

high correlation between single item scores and the sum score of the rest of the items are desirable ( Kelava & Moosbrugger,

2012 ). This indicates that differences in single item scores are reflected in differences in total sum scores ( Kelava & Moosbrugger, 2012 ). To assess diagnostic validity, Mann–Whitney U tests were used to assess the statistical significance of differences in

DiBAS-R sum-scores and item scores between participants with and without ASD. Mann–Whitney U tests is a rank based

procedure which is more appropriate with ordinal data ( Gravetter & Wallnau, 2009 ). Receiver operator characteristic analyses (ROCs) were used to calculate the sensitivities and specificities of the observed scores that were extracted from the

DiBAS-R subscales and total score using the diagnostic decisions of the multi-professional case conference as the relevant

diagnostic criterion. A ROC curve was plotted to visualize the relation of sensitivity and specificity over the entire range of

observed scores. The area under the curve (AUC), which indicates the general differentiating value of a test, was determined

( Goldhammer & Hartig, 2012 ).

A two-step exploratory procedure was used to determine the optimal cut-off. In the first step, separate potential cut-offs

for each subscale and the total score were determined using ROC analyses. In the second step, several combinations of cut-

offs were evaluated in terms of diagnostic values. The combination of cut-offs that achieved the most balanced sensitivity

and specificity was used. All further analyses were based on this cut-off. Cohen’ kappa was calculated to evaluate the

agreement between the diagnostic decisions of the DiBAS-R and those of the consensus conference.

For analysis of convergent and divergent validity and interrater reliability sum scores were treated as interval scaled

variables. Convergent validity was evaluated by calculating the Pearson correlations between the DiBAS-R sum scores and

the sum scores of established measures of ASD (PDD-MRS, SCQ-current, and ACL). Since all measures intend to measure

autism a meaningful positive correlation was expected. Discriminant validity was assessed by calculating the correlation

T. Sappok et al. / Research in Autism Spectrum Disorders 8 (2014) 362–375

between the DiBAS-R and MOAS total scores. As hint for divergent validity the correlation should be non-significant and

close to zero.

Interrater reliability was estimated by calculating the ICCs between the overall DiBAS-R total and subscale scores. ICC is a

common procedure to quantify the degree to which scores of a given instrument resemble in different ratings.

The scale’s feasibility was assessed using the proportion of completed ratings on scale and item level.

All tests except the assessment of convergent and divergent validity were performed two-tailed, and p < 0.05 was

considered statistically significant. All data analyses, with the exception of the factor analysis, were performed with SPSS

15.0. ROC curves were visualized using R ( R Core Team, 2013 ).

3. Results

3.1. Factor analysis

In the first step, all 20 items were used in the factor analysis. Three eigenvalues greater than 1 indicated a three-factor

solution (eigenvalues: 9.41, 2.71, 1.13, and 0.93), and exploration of the scree plot indicated a 2-factor solution. Examination

of the pattern matrix of the Geomin-rotated 2- and 3-factor solutions identified the 2-factor solution as the most appropriate

in terms of interpretability and the DSM-5 diagnostic criteria. Only one item (‘‘Does he use your hand as a tool to communicate his needs?’’) showed comparable loadings on both factors with a loading difference below 0.15. This item was

therefore removed, and the factor analysis was rerun with 19 items. For this solution, the RMSEA = 0.058, 90% CI [0.045,

0.070], the CFI = 0.98, and the SRMR = 0.052 indicating at least adequate model fit. However, a chi-squared test of model fit

was significant with a x 2 (134) = 233.3, p < 0.001. The items associated with social communication and interaction loaded on

the first factor, while the items associated with restrictive, repetitive behaviors and sensory aspects loaded on the second

factor. The two factors were correlated (r = 0.50). The factor loadings of the Geomin-rotated 2-factor solution are presented in

Table 2 . Items with loadings > 0.40 on one factor were grouped into subscales. The first subscale was termed Social Communication and Interaction (SCI) and the second subscale was termed Stereotypy, Rigidity, and Sensory Abnormalities

(SRS). The Social Communication and Interaction subscale consisted of 12 items. All items exhibited strong factor loadings that

ranged from 0.60 (DiBAS-R: 16 – ‘‘Does he nod to mean ‘yes’?’’) to 0.88 (DiBAS-R: 5 – ‘‘Does he show you things he likes or is

interested in to share enjoyment with you?’’). The Stereotypy, Rigidity, and Sensory Abnormalities subscale consists of 7

items. All items showed strong loadings on the second factor The internal consistencies were a = 0.91 and a = 0.84

for the Social Communication and Interaction and Stereotypy, Rigidity, and Sensory Abnormalities subscales, respectively.

The internal consistency of the entire scale was a = 0.91.

3.2. Item-analysis The item-validities, -difficulties, -variances, and the item-total correlations are summarized in Table 2 . Mean item values

for individuals with ID only were lower than those of the combined ID and ASD individuals ( Table 2 ). As calculated by Mann–

Whitney U tests, all 19 items exhibited significant differences between the ID/ASD and ID-only groups (all ps < 0.001). The

part-whole corrected item-total correlations for the Social Communication and Interaction subscale ranged from r it = 0.77

(DiBAS-R: 5 ‘‘Does he show you things he likes or is interested in to share enjoyment with you?’’) to r it = 0.52 (DiBAS-R: 16

‘‘Does he nod to mean ‘yes’?’’). The median was r it = 0.67. The item difficulties of this subscale ranged from 0.32 (DiBAS-R: 14

‘‘Does he look up and pay attention to you when you talk to him without calling his name?’’) to 0.79 (DiBAS-R: 7 ‘‘Does he

comfort others if they are sad?’’). The median item difficulty of this subscale was 0.56. The item-variances ranged from 0.78

(DiBAS-R: 2 ‘‘Can you tell how he feels by his facial expression?’’; DiBAS-R: 11 ‘‘Does he respond in a positive way when

somebody else approaches him?’’) to 1.91 (DiBAS-R: 12 ‘‘Does he refer to himself in the first person, e.g., ‘I’ and ‘me’?), and the

median was 1.18.

The part-whole corrected item-total correlations of the Stereotypy, Rigidity, and Sensory Abnormalities subscale varied

between r it = 0.47 (DiBAS-R: 6 ‘‘Does he repeat certain words in exactly the same way?’’) and r it = 0.76 (DiBAS-R: 17 ‘‘Does he

show odd movements of his body, such as rocking, finger flapping, walking on tiptoes, or spinning around his body’s axis?’’).

The median was r it = 56. The item difficulties ranged from 0.18 (DiBAS-R 13: ‘‘Does he like to smell or tap at objects/walls?’’)

to 0.52 (DiBAS-R: 4 ‘‘Are there particular rituals that are important for him?’’), and the median was 0.36. Item 17 (‘‘Does he

show odd movements of his body, such as rocking, finger flapping, walking on tiptoes, or spinning around his body’s axis?’’)

exhibited the highest item-variance (1.56), and item 13 (‘‘Does he like to smell or tap at objects/walls?’’) exhibited the lowest

item variance (0.93). The median of the item-variances of the Stereotypy, Rigidity, and Sensory Abnormalities subscale items

was 1.35.

3.3. Diagnostic validity

The DiBAS-R total scores for the ID/ASD combined group and the ID-only group are summarized in Table 3 . Mann–

Whitney U tests indicated significant differences between participants with and without ASD in the DiBAS-R total score,

T. Sappok et al. / Research in Autism Spectrum Disorders 8 (2014) 362–375 369 Table 3

Means and medians for the DiBAS-R sum scores differentiated for adults with and without ASD.

Scale Sum scores p a

Total (n = 196) ASD/ID (n = 67) ID-only (n = 129)

DiBAS-R-total M (SD) 27.9 (12.9) 39.4 (8.3) 22.0 (10.8) < 0.001

Mdn 28.0 40.0 20.0

SCI M (SD) 20.2 (9.3) 26.8 (5.6) 16.9 (9.0) < 0.001

Mdn 21.0 27.0 16.0

SRS M (SD) 7.7 (5.7) 12.6 (5.0) 5.2 (4.1) < 0.001

Mdn 6.0 13.0 4.0

Note. SCI = DiBAS-R Social Communication and Interaction subscale; SRS = DiBAS-R Stereotypy, Rigidity, and Sensory Abnormalities subscale; M = mean;

SD a = standard deviation; Mdn = median.

As result of Mann–Whitney U Test.

Fig. 1. ROC curves for the DiBAS-R total and subscale scores using the diagnosis of ASD as the criterion.

U = 943.50, Z = 8.97, p < 0.001, the Social Communication and Interaction subscale score, U = 1598.50, Z = 7.23, p < 0.001,

and the Stereotypy, Rigidity, and Sensory Abnormalities subscale score, U = 1151.50, Z = 8.43, p < 0.001. All means and

medians of the total and subscale scores were significantly greater in the subsample with ASD compared to the subsample

without ASD.

ROC analysis of the DiBAS-R total scores resulted in an AUC of 0.89, 95% CI [0.85, 0.94], p < 0.001. The AUC for the Social

Communication and Interaction subscale was 0.82, 95% CI [0.76, 0.87], p < 0.001, and the AUC for the Stereotypy, Rigidity,

and Sensory Abnormalities subscale was 0.87, 95% CI [0.82, 0.92], p < 0.001. All estimations indicated good discriminative

ability. The ROC curves are presented in Fig. 1 .

The sensitivity and specificity values and Cohen’s kappas for three possible cut-off values for each subscale are presented

in Table 4 . Several combinations of cut-offs were evaluated in terms of sensitivities and specificities. A combination of cut-off

scores of 29 points on the total scale, 21 points on the Social Communication and Interaction subscale, and 5 points

Table 4

Psychometric properties of the DiBAS-R for different cut points.

DiBAS-R total SCI SRS Combined cut point

Cut-off 28 29 30 20 21 22 4 5 6 (29 – 21 – 5)

Sensitivity % 89.6 88.1 85.1 89.6 86.6 83.6 95.5 94.0 91.0 80.6 Specificity % 69.8 72.1 74.4 60.5 65.1 69.0 43.4 54.3 63.6 80.6

Kappa 0.53 0.55 0.55 0.43 0.46 0.48 0.31 0.40 0.48 0.59 Note. SCI = DiBAS-R Social Communication and Interaction subscale; SRS = DiBAS-R Stereotypy, Rigidity, and Sensory Abnormalities subscale.

T. Sappok et al. / Research in Autism Spectrum Disorders 8 (2014) 362–375

Table 5

Convergent validity of the DiBAS-R with the SCQ-current, the PDD-MRS, and the ACL.

DiBAS-R SRS DiBAS-R total SCQ current PDD-MRS ACL n = 196 n = 196 n = 87 n = 77 n = 91

DiBAS-R SCI 0.47 ***

DiBAS-R SRS 1 0.77 ***

DiBAS-R total – 1 0.52 ***

0.59 *** Note. SCI = DiBAS-R Social Communication and Interaction subscale; SRS = DiBAS-R Stereotypy, Rigidity, and Sensory Abnormalities subscale.

** p < 0.01.

*** p < 0.001.

for the Stereotypy, Rigidity, and Sensory Abnormalities subscale resulted in the most balanced sensitivity and specificity

values. Using these combined cut-off values, 80.6% of patients with ASD and 80.6% of the patients without ASD were correctly recognized by the DiBAS-R. The positive predictive value was 68.4%. The DiBAS-R classification and the diagnostic decision agreed in 80.6% of all individuals of the current sample (n = 158). The Cohen’s kappa was

3.4. Convergent and discriminant validity

The results of the correlation analyses assessing convergent validity are summarized in Table 5 . The DiBAS-R total score was significantly correlated with the sum scores of all of the established diagnostic instruments

that were available for the participants (r 0.5, p < 0.001, one-tailed). The overall sum scores exhibited the highest correlation with the ACL sum score, r(89) = 0.59, p < 0.001, one-tailed. The correlations of the DiBAS-R subscale scores with

the other established measures ranged between r(75) = 0.35, p < 0.01, one-tailed (PDD-MRS and DiBAS-R Social Communication and Interaction subscale score) and r(89) = 0.53, p < 0.001, one-tailed (ACL sum score and DiBAS-R Social

Communication and Interaction subscale score). Analyses of divergent validities revealed non-significant correlations between the MOAS and the DiBAS-R total score r(79) = 0.08, p = 0.23, one-tailed, ns, the DiBAS-R Social Communication and

Interaction subscale total score r(79) = 0.11, p = 0.17, one-tailed, ns, and the DiBAS-R Stereotypy, Rigidity, and Sensory

Abnormalities subscale total score r(79) = 0.01, p = 0.46, one-tailed, ns.

3.5. Interrater reliability

The ICC coefficients for estimation of interrater reliability (n = 36) were 0.88, 95% CI [0.78, 0.94], p < 0.001 for the DiBAS-R

total score, 0.72, 95% CI [0.51, 0.85], p < 0.001 for the Social Communication and Interaction subscale score and 0.78, 95% CI

[0.60, 0.88], p < 0.001 for the Stereotypy, Rigidity, and Sensory Abnormalities subscale score.

3.6. Feasibility

The scale’s feasibility was 100%, and 196/219 of the participants completed the entire questionnaire.

4. Discussion

ID and ASD co-occur at high rates. The DIBAS-R was designed to enable quick screening for ASD in the highly vulnerable ID

population. Factor analysis indicated two subscales, the DiBAS-R Social Communication and Interaction and the DiBAS-R