Test-retest Reliability and Measurement Invariance across time of the Quantitative-CHecklist for Autism in Toddler


Abstract


The study evaluated the stability over time of the Q-CHAT (i.e., Quantitative-CHecklist for Autism in Toddler), a screening tool for the early detection of Autism. The Q-CHAT is filled by parents when the child is from 18-36 months old, and whether a risk score is found, the clinician refers the child to a more in-depth diagnostic evaluation. Previous studies demonstrated the measure test-retest reliability on a small time interval (1-6 months). Therefore, the present study tested the stability of the scores considering a larger time interval of 18 months. No previous studies have tested its measurement invariance over time. Since the tool is filled by parents during a large time span (18-36 months of life), it is imperative to understand whether there is an invariance over time in their evaluations. The Italian version of the Q-CHAT was completed by 282 parents of children with no pre-existing signs of risk of Autism. The Q-CHAT was administered when children were 18 months (T1) and then 18 months later (T2). The intraclass correlation coefficients for the test-retest reliability ranged from sufficient to moderate. The measurement invariance across time revealed a tolerable configural and metric invariance. Contrary, the scalar invariance was not met meaning that the means of the constructs are not invariant over time. The findings give a further demonstration of the reliability of the Q-CHAT. They give evidence that changes in the scores would reflect real changes in the construct itself, and not in the way individuals interpret the measure items.

DOI Code: 10.1285/i20705948v14n1p146

Keywords: autism spectrum disorder; Quantitative-CHecklist for Autism in Toddler; Q-CHAT; test-retest reliability; measurement invariance across time; validation

References


Allison, C., Baron-Cohen, S., Wheelwright, S., Charman, T., Richler, J., Pasco, G., and Brayne, C. (2008). The q-chat (quantitative checklist for autism in toddlers): A normally distributed quantitative measure of autistic traits at 18–24 months of age: Preliminary report. Journal of autism and developmental disorders, 38(8):1414–1425.

Anderson, D. K., Liang, J. W., and Lord, C. (2014). Predicting young adult outcome among more and less cognitively able individuals with autism spectrum disorders. Journal of Child Psychology and Psychiatry, 55(5):485–494.

APA (2013). The Diagnostic and Statistical Manual of mental Disorders-5. APA.

Baron-Cohen, S., Allen, J., Gillberg, C., et al. (1992). Can autism be detected at 18 months? the needle, the haystack and the chat. British journal of psychiatry, 161:839– 839.

Ben ́ıtez-Burraco, A. and Progovac, L. (2020). A four-stage model for language evolution under the effects of human self-domestication. Language & Communication, 73:1–17.

Capirci, O., Contaldo, A., Caselli, M. C., and Volterra, V. (2005). From action to language through gesture: A longitudinal perspective. Gesture, 5(1-2):155–177.

Chang, C.-C., Wu, T.-H., Chen, C.-Y., Wang, J.-D., and Lin, C.-Y. (2014). Psychometric evaluation of the internalized stigma of mental illness scale for patients with mental illnesses: measurement invariance across time. PLoS One, 9(6):e98767.

Cheung, G. W. and Rensvold, R. B. (2000). Assessing extreme and acquiescence response sets in cross-cultural research using structural equations modeling. Journal of cross- cultural psychology, 31(2):187–212.

Dawson, G., Rogers, S., Munson, J., Smith, M., Winter, J., Greenson, J., Donaldson, A., and Varley, J. (2010). Randomized, controlled trial of an intervention for toddlers with autism: the early start denver model. Pediatrics, 125(1):e17–e23.

Devescovi, R., Monasta, L., Bin, M., Bresciani, G., Mancini, A., Carrozzi, M., and Colombi, C. (2020). A two-stage screening approach with i-tc and q-chat to identify toddlers at risk for autism spectrum disorder within the italian public health system. Brain Sciences, 10(3):184.

Haine-Schlagel, R., Rieth, S., Dickson, K. S., Brookman-Frazee, L., and Stahmer, A. (2020). Adapting parent engagement strategies for an evidence-based parent-mediated intervention for young children at risk for autism spectrum disorder. Journal of Com- munity Psychology.

Harrop, C., McConachie, H., Emsley, R., Leadbitter, K., Green, J., Consortium, P., et al. (2014). Restricted and repetitive behaviors in autism spectrum disorders and typical development: Cross-sectional and longitudinal comparisons. Journal of autism and developmental disorders, 44(5):1207–1219.

Heal, R. and Twycross, A. (2015). Validity and reliability in quantitative research. Evid.-Based Nurs, pages 66–67.

Honey, E., Leekam, S., Turner, M., and McConachie, H. (2007). Repetitive behaviour and play in typically developing children and children with autism spectrum disorders.

Electronic Journal of Applied Statistical Analysis 19 Journal of autism and developmental disorders, 37(6):1107–1115.

Hu, L.-t. and Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural equation modeling: a multidisciplinary journal, 6(1):1–55.

Istvan, E. M., Nevill, R. E., and Mazurek, M. O. (2020). Sensory over-responsivity, repet- itive behavior, and emotional functioning in boys with and without autism spectrum disorder. Research in Autism Spectrum Disorders, 75:101573.

Koo, T. K. and Li, M. Y. (2016). A guideline of selecting and reporting intraclass corre- lation coefficients for reliability research. Journal of chiropractic medicine, 15(2):155– 163.

Lameira, A. R. and Call, J. (2020). Understanding language evolution: Beyond pan- centrism. BioEssays, 42(3):1900102.

Lecciso, F., Levante, A., Signore, F., and Petrocchi, S. (2019). Preliminary evidence of the structural validity and measurement invariance of the quantitative-checklist for autism in toddler (q-chat) on italian unselected children. Electronic Journal of Applied Statistical Analysis, 12(2):320–340.

Lecciso, F., Petrocchi, S., Savazzi, F., Marchetti, A., Nobile, M., and Molteni, M. (2013). The association between maternal resolution of the diagnosis of autism, maternal mental representations of the relationship with the child, and children’s attachment. Life Span and Disability, 16(1):21–38.

Lee, S. T. (2018). Testing for measurement invariance: Does your measure mean the same thing for different participants? APS Observer, 31(8).

Leekam, S., Tandos, J., McConachie, H., Meins, E., Parkinson, K., Wright, C., Turner, M., Arnott, B., Vittorini, L., and Couteur, A. L. (2007). Repetitive behaviours in typi- cally developing 2-year-olds. Journal of Child Psychology and Psychiatry, 48(11):1131– 1138.

Lord, C., Rutter, M., DiLavore, P., Risi, S., Gotham, K., Bishop, S., et al. (2012). Autism diagnostic observation schedule–2nd edition (ados-2). Los Angeles, CA: Western Psy- chological Corporation.

Magiati, I., Goh, D. A., Lim, S. J., Gan, D. Z. Q., Leong, J., Allison, C., Baron-Cohen, S., Rifkin-Graboi, A., Broekman, B. P., Saw, S. M., et al. (2015). The psychometric properties of the quantitative-checklist for autism in toddlers (q-chat) as a measure of autistic traits in a community sample of singaporean infants and toddlers. Molecular autism, 6(1):40.

Mohammadian, M., Zarafshan, H., Mohammadi, M. R., and Karimi, I. (2015). Evaluat- ing reliability and predictive validity of the persian translation of quantitative checklist for autism in toddlers (q-chat). Iranian journal of psychiatry, 10(1):64.

Park, S., Won, E.-K., Lee, J. H., Yoon, S., Park, E. J., and Kim, Y. (2018). Reliability and validity of the korean translation of quantitative checklist for autism in toddlers: a preliminary study. Journal of the Korean Academy of Child and Adolescent Psychiatry, 29(2):80–85.

Perry, A., Cummings, A., Geier, J. D., Freeman, N. L., Hughes, S., LaRose, L., Man- aghan, T., Reitzel, J.-A., and Williams, J. (2008). Effectiveness of intensive behavioral intervention in a large, community-based program. Research in Autism Spectrum Dis- orders, 2(4):621–642.

Petrocchi, S., Labrie, N. H., and Schulz, P. J. (2020a). Measurement invariance of the short wake forest physician trust scale and of the health empowerment scale in german and french women. Journal of health psychology, 25(4):558–569.

Petrocchi, S., Levante, A., and Lecciso, F. (2020b). Systematic review of level 1 and level 2 screening tools for autism spectrum disorders in toddlers. Brain sciences, 10(3):180.

Pornprasertmanit, S., Miller, P., Schoemann, A., Rosseel, Y., Quick, C., Garnier- Villarreal, M., Selig, J., Boulton, A., Preacher, K., Coffman, D., et al. (2015). Package ‘semtools’. Retrieved in.

Renty, J. and Roeyers, H. (2006). Quality of life in high-functioning adults with autism spectrum disorder: The predictive value of disability and support characteristics. Autism, 10(5):511–524.

Robins, D. L., Casagrande, K., Barton, M., Chen, C.-M. A., Dumont-Mathieu, T., and Fein, D. (2014). Validation of the modified checklist for autism in toddlers, revised with follow-up (m-chat-r/f). Pediatrics, 133(1):37–45.

Robins, D. L. and Dumont-Mathieu, T. M. (2006). Early screening for autism spectrum disorders: update on the modified checklist for autism in toddlers and other measures. Journal of Developmental & Behavioral Pediatrics, 27(2):S111–S119.

Robins, D. L., Fein, D., Barton, M. L., and Green, J. A. (2001). The modified checklist for autism in toddlers: an initial study investigating the early detection of autism and pervasive developmental disorders. Journal of autism and developmental disorders, 31(2):131–144.

Rodgers, M., Marshall, D., Simmonds, M., Le Couteur, A., Biswas, M., Wright, K., Rai, D., Palmer, S., Stewart, L., and Hodgson, R. (2020). Interventions based on early intensive applied behaviour analysis for autistic children: a systematic review and cost-effectiveness analysis. Health technology assessment (Winchester, England), pages 1–306.

Rosseel, Y. (2012). Lavaan: An r package for structural equation modeling and more. version 0.5–12 (beta). Journal of statistical software, 48(2):1–36.

Ruta, L., Arduino, G. M., Gagliano, A., Apicella, F., Leonardi, E., Fam`a, F. I., Cheri- coni, N., Costanzo, V., Turco, N., Tartarisco, G., et al. (2019a). Psychometric proper- ties, factor structure and cross-cultural validity of the quantitative checklist for autism in toddlers (q-chat) in an italian community setting. Research in Autism Spectrum Disorders, 64:39–48.

Ruta, L., Chiarotti, F., Arduino, G. M., Apicella, F., Leonardi, E., Maggio, R., Car- rozza, C., Chericoni, N., Costanzo, V., Turco, N., et al. (2019b). Validation of the quantitative checklist for autism in toddlers (q-chat) in an italian clinical sample of young children with autism and other developmental disorders. Frontiers in psychia- try, 10:488.

Sinai-Gavrilov, Y., Gev, T., Mor-Snir, I., Vivanti, G., and Golan, O. (2020). Inte- grating the early start denver model into israeli community autism spectrum dis- order preschools: Effectiveness and treatment response predictors. Autism, page

Thabtah, F. and Peebles, D. (2019). Early autism screening: A comprehensive review. International journal of environmental research and public health, 16(18):3502.

Tregay, J., Gilmour, J., and Charman, T. (2009). Childhood rituals and executive functions. British Journal of Developmental Psychology, 27(2):283–296.

Tyrell, F. A., Yates, T. M., Widaman, K. F., Reynolds, C. A., and Fabricius, W. V. (2019). Data harmonization: Establishing measurement invariance across different assessments of the same construct across adolescence. Journal of Clinical Child & Adolescent Psychology, 48(4):555–567.

Volterra, V. (1981). Gestures, signs, & words at two years: when does communication become language? Sign Language Studies, (33):351–362.

Widaman, K. F., Ferrer, E., and Conger, R. D. (2010). Factorial invariance within longitudinal structural equation models: Measuring the same construct across time. Child development perspectives, 4(1):10–18.

Widaman, K. F. and Reise, S. P. (1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain.


Full Text: pdf
Bookmark and Share


Creative Commons License
This work is licensed under a Creative Commons Attribuzione - Non commerciale - Non opere derivate 3.0 Italia License.