Treatment Impact Measures

New MassHealth regulations require the periodic administration of treatment impact measures to objectively evaluate the effectiveness of ongoing services. In response, the Commonwealth has identified three well-established instruments commonly used in autism services: the Autism Treatment Evaluation Checklist (ATEC), the Autism Spectrum Disorder–Diagnosis for Children (ASDQ), and the Pervasive Developmental Disorder Behavior Inventory (PDD-BI). Each measure differs in cost, administration burden, and psychometric rigor, which has implications for their use in applied clinical settings.

The ATEC is a caregiver-report measure that is publicly available at no cost and can be completed in approximately 10 to 15 minutes. Its primary strength lies in its practicality and sensitivity to change over time, making it well suited for repeated administration as required under MassHealth guidelines. The results of the ATEC are promptly compiled into an eight-page report and transmitted within three (3) hours of completion. While the ATEC is not norm-referenced and relies exclusively on caregiver report, research indicates acceptable internal consistency, defined as the degree to which items within a measure consistently assess the same underlying construct, suggesting that its subscales function coherently across domains. The ATEC also demonstrates reasonable construct validity, meaning it measures the areas of functioning it is intended to assess, supporting its use for monitoring treatment-related change over time. These characteristics support its use as a cost-effective outcome measure in routine service delivery. 

The ASDQ is a brief, low-cost instrument designed primarily for screening and symptom monitoring. Administration typically requires 10 to 20 minutes. Psychometric studies support adequate internal consistency, indicating that the items reliably reflect autism-related characteristics, as well as acceptable concurrent validity, defined as the degree to which a measure correlates with other established instruments assessing similar constructs. However, the ASDQ was not developed as a comprehensive treatment-response measure. As such, it may be less sensitive to incremental functional gains over time and is best conceptualized as a supplemental tool rather than a primary measure of treatment effectiveness.

In contrast, the PDD-BI is a comprehensive, norm-referenced caregiver-report measure with strong psychometric properties. It demonstrates excellent internal consistency, as well as strong test–retest reliability, defined as the stability of scores when the same respondent completes the measure across different time points. The PDD-BI also shows robust validity, meaning it accurately measures both adaptive and maladaptive behavior domains relevant to autism spectrum disorders and is sensitive to meaningful treatment effects. However, the PDD-BI is proprietary, associated with higher costs, and requires 45 minutes or more to complete, in addition to scoring time. These factors limit its feasibility for frequent reassessment but make it well suited for baseline evaluations and periodic comprehensive reviews.
Taken together, these instruments represent a range of options that balance psychometric rigor with feasibility. Selection of a treatment impact measure should consider regulatory requirements, clinical utility, caregiver burden, and the need for repeated measurement over time.

Below is a side-by-side comparison of ATEC, ASDQ, and PDD-BI, focused specifically on cost, administration burden, and psychometric strength.

____________________________________________________________________________________________________________________

Comparison of Common Autism Outcome Measures

____________________________________________________________________________________________________________________

References

Cohen, I. L., & Sudhalter, V. (2005). Pervasive Developmental Disorder Behavior Inventory (PDD-BI): Professional manual. Psychological Assessment Resources.
NCBI - WWW Error Blocked Diagnostic

Cohen, I. L., Schmidt-Lackner, S., Romanczyk, R., & Sudhalter, V. (2010). The PDD Behavior Inventory: A rating scale for assessing response to intervention in children with pervasive developmental disorder. Journal of Autism and Developmental Disorders, 40(11), 1339–1350. 
https://pubmed.ncbi.nlm.nih.gov/12708578/

Geier, D. A., Kern, J. K., & Geier, M. R. (2013). A comparison of the Autism Treatment Evaluation Checklist (ATEC) and the Childhood Autism Rating Scale (CARS) for the quantitative evaluation of autism. Journal of Mental Health Research in Intellectual Disabilities, 6(4), 255–267. 
https://doi.org/10.1080/19315864.2012.681340

Magiati, I., Moss, J., Yates, R., Charman, T., & Howlin, P. (2011). Is the Autism Treatment Evaluation Checklist a useful tool for monitoring progress in children with autism spectrum disorders? Journal of Intellectual Disability Research, 55(3), 302–312.
https://doi.org/10.1111/j.1365-2788.2010.01359.x

Matson, J. L., & González, M. L. (2007). Autism Spectrum Disorder–Diagnosis for Children (ASD-DC): Development and initial psychometric analysis. Research in Autism Spectrum Disorders, 1(3), 231–242. 
https://pubmed.ncbi.nlm.nih.gov/17981435/

Rimland, B., & Edelson, S. M. (1999). Autism Treatment Evaluation Checklist (ATEC). Autism Research Institute.
https://autism.org/autism-treatment-evaluation-checklist/