Is intervention fadeout a scaling artefact?

Abstract

To determine whether scaling decisions might account for fadeout of impacts in early education interventions, we reanalyze data from a well-known early mathematics RCT intervention that showed substantial fadeout in the two years after the intervention ended. We examine how various order-preserving transformations of the scale affect the relative mathematics achievement of the control and experimental groups by age. Although fadeout was robust to most transformations, we were able to eliminate or even reverse fadeout by emphasizing differences in scores near typical levels of first-graders while treating differences elsewhere as unimportant. Such a transformation lowers treatment effects at preschool age and raises them in first grade, relative to the original scale. The findings suggest substantial implications for interpreting the effects of educational interventions.