NBER posted an RCT of Saga Technology, a program that provides 9th graders with high-dosage math tutoring and engagement with educational technology. Quick take: High-quality RCT in New York City and Chicago finds sizable gain (~86% of a grade-level) in math achievement on end-of-year district tests.
Program:
Students in the program alternate (on successive days) between working with an in-person, trained tutor and individually engaging with an education technology platform. The student-tutor ratio is 4:1. The program takes place 50 minutes a day, 5 days a week, for the full 9th grade school year.
Study Design:
The study sample comprised 2,065 incoming 9th graders in 6 high-poverty high schools (3 in NYC, 3 in Chicago). 81% were Black or Hispanic; 91% were low-income. Based on careful review, this was a well-conducted RCT (e.g., baseline balance, low attrition, prespecified outcomes and analyses).
Findings:
The study found a sizable, statistically-significant impact on end-of-year math scores on district tests. The effect size was 0.19 - which represents an 86% improvement over the annual gain in math otherwise expected for 9th graders.
The study also found significantly improved math GPA and fewer math course failures through fall of 10th grade, after which COVID interrupted study follow-up.
Comment:
Prior RCTs have found sizable, sustained effects for Saga's traditional program (2:1 tutoring ratio, no tech component). Saga Technology's impacts are almost as big, but the cost is 40% lower (around $2600 per tutored student).
Taken together, these studies constitute strong, replicated RCT evidence - providing confidence that schools which adopt and faithfully implement Saga's traditional or technology model will see meaningful math gains for low-income students.
Disclosure: my former employer, Arnold Ventures, helped fund this study.