NBER posted an RCT of Saga Technology, a program that provides 9th graders with high-dosage math tutoring and engagement with educational technology. Quick take: High-quality RCT in New York City and Chicago finds sizable gain (~86% of a grade-level) in math achievement on end-of-year district tests.

​

Program:

​

• Students in the program alternate (on successive days) between working with an in-person, trained tutor and individually engaging with an education technology platform. The student-tutor ratio is 4:1. The program takes place 50 minutes a day, 5 days a week, for the full 9th grade school year.

Study Design:

• The study sample comprised 2,065 incoming 9th graders in 6 high-poverty high schools (3 in NYC, 3 in Chicago). 81% were Black or Hispanic; 91% were low-income. Based on careful review, this was a well-conducted RCT (e.g., baseline balance, low attrition, prespecified outcomes and analyses).

Findings:

​

• The study found a sizable, statistically-significant impact on end-of-year math scores on district tests. The effect size was 0.19 - which represents an 86% improvement over the annual gain in math otherwise expected for 9th graders.

• The study also found significantly improved math GPA and fewer math course failures through fall of 10th grade, after which COVID interrupted study follow-up.

Comment:

​

• Prior RCTs have found sizable, sustained effects for Saga's traditional program (2:1 tutoring ratio, no tech component). Saga Technology's impacts are almost as big, but the cost is 40% lower (around $2600 per tutored student).

• Taken together, these studies constitute strong, replicated RCT evidence - providing confidence that schools which adopt and faithfully implement Saga's traditional or technology model will see meaningful math gains for low-income students.

• Disclosure: my former employer, Arnold Ventures, helped fund this study.