US Department of Labor (DOL) published RCT results for DOL's Ready to Work (RTW) Partnership Grants, providing employment services to long-term unemployed or underemployed workers. Quick take: High-quality RCTs of 4 RTW projects find no discernible positive earnings effects over 3-4 years.

​

Program:

​

• The RTW program, operating from 2015-2019, awarded grants to local agencies to provide un- or under-employed workers with customized services including staff guidance on career planning, occupational and work-based training, employment readiness courses, and job search assistance.

 

Study Design:

​

• DOL chose 4 local RTW programs for the RCTs: (1) Maryland Tech Connection (MTC); (2) JVS programs (Skills to Work in Technology & Job Search Accelerator) in San Francisco; (3) Finger Lakes Hired (FLH) in Monroe County NY; and (4) Reboot Northwest (Reboot NW) in Portland OR & Vancouver WA. Each program had flexibility to develop and customize services to meet local needs.

​

• The RCTs had samples of 1,029 (MTC), 993 (JVS programs), 610 (FLH), and 980 (Reboot NW) workers, and were well-conducted based on careful review (e.g., baseline equivalence, negligible attrition).

 

Findings:

​

• Unfortunately, none of the RCTs found discernible positive earnings impacts over follow-up periods of 3-4 years, and one program - MTC - had a statistically significant adverse impact over the pre-specified primary follow-up period (quarters 5-10). The graphs tell the story:

​

​

​

​

​

​

​

​

​

​

 

 

 

 

 

 

 

 

 

 

Comment:

​

• I believe the disappointing results reflect a flaw in how federal agencies often select programs for large RCTs: They underestimate the challenge of finding programs that are truly effective (many plausible-sounding ideas just don't work in practice).

​

• If the goal is - as I think it should be - to build the body of proven-effective programs which, if scaled-up, could help tackle major social problems, the feds would do better to focus large RCTs on programs that don't just sound like good ideas (as with RTW) but are backed by highly promising prior evidence - e.g., from smaller RCTs or quasi-experiments. Doing so could achieve a much higher success rate, as discussed here.