WHY A CONTROL GROUP IS NEEDED TO DETERMINE WHETHER A SOCIAL PROGRAM IS EFFECTIVE, in 4 simple charts. The example I’ll use is HHS’s Comprehensive Child Development Program (CCDP), which provided intensive case management services to low-income families with young children.

​

• As shown in chart 1 (below, left), mothers in CCDP saw their employment rate more than double over the 5y after program entry. But CCDP was evaluated in a rigorous study that had a randomly-assigned control group of families, & mothers in the control group saw an almost identical gain (below, right).

​

​

​

​

​

​

​

​

​

​

​

​

• So CCDP had no effect on mothers' employment, versus an equivalent group of mothers who didn't participate. If CCDP had been evaluated in the usual non-rigorous way - examining employment outcomes without reference to a control group - it would've been (erroneously) deemed highly effective.

• Same story with child outcomes. Chart 3 (below, left) shows that, among children in CCDP, the percent scoring "at risk" in cognitive development & behavior fell sharply from ages 2&3 to age 5, during their time in the program.

​

​

​

​

​

​

​

​

​

​

​

​

​

• But control-group children saw almost identical gains (chart 4, above right). Conclusion: Simply looking at before & after improvement in participant outcomes, without respect to an equivalent group of nonparticipants (controls), often yields the wrong conclusion about program effectiveness.

• Here's a link to the HHS/ACF study report.