Psychiatric Services published an RCT of a "Pay for Performance" approach to delivering care to adults with mental illness, aimed at reducing cost. Quick take: High-quality RCT finds no cost savings over 3 years - and a pre-post study, vs RCT, would've incorrectly found big savings.

​

Program and Study Design:

​

• The study randomized 652 adults with serious mental illness and a history of high care utilization to: (1) case management and housing subsidies provided by the county (control group) vs. (2) similar services provided by a contractor compensated via Pay for Performance [P4P] (treatment group).

• Under P4P, the contractor received financial rewards (penalties) if costs were below (above) projections. Based on careful review, this was a valid RCT (e.g., baseline balance, no attrition), although it only measured cost outcomes and not patient health or program implementation.

Findings:

• The study found no discernible impact on per-person cost of 24-hour psychiatric services over the 3 years after program inception (see graph below, based on report table 3). Note the major (>50%) drop in cost for both the treatment and control groups over the course of the study.

Comment:

​

• The likely cause of the drop is "regression to the mean" - i.e., patients were selected for the study based on high care utilization at baseline, and their average care utilization naturally declined to a less extreme level in subsequent years with or without a P4P approach.

• This is a reason why pre-post studies (without a control group) often yield the wrong answer. In this case, a pre-post study - examining whether costs were lower after program inception vs before - would've erroneously concluded P4P produced big savings.