Learning about Data Use in Pay for Success through the HUD-DOJ Demonstration

Learning about Data Use in Pay for Success through the HUD-DOJ Demonstration

In this column, Marina Myhre, social science analyst in the Program Evaluation Division of HUD's Office of Policy Development and Research, discusses the report, “Data Use and Challenges in Using Pay for Success to Implement Permanent Supportive Housing: Lessons From the HUD-DOJ Demonstration.”

What is Pay for Success (PFS)?

Pay for Success (PFS) is a mechanism by which the local jurisdictions work with intermediaries to fund interventions intended to create positive outcomes related to a particular issue that a community is facing, such as the cycle of criminal justice and homelessness for a particular population. PFS is an approach where a local jurisdiction pays for outcomes achieved rather than paying directly for the services provided. PFS deals are complicated because they involve multiple actors, such as governments, funders, financial intermediaries, knowledge intermediaries, service providers, and an independent evaluator. Under this structure, private investors, rather than local jurisdictions, pay the upfront costs for the interventions. Then, if the interventions are successful in creating the desired outcomes, the end payor, which is usually the local jurisdiction, pays for the “success.” In the HUD-DOJ Permanent Supportive Housing (PSH) PFS Demonstration, the intervention is permanent supportive housing, and the target group is a population experiencing chronic homelessness who are the highest users of jails, shelters, emergency rooms, and other costly crisis services. In this case, “success” is expected to be measured by reductions in homelessness and recidivism and improved health outcomes.

What are the potential benefits of using a PFS approach to financing a project or program, such as a permanent supportive housing project?

One potential benefit of PFS is that the risk is shifted to the private investor and away from the government. Another benefit is that this structure solves the "wrong pockets" problem, where the entity that pays for the intervention is not the entity that experiences the financial benefits from its success. For instance, a homeless service provider may pay for permanent supportive housing, however, this intervention may result in lower costs for emergency health services, jails, or prisons. So, by preventing recidivism, the homeless service provider is saving money for the ER, jail, or prison. Every entity, including the homeless service provider, has its own budget, and the homeless service provider is not able to recognize the return on investment within its own budget (or the budget of the local jurisdiction’s agency that typically funds homelessness services). Under PFS, rather than one organization worrying about their budget while another experiences costs savings, the overall system and the jurisdiction reap the benefits.

What is the typical structure of a PFS model?

The typical structure is that you have an intermediary, such as a nonprofit, that brings together all of the stakeholders or partners, including the local jurisdiction/government, the service providers, and private funders or foundations. The intermediary conducts a feasibility analysis. If the program is determined to be feasible, the program partners structure the transaction (develop a legal contract) under which the private funders or foundations agree to pay for the costs to implement the specified intervention, and the local jurisdiction or end payor agrees to repay the funders if certain success thresholds are met. After the intervention is implemented, an independent evaluator measures the success of the intervention and determines whether it was successful under the terms of the contract.

What kinds of data are needed to evaluate the success of a PFS program?

Initially, in the feasibility analysis phase of a PFS program, data are needed to help identify the size and scope of the target population and examine possible interventions and service providers that could promote positive outcomes. Data are also needed during this phase to estimate expected outcomes, as well as calculate potential costs and benefits for local jurisdictions that would be making success payments. Then, in the transaction structuring phase, data are needed to further refine the target population, if necessary, and help establish success metrics and performance thresholds for success payments. When the program is actually implemented, participating organizations may need data to find clients, track progress on outcomes, and document performance thresholds. Finally, independent evaluators need data in the evaluation phase to measure whether the interventions were successful, as defined by the evaluation metrics outlined in the contract.

What types of challenges have organizations involved in PFS programs faced when attempting to access or use data to measure success?

In order to define and evaluate success, the different agencies that are involved have to be able to collaborate. The participating agencies may also need to collect data from several different systems. If the program requires the use of health data, the agencies must be careful not to violate Health Insurance Portability and Accountability Act (HIPAA) requirements, and with any personal data, the agencies must address data privacy concerns. For example, data sharing between a correctional system, such as a jail or prison at a city, county, or state level, and a health department to look at EMS calls and ER service provision, can be very complicated. These entities are generally not used to working together and linking their data.

Are there any approaches that can mitigate data-related challenges in PFS?

Yes, stakeholder engagement is critical, with stakeholders representing the agencies that own data in the criminal justice, housing and homelessness, and health domains, forming working groups to facilitate access to data. Maintaining active partnerships was critical for data access, negotiating data sharing agreements, and allocating staff time to fulfill the data requests. In one case, the intermediary played the key role in getting stakeholders to work together. In other cases, subgrantees or other project partners from the local jurisdiction were responsible for establishing stakeholder working groups and overcoming data-related challenges.