Evaluating Cohort #1 of the Moving to Work Demonstration Program Expansion: Understanding the Effects of PHA Flexibility for Small Public Housing Agencies

Research Design, Data Collection & Analysis Plan (RD/DCAP)

The focus of the evaluation of the first cohort of MTW expansion agencies is to explore how smaller PHAs use the flexibility offered by their new MTW designation to achieve the statutory objectives of the MTW demonstration and what the consequences of that flexibility are for housing authority operations and tenants. The evaluation is structured as a randomized controlled trial (RCT). The PHAs selected to participate in the MTW demonstration for this first cohort of agencies were randomly selected from a pool of eligible applicants. The RCT design allows the study to measure the impact of the offer of MTW designation by comparing the outcomes of the new MTW agencies (treatment group) to the outcomes of the eligible applicants that were not selected (control group).

The Research Design, Data Collection and Analysis Plan (RD/DCAP):

• articulates the key research questions,
• describes the research sample,
• outlines the approach for conducting the process study, including the data collection methods, and the process study data analysis plan,
• outlines the approach for conducting the impact analysis, including the outcomes measures of interest, relevant data sources, and the analytic approach, and
• presents the timeline for data collection, analysis, and reporting.

Back

Reports

Evaluating MTW Flexibility for Smaller PHAs: Baseline Report. This report, entitled Evaluating MTW Flexibility for Smaller PHAs: Baseline Report, is the first of five annual reports that will document the evaluation of the first cohort of the MTW Expansion. This first cohort of the MTW Expansion includes 31 smaller PHAs, defined as PHAs administering 1,000 or fewer combined units, and tests the overall impact of MTW flexibilities for smaller PHAs. The overarching research question to be answered with this first cohort is: how does providing MTW regulatory flexibility impact smaller PHAs? Using data from PHA applications and telephone interviews, the Baseline Report explores PHA motivations for participating in MTW and the interrelated objectives that these new MTW agencies hope to pursue using the flexibilities afforded through their MTW designation to improve program operations for tenants. In addition, the report lays the groundwork for future reports by detailing the outcome measures that will be used to assess the impact of MTW designation and by documenting and confirming the soundness of the experimental research design for this evaluation.

Evaluation of the Moving to Work Flexibility Cohort: First Year of MTW Eligibility. The evaluation of the first cohort of agencies participating in the MTW expansion focuses on how smaller PHAs use the flexibility offered by their new MTW designation to achieve the statutory objectives of the MTW program and what the consequences of that flexibility are for housing authority operations and tenants. By the end of April 2022, all 31 PHAs in the Flexibility Cohort had fully executed their Annual Contributions Contract (ACC) amendment that finalized their designation as an MTW agency, but only 11 PHAs had an approved MTW Supplement to their PHA Annual Plan and were thus eligible for beginning to implement MTW flexibilities. Nonetheless, all 31 PHAs continued to advance their planning efforts over the course of this first year of MTW designation, refining the set of MTW activities that they intend to pursue and considering how the MTW funding flexibility would factor into these activities. The onboarding process has been slower than anticipated, and while PHA’s enthusiasm for the MTW program has not waned, the reality of implementing big changes in small agencies has been a central challenge for these 31 PHAs in this first year.

Evaluation of the Moving to Work Flexibility Cohort Year 2 Report. The evaluation of the first cohort of agencies participating in the MTW expansion, referred to as the Flexibility Cohort, focuses on how smaller PHAs use the flexibility offered by their new MTW designation to achieve the three statutory objectives of the MTW program: cost effectiveness, self-sufficiency, and housing choice. The report builds on the Baseline and Year 1 Reports, detailing the early implementation experience (process study) of the Flexibility Cohort PHAs and estimating program impacts (impact study) to date on outcome measures associated with the three statutory objectives. By the end of 2022, 26 of 31 PHAs in the Flexibility Cohort had an approved MTW Supplement to their PHA Annual Plan and were thus eligible to begin implementing MTW flexibilities. Among the 26 PHAs, 236 MTW Waivers were approved for implementation, with the median number of approved MTW Waivers per PHA at 8. The PHAs are making progress towards the three statutory objectives: 24 PHAs have approved waivers for cost-effectiveness activities, 20 PHAs for self-sufficiency activities, and 19 PHAs for housing choice activities. However, PHAs are only in the beginning stages of implementation with only 13 PHAs having approved MTW waivers for at least six months. The PHAs have taken a more conservative approach when implementing waiver activities, aiming to learn from initial implementation before moving forward with a full suite of waiver activities. Therefore, it is still too early to observe program impacts and statistically significant impacts estimated in this report are likely due to chance.