Evaluation Tools for Community Development and Health

Evaluation Tools for Community Development and Health

Success Measures^® developed the Success Measures^® Health Outcome Tools to help organizations collect evidence of health-related outcomes for their programs, such as an initiative to improve food access through a free farmers market.

As the affordable housing and community development sectors in the United States have become more aware of the intersection of health and housing, organizations have increasingly sought to implement programs that integrate their work with health-related programming. Often, however, entities outside of the medical field find evaluating and documenting health-related outcomes challenging. To help nonprofit organizations collect evidence of health-related program outcomes, NeighborWorks America’s evaluation resource group, Success Measures^®, developed the Success Measures^® Health Outcome Tools. On April 10, 2018, Success Measures^® presented the webinar “Success Measures Evaluation Tools for Community Development and Health,” during which speakers Dr. Melissa Nemon, owner and founder of Nemon Consulting LLC, and Carrie Copeland of Cornerstone Community Housing (CCH) discussed the Health Outcomes Demonstration Project.

The Tools and the Social Determinants of Health

NeighborWorks began developing the Health Outcome Tools in response to requests from community development organizations that wished to work with health partners but lacked the tools to specifically measure health-related impacts. Nemon, a consultant with the Health Outcomes Demonstration Project, attributed this interest in health-based partnerships to a growing awareness of the social determinants of health over the past decade. The Centers for Disease Control and Prevention defines the social determinants of health as “the complex, integrated, and overlapping social structures and economic systems that are responsible for most health inequalities.” Success Measures^®, which has developed more than 350 measurement tools pertaining to revitalization, financial capability, and resident engagement, began work on the new health evaluation tool in 2014. Field testing in English and Spanish was completed in October 2015.

The process resulted in 68 new data collection tools that can be used to gather both primary and secondary data through numerous formats, including surveys, worksheets, focus groups, calculators, observations, and interviews. Primary data are obtained directly from program participants; for example, by conducting an interview with questions provided by the tools, whereas secondary data are obtained from sources such as administrative records by using worksheets. Most of the tools consist of pretested questions about a given subject and are typically between 8 and 12 questions long. Although the tools are appropriate for analyzing nonmedical programs sponsored by healthcare organizations, the tools are intended to be used by laypeople, making them more accessible to nonclinical entities. Topics cover a broad spectrum of health outcomes and address changes in areas that traditionally are difficult to quantify, such as attitudes and beliefs. Because the tools are intended to measure changes for individuals and communities as well as direct and indirect benefits, they can help organizations articulate the value of their work.

The tools have thus far been subjected to one pilot test, and additional evaluation is ongoing. The first study, the NeighborWorks America Health Outcomes Pilot, was conducted with 11 organizations over 6 months in 2016. The second, the three-year Health Outcomes Demonstration Project, is a joint effort by NeighborWorks America and Enterprise Community Partners through 2019. In this project, Success Measures^® is providing 20 nonprofit community development organizations with grant funding, technical assistance, peer learning, and the Health Outcome Tools. The organizations are using the tools to plan and implement a health outcome evaluation of their own work.

Evaluation in Action at Cornerstone Community Housing

CCH is a nonprofit affordable housing developer and service provider operating in Lane County, Oregon since 1992. CCH owns and operates mixed affordable housing complexes in rural Oregon, serving families and individuals earning between 30 and 60 percent of the area median income. CCH is using the Health Outcome Tools to evaluate two programs: Extra Helping, a program that distributes free produce, and CCH’s coaching program, which helps clients set and meet wellness and parenting goals. Copeland, CCH’s impact and evaluation manager, spoke about her organization’s experiences. “[The demonstration] is helping us think about data differently,” explained Copeland. “We’re excited to build smarter, win-win programs with clinical health providers.”

Copeland spoke specifically about Extra Helping, describing the program as a free farmers market held twice a month in a community space in collaboration with the local food bank, during which staff give visitors referrals to other services. The events are intended to increase access to healthy food, improve social cohesion, and contribute to resident awareness of resources. To evaluate the program, CCH took questions from nine different tools and combined them into a single paper survey, which the organization distributed at two Extra Helping events.

Staff found that the answers “flew in the face of what we had assumed,” reported Copeland. They found that participants do feel more socially connected to the community and had a slightly increased incidence of vegetable and fruit consumption compared with nonparticipants. They also found that food is the first thing that most participants gave up to afford rent, whereas nonparticipants tended to give up transportation first. “We decided we really needed to tell funders this,” said Copeland, “because it demonstrates that we really need to bring services to the housing communities.”

Copeland concluded by saying that the tools are helping CCH learn how to “speak the language” of clinical providers, a useful skill because of the increased work the nonprofit is doing with such entities. “It’s also helped us have conversations with property owners, funders, and legislators,” she said. “We’ve been using the data to talk about the need for social service supports.”

Using the Success Measures^® Health Outcome Tools

Nemon provided evaluation planning tips for the use of the tools. “Figure out what you want to know and why, and then stick to it as you select tools,” she emphasized, explaining that while some details are interesting to know, it is vital to limit the scope of the evaluation and be realistic about the time and effort it will take. The tools are best applied using a participatory approach, and piloting is vital to a successful application.

The Health Outcome Tools are available for free download. Their use is restricted to noncommercial purposes, with attribution and inclusion of the copyright.