Community Health Workers Toolkit

Evaluation Perspective

Program evaluation and outcome measurement are crucially dependent on perspective. What makes sense to measure for the program overall might not be the best measurement for clients or CHWs. The most common perspectives in developing a measurement plan are:

CLIENT

This perspective centers the clients. Initial planning could include potential clients in a focus group or other planning committee to choose measures that are the most important for the clients. Measures using this perspective would center the client and be based on the individual, most likely comparing the same person at different points in time.

COMMUNITY HEALTH WORKER

This perspective might be the most useful for measuring performance of CHWs or making decisions about the allocation of personnel and resources across various sites or programs. Measures using this perspective would center the CHW and so be measured at that level, either comparing the same CHW over time, or comparing among different CHWs.

PROGRAM

This perspective would allow metrics to be calculated for an entire program. An important consideration is how different aspects of the program are combined. For example, some clients are enrolled for 30 days compared to another set enrolled for 90. Consistency in measurement, especially in the timing, is critically important. Having one program measure 30-day emergency room usage and another measure 90-day emergency room usage means that some measures may need to be adjusted (multiply the 30-day rate by 3) with notes in the report(s). Some outcomes measured from the client or CHW perspective may be aggregated to provide a program perspective. Others, such as costs, can only be meaningfully measured at the program level.

PAYER

This perspective is most likely to be used by funders of CHW programs. Using this perspective implies that outcomes important to funders, such as costs, savings, and missed appointments, are specific to the location or funder. Many of these measures will overlap with measures used for the program perspective but will be calculated differently. For example, if your program has two funders, measures of the percentage of missed appointments for all clients will need to be separated into funder groups.

COMMUNITY

This perspective uses all available information to assess the overall outcomes at a community level. For smaller programs, especially in more populous communities, this will be a more difficult perspective to use. Measurement can be of community-level outcomes, such as the percentage of people reporting unmet health care needs or the percentage of people waiting for housing. It can also take information from the other levels, such as program costs, payer costs/savings, and combine that with other information, such as reduced Medicaid expenditures, or lower unemployment payments.

Choosing the perspective is not a one-time-only decision. There may be reasons to use multiple perspectives. For example, programs might use client perspective measures when discussing the program with other community organizations, or the payer perspective when writing a report for the payer. Thinking through and writing out the different perspectives and when they will be used can have a dramatic impact on reducing workload later when programs must be justified to different stakeholders. After identifying the appropriate perspectives, program developers can move onto selecting, testing, and using different measures.

Evaluating programs is important for sustainability as well as improvement. The Online Resources section provides available resources for conducting evaluations of CHW programs. However, general principles to keep in mind are:

  • Use a comparison group. Having a plan to compare outcomes across people who are/aren’t in the program and time can strengthen the impact of any evaluation.
  • Measure the process. Measuring the process, numbers of visits, referrals, etc., will help assess how closely the program meets the plan. This is especially important for programs addressing social determinants of health since many community resources require extension coordination and time to manage.
  • Measure the outcomes. Make sure that at least some of the measures are validated (have been used in other evaluations) to make comparisons easier.