Defining Your Appropriate Evaluation Measures
There is no one “right” evaluation. Rather, a host of evaluation questions may arise over the life of a program that might reasonably be asked at any point in time. Addressing these questions about program effectiveness means paying attention to documenting and measuring the implementation of the program and its success in achieving intended outcomes and using such information to be accountable to key stakeholders. Program evaluation is a tool with which to demonstrate accountability to an array of stakeholders who may include funding sources, policymakers, state, and local agencies implementing the program, and community leaders. Depending on the needs of stakeholders, program evaluation findings may demonstrate that the program makes a contribution to reducing morbidity and mortality or relevant risk factors; or that money is being spent appropriately and effectively; or that further funding, increased support, and policy change might lead to even more improved health outcomes. By holding programs accountable in these ways, evaluation helps to ensure that the most effective approaches are maintained and that limited resources are spent efficiently.
SAE’s preferred approach is to develop an evaluation team that includes internal program staff, external stakeholders, and an SAE evaluator with subject matter and evaluation expertise that is aligned with the proposed project. The underlying logic of SAE’s Behavioral Health Evaluation Unit’s (Evaluation Unit) Framework is that good evaluation does not merely gather accurate evidence and draw valid conclusions but produces results that are used to make a difference. SAE frames each evaluation project using the following guides:
- Utility: Who needs the evaluation results? Will the evaluation provide relevant information in a timely manner for them?
- Feasibility: Are the planned evaluation activities realistic given the time, resources, and expertise at hand?
- Propriety: Does the evaluation protect the rights of individuals and protect the welfare of those involved? Does it engage those most directly affected by the program and changes in the program, such as participants or the surrounding community?
- Accuracy: Will the evaluation produce findings that are valid and reliable, given the needs of those who will use the results?