Michael Sikes, Ph.D.
Evaluation, Research, and Consulting
Evaluation and Program Planning
Many programs have an evaluation of some form or another. Such evaluations can have value, depending on how carefully they are conducted. But ultimately, the most consistent difference between useful evaluations and the other kind is whether they are driven by an evaluation plan—a detailed description of what the evaluation will discover, how (with what tools and procedures), when, and with what personnel and resources.
An evaluation plan:
- Ties program objectives and activities to observable, measurable indicators and data.
- Helps test program logic and theory of action in real time.
- Provides input for mid-program corrections, as needed.
- Informs ongoing project implementation.
- Provides learning opportunities for those involved in the program.
- Builds the knowledge base of effective program practices.
Many programs, if not most, can benefit from an evaluation perspective from the beginning. Such a perspective incorporates several important stages:
Theory of Change
An important, possibly essential component of either an evaluation plan or a program plan is a theory of change. This is a description of the relationships among program activities and intended outcomes. It is different from its better-known counterpart, the logic model (see below), and in some ways precedes it because it articulates the all-important how and why of these interconnections.
Developing a theory of change is not an incidental process. It can involve program planners and stakeholders in hours of collaborative planning and thinking about internal program logic. The nucleus of such work is drilling down to the vital outcomes that are mission-critical, those whose attainment is essential to the success of a program or project. This identification is followed by a process of backward mapping—working backward from the outcomes to the intermediate and early steps that are key to their attainment. The evaluator's role in this process may be that of facilitator, or it may be as a methodological advisor. The end result is usually a kind of vertical flowchart, with the activities and intermediate steps toward the bottom and the outcomes at the top.
Not all evaluations use theory of change. Indeed, not all programs use it. But substantial experience in programs across sectors and throughout the world suggests that where it is used, programs and evaluations are more likely to bring about intended, useful consequences.
Logic Models: Indispensable Evaluation Tools
Almost any program or program evaluation will benefit by a clear articulation of what the program is supposed to accomplish, and as importantly, how the various stages of the program should lead to these accomplishments. Yet quite often, this relatively natural and simple component is missing from the program plan or the evaluation plan. A program logic model is the optimum tool for this purpose.
Going beyond the theory of change but building from it, a logic model is a representation, most often in the form of a table, of the program's goals, objectives, or outcomes, along with the various resources or components—often variously called activities, inputs, or outputs—that will be provided in the program to bring these objectives about.
An essential part of any logic model is of course logic—the if/then kind: If the program marshals these resources, then these results are likely to result. Such assumptions should rest in knowledge of previous programs or in research in the program area. For example, the logic model of a program designed to develop creative thinking should be grounded in the literature on creativity and learning. And as noted previously, an important element within program logic is one or more theories of change about the causal relationships between program components and their probable outcomes. Although distinctions vary, a logic model is generally a more public document, one that can quickly communicate the overall direction of the project to a variety of stakeholders.
An evaluator with experience in crafting logic models can provide valuable assistance in the program design phase, another reason that evaluation should be built into a program from the beginning stages of conceptualization and planning.
From a Logic Model to an Evaluation Plan
A logic model is useful in itself, but imagine a logic model that connects dynamically to every component of the evaluation plan. Click on any part of the model—an activity, an input, or an outcome—and a hyperlinked navigational system takes you to the detailed evaluation plan for that component, including the relevant indicators, data sources, timeframe, and other parts of a well-thought-out plan. This makes your logic model a portal into your evaluation landscape, which is updated frequently in response to new evaluation tasks and new data.
Sounds useful? It is. Such a roadmap can track every component being evaluated, the resources being used, and the progress to date, not only in terms of evaluation tasks completed but even in terms of findings. By providing information on the fly, this can be an enormously important tool for clients and managers of large, complex evaluation projects. Sounds complicated? Not so much. The tools that you need are readily at hand. A spreadsheet program like Microsoft Excel, some basic functions and macros are all that are required.
Building such a plan is like drawing the blueprint for a building, except the blueprint expands as the building becomes a reality.