Home > Preventing Child Abuse & Neglect > Evaluating Prevention Programs > Steps in Evaluating Prevention Programs

Steps in Evaluating Prevention Programs

• Step 1: Prepare for the evaluation
• Step 2: Develop a program logic model
• Step 3: Develop an evaluation plan
• Step 4: Collect data
• Step 5: Analyze data and report results

A successful evaluation begins with good planning and preparation. There is much to be done before the first data are collected, beginning with careful thought to what it is you want to know about the program and what you intend to do with the information once you gather it. Following are the key steps in evaluating a child abuse prevention program.

Step 1: Prepare for the evaluation

Involve staff, service providers, and program participants in the design process. Engage them in discovering the important questions to be addressed. Ask staff what obstacles they foresee to evaluating the program and how they think the results should be used. Identify who else is going to use the evaluation, and understand what each group of users expects to learn from the process.

You may also wish to conduct a review of the literature. Learn from how other programs have conducted their evaluations, including design framework, instruments, and data collection procedures. Many of the resources listed in this section may assist you in this process.

(Back to Top)

Step 2: Develop a program logic model

A logic model of your program illustrates the relationship between program inputs (resources, staff, curriculum), outputs (services you are providing), and the expected outcomes for participating families. A logic model includes several levels of expected outcomes. Immediate outcomes include changes in knowledge, skills, and attitudes (increased knowledge of positive discipline techniques, for example). Intermediate outcomes reflect changes in applied skills and behavior (for example, increased use of positive discipline skills). Long-term outcomes (or impact) are what you hope the program will ultimately achieve (i.e., a decrease in the incidence of child abuse and neglect).

• Logic Model Builder
  Child Welfare Information Gateway
  This user-friendly tool takes you through the process of developing a logic model and includes program evaluation tools specifically related to family support and prevention services.
  
  
• Point K Learning Center
  Innovation Network
  Practical tools and resources for nonprofit planning, evaluation, and action. (Free registration required.)
  
  

(Back to Top)

Step 3: Develop an evaluation plan

Use the logic model to identify your expected outcomes and the strategies you will use to achieve them. From these, you will identify measurable indicators of success. For example, prevention programs might measure decreases in parenting stress or increases in parents' knowledge about child development, maternal social support, family functioning, marital satisfaction, child health status, or home safety among participants. View a list of common risk and protective factors for child maltreatment.

You will also need to select an appropriate research design. There are three basic types of evaluation designs that are typically used in outcome evaluation.

• Experimental. Families are randomly assigned to experimental and control groups, with program services provided only to families in the experimental group. Experimental designs are the strongest framework for establishing causal relationships and have the highest credibility. These designs also tend to be more costly and time-consuming.
  
  
• Quasi-experimental. These designs may use either multiple groups for comparison or multiple data collection points over time, but do not assign families to groups randomly.
  
  
• Nonexperimental. Nonexperimental designs (which do not use multiple groups for comparison or collect data over time) cannot establish a direct connection between program services and family outcomes, but can provide valuable information about processes.

Your research questions, as well as time and resources available, will be the major considerations driving your design selection.

Using Resilience as a Framework for Evaluating Safe Start Outcomes (PDF - 139 KB)
Association for the Study and Development of Community (2003)
Explains the resilience framework, discusses methodological and theoretical considerations for evaluation, and describes the challenges to adopting the framework when evaluating the Safe Start program.

(Back to Top)

Step 4: Collect data

There are a number of data collection methods available to support program evaluation. These include:

• Surveys and questionnaires
• Interviews
• Standardized tests and assessments
• Observations
• Focus groups
• Case studies
• Program records
• Existing data

Logic Model Builder
Child Welfare Information Gateway
This user-friendly tool takes you through the process of developing a logic model. The Logic Model Builder helps users identify the data they need to measure, as well as specific data collection resources to measure progress and identify outcomes.

Evaluating and Reporting Outcomes: A Guide for Respite and Crisis Care Program Managers
ARCH, Chapel Hill Training-Outreach Project (2002)
Evaluation tools for respite and crisis care programs, including published scales that can be purchased, scales that can be photocopied, and instructions for creating customized scales.

Measurement Issues in Child Maltreatment and Family Violence Prevention Programs
DeVoe & Kantor
Trauma, Violence, and Abuse: A Review Journal, 3(1), 2001
View Abstract
Offers guidelines for selecting appropriate domains for evaluation and information on common instruments measuring family and child well-being, parenting, stress and coping, and family violence.

Program Information Management System (PIMS)
Healthy Families America
Describes a computerized data collection, management, and reporting tool that enables sites to manage and report on the community programs and participant services they provide.

(Back to Top)

Step 5: Analyze data and report results

In addition to reporting results to external stakeholders, it is important to use the knowledge gained through the evaluation process to improve services.

Development and Implementation of a Quality Assurance Infrastructure in a Multisite Home Visitation Program in Ohio and Kentucky
Ammerman, Putnam, Kopke, Gannon, Short, Van Ginkel, et al. (2007)
In Healthy Families America Initiative: Integrating Research, Theory, and Practice
View Abstract
Describes a web-based system and quality assurance procedures designed to collect data and document outcomes from a home visitation program.

(Back to Top)

Updated on September 18, 2008