Tips for Assessing Evidence-Based Practices
In child welfare, as in any field, most practices are not easily determined to be evidence-based or not evidence-based. Each practice must be assessed along a continuum, from those that are supported by extensive, well-conducted research to those that are untested and based on questionable application of theory.
Here are some questions to consider when assessing identified practices:
- Do the outcomes described apply to the problem or issue that was the basis of the literature search?
- Are the strategies and outcomes described consistent with the values of the communities, families, and children that will be served?
- How long are the outcomes of the intervention sustained?
- Is the practice informed by accepted child welfare theory?
- If another entity has identified the practice as evidence-based or a best practice, how credible is the source?
- What criteria did that entity use to make this determination?
- Is the purpose of the study clear?
- Is the sample size large enough to be convincing?
- Was the sample representative of the larger population (e.g., random selection)?
- Was the attrition rate acceptable? (How many people dropped out of the study from the initial group? For what reasons?)
- Did the researchers use a comparative research design? This might mean:
- The group that received services was compared to another group that did not (control group), and subjects were assigned to the two groups randomly ("experimental design"—this is preferable).
- The group receiving services was compared to a control group, and the subjects were matched by selected characteristics.
- No control group was used, but changes in families receiving services were measured using a consistent pre- and posttest.
- Are the conclusions supported by research results? Does the evaluation reflect more than subjective observation? Does the data presented actually support the findings or just describe the population?
- Do the researchers discuss the limitations of their findings? Do the researchers present the findings objectively?
- Have the researchers taken into account other factors that may have contributed to the outcomes?
- Has the research been externally or peer reviewed? Have the results been replicated?