- » Emerging Practices in the Prevention of Child Abuse and Neglect
- » Future Directions
Emerging Practices in the Prevention of Child Abuse and Neglect
Thomas, D., Leicht, C., Hughes, C., Madigan, A., Dowell, K.|
|Year Published: 2003|
The Emerging Practices in the Prevention of Child Abuse and Neglect project was initiated by the Children's Bureau as a collaboration between the agency and the professional community to generate new information about effective and innovative approaches to the prevention of child maltreatment. While the information contained in this report contributes to an ever-broadening body of knowledge about the type and range of programs in the United States for the prevention of maltreatment, it is clear that much more can and must be learned about the effectiveness of these programs in terms of what works and for whom. This section discusses the need to expand existing knowledge about effectiveness, describes the goal of integrating research into practice, and identifies potential next steps for this project.
Need to Expand Existing Knowledge About Effectiveness of Prevention
Existing knowledge about the efficacy of prevention in the field of child maltreatment is limited; clearly, all the major prevention models and strategies could benefit from more rigorous study. Based on the evidence available today, home visitation programs have demonstrated some capacity to improve family functioning and wellness, reducing the risks faced by children. While there is evidence that a few home visitation programs actually reduce the number and severity of subsequent child abuse reports, evaluations of other home visitation programs have not measured program impact on incidence of maltreatment or have found no differences over time between experimental and control groups.
The record for parent education programs, which strive to increase family wellness by improving the knowledge and skills of parents, is neither rich nor, on the whole, particularly compelling. Though numerous studies of parent education programs have demonstrated positive findings, these findings have largely been limited to short-term gains among participants in parenting knowledge, skills, and abilities. The issue of statistical versus clinical significance is an important one. Many of the studies report findings of statistically significant differences between "treatment" and comparison groups on these measures. The question to be asked, however, is whether observed differences in short-term knowledge gains are meaningful in terms of actually protecting children, i.e., are the knowledge gains correlated with fewer actual abuse incidents? Taken as a whole, little is known about the impact of these programs on child maltreatment in the long term.
Programs for children and parents that are designed to raise awareness about child sexual abuse is another area where there has been a recent and relatively concentrated research focus. Available research suggests that such programs, like parent education programs, can be successful at imparting information and changing behavior, but there is little evidence to conclude that these programs actually prevent child sexual abuse. Though a few very recent studies have examined the relationship between knowledge and behavior and subsequent incidence of maltreatment, leading to some encouraging findings, these studies have been idiosyncratic and the results, therefore, are not widely generalizable.
Very little is known about the effectiveness of universal initiatives that seek to prevent maltreatment by raising public awareness. Measurement of the effectiveness of these initiatives faces its own unique complexities. In the particular area of Shaken Baby Syndrome, for example, understanding and recognition among medical professionals of the constellation of injuries that constitutes the syndrome continues to progress, steadily increasing the reliability of the diagnosis. There is currently no national mechanism for counting cases, however, so the true incidence of the syndrome is not well understood. Until such time, evidence of the effectiveness of initiatives to prevent the syndrome will continue to be limited.
Still, in a recent discussion of the historical "waves" of prevention, Daro and Donnelly conclude that there has been substantial progress for prevention as a concept over the last three decades. Progress can be found today in stronger, more diversified partnerships, increasingly rigorous research, greater pooling of resources across agencies, and more unified thinking and sense of purpose (Daro & Donnelly, 2002).
Integrating Research Into Practice
Research and evaluation studies provide program administrators, policy makers, and service providers with information and insight into whether programs work and for whom. Rather than relying upon anecdotal evidence or intuition, programs are able to link services with performance measures and outcomes. The results can be used to revise or refine specific approaches, policies, and practices to ensure better outcomes for children and families.
There has been a long history of tension among researchers and service providers, however, with both often feeling disconnected from one another. Service providers and program administrators sometimes feel as if research is artificial or inapplicable, or that it applies to services that are materially different from what their programs offer. In addition, many programs lack the funding to cover the cost of evaluations, or sometimes evaluations are conducted as an afterthought to program design. It is important for researchers to conduct applied studies in real-world settings. At the same time, it is important also for service providers to be open to accepting results that may call for changes in their delivery methods or mechanisms.
Research has produced considerable, new information in recent years about both specific types of services, such as home visitation programs, school-based sexual abuse prevention programs, or parent education programs. In addition, some studies have focused on the effects of specific elements of programs, such as the intensity or length of services. However, it is difficult for program staff, who do not necessarily speak the language of researchers, to sift through the available research and determine the potential impact that research results may have on their programs. Bridging this gap continues to be a critically important challenge.
Though the notion of demonstrating impact through evaluation has made great strides, the maltreatment prevention community needs to continue to engender a results-based approach to management. Managing for results focuses an organization on its specific goals and objectives and entails the selection of appropriate performance measures and the use and the reporting of those measures for purposes of ensuring program accountability and promoting effective and efficient allocation of resources. This process should be engaged at the earliest possible point, as part of development and implementation of the entire program concept.
Though service providers increasingly recognize the importance of evaluation and performance measurement, and the connection between demonstrating impact and funding security, there is a present and understandable gap between the recognition and will of providers and their actual capacities. Thus, the field needs to continue to promote and encourage collaboration between service providers and local entities holding evaluation research capabilities, whether in arms of State and local government, universities, or other organizations. These connections are fundamentally important to professionals in the field who share a common interest in understanding the dynamics of what works in prevention and for whom.
Emerging Practices: A Pilot Test and Next Steps
The Emerging Practices in the Prevention of Child Abuse and Neglect project represents a first-ever deployment of a federally funded program nomination procedure specifically targeted toward programs for the prevention of child maltreatment. Working with a diverse pool of nominations, the Advisory Group met in October and November 2002 to review each nomination and to reach consensus regarding the final disposition and categorization of each nominated program. Nominations of programs submitted as "Effective" were reviewed with an emphasis on the quality of the methodological design and the integrity of the resulting program outcomes. For those submitted as "Innovative," reviewers looked for new, creative ideas and strategies for preventing child abuse and neglect.
The project faced considerable challenges, and there were important lessons learned for any future use of this methodology. One particular challenge for this project was the reality that prevention programs are diverse. The population of existing programs that should fall within the parameters of a project like this is not easily identifiable. The most difficult of all inclusion issues was whether the population of programs of interest to this process should include only those programs that define themselves as child maltreatment prevention programs. What about programs that are not specifically geared toward maltreatment prevention, but have the capacity to prevent maltreatment just the same by increasing family capacity, such as a parent-child reading program? The answer to this question drives a number of tasks that are critical to deploying any program nomination procedure, first and foremost of which is how national outreach and promotion of the project is conducted.
Under this first effort, the nomination process was intentionally designed not to be unduly restrictive, but rather to cast a wide net. The goal was to learn more about "what's out there" in terms of new programs on the prevention landscape, as well as to generate a sufficient number of nominations with which to work. Thus, the two principal nomination categories—Effective and Innovative—were broadly circumscribed in a way that would open the process to the maximum extent possible, leaving the burden of selection to the panel of expert reviewers.
During the course of the review, however, it became apparent that a substantial number of nominations did not meet the criteria for "Innovative" because of program age and activities, or the criteria for "Effective" due to inconclusive outcomes based on methodological design considerations. However, many of these programs either had interesting and unique aspects or had made a concerted effort to undertake research and evaluation with limited resources. The Advisory Group concluded that these programs had features that would be informative to the field. Consequently, the Advisory Group wanted to recognize the "noteworthy aspects" of those programs, especially those that made a good effort at evaluation and presented positive preliminary results, as well as programs with some unique aspect that could possibly be replicable or programs that could become candidates for more rigorous evaluation.
A careful review of the nomination categories utilized under this project will precede any future use of this methodology, with specific consideration given to clarifying and expanding the framework of categories to reflect the universe of prevention programs more precisely. This would require potential nominators to consider their programs against more specific requirements and standards, ensuring more unified, homogenous categories of programs for consideration. In addition, within categories, it may be possible to implement a numeric scale or rating system that would reduce interpretability and maximize objectivity, standardization, and interrater reliability.
A nomination process like this one is inevitably vulnerable to self-selection biases. While this report presents new information on numerous programs, it also is true that a number of widely known and respected programs were not nominated. The omission of major program models in this first-ever effort has implications for any future deployment of this methodology in terms of outreach and promotion. Any future use of the methodology may utilize an invitation procedure to ensure that known models are aware and have sufficient time to respond.
Finally, the review process was limited to information in accordance with submission requirements identified in the nomination instrument, supplemented, if available, by any reports containing supporting evaluative analysis. It is not altogether clear, however, whether all programs made the best possible case for their nominations. Some nominations, in fact, were not supported by sufficiently apparent internal logic in terms of the connections between what they do, why, and what they intend to accomplish. Site visits to programs were not possible in this initial effort, but will be considered in the future to collect additional information about programs in terms of services, delivery, setting, and participants.
Summary Observations About The Programs
Anticipated from the start, the programs nominated for consideration under this project are quite diverse with respect to the populations with which they work, the specific strategies they employ, and the level of intensity with which they involved families. A common thread among most, however, is an attempt to change both parental knowledge and practice by building relationships through some form of interaction between a family and a teacher, home visitor, or counselor. Research suggests that the issue of "dosage," or the intensity and duration of involvement with families, is a crucial one that drives the potential for change and improvements in functioning. Particularly for vulnerable populations, there may need to be significant involvement (relatively high dosages) to produce sufficiently meaningful and durable changes. One size is not likely to fit all; and different strategies may serve very different purposes.
While these programs had specific components, their successes are likely to be, to a significant degree, a function of the skills of their frontline staff at engaging families who are the target for the intervention, as well as in relationship development, communications, diagnosis, and modeling. Future exploration of effective practices will need to look beyond specific programs to the people who operate them and their skills in these particular domains, including how the program helps promote those skills among staff.
How This Report Will Be Used
The Emerging Practices in the Prevention of Child Abuse and Neglect project is one important component of a larger Child Abuse Prevention Initiative to promote greater visibility for child abuse prevention program and activities in 2003-2004. With support from the U.S. Department of Health and Human Services, this Initiative includes a series of events and partnerships with the broader child abuse prevention community to raise awareness of the issue in a much more visible and comprehensive way than ever before.
Based on the results of the nomination process, it is clear that more support is needed to develop a stronger evidence base for promising prevention program strategies. Part of the overall strategy needs to include increasing awareness among service providers about resources and opportunities that are already available at the Federal, State, and local levels. Over the last few years, the Children's Bureau has provided technical assistance on evaluation to all its discretionary grant programs. In addition, the FRIENDS National Resource Center for the Community-Based Family Resource and Support (CBFRS) Program provides training and technical assistance to the lead agencies in each State around the continuum of evaluation approaches.
In its yearly funding announcement, the Children's Bureau has emphasized that applications should include a clear logic model that presents the conceptual framework for the program, and explains the linkages between the problems and conditions that are the focus of the program and its goals and objectives. Through these efforts, the Children's Bureau hopes to encourage prospective applicants and funded programs to include a focus on outcomes and evaluation as an integral part of their policy and program planning.
As part of the Prevention Initiative, the Children's Bureau plans to support future work based on the findings of this report that can contribute to advancing theory, policy, and evidence-based practice in child abuse prevention. Certainly, more resources will need to be available in order to improve the capacity of prevention programs for demonstrating effectiveness in reducing child maltreatment. The Children's Bureau will be exploring various mechanisms to further this work through various partnerships with other Federal agencies and through the existing Federal Interagency Workgroup on Child Abuse and Neglect. Various State and local public and private agencies and foundations also have recognized the importance of prevention and have made commitments to support similar efforts.
This report will be used as a catalyst to launch an important discussion about short- and long-term goals for the prevention field. The primary goals of this discussion are to:
- Consider the gaps in our knowledge about the effectiveness of maltreatment prevention;
- Identify future directions and priorities for research; and
- Link research findings to the creation of effective, new prevention programs.
Federal, State, and local prevention organizations will need to continue to work together to strengthen the knowledge base. Much more work is needed to develop a national research agenda for child abuse prevention that identifies gaps, prescribes new questions to be addressed, and presents potential programs and program sites on which to focus future resources. The Emerging Practices in the Prevention of Child Abuse and Neglect project has provided some of the first steps and identified some research directions from which to start.