Continuous Quality Improvement

View an
Example from
the Field

Definition

Continuous quality improvement is the complete process of identifying, describing, and analyzing strengths and problems and then testing, implementing, learning from, and revising solutions. It relies on an organizational and/or system culture that is proactive and supports continuous learning. Continuous quality improvement is firmly grounded in the overall mission, vision, and values of the agency/system. Perhaps most importantly, it is dependent upon the active inclusion and participation of staff at all levels of the agency/system, children, youth, families, and stakeholders throughout the process (National Child Welfare Resource Center for Organizational Improvement and Casey Family Programs, 2005).

To implement continuous quality improvement, organizations should form a team that has knowledge of the system needing improvement, define a clear aim, understand the needs of those served by the system, and identify and define measures of success. In addition, organizations can advance toward continuous quality improvement by brainstorming potential change strategies; planning, collecting, and using data for effective decision-making; and applying the scientific method to test and refine changes (Iowa State University, 2006).

Involving stakeholders, creating a theory of change, and applying a continuous quality improvement framework to infrastructure components and systems of care principles are represented in the following illustration, developed by Robert Friedman (2005) at the University of South Florida.

Why Use a Continuous Quality Improvement Toolkit?

Having a robust continuous quality improvement system is important to consistently improve services and supports for children and families, and to ensure that you are making the most effective use of your resources. As service interventions evolve, your continuous quality improvement program can make sure that delivered services are relevant and contributing to positive results.

In today’s economic and competitive environment, it is also important to be able to justify why investments should continue to be made in human service systems. As administrators are determining how to spend funds given by a legislative body, it is equally important that the front line workers and supervisors of those workers are able to see the impact that services are having on the consumers of those services. Furthermore, your continuous quality improvement program can track the improvements made by your system in response to the Federal Child and Family Services Reviews and the Program Improvement Plan in response to the Federal review. This toolkit will provide you guidance in the kinds of questions you should examine depending on where you are in developing a continuous quality improvement system, the kinds of activities you may want to explore, a rich set of resources from communities that have launched a continuous quality improvement effort, and sage advice from those whom have led such efforts.

Systems of Care Principles and Values

The following are just a few ways systems of care principles and values might be evident in your community’s continuous quality improvement:

  • All key stakeholder groups determine measures to be used.
  • Your system of care governance body uses data from your continuous quality improvement program to inform all major decisions they make.
  • Processes and instruments gather information on short-term, intermediate, and long-term outcomes.
  • Results are used continually to improve systems of care for clients and families, practitioners, administration or governance, and policy.
  • Continuous quality improvement activities are integrated into all aspects of systems of care.
  • Client outcomes, program performance, and system measures are part of the continuous quality improvement process being assessed.
  • Personnel within systems of care have maximum access to data.

Goals

  • Creating a continuous learning environment.
  • Using data consistently to guide fiscal and programmatic decision-making.
  • Utilizing a management information system that can track data across agencies, if possible, and can produce data to inform decisions.
  • Developing a process for monitoring cross-agency data, if a common interagency management information system is not an option.
  • Developing evaluations that incorporate systems of care principles.
  • Developing processes for using data and outcomes to improve agency processes, procedures, and functions.

An important first step in developing an interagency continuous quality improvement program is to take a close look at what data, results, and evaluation groups already exist in other agencies and systems. For example, are there available existing management information systems or other existing resources that can be drawn upon? It is also important to look at the costs of implementing a continuous quality improvement program including the necessary staff and policies to support the work. Finally, many agencies have some sort of management information system that may be tied to a variety of funding sources. Moving from that singular agency management information system to one that involves stakeholders from different agencies and includes family involvement will be a necessary cultural shift for many agencies. Getting advice from communities and agencies that have made the shift to a comprehensive continuous quality improvement system can be valuable information that will inform your own early planning.

Activities, Questions to Consider

Continuous quality improvement pre-planning tasks and activities in your community might include:

  • Assessing existing agency continuous quality improvement processes that can be enhanced.
  • Looking to existing continuous quality improvement efforts in your community to see how they developed their system, its relevance to what you want to accomplish, what has changed because of their continuous quality improvement program, and what steps they took to put the system in place to see how it is working today.
  • Identifying staff and consultants with expertise in continuous quality improvement and child welfare who are people oriented to assist in developing continuous quality improvement processes within systems of care.
  • Building flexibility into any continuous quality improvement program.
  • Identifying members who are interested in serving on a continuous quality improvement committee.
    • Has a committee responsible for evaluation, data, and continuous quality improvement been established?
Voices from the Field

Voices from the Field: Pre-planning Phase

"Make sure that the evaluation team IS a part of the early programming/strategies in order to help align needs assessments and other evaluation components, such that there is a greater likelihood that data collected aligns with the goals of the system of care work."

– Lead Evaluator, North Carolina System of Care

“Some of the first steps for our Continuous Quality Improvement work were learning about what data we had on hand, understanding how to pull data from various sources, then communicating and working with staff and partner organizations. Next we needed to see what data was relevant to staff and partner organizations and make sure that the data we collected and used was good data.”

– Research/Evaluation Manager, Contra Costa County, California

“System of care principles were at the center of our work from the beginning. Our original logic model had in the first column each principle then bullet points with activities related to each principle.”

– Lead Evaluator, Kansas System of Care

Key Partners

The people who perform continuous quality improvement duties in your community might include:

  • Interagency data team
  • Data collectors and data analyzers
  • Administrative support
  • Service delivery staff
  • Continuous quality improvement personnel, consultants, and/or University evaluators

Sustainability

  • Take the time to assemble the right people to explore a continuous quality improvement program, including interagency partners, evaluators and family members involved in your system.
  • Do the proper research on existing high performing continuous quality improvement programs that may be applicable to your system.
  • Look at the existing data being collected across systems. Be aware of duplication and try to assure that the ultimate interagency data set gives a picture of the whole system.

Resources

Contra Costa County, California-System Improvement Plan 2010
This 39-page report looks at three Federal and State mandated areas of responsibility: safety, permanency and well-being of children. Using the performance indicators of the child and Family Services Reviews, it tracks the county’s past performance, Improvement Goal and Current Performance detailing strategies used to reach the performance level. This is a very solid presentation of material that shows the comprehensive nature of their continuous quality improvement program.

This phase is marked by taking a close look at what data you already have, where the data exist, in what form the data exist, and what, if any, duplication there is in the data. It is important to inventory the kinds of resources available as you think about a continuous quality improvement system. What kind of staffing exists for your continuous quality improvement work? Do you have the necessary hardware and software? It is also a time to bring people together to look at what you want to measure and how you will go about collecting the necessary data. Start to inform all involved parties that you are thinking of developing a robust continuous quality improvement system and would like their ideas and involvement. Furthermore, you may want to draw on examples of successful continuous quality improvement programs in your community where data is used to improve performance in order to build support for your emerging effort. Finally, it is a time to start thinking how you want to use the collected data and how you intend to analyze the information that you have on hand.

Activities, Questions to Consider

Continuous quality improvement planning tasks and activities in your community might include:

  • Establishing protocols for collecting and sharing information.
  • Determining what data are collected, how they are used, who has access to them, where they are housed, and who owns them.
    • Has a set of measures been developed to determine whether current and future investments yield expected results?
    • Is the information from evaluation activities used to inform and improve the delivery of services and supports to the population of focus? Are direct service workers able to view progress?
  • Establishing information system support and management.
  • Using your continuous quality improvement program to celebrate successes and to inform the status of your system in a strength-based manner.
  • Establishing feedback loops for the quality assurance process while monitoring function and including families.
  • Diagnosing, processing, and developing a remediation plan (continuous improvement, management benchmarks, and goals).
  • Identifying domains to be measured.
  • Establishing processes for identifying problems, solutions, and managing the quality improvement process.
  • Conducting geo-mapping data. Geo-mapping is a tool used to show the location and density of children, youth, and families in your system of care in order to maximize the allocation of services and resources.
  • Identifying resources and gaps in resources.
  • Identifying other research and evaluation resources.
    • Will quantitative and/or qualitative methods be used?
Voices from the Field

Voices from the Field: Planning Phase

“One powerful impact on our system of care was the development of logic models for each system of care principle. We worked with staff to hone logic models which led to high acceptance of logic modeling because of staff involvement in the development of them. This early work helped the system of care community to see what was going on and to guide their ongoing work and where they needed to go in the future…It is important to work with all stakeholders to understand the evaluation process. Explain what you can do with evaluation and reports. Teach people how to use data. Spend time in conversation – How does it relate to what we are doing here? Also learn from the community on how to best report data. Don’t assume you know the answers…We had family partners involved at the State and local level in our continuous quality improvement work. They looked at community profiles, qualitative levels, provider reports and offered feedback. All planning sessions had family partners involved. Families remained involved in the system of care work the longest of any stakeholder group. We used the logic model approach in developing the family navigator positions. This helped to develop a common understanding of the family navigator. We used logic modeling for seeking funding sources including the Department of Social & Rehabilitation Services which funded the family navigators for the first year. Providers would not participate in embracing the family navigators, they needed specific contract language that required them to work with the family navigators.”

Lead Evaluator, Kansas System of Care

“The continuous quality improvement work was an integral part of developing our system of care. Since the agency has bought into the system of care principles everything that we do is tied to those principles. We looked at developing measurable, realistic system change goals and process practice goals. We integrated continuous quality improvement into the State improvement plan which feeds into the Federal Child and Family Services Review and Program Improvement Plan. All of the work of our system of care thus was funneled into responding to the state improvement plan and thus the Federal Child and Family Services reviews and Program Improvement Plan. We used three tools to develop our state improvement plan. First, a thorough self-assessment instrument where we were looking at how we were doing with disproportionality issues; then a peer quality case review, where we wanted to see places we were weak and then look for counties that were strong in this area and bring them in to raise our quality in those areas needing attention; finally, the whole system improvement plan which generates much attention and energy… As you help the agency move more toward data development and synthesis and it shows results, you are asked to analyze more and more. People have reframed how they think about change. People are more empirically-oriented. They may ask now, Does this work? It is much more of a heart and head approach to the work versus a more heart only approach to what we do next. There is a clear shift in how people think.”

– Research/Evaluation Manager, Contra Costa County, California

“Our extensive blue printing process detailed what had been accomplished and still needed to be done each year from the inception of our system of care… When needs assessment questions were completed for what was actually being done they had a big impact.”

– Lead Evaluator, North Carolina System of Care

Key Partners

The people who perform continuous quality improvement duties in your community might include:

  • Interagency data team
  • Data collectors and data analyzers
  • Administrative support
  • Service delivery staff
  • Continuous quality improvement personnel, consultants, and/or university evaluators

Sustainability

  • The more you educate all stakeholder groups about the value of a continuous quality improvement program, the more valuable it will become to all concerned. The more valuable the continuous quality improvement program becomes to people the more likely they will fight to keep it as a part of the system you are creating.
  • Once you have built an understanding of what a continuous quality improvement will do for your system of care it then becomes equally important to have all stakeholder groups engaged in developing the benchmarks and outcomes you want to track in your system.
  • Developing logic models for each activity is another way to show all concerned how the change process will work and where you ultimately want to be with each activity.

Resources

Contra Costa County, California – Using Data for Sustainability (PDF - 1.0 MB)
This 27-slide PowerPoint presentation first lays out the foundation necessary for a vibrant continuous quality improvement program by paying attention to the population of focus, readiness, necessary partnerships, systemic and individual impacts, and approach in collecting and using the data. The slides then go on to address the various forms that data is collected and utilized to make the case of sustaining the work of the system of care.

Kansas – Activity Models (PDF - 74 KB)
This is a comprehensive set of 6 Activity Models (Statewide Implementation, Cultural Competence, Social Marketing, Support to Local Steering Committees, Family Involvement and Training) that detail the work required to fulfill the goals for each activity. Each activity model has a specific goal then 4 headings; Inputs, Tasks/Strategies, Outputs and Outcome Measures, then the specific activities under each heading. The activity models are a powerful communication tool and are very useful in guiding the discussion from the theory of change through identification of need and population of focus, key resources and participants, activities, identification of short-, medium and long-term expectations as well as the actual performance outcome measures.

In this phase of a continuous quality improvement system it is important to see how all the parts of the system are working together. Adjustments and improvements may need to be made to strengthen the system. It is also a time that as data are collected, cleaned and synthesized, that all stakeholder groups be involved in receiving the data as well as have a chance to ask questions of the data so as to inform their work. Making sure that supervisors and front line workers as well as interagency partners and families have access to the data is very important. At the same time, you want to be sure that administrators and policymakers are also able to see the information so they can make prudent decisions based on solid information gleaned from the identified individual and system outcomes.

Activities, Questions to Consider

Continuous quality improvement implementation tasks and activities in your community might include:

  • Identifying emerging trends in the environment (e.g. population, fiscal).
  • Connecting to universities and other programs and determining key success factors (e.g. what is adoptable versus adaptable, evidence-based practices).
  • Monitoring service utilization.
  • Creating an interagency management information system, if possible. If one already exists, work to enhance the membership to include all system of care stakeholder groups.
    • Has an interagency team been created to determine the ability to purchase and administer an interagency management information system, or discuss other methods to streamline and integrate information about the population of focus?
  • Conducting training of all relevant stakeholder groups and conveying the importance to the system of the data being collected as part of a learning community culture being developed and a sustainability strategy.
  • Ensuring continuous quality improvement is integrated with the rest of the initiative.
  • Integrating national, State, and local evaluations like the Child and Family Services Reviews, State Improvement Programs, and the Program Improvement Plan.
  • Identifying trends and presenting information to various key stakeholder groups for dissemination and feedback purposes.
    • Are you regularly presenting your data results to stakeholder groups in a timely manner?
    • Are you using charts, graphs, and anecdotal evidence to amplify the information presented?
  • Providing support.
  • Embracing mistakes as important steps to success.
    • Are you offering analysis of the data that does not impugn the work of the front line staff but rather helps them see the strengths of their efforts and how they can improve?
Voices from the Field

Voices from the Field: Implementation Phase

“As the lead evaluator, I was a member of administrative team from the beginning. My office was just down the hall from the Child Welfare Agency Director. The administration embraced evaluation and the evaluator position…Having also been trained as a clinical psychologist helped me connect with the case workers. Get in and understand the work of the agency. I recommend that you find out what the agency is doing, how they are doing it, how does the clinical process work? What are the goals and mission for the organization? You may have to learn new things …get involved. It is easy to isolate yourself but by meeting the staff, presenting to the staff, you build a relationship that will help shift how people think about evaluation. You will start to have people ask, what about this? Also must be open to local factors. All of this will pay big dividends. As staff has gotten to know me they have started to e-mail me questions about the use of data…”

– Research/Evaluation Manager, Contra Costa County, California

Key Partners

The people who perform continuous quality improvement duties in your community might include:

  • Interagency data team
  • Data collectors and data analyzers
  • Administrative support
  • Service delivery staff
  • Continuous quality improvement personnel, consultants and/or university evaluators

Sustainability

  • The more you use the data the more powerful they become in the decision-making processes of your system.
  • Be creative in how you exhibit data. In reports, presentations, and other venues, using pictures, pie charts, graphs, and other visual representations of data used can help translate hard numbers into meaningful pieces of information for all audiences.
  • Combining data with stories of real people can make your data come alive and help interested parties connect more with hard data.
  • Make sure you share your data with your social marketing team so they can broadly disseminate the data to the media, general public, and other strategic groups of people who will be interested in what is happening in your system of care. The more you inform the public in a meaningful way about how you are using your data, the more likely you will be able to sustain your effort beyond any grant funds.

Resources

Contra Costa County, California – Emancipated Youth Presentation (PDF - 344 KB)
This 34-slide PowerPoint presentation is a great example of the kind of analysis that you can do with a robust set of data. The focus here is on youth involved in child welfare who are in transition to emancipation. This kind of data presentation could be very useful to administrators, policy makers, front line staff, community stakeholders, and youth groups in trying to find solutions for this special population of youth.

Jefferson County, Colorado – Advisory Meeting Presentation (PDF - 74 KB)
This 21-slide PowerPoint presentation to the system of care advisory board shows baseline and one year later data on staff and family perceptions of cultural competency and family engagement. Also shown were the various interventions that took place between the two points of inquiry. The impact of those interventions had some profound positive impacts on staff and families related to cultural competence and family involvement.

Jefferson County, Colorado – On the Job Training (PDF - 22 KB)
This one-page evaluation form lists the major competency and content areas of the training and asks the attendees to rate how they responded to the specific content areas before and after the training in order to gage the impact of the training.

Jefferson County, Colorado – Over Representation Report (PDF - 94 KB)
This 9-page data analysis of various cultural, racial and ethnic populations of families involved in the child welfare system sets the stage for internal and cross agency discussions and possible training opportunities to address issues of cultural competency in serving families engaged in the child welfare system.

Regardless of how well your continuous quality improvement program is working, it is important to regularly look at the outcomes being measured and the tools used to measure the outcomes to see if they need revision. As your system matures with added new services, supports or policy changes, and/or new knowledge, it is incumbent on a continuous quality improvement system to adjust the benchmarks and outcomes it seeks to learn about. You also may want to check if your continuous quality improvement program goals/measures are aligned with agency ones, so you can show how the continuous quality improvement program contributes to the larger goals of the agency/system.

Activities, Questions to Consider

Continuous quality improvement tasks and activities in your community might include:

  • Making sure that the continuous quality improvement program is working.
  • How has the continuous quality improvement program benefited various stakeholders?
  • Are all stakeholder groups participating in setting the desired outcomes?
  • Are all stakeholder groups seeing the results of the data generated by the continuous quality improvement program?
  • Do you have proper staffing to support the continuous quality improvement program?
  • Are you going after various Federal and foundation funding opportunities to enhance the work of the continuous quality improvement program?
  • Are trainings offered that will help front line staff and others who may have to collect the data learn about the benefits of doing such work?
  • Do you have access to external evaluators to give you an objective view of the quality of your continuous quality improvement system?
Voices from the Field

Voices from the Field: Continuous Quality Improvement

“We have had family partner and community involvement in all aspects of the continuous quality improvement work. We have regular community meetings twice a year in all 12 catchment areas of the county. We always respond to provider and community requests for information. We have self-assessment and self-improvement processes in place to continually grow as an agency. We consistently ask from all stakeholder groups, what are we doing well? What needs improvement? What can we do differently? It behooves the evaluation team to become more knowledgeable about the financial end of the agency’s work. Integrating the fiscal side of systems and programs into evaluation is invaluable. For example, you can start to ask questions like, how well are we using funds? Learn more about the fiscal end of operations in child welfare practices. If you do not learn this aspect of the agency you will not be offering a complete service and will not be seen as valuable to the agency…Another pitfall is that you only give data to upper level management. It is critical that you see your role to also offer data back to line staff and also to partner agencies. Sometimes this will take a lot of time and certainly in a big county such as Contra Costa you may have to break down the data to adjust to various geographical areas, but it is worth the effort. You will build broad support and community partnerships will develop. Working with different board members, developing community surveys and performing various data analysis are all valuable resources you can offer.”

– Research/Evaluation Manager, Contra Costa County, California

“When you are trying to achieve systems change and engage new stakeholders, lay out a clear plan for the process you want to follow. Identify goals, benchmarks for success and track progress and communicate all this to everyone involved…Make sure what you are measuring is what you are doing, not what you hope to effect later. We modified our theory of change to be more in line with what we were actually doing. Down the road these infrastructure changes should lead to other changes like child and family outcomes…Important to work with all stakeholders to understand the evaluation process. Explain what you can do with evaluation and reports. Teach people how to use data. Spend time in conversation – How does it relate to what we are doing here? Also learn from the community on how to best report data. Don’t assume you know the answers!”

– Lead Evaluator, Kansas System of Care

“As evaluators, it is important to build relationships versus just stay in your office and do the work. Results of our continuous quality improvement work were always presented to the system of care workgroup versus mailing the results to them.”

– Lead Evaluator, North Carolina System of Care

Key Partners

The people who perform continuous quality improvement duties in your community might include:

  • Interagency data team
  • Data collectors and data analyzers
  • Administrative support
  • Service delivery staff
  • Continuous quality improvement personnel, consultants, and/or university evaluators

Sustainability

  • The more data are seen and used by all concerned stakeholders in the system of care the more likely they will become an integral part of the business of your system.
  • The evaluation team should be seen as ambassadors of the data. The more that the data can be explained to stakeholder groups, the more likely the data will be understood and thus utilized in their day-to-day work.
  • Keep the data analysis simple, yet honest and specific.
  • Be creative in the use of graphs, pie charts, and other visuals in presentation of the material.
  • Combine real life stories with data to make your point.
  • Never be satisfied with the current continuous quality improvement program. Always have an eye toward year to year improvement.

Resources

Jefferson County, Colorado – Goals Survey, Measures (PDF - 40 KB)
This two-page survey is given to all members of the various system of care committees to help assure that each committee is achieving the desired results. Areas covered in the survey include adherence to goals and purpose, Identity & Membership, Productivity and Accountability. The survey was issued each year to measure change within the various committees.

Jefferson County, Colorado – Volunteer Program Pre-Evaluation (PDF - 12 KB)
This is a 1-page survey to gauge a person’s readiness to be a mentor in the Jefferson County Department of Family Services’ Family Navigator Program.

Jefferson County, Colorado – Volunteer Program Post-Evaluation (PDF - 14 KB)
This is a 2-page survey to gauge the readiness of a mentee to work with a mentor in the Jefferson County Department of Family Services’ Family Navigator Program.

Jefferson County, Colorado – Family Mentor Volunteer Evaluation (PDF - 14 KB)
This is a 2-page evaluation filled out by family mentors to ask them questions related to their experience as mentors. Questions ranged from were they properly trained for the family mentor position to were they acknowledged by the agency for their work with families.

North Dakota – Evaluation Logic Model (PDF - 26 KB)
This 2-page logic model details the approach of four Native American communities in North Dakota. The logic model is divided into Need/Goal, Resources, Process and Outcomes.

North Carolina – Diversity Training Evaluation Form (PDF - 100 KB)
This 3-page pre/post evaluation form helps show the impact that a cultural diversity training has on participants.

Pennsylvania – Assessment Tool (PDF - 57 KB)
This 11-page tool is intended to provide a framework for discussion and organization of local system strengths and needs as well as a means of defining both common and unique training and technical assistance needs. Additionally, this tool and the process that accompanied its implementation were used as a means of gathering valuable information across county systems that was then compiled and provided to State officials with regard to policy and/or regulatory issues.

( Back to the top )

Example from the Field

Site

Contra Costa County, California

Goal

The goal of the Systems of Care Evaluation Team was to continuously track and monitor the activities that were set forth in the strategic plan. Additionally, the team made adjustments in policy, practice, data collection strategies, and reports, and then revised implementation strategies based on the continuous feedback from both the program staff and the evaluators. This was the essence of continuous quality improvement, which joined evaluation, program, and management personnel to use real time information to review and revise implementation strategies across the areas addressed in the strategic plan. The plan was a working document which was reviewed and revised at each meeting of the team. The team continuously reviewed successes and challenges and developed strategies to address them. The continuous quality improvement plan helped to create a shared vision across the team and ultimately across the system of care.

Strategy/Approach

Continuous quality improvement has been a major focus for the Contra Costa County Systems of Care Team since the very beginning of the initiative. Contra Costa County created an evaluation team composed of data managers and analysts working within the county’s child welfare system, as well as external evaluators from the University of California-Berkeley. This structure allowed timely extraction of data from the child welfare system’s database, and an intimate understanding of the limitation and reporting structures of the archived data. In addition, the team approach included evaluation expertise from a respected university in the community. The evaluation team provided information to a broad spectrum of people within the local government as well as the community. The evaluation team reported on the initiative’s progress to agency managers, supervisors, case workers, agency partners who sit on the system of care interagency council, and to families in the community. These data and their analyses gave them the ability to track progress, consider successes, and identify and address barriers more quickly.

Additionally, the county created an evaluation subcommittee called the Systems of Care Evaluation Policy Team that included the local evaluation team, external evaluator, and the program staff. This subcommittee met monthly, or more, to develop, review, and execute activities to ensure the quality of services provided. Initially, the subcommittee analyzed child welfare data to identify four populations of focus with the most need within the county child welfare system: multi-jurisdictional youth, transition-aged youth, youth at risk of multiple placements, and youth with three or more placements in a six month period. Not only did the evaluation team identify populations of focus and shaped other planning activities, it also used performance measures to track progress toward their goals. Assessments (e.g., to identify children at risk of multiple placements) were often reviewed and revised based on data from the population of focus. The evaluation team also collected and reported out on the process and practice elements within their work. For example, the evaluation team tracked several measures that could lead to modifications in the Team Decision-Making process, such as whether the meetings have active participation from all relevant parties, participant satisfaction with the meetings, and whether the meeting assists in linking youth with resources to support the transition out of foster care. The evaluation team also tracked outcomes that were relevant to each population of focus and systems of care activity, such as a reduction in the number of youth who are in three or more placements over a six-month period, more integrated case planning processes for multi-jurisdictional youth, and more youth linked to necessary resources upon emancipation.

Systems of Care Principles

The systems of care principle that guided this approach was that of accountability. In order to ensure that the project stayed on track and was measurable, a team approach was utilized.

Lessons Learned

Contra Costa County learned that there is an immense need for a committed evaluation team, and that the team is most effective if it includes both external and internal evaluators. Contra Costa discovered that establishing continuous quality improvement is a difficult and time-consuming process, but the result of sustaining true change in the child welfare system is worthwhile

Facilitators of Positive Change

Contra Costa County used systems of care committed funds to help develop and improve the internal evaluation team, as well as bring in external evaluators. While evaluators helped guide the process, the development of an evaluation subcommittee was also integral to the success of continuous quality improvement.

It is important to have family partners involved in continuous quality improvement from the outset. Their input and participation gives the other professionals strategies for how best to report data so that they are meaningful to families and it keeps the language and the process realistic.

It was important to spend time and train all the members of an evaluation team and those addressing continuous quality improvements in the language of evaluation. The data provided and reported were only as valuable as their ability to truly inform all of those who read it so they could be used to improve performance and assist managers with resource decisions. Additionally, these data were our best marketing tool for telling the story of the success of systems of care. Partners, current and potential funders, and the community and families needed this information. Joined with the stories of the successful families, these data helped tell the story of success, improved the quality of services at all levels, and engaged agency partners and the community in creating a sustainable system of care.

Acknowledgements

The National Technical Assistance and Evaluation Center for Systems of Care would like to thank the numerous individuals whose hard work and dedication made this toolkit a possibility. Thank you to the Children’s Bureau staff, specifically Bethany Miller and Pamela Johnson (retired), Federal Project Officers, for their guidance and support throughout the writing and editing process. A special thank you goes to the following people for graciously volunteering to be interviewed for the Voices from the Field Section of the toolkit: Pat Harrington, Contra Costa County, California; Peggy Taylor, Kansas; and Liz Snyder, Duke University, North Carolina. Many thanks to the representatives from the nine Systems of Care grant communities of the Improving Child Welfare Outcomes through Systems of Care demonstration initiative who willingly shared sample products and tools which have been highlighted in the Resources Section. Finally, we thank the members of Infrastructure Toolkit Workgroup, who all brought their expertise to the table:

  • Susan Franklin, Program Manager, Jefferson County, Colorado, Department of Human Services
  • Beth Evans, Children and Family Services Administrator, Kansas Department of Social and Rehabilitation Services
  • Angela Braxton, Parent Leader, Kansas
  • Peggy Taylor, Evaluator, Kansas
  • Angela Mendell, Casework Supervisor, Bladen County, North Carolina
  • Marie Parrott-Withers, Parent/Provider, North Carolina
  • Helen Spence, System of Care Outreach Coordinator, Foster/Adoptive Parent, Dauphin County, Pennsylvania
  • Ervin Talley, Community Member, Bedford-Stuyvesant, New York City, New York
  • Kamelia No Moccasin, Oglala Lakota Tribe, South Dakota
  • Paula Loud Hawk (deceased), Caseworker, Lakota Oyate Wakanyeja Owicakiya Pi Okolakiciye (Helping Children of the Lakota People), Oglala Lakota Tribe, South Dakota
  • Nicole Bossard, Technical Assistance Team Leader, National Technical Assistance and Evaluation Center
  • Gary De Carolis (Chair), Senior Consultant, National Technical Assistance and Evaluation Center
  • Elleen Deck, Technical Assistance Liaison, National Technical Assistance and Evaluation Center
  • Janet Griffith, Senior Fellow, National Technical Assistance and Evaluation Center
  • Ethleen Iron Cloud Two Dogs, Technical Assistance Liaison, National Technical Assistance and Evaluation Center
  • Caitlin Murphy, Analyst, National Technical Assistance and Evaluation Center

Resources

Friedman, R. (2005, February). Effectively serving children with special needs and their families: What does it take? Paper presented at the 2005 Systems of Care Communities Meeting, Dallas, Texas. Retrieved November 15, 2006, from http://rtckids.fmhi.usf.edu/presentations.cfm.

Iowa State University. (2006) Continuous quality improvement (CQI).Retrieved November 20, 2006, from www.fpm.iastate.edu/worldclass/cqi.asp

National Child Welfare Resource Center for Organizational Improvement and Casey Family Programs. (2005). Using continuous quality improvement to improve child welfare practice.Retrieved December 13, 2006, from http://muskie.usm.maine.edu/helpkids/rcpdfs/CQIFramework.pdf