Defining Accountability

Whether in relation to criminals or lawmakers, U.S. or foreign government, corporations or public institutions, holding people and entities accountable encompasses such a variety of goals and activities that arriving at a comprehensive definition is difficult. According to one policy expert (Koppell, 2005, p. 95), "Layering every imagined meaning of accountability into a single definition would render the concept meaningless." Instead, Koppell presents a typology of accountability that features five dimensions:

• Transparency ("Did the organization reveal the facts of its performance?").
• Liability ("Did the organization face consequences for its performance?").
• Controllability ("Did the organization do what the principal desired?").
• Responsibility ("Did the organization follow the rules?").
• Responsiveness ("Did the organization fulfill the substantive expectation/demand/need?").

Koppell emphasizes that no single organization will focus equally on all dimensions of accountability. Behn (2001) identifies four main categories of issues to which people and institutions are held accountable: finance; fairness; abuse of power; and performance. Although all categories serve important purposes, for accountability as a principle within the system of care framework, performance is the main issue, as reflected in the definition of accountability used in the Improving Child Welfare Outcomes Through Systems of Care demonstration initiative: "Accountability refers to the continual assessment of practice, organizational, and financial outcomes to determine the effectiveness of systems of care in meeting the needs of children and families" (Child Welfare Information Gateway, 2008).

Accountability within large systems is not achieved with a single act. As Figure 1 illustrates, accountability is a process with three important phases:

• Planning
• Measuring
• Applying the Information

Figure 1: Phases of Accountability

D

Planning. Holding systems accountable requires measuring the extent to which they are successful. Measuring the success of systems requires establishing clear outcomes and a plan for reaching them. Formulating strategic plans, logic models, and theories of change helps agencies and institutions connect activities to outcomes. As plans start to take shape, strategies to measure progress also should be determined (Sahonchik, Frizsell, & O'Brien, 2005). In addition to decision-makers, planners should include individuals who do the work (staff), receive its benefits (families), and measure its effectiveness (evaluators and others involved in accountability).

Although planning is the first step toward establishing an accountable system of care, reviewing and revising system planning should be ongoing (Sahonchik et al., 2005).

Measuring. Good measurement is the foundation of accountability. True accountability includes measuring change as it happens as well as measuring the outcomes of change. The systems of care theory of change depicted in Figure 2 proposes that system and practice changes result in improved child and family outcomes. Hence, indicators should be established to measure change in system policies, procedures, and practices as well as change in outcomes for children and families.

Figure 2: Theory of Change for Systems of Care

D

Child and Family Outcome Measures. Many child and family measures are associated with long-term outcomes. Accountability often begins with these measures since they represent the goal of the agencies: to improve safety, permanency, and well-being for the children served. Many of these measures are collected from large administrative datasets housed in agencies' electronic information systems. While important, and often mandated by Federal or State agencies, these are not the only kinds of indicators of child and family outcomes. Agencies can obtain valuable information about the population served, or a subpopulation of interest, through a variety of other methods, including focus groups, interviews, and evaluations designed to track cohorts over time.

System and Practice Measures. Systems can be complex, including not only the children and families served, but also staff, partner agencies, and communities, as well as all the practices and operations that contribute to making the system work. If accountability is embedded in the system, both outcomes for children and system change can be measured. Measuring complex system change may require measuring many individual aspects of the system, such as how a new tool works, if training was successful, or how staff members' attitudes about cultural competency change over time. These individual measures should be as accurate as possible, but they do not have to be overly complex. Often, these measures include information that already is, or could easily be, routinely collected. The measures may be quantitative (e.g., how many staff attended a brown bag session on diversity or participant satisfaction with the training, indicated on a scale of 1-5) or qualitative (e.g., response from caseworkers who are testing a new assessment tool or focus group findings about new ideas for foster parent recruitment). Engaging child welfare staff in evaluation planning can be helpful for identifying ways to measure an intended change as well as maintaining accurate records throughout an evaluation project.

Even if collecting measures associated with mundane activities is easy, collection should be planned and systematic and all measures should link to the larger outcomes a system is trying to achieve. Often referred to as performance measurement, a structured process using empirical indicators should be established to allow child welfare and partner agency staff to track whether an agency's activities and services are being provided, used, and ultimately improve the lives of children and families (Bazemore, 2006; Rossi, Lipsey, & Freeman, 2004).

In addition to being able to accurately assess system progress, measures must lend themselves to the next step in accountability, applying the information.

Measures must be:

• Thorough enough to be useful to people at all levels of the system and those outside the system without being burdensome.
• Directly related to agency/program/project goals and expectations.
• Flexible and fast enough to guide real-time decisions and test new ideas quickly (Langley, Nolan, Nolan, Norman, & Provost, 1996)
• Continuously collected.

Applying the Information. To advance from simply measuring to practicing accountability, the information must be shared and used to improve the activity, program, or system. The information must be analyzed and synthesized for multiple communication formats to support a variety of tasks, including:

• Testing the theory of change on which the program or project is based.
• Informing staff about how their practices affect individuals served.
• Helping administrators and decision-makers shape policy.
• Informing families, the community, and the public about, and engaging them in, the work.

Communicating information about performance to a variety of stakeholders is critical for agencies trying to effect significant system change, especially when they are relying on a broad-based collaborative to oversee the change. Small indicators may show improvement long before large, system-wide results are seen. Even if progress is not apparent, sharing data advances system improvement by:

• Creating early opportunities to review and revise system improvement activities that are ineffective.
• Allowing a wider group of stakeholders to participate in improvements.
• Demonstrating an agency's willingness to be transparent to staff and community partners, resulting in increased trust and commitment.
• Showing those who may perform most of the data collection (i.e., caseworkers) how the information is used and how improvements in practice can generate improvements in outcomes.
• Making accountability a routine part of agency operations by building evaluative capacity throughout the system.

Applying the information must be as inclusive as possible and based on sound leadership. Leaders must be willing to use what is learned through measurement to make changes, holding the agency and their own leadership accountable. However, accountability and leadership both require the participation of a broad group of stakeholders who share responsibility for outcomes (Schorr, 2006).

(Previous) (Next)

This material may be freely reproduced and distributed. However, when doing so, please credit Child Welfare Information Gateway.