for State and Local Leaders
By Sara D. Watson
1000 Vermont Ave., NW
Washington, D.C. 20005
Fax: (202) 628-4205
Funded by the National Governors’ Association
Center for Best Practices
through a grant from the
U.S. Department of Health and Human Services
Office of the Assistant Secretary for Planning and Evaluation
© 2000 The Finance Project
This guide was produced with the support of the U.S.
Department of Health and Human Services, Office of the Assistant Secretary for
Planning and Ev aluation, through funding to the National Governors’
Association, Center for Best Practice. The
author gratefully acknowledges the ideas and support of Ann Segal and Martha
Moorehouse of USDHHS; Evelyn Ganzglass and Helene Stebbins of NGA; and Cheryl
D. Hayes and Barry Van Lare of The Finance Project.
aluation, through funding to the National Governors’ Association, Center for Best Practice. The author gratefully acknowledges the ideas and support of Ann Segal and Martha Moorehouse of USDHHS; Evelyn Ganzglass and Helene Stebbins of NGA; and Cheryl D. Hayes and Barry Van Lare of The Finance Project.
In addition, the following people generously gave of their time and expertise to provide the advice in this publication, and to share their comments and suggestions on the draft guide. Many thanks go to them for their hard work to improve the lives of children and families, and their willingness to share their insights. In particular, the author wishes to acknowledge Mark Friedman for his ground-breaking work in this field; and Cornelius Hogan for his leadership at the state, national and international level.
Peter Beeson – Administrator, Strategic Management Services Division, Nebraska Department of Health and Human Services
Juanita Blount-Clark – Director, Division of Children and Family Services, Georgia Department of Human Services
Neil Bryant – State Senator, Oregon State Legislature
John Dorman – Director, Office of State Planning, North Carolina
Edward Harmeyer – Deputy Director, Office of Statewide Performance Review, Illinois
Richard Larison – Director, Office of Statewide Performance Review, Illinois
David Murphey – Senior Policy Analyst, Vermont Agency of Human Services
Cheryl Mitchell – Deputy Director, Vermont Agency of Human Services
Sandra Moore – Executive Director, Family Investment Trust, Missouri
Jessie Rasmussen – Director, Iowa Department of Human Services
Steve Renne – Deputy Director, Missouri Department of Social Services
Kenneth Seeley – Director, Colorado Foundation for Children and Families
Gary Stangler – Director, Missouri Department of Social Services
Jeffrey Tryens – Executive Director, Oregon Progress Board
Martha Wellman – Office of Program Policy Analysis and Government Accountability, Florida
Sandra Wilkie –
Senior Policy Analyst, Family Investment Trust, Missouri
Phyllis Becker – Communities in Schools, Missouri
Jane Campbell – County Commissioner, Cuyahoga County, Ohio
Susanne Daily – Assistant County Administrator and Executive Director, Office of Children and Family First, Montgomery County, Ohio
Tana Ebbole – Executive Director, West Palm Beach County Children’s Services Council, Florida
Sara Hoffman – Assistant County Administrator, Contra Costa County, California
Molly Irvin – Evaluation and Assessment Manager, Family and Children First Council, Cuyahoga County, Ohio
Richard (Jake) Jacobsen – Director of Social Services, Mecklenburg County, North Carolina
Thomas Kelley – Assistant Director, Office of Family and Children First, Montgomery County, Ohio
William Laaninen – Staff Director, Skagit County Community Network, Washington
Christina Linville – Deputy County Administrator, Contra Costa County, California
Michael Monteith – Assistant City Manager, Hampton, Virginia
Bette Meyers – Deputy Administrator, Health and Human Services, Cuyahoga County, Ohio
John Skidmore – Assistant Director, Department of Social Services, Mecklenburg County, North Carolina
Robert Stoughton – Research Administrator, Office of Family and Children First, Montgomery County, Ohio
– Director, Budget and Management, Cuyahoga County, Ohio
Ira Barbell – Senior Associate, Annie E. Casey Foundation (who suggested the title)
Robert Behn – Director, The Governors Center at Duke University, Terry Sanford Institute of Public Policy
Susan Christie – Director of Organization and Professional Development, American Public Human Services Association
Judy Chynoweth – Executive Director, Foundation Consortium
Mark Friedman – Director, Fiscal Policy Studies Institute
Beverly Godwin – Director, National Partnership for Reinventing Government
Cornelius Hogan – consultant, Annie E. Casey Foundation/former director, Vermont Agency of Human Services
Anne Kubisch – Co-director, The Aspen Institute Roundtable on Comprehensive Community Initiatives
Lizbeth Leeson – consultant, Community Based Innovations, Michigan
Shelley Metzenbaum – Director, Performance Management Program, John F. Kennedy School of Government, Harvard University
Jolie Bain Pillsbury – consultant, Sherbrooke Consulting
Connie Revell – Director, Community Choices 2010, Washington /former director of Oregon Options
Phyllis Rozansky – Director, Pathways
– special reporter, Governing
Across the nation and across the world, the concept of using results to drive and measure success in supports for children and families is taking hold. From the most senior leaders to service providers, teachers, advocates, and others, this massive shift in thinking and working has begun to permeate the structures that support children and families to have the best possible outcomes.
The National Governors’ Association has taken a leading role in assisting its members design and implement results-based decisionmaking systems across a variety of policy areas. Since the mid-1990s, The Finance Project has been a leader in this field by conceptualizing and developing materials that present a framework for results-based planning, budgeting, management and accountability – what we term “results-based decisionmaking” – and that begin the shift from theory to practice.
Both organizations are pleased to continue this tradition with a new wave of publications that reflect the hard-won experiences and lessons learned of veterans in the field. The newest one is Informed Consent: Advice for State and Local Leaders in Implementing Results-Based Decisionmaking, by Sara Watson. This paper was prepared with support from the U.S. Department of Health and Human Services/Office of the Assistant Secretary for Planning and Evaluation, through a grant to the National Governors’ Association Center for Best Practices. Evelyn Ganzglass and Helene Stebbins at NGA were instrumental in bringing this project to fruition. In addition to this full-length version published by TFP, a shorter version will be published jointly by NGA and TFP.
This guide gives targeted, strategic advice on implementing different approaches to results-based decisionmaking. It aims to help state and local leaders answer questions such as “should we do this?” “can we do this?” “how do we do this?” “how long will it take?” and “what can we expect?” It discusses using results (and all variations thereof) to develop an agenda to improve the lives of children and families; to align resources to support that agenda; to align management practices and organizational cultures with that agenda; and to measure performance and hold organizations and individuals accountable for improvement. It also provides suggestions for eliciting the support of key stakeholders, such as executive and legislative branch officials, the media, and communities. It goes beyond descriptions of state and local experiences, to draw the political and strategic lessons that can help state and local leaders avoid pitfalls and move ahead.
The guide is based on extensive interviews and discussions with over 50 leaders in the field, including those working across sites and in national, state and local leadership positions. We are grateful to these individuals for generously sharing their insights and ideas.
We hope this guide is useful to those who are pushing
forward and those who are just trying to hold onto hard-won gains; those who
are in visible positions of leadership and those who are the behind-the-scenes
heroes. Their wisdom and
hard work will help us all move towards better lives for children and
Cheryl D. Hayes
Should we do it? Can we do it? How do we do it? How long will it take? How do we win people over? What will we get out of it? And what do we risk? State and local leaders are asking these questions as they consider or implement a shift to using results to change the way states and communities support families – what this guide terms “results-based decisionmaking.” The guide aims to help these leaders make informed decisions about moving to results-based decisionmaking, based on the advice of people who have led these and other change processes. It is not a step-by-step cookbook, as every state and locality needs to design its own system. But it does provide some advice and insights on avoiding common pitfalls, preparing key constituencies, and how to move forward.
Before launching into advice on how to do
results-based decisionmaking, a
common first question is whether this latest reform will last longer than many
similar-sounding predecessors. There
is considerable optimism that this approach will endure, because the public is
demanding a higher level of accountability than ever before;
there is more precedence, in welfare reform, education and other policy
areas; policymakers have learned from past initiatives, and the dramatic
changes in information technology mean it is more feasible simply to collect
and analyze the necessary data.
The guide provides advice on a number of initial
decisions or considerations as leaders implement their own results-based
Structure: There are a number of options for structuring the overall approach. Some states focus on inter-agency changes that can affect core functions but may not emphasize significant community involvement or cross-agency results. Other states have focused on empowering community collaboratives to take responsibility for cross-agency results. Still others have organized their approach around individual populations, such as young children. Each method has advantages and disadvantages.
and risks for state and local leaders:
There seem to be three main reasons why state and local leaders
pursue results-based decisionmaking.
They believe it will improve the lives of children and families; it
responds to voters’ demands for results; and/or it fits with their own
beliefs about improving government and communities’ support for children
and families. At the same
time, there are also three main risks to pursuing this path:
Leaders may be criticized for revealing or achieving poor results;
inevitably, someone, somewhere will make mistakes in implementing this new
way of work; and investing in this effort, as with any other initiative,
creates at least a short-term opportunity cost
(however, advocates would also state that improved results
eventually frees up resources).
Leadership: One of the most universal conclusions from people experienced in RBD is that it will succeed only with sustained leadership from the most senior levels. Advice on what to do in the absence of complete support and commitment focuses on “starting where you are” – for example, if leadership exists on the county level, start there.
Initial focus: Deciding where to start also comes down to the advice from Washington state’s Joe Dear of “Use what you have. Start where you are. Do what you can.” Some recommend starting where there is a crisis, others on where there are successes on which to build.
All at once, or phasing it in?: Another common question is whether to jump in to the RBD “pool” with both feet, or start with one toe at a time. States and localities have pursued both paths.
Bottom-line advice: Experienced leaders from many perspectives provided a set of advice to help initiatives start well and avoid getting bogged down:
Start with a vision specifically stated in terms of results for children and families.
Move rapidly beyond the initial vision and mission stages by building on what already exists.
Treat results-based decisionmaking as a better way of doing the existing work, not just another process added to the existing ones.
Don’t make organizational changes until and unless there is a compelling need.
Recognize that changes in results are more than the sum of changes in program performance measures.
All parties must have something to gain – and something at risk.
Don’t let data needs bog down the process -- start with the available data and build from there.
The best way to move forward is to take small steps at a brisk pace.
Focus can be more powerful than money
-- improving results doesn’t necessary require large infusions of
the next election – how far can a state or city/county get in three
years?: Realistically, many
leaders need to think about demonstrable success in two to three years. Within that time, they can aim to produce a state- or
city/county-wide vision for results, publicize information on the status of
children and families, establish new results accountability systems for
specific areas, and can perhaps demonstrate actual improvements in results
for very specific populations.
Instituting results-based decisionmaking requires the
positive involvement of a wide variety of constituencies.
The guide gives brief advice on ways to elicit support from agency staff,
community leaders, elected officials, budgeting and financial management staff,
auditors, the media, advocacy and civic groups, labor and business.
The advice helps leaders identify and overcome expected reasons for
opposition, as well as develop aspects of RBD that appeal to the interests of
each constituency in better lives for its citizens.
The guide then explore four major phases of
results-based decisionmaking and provides advice on structure and
Strategic Planning: A first stage in using RBD is planning – choosing results, strategies to achieve the results, and ways to measure progress. The guide provides examples of planning frameworks as well as advice on structuring a planning process that elicits support but moves promptly to implementation.
Aligning Resources: Most states have moved away from the idea of completely changing from line-item or program budgets to results-based budgeting. Instead, they have taken smaller steps to align their resources with chosen results. Leaders emphasize changing the budgeting conversation instead of the rules – rather than trying to amend formal, entrenched processes, they suggest ensuring that budget discussions within the executive branch, and between the executive and legislative branches focus on how a particular resource request will move the state or city/county ahead on a particular result.
Changing Management and Culture: Implementing results-based decisionmaking creates a riskier and less controlled environment for senior decisionmakers, middle managers, front-line workers, and communities. Creating a fair and effective RBD system, and encouraging people to embrace this system enthusiastically is perhaps the hardest challenge of all. Advice here emphasizes helping to reduce the fear, even if one cannot eliminate the risk; presenting a RBD initiative as a shared effort to help families, rather than as a way to “catch” wrong-doers; providing workers with the praise and support necessary in a new environment; creating safe support groups; starting with volunteers; and recognizing that change takes time.
Assuming Accountability/Responsibility for Results: The bottom line for results-based decisionmaking is using data to improve the efforts to better the lives of children and families. While relatively few places have gone through a complete accountability cycle of planning, aligning resources, implementing management changes and taking responsibility for improvement in results, there is enough wisdom and experience to offer some insights. The most prominent is to recognize that there needs to be an explicit set of steps between collecting data on performance, and taking actions because of that performance. A wide variety of consequences can support improvement at the individual and organizational levels, and moving too quickly to an inappropriate consequence can make the situation worse, not better. The array of possible consequences includes:
· Private reward and pressure, by supervisors and peers
· Public reward and pressure
· Tangible rewards for success
· Increased autonomy
· Increased assistance
· Reduced autonomy/increased oversight
· Reduced or transferred funding
· Changing or terminating employment
Another key insight is that “consequences” need to apply both to those who are expected to produce results, and those to whom they report – too often RBD systems leave all of the risk and exposure to the results producers, creating an unbalanced relationship. And it is important to realize that setting specific performance targets, as with much of results-based decisionmaking, is still a combination of art and science. While baseline data can help, it is important to consider resources, challenges, and other factors in deciding what to expect.
Results-based decisionmaking has the power to transform formal agencies, the role of communities and the lives of children and families. It can rebuild public faith in the ability of government to partner with communities to support families. It can energize tired workers and advocates who can now see progress. And it can catalyze needed changes among those who at last are rewarded not only for following the rules but for using their creativity and energy to create change. But as with any change, there are risks. This is still a learning process, a huge experiment, albeit one that resonates deeply with many who have struggled for decades to improve the lives of children and families. Their advice and ongoing experiences may help communities, states and other nations find better ways of using financial and human resources to achieve better lives for children and families.
Table of Contents
Setting the Stage
3. Déjà vu All Over Again?
2. Rationale and Risks For State and Local Leaders
4. Initial Focus
5. All At Once, or Phasing It In?
Bottom Line Advice
Looking to the Next Election
Laying the Groundwork with Key Constituencies
1. Agency Staff
3. State Legislators and City/County Councils
4. Budgeting and Financial Management Staff
6. Internal Systems (Personnel, Procurement, etc.)
7. The Judiciary
9. Advocates, Civic Groups, Advisory Committees and Citizen Commissions
10. Public Employee Unions
Assuming Accountability/Responsibility for Results
Changing Management and Culture
Setting the Stage
This guide was born out of a deep desire on the part of people implementing and supporting results-based decisionmaking systems across the country – and indeed across the world – to gather “lessons learned” on how to do it, and what to expect. Implementing results-based decisionmaking (RBD) is still a grand experiment. No nation, state or locality has a complete system in operation, and indeed only a few places have gone through a full accountability cycle of measuring performance and making changes based on performance. However leaders in many places have begun to make profound changes in the ways that governments and communities support families and in the ways they are responsible for performance. There are success stories and rueful stories – victories and mistakes. There is much to learn about, and learn from, in steering a true course to using results. Before launching into a discussion of how to do results-based decisionmaking, some introductory remarks may be helpful.
First, a definition: in this guide, “results-based decisionmaking” is a short-hand phrase for a wide variety of approaches to focus on results, rather than activities, as the goal of public acts and expenditures. The exact design can take a number of forms, but here it encompasses choosing a set of results and planning supports and services to achieve those results; allocating resources as needed to achieve results; managing public and private supports and services in ways that promote achievement of results and encourage workers to take responsibility for their performance; and being responsible for improving performance over time. The structure has some fundamental elements, such as a commitment to use results to improve, not just to plan or budget; and a linked agreement to allow people who are responsible for results more discretion and autonomy in exchange for more responsibility for performance.
The audience for this guide is state and local officials who are trying to develop or strengthen their results-based decisionmaking systems. While communities and neighborhood leaders are crucial parts of these systems, this guide focuses on the concerns of these appointed and elected public leaders.
The purpose of this guide is to give strategic advice about what to expect and how to successfully design and implement results-based decisionmaking systems. Much has been written about the obstacles and challenges to using RBD, as well as how to design and measure specific indicators. This guide goes beyond obstacles to give some advice on how to overcome them. It also goes beyond a discussion on measuring indicators to discuss how to use them to improve performance. It is designed to help leaders make informed decisions about how much to invest in these ideas, what they can expect to gain and what they risk by doing so. While the guide focuses on RBD for systems serving children and families, the advice applies across other policy areas as well. This guide also assumes that readers are already familiar with the basic language, theory and rationale for results-based decisionmaking – they have taken “Outcomes 101” – and are looking for informed advice on what to do – and expect – next.
The advice in this guide is not dependent on a state or locality adopting a particular results framework. It does not describe the different approaches taken by states and localities in any detail, since there are many such studies. It does assume that any results-based decisionmaking system has some common elements, such as those listed above.
The section below describes several different approaches to using results-based decisionmaking, but two variations in particular stand out. Making the best use of these approaches will be a recurring theme. Some initiatives emphasize changing formal state or county agency structures to focus on results and then to use them for planning, budgeting and management purposes. This approach tends to treat agencies as separate entities, sets agency-specific measures as the targets for change, and is less focused on community mobilization and involvement. The second approach strongly emphasizes mobilizing communities to take a more active role in improving results for children, youth and families. It often starts with a state-wide list of results and includes a state or county structure that enables agencies to work across boundaries. However, most of these initiatives involve smaller, “side” funds rather than the major agency operations and budgets.
This guide is written from the standpoint that an effective results-based decisionmaking system combines the best of the various reform approaches. However, the advice here applies regardless of the exact structure.
This is not a cookbook or a step-by-step guidebook, but a compendium of advice drawn from experience or best strategic thinking. It is based on dozens of interviews and document reviews from people who are implementing these ideas across the country, and across the world. While each place is unique, the concerns and strategies are similar, from Tillamook, Oregon; to New York City; to Oslo, Norway.
This guide is organized into four major parts. This introduction concludes with some definitions used in the guide, as well as a brief discussion of why the current wave of results-based decisionmaking reforms is expected to endure longer than many of its predecessors. The second part gives some overall advice about structuring a results-based decisionmaking system. The third part looks at how to lay the groundwork for results-based decisionmaking with specific constituency groups, such as the media, auditors and legislators. The fourth part gives suggestions for implementing four distinct (but overlapping) phases of results-based decisionmaking: strategic planning, resource allocation, management/administration and culture and accountability/responsibility.
Every organization that is pursuing results-based decisionmaking uses its own jargon and framework. To streamline the writing of this guide and promote clarity, it is helpful to define the terms used here (this terminology is based on the framework first developed by Mark Friedman). Other terms, such as strategies or activities, have their commonly understood meaning.
Result: A broad condition of well-being for children, adults, families, or communities, sometimes also known as an outcome. To prevent repetition of “results, indicators and performance measures,” this guide will also use “result” as an umbrella term covering all three terms when appropriate. (examples: all children born healthy, youth becoming productive adults, supportive communities)
Indicator: A measure which helps quantify the achievement of the desired result for community-wide populations. These measures are usually so broad that progress requires actions across agencies and systems. (examples: rates of low-birthweight births, youth graduating from high school, family income)
Performance measure: A measure which helps quantify movement towards the indicator for specific target populations, or one that measures the level of activity, efficiency, capacity or quality of a service or intervention. The latter are also called measures of process, activity or effort. These measures can be affected by individual organizations or actors. Some measures can be either indicators or performance measures, depending on the number of people included in the data (those in one program or across a state), and how the measures are used (to hold one agency accountable, or mobilize a whole community). (examples: academic achievement among students in a program aimed to boost achievement, number of participants in such a program)
Accountability: Having some responsibility for taking actions to improve results, indicators and/or performance measures, with some consequences for performance.
Community collaborative: A community-based group that has some authority to make decisions about supports for children and families, and some accountability for improved status of children and families. This term is meant to encompass community-based groups that have both formal and informal status.
Results-based decisionmaking: The process of using results, indicators and performance measures to improve child and family well-being. Includes using these for strategic planning; aligning resources; changing organizational management and culture; and accountability.
Finally, before launching into a description of how to implement results-based decisionmaking, it may be helpful to answer a key question. Many leaders who are trying to implement results-based decisionmaking have encountered skeptics who have seen other performance management trends come—and go. Variously titled Total Quality Management, Performance-Based Budgeting, Management By Objective, or other names, this latest round of work follows a significant history of similarly-sounding efforts. One of the first questions they need to answer – for themselves as well as for their critics – is whether this latest wave of reforms will last longer than the many initiatives that have preceded it. After all, as William Faulkner said, “The past isn’t dead – it isn’t even past.”
Fortunately, there is considerable reason for optimism
that this approach will be different. Jonathan
Walters, a reporter for Governing
magazine states flatly, “even though arguing the pros and cons of
performance measurement is lots of fun, and even though you’ve been
subjected to countless, cascading come-and-go measurement fads for the past 30
years, this time it’s a moot point. Chances
are that when performance measurement rolls your way, it won’t be an
While it will surely be refined and “tweaked” along the way, the core
ideas seem here to stay. The
reasons for this optimism include:
Public demand for accountability: State and local leaders note a heightened level of public expectation for knowing what they are accomplishing with public funds. This is markedly different from previous decades, and has even spread to other countries that have traditionally funded social services based on moral rather than effectiveness imperatives. Martha Wellman from Florida’s Office of Program Policy Analysis and Government Accountability notes that “some agencies believed that they would never get another drop of revenue unless they could show the results of their program.” As Sid Gardner points out, two-thirds of workers have some connection between their pay and performance, and they have come to expect this of the others.
Emphasis on a change in status or condition: This focus on results is different from many other initiatives because it has at its center an emphasis on an end result – a better environment and improvements in the status of children and families. Although other efforts, such as Performance-Based Budgeting, sounded similar, looking beneath the labels often revealed that the focus was still on measures of activity or process, rather than result.
Precedents: There are now several major national programs or reform movements that may help spread the idea of results-based decisionmaking. Welfare reform (and related programs, such as the Workforce Investment Act) is encouraging one large public system to re-invent itself to measure progress based on results, to re-think the mission of its employees, and to take responsibility for performance. The entire system of public education is also undergoing a similar transformation, over a longer period of time. Having effected these reforms in a few systems, leaders may be more able to spread these changes to others.
Experience over time: There is some evidence that enough people are accumulating enough knowledge and experience to help these ideas “stick.” Many reforms require continued exposure to the ideas over time, and this latest round of efforts to use results may find more fertile ground because of past attempts at reform. It remains to be seen if the country has yet reached what Malcolm Gladwell calls a “tipping point,” where ideas truly catch fire, but they are certainly being proposed and considered in a variety of venues.
Privatization of public services: Another major difference in the public sector environment between now and the last wave of performance reforms is the privatization of public services. To take the chances necessary for success, people must be motivated to be more productive, and they must have the freedom to act accordingly. Privatization usually means competition, which inspires motivation for change. This has been a key factor in eliciting public employee union support for results-based decisionmaking. Privatization also sometimes provides opportunities to allow the freedoms necessary to pursue better performance, without having to take the time to change rigid public procurement, personnel and other regulations.
New faces in government: These ideas are also starting to permeate through the graduate schools and work environments that prepare public officials and leaders. While there is mixed opinion about the impact of term limits, one possibility is that it will enable people to earn elected offices who expect to have good information, who have experience with being held accountable for performance, and who would come to expect the same of public systems.
Better data systems: Finally, data collection capacity has increased dramatically, enabling states and localities to collect and analyze more data and turn it around quickly to front-line workers and communities. This has allowed more people to see the value of the data in evaluating their own community’s situation and their own work. Providing this data on a local and quick turn-around basis has “let a genie out of the bottle” that will not go back in.
One can still be skeptical about the power of results to overcome decades – even centuries – of entrenched thinking about how public systems and communities can support families. And indeed, moving to results-based decisionmaking alone is not sufficient to improve results. But the appeal of the new ideas and the changing circumstances give hope that this is one reform movement that can make a difference.
As state and county leaders consider how to structure
the use of results to improve the conditions of children and families, they need
to make a number of initial decisions.
Structure: There seem to be a lot of ways to use results in a formal system; what are the options for structuring the overall approach?
Rationale and risks for state and local leaders: Do leaders really want to do this? Why do state and local leaders pursue results-based decisionmaking, and what do they risk?
Leadership: Every guidebook says that using results effectively requires leadership. But what if all of the right people aren’t in place?
Initial focus: Using RBD seems to be a huge undertaking. How can state or local leaders decide where to start?
All at once or phasing it in?: Should a state or locality start the RBD initiative all at once, or phase it in over time?
There seem to be a lot of ways to use results in a formal system; what are the options for structuring the overall approach?
One of the first areas of consideration for a state or local leader will be what overall structure to use to implement results-based decisionmaking. State and local initiatives to use results-based decisionmaking currently take a variety of forms, with the most distinct difference between approaches that place more emphasis on formal agency changes, and approaches that emphasize community mobilization and responsibility. While each of these approaches has distinct advantages, as they are currently implemented, each also has certain drawbacks.
There can be several different ways to infuse results in
public decisions about the best ways to support children and families.
Many states and localities are implementing several forms of
results-based decisionmaking at the same time; the key is to consciously develop
a structure that uses the best of each approach.
Distinct approaches for implementing RBD include:
Initiatives that focus on public
agencies’ use of results-based decisionmaking (with less emphasis on
Initiatives that focus on devolving responsibility for improving results
to communities (with less emphasis on
Initiatives that concentrate on using results-based decisionmaking with specific populations that cut across agency and community
boundaries (instead of encompassing all children and families); and
Initiatives that are started and led by the legislative
or executive branches.
A. Focus on agency change: This approach emphasizes changes within executive branch agencies at the federal, state or county level. Texas, Florida and Washington state are often mentioned as leading examples of this type of approach. Agency staff develop results and measures of performance for a single agency; many agencies are using these measures to improve services through the use of performance contracts with public and private providers. When Governing magazine reporters Katherine Barrett and Richard Greene surveyed all 50 states and 35 major cities, they found that every one was either implementing this type of initiative or expressed a desire to do so.
The agency-focused approach to RBD is powerful in many ways. It has the potential to address the mainstream sources of funds and affect large numbers of families, and it dominates the public service industry and the public administration literature.
However, there are three major concerns about the current status of this type of reform. First is the focus on single-agency change. Most important results require cross-agency work and indeed work outside agencies. Yet this form of RBD often focuses on measuring performance only within individual agencies or even individual programs. It does not always claim or support a vision in terms of broad results for children and families. However, there are exceptions to this situation, and proponents of this approach say that states and localities are moving to address these two issues in their use of RBD. A second is that while some states and cities using this approach are improving their management of public programs, others are not yet using it to make significant decisions, including service delivery and resource allocation decisions. This is perhaps because leaders are required to operate under the same old planning and allocation rules that do not encourage flexibility to achieve results. Asking agencies to develop elaborate plans, but not then enabling and encouraging them to use these plans to change policies and budgets to achieve results, is not likely to improve performance.
A third issue is that this approach to using results tends not to focus on the role of communities. Even in states that have both agency-driven reform initiatives and formal processes to create local collaboratives and mobilize communities (the second major reform approach, described below), there is often little contact between people in the two reform approaches. Agencies do not benefit from the good ideas and energies of communities, and community-driven reforms do not affect decisionmaking in the major systems. And yet, states could benefit greatly from drawing on the advantages of each. In the eyes of many RBD advocates, these are important omissions and reduce the chances that this approach will profoundly affect families, or endure beyond changes in political leadership. This approach could be much more powerful if it were coupled with an approach that mobilizes communities and focuses not just on limited performance measures but on cross-agency results. Later in this guide is a detailed discussion of how communities and agencies could strengthen their partnership in results-based decisionmaking.
B. Focus on community mobilization: This approach to RBD emphasizes moving responsibility for results from centralized, formal agencies to communities. It often involves a state-wide organization that brings public and private leaders together to pursue cross-agency work to move towards state-wide results. It also involves establishing community collaboratives (composed of leaders from many different sectors of the community) that are supported in taking a more active role in decisions about public funds and private actions to help families. Vermont, Missouri, Maryland, Washington and Oregon have established strong community collaboratives and given them resources to fund services and/or recommend policy changes to move towards better results.
This community-focused approach to RBD, as it is often currently implemented, has some limitations. It usually focuses less on changing mainstream agency policies and sources of funds, and as a result local implementers – especially local government officials – are often in the position of being expected to change major results with tiny amounts of money and little authority. It can be extremely difficult and contentious to determine which leaders, organizations, governmental entities or others at the community level should be given this new responsibility – the county, school district, city, an entirely new jurisdiction, etc. – and how they coordinate with existing governmental structures. Leaders are also facing a difficult struggle to keep communities engaged in this difficult work. Juanita Blount-Clark, director of Georgia’s Division of Children and Family Services, says “It’s a race to keep the community volunteers going long enough to see changes in results.” Each of the states that has pursued this approach has also been faced the dilemma of asking communities for a level of accountability that formal state agencies have not yet achieved.
However, this approach does take advantage of the increasing recognition of the importance of communities in supporting families. Many agency leaders have come to recognize that agencies are not the sole or even the major factor in improving child and family results in many communities. For example, improving agency performance is a usually necessary but almost never sufficient step to reducing teen pregnancy, etc. Each of the states using a strong community-focused approach to RBD can point to examples where community collaboratives have contributed to improved results for specific populations, and a few, such as Vermont, have even seen changes in results for statewide populations.
As stated above, both of these approaches address important issues in improving the results of children and families; the key is using the best of both. For example, Missouri has developed “Show Me Results” covering all aspects of the quality of life. It has also developed a state-wide network of local collaborations, Caring Communities, which has developed “Core Results” for families and children. Leaders in both initiatives are now examining how the Caring Communities’ community-wide strategies, Core Results, Show Me Results and state agency objectives all fit together (see appendix 1).
C. Focus on specific populations: Other places have taken a different tack, focusing on cross-agency and community mobilization work but for selected populations. Three examples are North Carolina’s Smart Start (a major initiative of Governor Hunt), California’s Proposition 10 (a public ballot initiative), and Florida’s School Readiness Councils (enacted by the legislature). All set state-wide results and create local groups charged with improving results at the community level. This approach to RBD has the advantage of focusing on a specific population, and riding a wave of public interest in that population. However, all of these are relatively new, so it is yet to be seen how they impact significant results for state-wide populations.
D. Legislative or executive branch structure: Another way to approach institutionalizing results-based decisionmaking is to emphasize legislative or executive branch leadership initially. While ultimately both branches will need to use results in order to incorporate them into decisions that effect significant and enduring changes, the initial focus can come from either side. While putting these requirements in statute can give them an institutional basis, they can also establish rules that are difficult to change with new developments and new information. At the same time, in order to alter major allocations of resources, the legislature must be involved.
Therefore, most places have started with executive branch leadership, either in partnership or with the encouragement of the legislative branch. In Illinois, several legislators drafted legislation requiring the use of results in agency decisions but put it aside once they saw agencies following this path on their own. In Washington State, legislation creating a community-focused results-based decisionmaking system has institutionalized an effort that could have been lost over changes in political leadership; however, it also codified a variety of requirements that have been challenging to follow. Several states, such as Missouri, started with executive branch leadership and only later codified or attempted to codify general requirements into legislative statute.
What is encouraging about all of these approaches to
implementing RBD is the potential for them to learn from each other, and to
grow towards each other. Juanita
Blount-Clark, who is now a state agency division director and was formerly the
head of the cross-agency Family Policy Council, has been a leader in
implementing both a agency-focused and a community-collaborative-focused
approach. She emphasizes the need
to integrate them and use the best of each.
Once a leader understands the basic structure, the
first questions are, do we really want to do this? Why bother? Why go through
the hassle and risk of pursuing this new system? Leaders who are considering
how to enact or strengthen a results-based decisionmaking approach will need
to consider what they want (and can expect) to achieve by pursuing this path,
and what they risk by doing so. Any
effort to change the status quo will encounter resistance, and any effort
expended on this initiative has opportunity costs.
While many sources list general descriptions of what one could hope to
gain by pursuing RBD, as well as the cautions about implementing it, this
section focuses on the political gains and risks that might be expected.
could a state or city/county leader expect to gain by pursuing
There seem to be three main reasons why leaders pursue results-based decisionmaking. While there can be quick successes, leaders need to bear in mind that implementing RBD is a long-term process.
It will address the public’s demand for better results for children and families. While many people seem content with the country’s overall condition, there are certain areas that continue to capture the public’s attention, such as crime and education. Significant improvements in these areas can help leaders gain public support. For example, police departments in many cities, including Boston, New York and Long Beach, CA, have explicitly credited their use of RBD systems with contributing to significant declines in the rates of a variety of serious crimes. While many factors may have been involved in the decline, there is support for linking reduced crime rates with their efforts to measure results and align resources to support those results.
As another example, a variety of state and local school systems have used RBD systems as a tool to improve educational achievement. Texas leaders have publicized the state school system’s focus on results, and the new superintendent in Montgomery County, Maryland (population 800,000) has staked his reputation on a highly publicized initiative to “Build a System of Shared Accountability.” In particular, this superintendent sees the system as a way to focus attention on specific subgroups of students, especially racial minorities, in order to improve their test scores.
It will satisfy the public’s demand for government to work in new ways, and for government and communities to work together more effectively. While standard political advice suggests avoiding specific achievement targets, dates, and lines of responsibility, results accountability requires it. As Robert Behn notes in a forthcoming book, “Clarifying objectives is managerially sound but politically irrational…From experience, elected officials have learned that they can win more praise, support and votes by being fuzzy about what results government will produce then they can by being specific.”
Moving to results-based decisionmaking flies in the face of this conventional wisdom. Leaders must be prepared for the inevitability that initial measurements of child and family well-being will show deficiencies. While waiting for actual results to improve, many leaders have found public support for demonstrable efforts to improve public systems in ways that make sense. In short, there may be public support just for measuring and reporting the numbers, even if the numbers themselves are not good news.
When Portland, Oregon mayor Vera Katz announced at a meeting of the U.S. Conference of Mayors that she would produce a city-wide report card on the status of children and families, she was greeted with disbelief. Many mayors were concerned that she would be criticized for numbers that largely reflected factors outside her control and before her tenure. But she replied that she felt voters would support her efforts to know and publicize performance, while giving her time to improve them. And indeed, in her re-election campaign after the report card was released, there was some concern over the numbers, but voters strongly supported the production of the report card. By the next campaign, however, she will likely be expected to show meaningful improvements. The trick will be to determine and convince the public as to what improvements are reasonably within the scope of city programs.
Many observers of emerging efforts to use results believe that it can also encourage – even compel – individuals and agencies to collaborate when they had been resolutely independent. The results effort in Vermont brought the Agency of Human Services and the Department of Education together, and a new joint legislative committee in Maryland has brought child- and family-serving agencies to testify jointly on what the state needs to do to ensure that all children arrive at school ready to learn.
It resonates with their own beliefs about how to support children and families. For a number of politicians, these ideas resonate with their own personal beliefs about the role of government and community, and how both can most constructively support a better society. These ideas provide possible avenues to respond to their dissatisfaction with public processes and governance. This rationale necessarily complements the other two; using RBD is a long-term process which may not cause immediate improvements in the life situations of children and families and which will almost certainly generate controversy. Policymakers, therefore, need to believe in this process and be willing to stand on their belief that this is a better way. While some leaders will pick the fruit of these trees, some plant the orchard even while knowing that others may reap the harvest. When Rick Larison, the director of Illinois Governor George Ryan‘s Office of Statewide Performance Review, was asked why the Governor supported a new state-wide initiative to use results, he replied, “The Governor has a long history of being a state official. He wants his legacy to be change in the long-term operation of government. Of course, he wants the government’s interaction with the ‘man in the street’ to improve in the short term, but he’s also willing to plant the apple tree, knowing that he might not reap all of the apples.”
does a state or local leader risk by pursuing results-based
Pursuing results-based decisionmaking requires
using up a certain amount of political and other resources, and it entails
risk. These risks generally fall
into three categories; much of the remainder of this guide discusses ways to
minimize these risks.
Criticism for poor results. As discussed above, reporting on specific results is inherently risky. If the numbers look bad, leaders can be criticized for poor performance. If they look too good, leaders can be criticized either for not analyzing the data on under-performing populations, or for adjusting statistics, or for setting standards too low. Ultimately, most leaders will need to show improved performance to win continued support. However, an effective results-based decisionmaking system has a number of elements that can help build the support necessary to withstand the inevitable setbacks. One is obtaining buy-in from a variety of stakeholders on both the process of RBD and the strategies chosen to achieve the desired results. If the relevant parties agree to take a particular course, it’s harder to blame one leader exclusively for slow progress towards the destination. A second element is shared responsibility between peer organizations and between higher levels of government and communities for progress on outcomes. While public officials will always retain ultimate responsibility for public efforts, partnering with communities helps increase their sense of ownership of the course chosen and progress made.
Mistakes made in implementing results-based decisionmaking. Adopting RBD generally needs to involve more freedom from rules and regulations in exchange for more responsibility for results. Yet one person’s hoary regulation is another person’s cherished protection against past or anticipated grievous errors or injustices. Determining how and how much to increase flexibility is a trial-and-error process. Inevitably, the people involved in this movement – from governors to front-line workers to community collaboratives – will make mistakes. As Robert Behn points out, asking someone to excel at both accountability for process (meeting all financing rules and other regulations) and accountability for results is like expecting someone to be both a top scholar and a top athlete – some people can do both perfectly, but most mortals cannot.
Strategies to reduce this risk include anticipating these errors as much as possible, knowing which types are more or less acceptable, and involving potential and actual critics early. For example, it may be more forgivable to fund a program that is shown to be ineffective than it is to exclude certain groups from services. It is also important to bear in mind that results can be misused, and RBD needs to be accompanied by certain safeguards: “The shift to results-based accountability cannot be allowed to substitute for rock-bottom safeguards against fraud, abuse, poor services and inequities or discrimination based on race, gender, disability or ethnic background.” 
The discussion below on the media and communications strategies addresses how to buttress the initiative against inevitable mistakes and criticism. While some leaders have found the media to be unrelenting critics, others have had success with bringing editors and reporters on board early in the process and keeping them informed in order to elicit more favorable coverage.
Opportunity cost. Implementing RBD successfully takes considerable political capital and other resources. If leaders decide to pursue this path, they will use up a certain amount of time, resources and energy that cannot be focused elsewhere. Also, people – public managers, workers, community volunteers, the media – have only so much time and energy to understand and cope with change. Asking them to institute these changes may mean delaying or foregoing other initiatives. Leaders need to make a conscious choice about the value of RBD and how it ranks with other goals. However, what makes this course of action so appealing is that implementing RBD can have wide-ranging impacts and improve virtually every sector of public and private life.
Based on these benefits and risks, leaders need to decide
if they are willing to invest in RBD, if this is the right time, and how to
structure their initiative to maximize the chances of success.
The advice in this guide is designed to help them make these informed
guidebook says that using results effectively requires leadership.
But what if all of the right people aren’t in place?
One of the most universal conclusions about using results effectively is that it requires people in key positions, from the state to the community level, who believe in this process and have the personal and technical skills to carry it out. Certainly some of the most successful initiatives, such as those in Oregon, Texas, Vermont and Missouri, have had such leaders in the governor’s office, key agencies, and neighborhoods. However, many other places may have senior and local leaders who do not choose results-based decisionmaking as one of their priorities, or elected or appointed supporters may transition out of office. In these places, leaders may wish to implement RBD but are unsure if they should make do with what they have, or wait until a more propitious time. How can they decide what to do?
The prevailing advice is to start where the leadership does exist – leadership with sufficient capacity not only to set results, but to take action to improve them. As Jolie Bain Pillsbury puts it, “start wherever you are.” If leadership exists at the state level, start there. If county or city leaders are excited, move ahead with implementing these ideas at that level. For example, Mecklenberg County, North Carolina, has created a results-driven system at the county level that has effected real change in how leaders measure success and plan resources. One caveat has been that instituting RBD cannot be solely a grass-roots movement. While efforts that focus only on agency change will miss the power of communities to change people’s lives, efforts that try to effect change solely from the community level may also encounter significant obstacles. In order for RBD to succeed with significant numbers of families, it must involve more than small, token pots of resources and the attention only of community volunteers. It must involve support from public organizations. However, organizations at the city, county or state level can make significant strides forward, even if they do not yet have the support of the higher levels of government.
4. Initial Focus
RBD seems to be a huge undertaking. How
can the state or county decide where to start?
Shifting to results-based decisionmaking may seem overwhelming at first. So many elements need to be aligned, and so many constituencies need to be included in the discussions and decisions. States and localities may consider a variety of ways to start this process – focusing on one population, or one agency, or one geographic area. The “bottom line” advice seems to be to “start where the opportunities are.” As Joe Dear (chief of staff to Washington Governor Gary Locke) put it, “Use what you have. Start where you are. Do what you can.”
In terms of which opportunities to pursue, there are two sets of advice. One, espoused by Connie Revell of Washington state, is to find a “good news” story – an outcome that is improving – and figure out how to spread credit for it and build on it.
The other advice is to find a “bad news” story.
Mark Friedman points out that what motivates change is dissatisfaction
with the status quo, so he recommends looking where there is significant
dissatisfaction to motivate action. Similarly,
Jeff Tryens of the Oregon Progress Board suggests three criteria to use in
deciding where to focus initial efforts:
Room for improvement – choose an area where the results are poor and likely to respond to change.
Public interest and key leadership – choose an area where there is significant public interest in improvement, and where there is sufficient leadership to carry out the plan of action.
Some agreement on effective strategies – it is also helpful if there is some clear indication on what strategies will be effective – and/or if it is possible to develop community consensus on what to do.
Using these criteria, Tillamook County in Oregon focused its initial efforts on teen pregnancy. Local leaders were shocked to find they had one of the highest rates in the state, the public was passionate about this issue, and while there was no single strategy, there was support for a variety of approaches. Focusing on this one problem helped dramatically reduce their teen pregnancy rates.
5. All At Once, or Phasing It In?
the state or local leadership start the RBD initiative all at once, or phase
it in over time?
Like a swimmer trying to decide whether it’s better to enter a cold pool one toe at a time or by jumping off the diving board, states and localities need to decide whether to enact RBD across an entire state or region at once, or phase it in. There are two very different schools of thought on this topic, and plenty of examples on both sides. Florida chose a phased-in approach for its agency-focused system, allowing two volunteer agencies to start and then adding others. Martha Wellman of the Florida Legislature’s Office of Program Policy Analysis and Government Accountability says the phased-in approach made the whole measurement approval process more manageable and enabled agency, governor’s office, and legislative staff to learn from their earlier experiences with RBD. In addition, some agencies simply were not as prepared with the data and other infrastructure necessary to participate successfully, and this gave them time to build that capacity.
However, leaders in other places are equally adamant that they needed to do it all at once, or the process would have taken too long. Also, they feared that some agencies would set up endless appeals to delay implementation, because the data are too hard to collect (“you can’t measure what I do”), because there are special demands on the agency that would impede its ability to comply with these new requirements, or because of other arguments. Texas and Louisiana are two states that started their agency focus all at once and leaders there maintain that this was the right approach for them.
The same logic applies if states and localities are using other structures for results-based decisionmaking – for example, Washington state and Oregon created a state-wide network of community collaboratives all at once, while Georgia and Missouri have grown their networks over time. The advantage of the former approach is that it creates a presence that is harder to eliminate, but it does strain resources and require tremendous commitment to build capacity over a large area all at once.
Start with a vision specifically stated in terms of results for children and families. While this may seem obvious, many results initiatives seem to start with a vision of improved process or streamlined government operations or single agency performance, rather than a larger vision of improving the lives of children and families. While all of these are important, they are all means to an end to improvements in children and family lives. In interviews, many people said that having this vision is a great help in bringing group discussions back to the focus of their work, and in encouraging reluctant participants. While people may disagree about the ideal structure for government operations, there is less disagreement about the ideal status for children and families (healthy, successful, caring, etc.). While a focus on process does not necessarily compel great changes, a focus on children and families can and has inspired great work.
Move rapidly beyond the initial vision and mission stages by building on what already exists. A related issue is how much time to spend developing the vision and mission for this type of initiative, and how quickly to try and move to making decisions and acting on them. While standard procedures for creating new initiatives start with developing a group vision for change, this stage can take up long periods of time and lose people in the process. One piece of advice is to acknowledge the need to start with this step, but take advantage of what others have done, both within the state or county and beyond. Use “visions” developed by comparable initiatives, as well as lists of key indicators developed by other states. Jonathan Walters points out that a common mistake is believing that each place is so unique that it cannot use what others have already done. The basic desires for child and family well-being are very similar across states and even countries. Groups can move relatively quickly to the decision and action phases, while reserving the option of reviewing their mission selection later. The chairman of the Montgomery County (OH) Family and Children First Council, Brother Raymond Fitz, followed this path, developing a county-wide set of results in less than three months. By keeping the list as a working draft, he was able to move the group along quickly to taking action to improve the results.
Treat results-based decisionmaking as a better way of doing the existing work, not just another process added to the existing ones. As long as RBD is seen as a separate way of making decisions, allocating resources, etc., on top of the current processes, it will feel like a burden, a temporary fix and an ineffective use of energy and attention. While political decisions will never be made solely on the basis of results, RBD needs to be viewed as a better way of doing the existing work, rather than simply as another set of procedures to follow and reports to file.
Don’t make organizational changes until and unless there is a compelling need. One common strategy to emphasize collaborative work to improve results is to change organizational arrangements to encourage people and institutions to work together more closely. Yet one of the most resounding pieces of advice, born out of hard-won experience, is not to spend time on making these types of changes, until initiative leaders have gone as far as they could within existing structures. Con Hogan, former director of Vermont’s Agency of Human Services, points out that organizational changes require work on everything from office space to stationery to new supervisory relationships that can sap crucial energy away from the ultimate purpose of the re-organization. Creating these new relationships and new hierarchies can generate so much resistance that leaders cannot even get to the stage of using results. In the jargon of facilitation, leaders can be forced to spend so much time “forming, storming and norming” that they do not reach the “performing” stage.
Recognize that changes in results are more than the sum of changes in performance measures. All of the individual programs that society could possibly support are not enough to effect changes in the broad results, across large populations. This society is engaged in a constant assessment about what interventions will have payoffs beyond the immediate measures of improvement. An old analogy for this dynamic is “stone soup,” a newer one is “broken windows,” and the latest term is “the tipping point.” The idea in all of these is that small steps can have a larger impact than just the sums of their individual changes. In the broken windows scenario, political scientists found that fixing small problems, such as graffiti on the New York City subway, ultimately led to reductions in bigger problems, such as crime. One key to successfully using limited resources to leverage larger changes is choosing what small changes – performance measures – will led to bigger effects.
All parties must have something to gain – and something at risk. In order for a results-based decisionmaking system to succeed, both those who “oversee” the results and those expected to produce them need to have something to gain, and something at risk. As Jolie Bain Pillsbury points out, too often all of the risk is borne by the results producers, with no exposure for the ones overseeing the process. The Oklahoma Department of Rehabilitation Services has developed a strong performance contracting system with its vendors working to help people with physical and mental disabilities obtain and keep jobs. The agency pays vendors a percentage of its per-client cost according to the following scale:
Determination of need – 10% of bid
Vocational preparation – 10%
Job placement – 10%
Four week of job retention – 20%
Stabilization on the job – 20%
Consumer rehabilitated – 30%
Part of what has made the program so popular with vendors as well as agency staff is that the state agency “risks” at least something by paying vendors for services before it knows if those services will pay off in job placements. The same dynamic needs to exist when a state is “contracting” with a community collaborative for results – instead of expecting the community to bear all the risk for intractable problems, the state needs to bear some of the risk by providing necessary infrastructure, data, resources and other support.
Start with the available data and build from there. One of the knottiest problems in using results is the need for accurate, timely data on the status of children and families. And different types of data are required – data that describe the overall condition of children across a state or county, as well as data that track the contributions and performance of individual programs and providers. Gary Stangler, director of the Missouri Department of Social Services, describes data as a “mirror” that when held up to communities inspires them to action. Sara Hoffman, the Assistant Administrator for Contra Costa County, California, points out that until data can show that using results improves the lives of children and families, these initiatives are solely dependent on rhetoric and personalities for public support -- “like Blanche DuBois, we’re ‘dependent on the kindness of strangers.’”
Yet, few data collection systems have been set up to collect results data, to collect data across systems to give a more complete picture, or to provide analyses within the three to six months that service providers need to make rapid adjustments. While RBD cannot succeed without good data, a common issue is the delay caused by trying to achieve optimal data collection. Again the advice here echoes the Nike slogan – “just do it,” or “start with what you have.” RBD initiatives can start with the populations, or the issues, for which there are data, and build capacity for collecting more powerful variables over time. In Iowa, the new director of the agency of human services, Jessie Rasmussen, is starting with pilot initiatives for small groups of families where performance data will be hand-collected if necessary. Vermont has pioneered making results data available at the community level, on the internet – without massive changes in their systems or technology.
Take small steps at a brisk pace. Leaders trying to choose a course of action are often faced with contradictory aphorisms. In this case, the dilemma might be summed up as between “a journey of a 1,000 miles starts with a single step” and “you can’t jump a chasm in two steps.” Should initiative leaders aim for large, institutional changes that set a precedent, or smaller changes over time? While each state or county will need to make its own decisions, the preponderance of advice seems to be, in the words of Jessie Rasmussen, “small steps at a brisk pace.” Mark Friedman echoes this idea, emphasizing that change comes less often through a “big bang,” than through the many smaller, everyday interactions between people – between governors and agency heads, supervisor and employees, workers and families, neighbors and friends. The section below on changing management culture will explore this idea in more detail.
Focus can be more powerful than money. Another common assumption is that improving results must require swift reallocation of significant funds. There is no doubt that some of the infrastructure work necessary to implement RBD requires immediate resources. And, improving results will ultimately require aligning resources to support those results. However, veterans of this process point out that too often an early focus on money is unnecessary and can create tensions and divisions that threaten the success of the initiative. Vermont’s Con Hogan and Cheryl Mitchell emphasize that what has changed outcomes in that state is not so much new or re-aligned money but more a constant focus on how each person’s or organization’s work contributes to the chosen results. They maintain that consistently focusing on how each legislator, employee and community member contributes to the results has made a major impact on the results themselves. Just as the Heisenberg Uncertainty Principle says that simply measuring something disturbs it, Vermont’s leaders maintain that asking the question has an effect on the answer.
Common sense dictates that results and systems that have taken decades to develop will not be changed overnight. However, most leaders of this movement – governors, city/county officials and grass-roots advocates – need to be able to demonstrate some progress within two to three years, in order to maintain public support. More specifically, elected officials, often with four-year terms, want to have some specific accomplishments they can use as part of their next campaigns. This is especially true if they were elected on a platform of improvements in the status of children and families or increased government accountability.
Of course, exactly how far one can go depends on a variety of factors: past history, current leadership, and willingness to invest time, political capital and other resources. However, advice from people who have created these initiatives and thought through this question suggests that leaders can expect visible progress in a number of areas:
State or county-wide vision for results: Leaders can substantially complete a process to elicit public input into, and support for, at least an initial set of results that reflect the overall vision for the initiative. While some ground-breaking states took several years to complete this pains-taking process, other places can build on their work and accomplish this in less time.
Data on the status of children and families: Even with imperfect data systems, states and counties can develop “report cards” that reflect the status of children and families – in absolute terms and in comparison with other geographic areas. This gives a baseline by which to measure future progress, and it demonstrates a public commitment to responsibility for performance. Beyond these basic data, leaders can produce information on how resources are aligned to support specific populations or results (a children and family budget). Many states and counties have attempted these; many are available on the websites listed at the end. Montgomery County, Ohio started producing annual data for 15 children and family results – even though in some cases the data are incomplete or show worsening results over time, publishing the data is a major step forward.
Results accountability for specific systems. Within this period of time, it is also possible to establish sets of accountability measures – results, indicators, performance measures – for specific agencies, systems or populations. If there is strong support within those entities, it may be possible to create an entire accountability cycle – setting results, implementing strategies, measuring results, taking action – within this period of time for very specific groups. Some states have pursued the creation of local collaboratives to assume more responsibility for results. With sufficient resources, states can create local collaboratives or endow existing groups with some form of these new responsibilities, but it will probably take longer than two to three years before they can take on responsibility for performance.
Improved results for specific populations. . The most impressive accomplishment, of course, is demonstrated improvements in the knowledge, skills, behavior or status of children and families. If leaders focus their attention on a particular population, in an environment where the results are poor, interest intense and strategies promising, they can expect to see some changes in smaller results within two to three years. One key to ensuring that even limited improvements are seen as successes, instead of falling short of larger goals, is not to over-promise. An intense intervention can expect to see significant changes with a small population, or a broader intervention can expect to see smaller changes within a large population. But absent major changes, luck and resources, it will take longer than two to three years to see significant improvements in a large number of people. A second key is placing new data in the context of past performance, so that small improvements – or even reductions in the rate at which performance is worsening – is viewed accurately.
States and counties have made different choices about where to place their initial emphasis. Some have pursued an agency focus exclusively, others have tried to create accountability systems for specific services or populations. Each will need to decide where is the best opportunity to establish a foothold in climbing this mountain – where is the best combination of room for improvement, public interest and promising strategies – that will build public support.
Instituting results-based decisionmaking into the fabric of government, and using it to change how formal agencies work and how communities can support families, requires the support of a wide variety of constituencies. Each plays a key role in using results – each can support it actively, oppose it actively or simply wear it down by passive resistance – and leaders who wish to institutionalize RBD need to consider carefully the interests of each and how they can be aligned with using results. While the length of this guide does not permit an extensive discussion of each, below is a brief discussion of why each constituency is important and some “bottom line” advice on how they can be involved in ways that encourage their support.
1. Agency Staff
Public agency staff – from directors to front-line
workers – are important because they control large amounts of resources and
touch the most vulnerable families in many ways. The section below on management and culture will describe in
more detail suggestions for eliciting agency support. But perhaps the most important point is to consider how to
approach and encourage agency involvement.
Too often, RBD is used to threaten and punish public staff.
But much of the advice from people experienced in the change process
boils down to avoiding this approach and instead, introducing RBD as a vehicle
for joint problem-solving. They
stress that it’s best to assume from the outset that staff want to do a better
job and that, given certain assurances that assuage their most basic fears,
staff can and will use this tool to do so.
Of course, this approach will not work with everyone, and the section on
accountability describes a series of consequences (beginning with simple peer
encouragement) that can increase the pressure to improve.
But the almost universal conclusion is that beginning with the
“stick” approach, rather than the “carrot” is almost certain to set up
walls of resistance that are difficult to scale.
While the previous reform attempts have focused on agency changes, a new focus in the past few years has been the mobilization of communities, both to contribute to improved results and to take on additional responsibility for results. While the role of agencies may seem obvious, there is less agreement on the roles of communities in a comprehensive results-based decisionmaking system.
Many results-based reform initiatives have focused more on agencies and less on community roles, for a variety of reasons. Agencies are reluctant to engage communities when they feel they will be criticized, or if doing so might mean giving up power and resources. Communities are inherently messy, and often fragmented along economic, racial, historical or personal lines. Bill Laaninen, director of the Skagit (WA) County Community Network (a local collaborative), says that “sometimes agencies and community groups don’t know how to do much with each other except fight, and we need to get beyond that.” On the agency side, Steve Renne, the deputy director of the Missouri Department of Social Services, says that his experiences have only strengthened his conviction that “results-based reforms will not significantly improve the lives of children and families without a major emphasis on community involvement and empowerment.”
The bottom line advice is that any results-based reform
initiative needs to find ways to combine the strengths of agency-based reforms
with the power of community involvement. To
do so, it may be helpful to sketch out a variety of roles that community
collaboratives could play in a comprehensive reform effort that attempted both
to change formal agency processes and mobilize communities to support families:
Contribute to the state-wide vision of results. In order to build support for the changes necessary to improve results, it is essential to involve the public in setting those results. Many states have used community forums, town hall meetings or other public venues to suggest and prioritize their list of results.
Provide ideas on ways to improve performance. If agencies wish to accomplish their desired results, they will need ideas for improving the services they provide. Community collaboratives are ideally positioned to provide suggestions for changes in service delivery that will improve performance.
Try out new service delivery ideas. Collaboratives can be a testing ground for new policies, practices and programs. They can often fund and implement new ideas faster than agencies, which may have less flexibility.
Watch for populations that fall through the cracks. Collaboratives can watch for indications that certain groups are experiencing poor results, before they show up in formal statistics.
Implement results accountability within a distinct program or population. Community collaboratives can be testing grounds for results accountability within a specific program or a specific population. They can fund or organize intensive interventions on a smaller scale to see if they can improve poor results. However, a common complaint is the seeming unfairness of holding community collaboratives to a higher standard of accountability than formal, more resource-rich agencies.
Find out the reasons behind performance. After collecting data on performance, it is essential to determine why a strategy performed as it did, and what are the appropriate next steps. Community collaboratives can be a valuable source of insights as to why a program performed well or not, and what could improve it.
Build support for results accountability.
Enlisting community support for this movement can help build a bulwark
against inevitable criticism. Community
collaboratives are often very deeply-rooted in their neighborhoods and can
explain these ideas, build support for them and provide protections against
criticism for inevitable mistakes – but only if they are brought in as
partners. For this reason, the
secretary of human services in Iowa is personally visiting civic groups to
explain their results-based decisionmaking initiative and build community
Build support for results accountability.
Enlisting community support for this movement can help build a bulwark
against inevitable criticism. Community
collaboratives are often very deeply-rooted in their neighborhoods and can
explain these ideas, build support for them and provide protections against
criticism for inevitable mistakes – but only if they are brought in as
partners. For this reason, the
secretary of human services in Iowa is personally visiting civic groups to
explain their results-based decisionmaking initiative and build community
A comprehensive results-based decisionmaking system can originate from either the executive or legislative side of government; however, fewer legislative members than agencies have embraced RBD.
One issue may be the expectation that senior leaders, such as legislators, focus on high-level results and allow implementers the flexibility to achieve those results. Many legislators are concerned that this approach would impair their ability to follow their basic roles and responsibilities, such as setting public policy, protecting the public from abuses and ensuring that certain specific policy goals are carried out. However, there are ways legislators can use RBD while still preserving their responsibilities. They can keep certain rules (such as civil rights protections), examine data on the status of specific subpopulations (to ensure they are being served well), and involve community members to ensure that results data presented at a highly aggregated level is consistent with local experiences. Louisiana State Representative Jerry Luke LeBlanc says, “This can be done in any state in the country, no matter what budget system they have. Legislators can move away from line item budgeting to using results without losing the ability to set public policy.”
Another issue has been how to build legislative support in an era of term limits. Oregon State Senator Neil Bryant suggests building a core of support for RBD within each district, and organizing local supporters to show legislators why this makes a difference in their neighborhood. This can convince individual legislators to support RBD and help create a community base of support that will encourage future office-holders to have that perspective.
Below are some ways in which legislators have started to
use RBD ideas:
Asking for performance data – carefully presented, tightly analyzed, limited in scope: Many legislators have become discouraged about using performance data, because the information they receive is so voluminous and raw that it is not helpful to them in making decisions. It may be helpful for them to recognize when they are getting too much raw data, and to ask for selected measures, presented in ways that they can interpret.
Asking questions at budget time: There is widespread acceptance that using results will not remove politics entirely or even mostly from budget decisions (this is discussed in more detail in section V). And there is much work to be done to ensure that results information is presented in ways useful to legislators. However, some agencies have started to provide results data with their budget requests, and legislators have started to ask for that data when making budget decisions. Former Vermont Agency of Human Services Director Con Hogan notes that providing legislators with results data on their districts as part of his budget testimony has been a powerful tool. Louisiana State Representative Jerry Luke LeBlanc has led the process for that state to ask for increasing amounts of results data with budget requests. The legislature started with asking agencies to provide simple program descriptions, and then moved to performance indicators and now broader goals and objectives. As Representative LeBlanc said, “Of all the budget reforms I’ve seen over the decades, this is the only one that will survive, because it just makes common sense. It is a win-win situation for everyone.”
Asking agencies to present joint reports: Vermont and Maryland are two states that have asked agencies to present joint reports on results, the strategies to achieve them, and the resources to support those strategies. The co-chairman of the Maryland Joint Legislative Committee on Children, Youth and Families, Rep. Mark Shriver, describes their effort: "The Joint Committee is institutionalizing in Maryland government the revolutionary idea that government agencies can and should be held jointly responsible for the well being of children."
Directing new roles for state and local officials that support RBD: Another role for legislators is determining key functions for state officials. In Louisiana, the legislature has helped redefine the role of the state auditor in order to support using results.
Looking for new ways to accomplish results: Legislators and other elected officials can also look for new ways to give “results producers” the flexibility, financial support or other resources necessary to accomplish the desired results. The Governor of Iowa has asked the state auditor to review existing governmental rules to see if any can be streamlined to increase flexibility. In Washington state, legislators have asked the local networks (community collaboratives) for recommendations for changes that will improve results in their community.
The responsibilities and experiences of budgeting and financial management staff often lead them to be especially skeptical of this type of reform, as they are dedicated to preserving fiscal accountability, accustomed to the existing budgeting process, and concerned about the cost of implementing these ideas. Ultimately, changing results will require the involvement of more than small, flexible pots of money that are easier to move around. But most places that have pursued RBD have not tried to make wholesale changes in the formal budgeting process. Instead, one piece of advice is to go as far as possible within the existing budgeting structure. Instead of trying to decategorize whole fund sources, states and localities have tried to direct resources to the chosen results within the existing rules or with a bit of additional flexibility.
A second piece of advice is to show budget staff how these changes make a profound difference in people’s lives. Unlike many program staff, they often do not have the opportunity to see how these proposed changes make sense in the community. Lizbeth Leeson, formerly with the Department of Mental Health in Michigan, says this experience made a profound difference in how their budget staff viewed these ideas: “We took chief accountants, who said these changes couldn’t be done, to see the programs in action. They made a complete turnaround, and on the way back, on the plane, they began to brainstorm the ways it could happen – and then they ran the show.”
What Can Legislators Do?
The National Conference of State Legislatures has developed a list of actions legislators can take to reinforce the effective use of results in the legislative process.
1. Articulate results in many different forums:
Use intent language:
3. Use appropriations language:
Use intent language or
Ask for meaningful data:
Express support for state
and community efforts.
Susan Robison, Improving the Well-Being of Children and Families: A Results Toolkit for State Legislators (draft), Denver: National Conference of State Legislatures, forthcoming.
As with budgeting staff, auditors have a responsibility for regulatory compliance that may be at odds with the increased flexibility required by results-based decisionmaking. Many results-based decisionmaking initiatives have struggled to match audit requirements with the front-line discretion often required to achieve good performance. One of the areas for future work in RBD will be exploring how to help audit staff perform their important functions – ensuring that public funds are spent in ways consistent with the public trust – while also allowing more freedom from restrictions that hamper improved results. The Government Accounting Standards Board is pursuing that question (www.seagov.org). A few pieces of advice have emerged:
Consider new roles for auditors. Audit functions are beginning to shift, to encompass not just financial audits but also performance audits that include results. While the Texas state auditor has embraced RBD ideas, he has also maintained the independence of the office in order to be able to play a role in verifying performance.
It is important in RBD systems to distinguish between public expenditures that may be unorthodox but still defensible, and those that violate basic fiduciary, civil rights or other laws. Explicit conversations with audit staff to elicit their support in achieving some flexibility in the first case, while preserving their ability to guard against the second, may help.
Consider “hold-harmless periods” – Louisiana officials found it helpful to work with auditors to establish a “hold-harmless” period while both program officials and auditors figured out their new roles. This gave the agency some measure of protection while it was working with the new performance system, and it helped the auditor adjust its expectations.
Many state auditors are popularly elected, and it may be helpful for them to make their support for improved government accountability a campaign issue.
As with budgeting staff, it may help to give audit staff opportunities to see the impact of increased flexibility in communities.
Internal Systems (Personnel, Procurement, etc.)
One of the biggest challenges facing leaders implementing new RBD systems is, as Barry Van Lare points out, the need to maintain the protections created by existing personnel, procurement and related systems while moving them towards a results-oriented style of work. Fortunately, these departments are feeling the overall public pressure on government to work in new ways, and increasingly their leaders are political appointees, subject to gubernatorial direction, rather than career civil servants. One strategy is to involve these leaders early in the process; often they are brought in only at the later stages, when buy-in is more difficult to achieve. Another is to quickly start measuring their processes (e.g., length of time to hire staff or order supplies), in order to have specific numbers to illustrate the problem and the need for improvement. A third is explicitly showing this staff how their work contributes to better results – as with auditors and other “support” personnel, people can often be convinced to take on the task of change if they can see how it contributes to a greater good.
While the courts have been less prominent players in results accountability than the executive or legislative branches, they are and can be involved both in the process of examining their own performance, as well as participating in broader efforts to improve cross-agency results. Leaders can pique judges’ interest in RBA on several grounds. One is that it can help them improve their own courtroom proceedings, which can be a factor in elected judges’ campaigns. Dallas County, Texas, has a performance measurement system that includes the county courts. When it tried to measure the outcomes and efficiency of the courts, it found that “costs are measurable but [the] quality of justice is not as easily measurable.” However, they are improving the system over time, and judges are now using their cost ratings as a factor in their election campaigns. There are potentially serious concerns about adverse effects, but the county director of budget and evaluation maintains that this process is not “distorting the judicial process by measuring only easily measurable items.”
Another way to interest and involve judges in this process is in their role as monitors of executive branch agencies, such as child welfare systems. The implementation of effective RBD systems can help them by giving them data that helps determine how well the agency is working. And they can encourage agencies to use RBD by asking for and using this data.
Institutionalizing results-based decisionmaking will require continued public support, despite the predictable mistakes and poor results before the smooth operations and good results come into being. States and localities that start to allow more front-line discretion in exchange for better outcomes will inevitably encounter bad judgments and questionable decisions. Bringing the media into the decision to pursue RBD, and the expected payoffs for this risky venture, can help encourage more informed coverage of the events, and prevent criticism for the obstacles and pitfalls that are an inescapable part of change. Gary Stangler, the director of the Missouri Department of Social Services says, “Everything we do has a communications strategy. The media like to find a scapegoat – it makes good news. The key is to find the right scapegoat – one that shows that the problem is often a difficult situation, not necessarily a person.” When Washington Governor Gary Locke began his system of results accountability, he met with media representatives to explain the rationale behind his plan, and to elicit their buy-in to this venture. Duke University’s Robert Behn quotes Governor Locke’s chief of staff Joe Dear as stressing that it is important to build up “money in the bank” – credibility with the media – in order to be able to depend on that credibility when it is needed. The best reporters and commentators also want to report knowledgeably on issues, so they may welcome an orientation that enables them to ask the right questions and correctly interpret what they see.
Many advocacy groups may cheer government’s increased sense of accountability and the inclusion of community groups into results-based decisionmaking. However, at some point, state and local leaders will need to make decisions about priority results and strategies, and those whose populations, problems or programs are not chosen as the first priority (or whose performance does not bode well) may object. One strategy that may help is to show how one result is connected to others, emphasizing that working on one issue (such as teen pregnancy) may also help improve others (such as child abuse and neglect).
Advisory committees that have a formal role may also be valuable “eyes and ears” on how a results-based system is working – what are the hidden costs and benefits that might not show up in formal studies, or that need to be documented before statistics are collected.
Another issue is that there are interest groups
that have a stake in the existing situation, perhaps because they provide
services for the target population, they feel they understand the existing
system or other reasons. In order
to win the support of these advocates, leaders need to think about how they fit
into the new environment and not just leave them to fend for themselves.
10. Public Employee Unions
Of particular concern in working with agencies is how to build support within public employee unions (including teachers’ unions). Many of the possible elements of results-based decisionmaking (such as rewards for performance to individual workers) can contradict long-held union rules (such as rewards for seniority). However, a few places have made strides in eliciting union support. Two ideas have been helpful.
First, the advent of privatization of formerly public responsibilities has exposed union workers to more competition than before. Having to compete with private organizations for everything from garbage collection to child abuse investigations has led unions to recognize the usefulness of RBD in increasing their members’ productivity and ability to compete successfully with outside firms.
Second, unions, especially teacher unions, also feel the pressure from the public to demonstrate wise expenditures of public funds. Voters are less willing to support new revenues for schools, including teacher salaries, without accountability for improved test scores. In Montgomery County, Maryland, the new superintendent has unveiled a strong accountability system, with the support of teacher unions, because the union was involved in the design and recognized that the public demanded accountability. The head of the local teachers’ union noted the dramatically different approach to collective bargaining in this new environment, “We championed and negotiated a contract that emphasized improvements in the quality of teaching and learning, including specific accountability at the district, school and classroom level.”
Employers are always essential partners and can play a variety of roles. Their actions can directly affect children and family well-being, they can share knowledge about how to improve performance, and they can build support for the implementation of RBD. Many places have found strong allies in businesses, because it increases their confidence that government is working in more effective ways. Con Hogan of Vermont found that talking about results for children and families helped overcome strong resistance to support for increased access to health insurance, because there was more trust that the state government was thinking like a business. One business leader said to him, “I don’t always understand exactly what you are doing, but I have more confidence in you now that you’re focusing on results, because you’re thinking like we do.” One caution about business involvement is that the analogies between business processes and government processes can go too far. There are important and sometimes little-understood differences between the business environment and the public sector environment that affect how leaders can implement changes. A business may be able to drop a product line if it is not productive, while governments cannot stop trying to reduce teen pregnancy, for example, or seek a different “customer” base.
All of these groups play crucial roles in supporting,
slowing down or stopping results-based decisionmaking systems.
Working with these groups, recognizing their interests and how to
accommodate them can help build their support.
The next several sections address four major dimensions of implementing a results-based decisionmaking system: planning for results; aligning resources to support the chosen results; changing management and organizational culture to support RBD; and assuming accountability or responsibility for results. While all of these dimensions overlap to some extent, they help separate out some distinct tasks and issues in using results. Within each dimension is a set of advice that can help strengthen a state’s or locality’s chosen approach to results-based decisionmaking.
Strategic PlanningAn initial stage in moving to a focus on results is to choose the desired results and then develop a plan to achieve those results. To develop this plan, leaders need to choose results, indicators and performance measures (or their own variations on these measures); decide what existing or new strategies are necessary to achieve those results (including necessary partners); develop an action plan to carry out those strategies; measure performance; and decide how to use the data to improve performance over time. An initial stage in moving to a focus on results is to choose the desired results and then develop a plan to achieve those results. To develop this plan, leaders need to choose results, indicators and performance measures (or their own variations on these measures); decide what existing or new strategies are necessary to achieve those results (including necessary partners); develop an action plan to carry out those strategies; measure performance; and decide how to use the data to improve performance over time.
Exactly who does what step and who has to be consulted (e.g., the entire state citizenry or just agency staff) depends on the RBD approach being implemented. For agencies that are fitting RBD into their existing structure, this phase might mean working within their existing advisory and decisionmaking structure and examining only how to measure progress at the agency and program levels. For a community-focused approach to results-based decisionmaking, this phase might involve extensive consultations with the public and creation of avenues for people to work across systems. This phase of using results has been relatively well-documented and fits well with a variety of strategic planning models. Many states and localities have had several years of experience with using results in a strategic planning process. Leaders from these places offer a variety of lessons learned.
Start with a comprehensive vision of results for children and families. Often these processes start with a vision focused on the organization or agency’s role or position, rather than what the participants want for children and families. Or it is limited only to what a single agency can do. Focusing on children and families motivates people and helps remind participants why they are doing this work. Developing not only individual agency or organization results, but a vision across agencies or organizations, encourages the collaboration necessary to achieve long-term change.
Align different approaches. Many states or counties have two, three or four different results-based decisionmaking approaches underway at any one time: agency-focused, community-focused; initiated by the executive or legislative branch; initiated by different levels of government (federal, state, local); targeted at one specific population; and/or one spearheaded by the government or a major private entity such as the Untied Way. The planning phase is a good time to consider how to align the different approaches to results-based decisionmaking that already exist. They may not necessarily need to be condensed into one approach, which can create more resistance. But leaving them all to function independently can be frustrating, demoralizing and wasteful. Some coordination will help make the best use of everyone’s energies. For example, if a state-wide family council has developed a set of results, can those results serve as the basis for agency measures? If agencies are seeking ways to improve performance, can they ask community collaboratives for input? If the state is using RBD with one system (welfare, public education), can it use that experience to help other systems?
Move briskly beyond results and indicators. Many planning processes get stuck after developing their results and indicators. Using strategies to develop the vision more quickly (such as those in the “top tips” section) can help keep the process moving to the action phase.
Develop a common language, or at least a translator. It is helpful to have a form of “Rosetta Stone” that can help translate across federal, state, local and private results language frameworks. While many different initiatives have their own language and may not want to adopt a single one, it is helpful to lay out the different words and pick one set for the sake of the planning process. This helps speed up the conversation and prevent misunderstandings or endless clarifications.
Develop an explicit model that shows how all of the pieces fit together. Many planning efforts have created extensive hierarchies of results and measures – vision, results, benchmarks, targets, process measures, etc. – but they never lay out all of these elements to show a consistent story. Or they do not lay out all of the strategies and activities to show how they are expected to “add up to” the larger changes in children and families. A logic model shows relationship between the broader results, the causes and conditions that affect those results, the strategies to address those causes and conditions, and the measure that will show progress. It can show how a particular agency or program fits within the larger vision, and it can show how the pieces of the agency or program fit together. This can point out gaps that need to be filled, areas where an agency or program perhaps should not be held accountable for a larger result, and areas where partnerships are needed.
Separate and support the different roles of results brokers, results peers and results producers. Any given person in a new RBD system will often have three different – and new – roles. In each role, the individual person may be acting alone (such as a front-line worker) or on behalf of an organization (such as an agency head) or with a collection of organizations (such as a governor). The current language does not yet have precise terms for these complex relationships, so the short-hand terms below are offered with some caution. The three roles are:
Broker of results produced by another entity (“results broker”): managing or overseeing the performance of other entities in producing results.
Partner in achieving results with another entity (“results peer”): working collaboratively with peers who have their own accountability for results.
Producer of results for another entity (“results producer”): having responsibility for producing changes in results for children and families.
For example, a state agency head might be a producer of results in terms of his/her accountability to the governor; a manager of results in terms of his/her leadership for the agency and its partners; and a partner with other agency heads in improving results that cut across agency lines. A community collaborative might be a producer of results for the state-level body that created it; a manager for the organizations carrying out contracted work; and a partner for other groups in the community. The section below on changing management and culture to support RBD explores these roles in more detail. In each case, these may be very new functions for the people involved, and they will need support to know when to wear each “hat” and how to function with these new responsibilities.
Select indicators carefully. A common dilemma is picking too many indicators, or ones that cannot be measured or that do not communicate well. Mark Friedman has identified three major criteria for indicators: communication power (does it send a clear message), proxy power (does it reflect accurately the status of children and families), and data power (are reliable data available).
Shelley Metzenbaum of the Performance Management Project at Harvard’s Kennedy School of Government provides an example of clear communication of results. Cambridge’s mayor gives a letter grade (A, B, C, etc.) for the “swimmability” of the Charles River every year. The event is closely watched by the public and appears on the front page of every newspaper. While the technical data for the water quality may be more accurate, the letter grade version of the data makes the impact and generates media coverage. When the Montgomery County (MD) Public School system released its first indicators – student test scores – there was an outcry because of all of the limitations of the scores and the difficulty of relying on only one variable. Their next proposal (appendix 2) developed a cohesive set of measures that together give a more holistic picture of performance. What has made this new approach much more appealing is the use of data that convey what people are concerned about and that communicate well with different groups.
Decide on the level of evidence required for promising strategies. An essential part of the planning process is identifying “promising” or “best” practices to achieve some result. Each place may have a different standard of evidence required to support a particular strategy. In Vermont Agency of Human Service programs, community instincts and preferences are given more credence, while in other programs (such as some federal substance abuse programs), nothing short of published results using rigorous control group procedures is allowed. However, leaders must bear in mind that some groups – such as Native American populations – have little published information on effective practices and so standards of evidence may necessarily be different.
Recognize that using indicators and performance measures is separate from evaluation. It is also important to note that tracking indicators and performance measures is not a substitute for evaluations that track if an intervention really did “cause” a better result or if it is only correlated with it. Since many human service interventions cannot adhere to the evaluators’ “gold standard” of control groups with randomized assignment, leaders must be careful about the exact claims of attribution that they make.
Develop a process to prioritize strategies. Once an organization has laid out its promising strategies, it needs to decide among them. While this is never a scientific process and involves intense public and political involvement, it may be helpful to develop a set of criteria by which to consciously assess which strategies most warrant investment. These criteria might include the strategy’s effect on key factors influencing the result, evidence of effectiveness, ease of implementation, financial feasibility, and political support.
Pick something and do it. Often groups get mired in the process of deciding on results and strategies, and a frequent piece of advice is to move briskly through this process and “just pick something and do it.” Con Hogan points out that “anything leads to everything” – focusing on one result will inevitably lead to improvements in others. So pick something where there can be a quick win. Using results becomes more comfortable over time, and there are many fears about this process that after some point will not be allayed by more planning but only by actual (positive) experience. So it is important to move ahead on using results in some way, even if all of the parameters are not fully or perfectly in place.
Create a plan to develop the needed data. As the group implementing RBD is moving ahead on some aspect, one major gap is usually the lack of complete data on performance. Mark Friedman recommends that indicators that communicate well and are good proxies, but that lack good data, are the primary candidates for what he terms a “data development agenda” – further work to develop the capacity to collect and analyze data. Leaders can announce what measures they will be developing while not waiting to start until they have a complete system.
Decide how public to make this RBD initiative. In some cases, implementers do not have a choice about how public to make their initiative to use results – Illinois Governor Ryan made implementing RBD a campaign issue and so people were immediately watching his work to keep those promises. But in other cases, leaders can choose how quickly to go public with their initiative. On the one hand, a very public initiative can help overcome resistors, who realize that the leaders have no choice but to move ahead. On the other hand, starting small and out of the limelight, as with the Massachusetts Department of Public Welfare initiative in the mid-1980s, gives leaders time to fix the mistakes before too much public scrutiny.
Once there are chosen results and desired strategies, a next phase is to start to align resources with the desired results (and other measures) to support the strategies and activities necessary to achieve those results. This dimension of RBD can take place at many levels – from small community-based projects to entire state budgets. When the latest round of using results started in the mid-to-late 1980s, there was some hope that a results framework could be used to transform the formal public budget systems. Ideally, executive and legislative budgets would be developed based on desired results and effective performance, rather than on what was funded last year or what is politically popular. This has proved to be immensely difficult, however, and most places now aim for more incremental changes. As Ron Snell of the National Conference of State Legislatures commented, “Thinking that managing by results would solve the budgeting problems of the early 1990s was the wrong road. What we have found is that results do enrich the policy and budgeting debate.” Therefore, rather than aiming for objective results to replace subjective politics in budgetary decisions, most leaders are aiming to use results to inform and affect the budget debate.
Short of allocating the entire budgets of large programs to certain results, there are a variety of steps that states and communities can take to align resources to support results. These steps can be more formal or informal, and address smaller parts of larger budgets, or larger parts of smaller budgets. People who have worked through attempts to shift decisions about resources to focus on results offer some advice about how to encourage this change:
Develop children and family budgets to show how resources support results. While there will probably always be a need for program budgets, new tools that show how resources contribute to improved results can help ask and answer these questions effectively. These budgets show how existing fund allocations line up with selected results. While governors and legislatures do not (yet) develop new budgets in this way, they do help point out how the government is (or is not) allocating its resources according to its stated results. These budgets generally take one or both of two forms. The first is a list of results, with programs and their associated budgets assigned to each. This lines up total budget with the outcomes, but does not recognize that any one program is likely to contribute to more than one result. Another version is to use a “check box” format to show how each program contributes to one or more result. These two together can help a state or locality show how its resources align with its chosen results. (There are two other theoretically possible but seldom-used formats. One is to allocate a program’s dollars to every result it influences, which means counting the same funds multiple times. The second is to allocate portions of each program’s budget to the result it supports. This prevents double-counting but requires making some seemingly arbitrary decisions.)
Change the budgeting conversation rather than the budgeting rules. Rather than attempting to change the formal budgeting process, which generates resistance from those who are trained in and supportive of that process, leaders have focused on changing the conversations within the executive branch, and between the executive branch and the legislative branch. Simply asking questions about performance, without trying to formally tie dollars to data, can help agencies focus on being able to produce better answers about performance.
Change the questions asked at every step. All of the interactions in state and local budgeting processes are opportunities to have these conversations. Governors talk to agency leaders, agency leaders talk to their middle managers, managers talk to front-line workers, agency staff talk to community members, agency officials talk to legislators and legislative staff, budget staff talk to program staff. If each of these interactions focuses on how funds can support better results, it will change how participants think about their impact on results. As part of routine evaluations at the Vermont Agency of Human Services, employees are asked how their work contributes to the state-wide results. Former AHS director Con Hogan is convinced that it is this focus, rather than formal legislative budget process changes, that has helped the state improve the status of its children and families.
Use results in budgeting conversations within the executive branch. A few states and localities have begun asking program staff to justify their funding requests in terms of results-based performance. In Illinois, when the governor sits down with agency directors to develop the budget requests, part of the conversation is the agency’s performance on results and indicators. While the final decisions are not yet tied solely to performance, this does serve to convey how serious the governor is about performance and to start to use results to make resource decisions.
Use results in budgeting conversations between the executive and legislative branches. Formal legislative budgeting processes have as yet changed little in terms of their use of results to make funding decisions. However, as with the executive branch, elected officials are beginning to ask questions about agency performance on results and indicators. Sandra Moore of Missouri’s Family Investment Trust points out that state agency budget requests must include an assessment of the item’s impact on the state’s “Show Me Results.” Again, while legislators may still make the final decisions based on a variety of factors, having legislators and other elected officials ask for the information and having agencies offer it even when not asked, helps to introduce results into the decisions.
Take specific steps short of complete decategorization with mainstream funds. Completely eliminating budgeting rules is probably not realistic, given the deep-rooted support for many categorical programs. But there are steps leaders can take short of eliminating cherished protections. One is to allow savings from improved results to be used for prevention of those “bad” results. Another is to combine similar funds into one pool – but too often these strategies only end up combining prevention funds that were fairly flexible and appropriately allocated in the first place and don’t include the remediation funds that have the most restrictions. A third step is to allow funds to be rolled over from one fiscal year to the next, as Iowa Decategorization Boards allow. This simple (but difficult, since it is prohibited by many state constitutions) step would provide more flexibility in directing budget resources towards improving results.
Try using small amounts of mainstream funds as specific incentives for improvement. While making the budget of an entire program contingent on better results may be difficult, it is often possible to allocate a small portion based on performance. Martha Wellman of the Florida Office of Program Policy Analysis and Government Accountability points out that even small amounts of money can leverage significant performance improvements. Last year the Texas legislature used a small portion of the higher education budget to allocate $1.5 million to institutions whose performance meet or exceeded standards, and $0.5 million to institutions whose performance improved. While it is a small percentage of the overall budget, the action seems to be significant enough to capture the attention of the higher education system.
Align smaller budgets with results. Apart from the process of establishing large agency or local budgets, staff within agencies are also starting to allocate funds based on desired results. This is happening both within agencies, and as agencies pass funds onto local collaboratives, for their decisions about how to achieve results on a local basis. Over the past decade, the Vermont Agency of Human Services has shifted its funding to emphasize prevention services in the area of its priority results – and it has seen those results improve substantially. Missouri has decategorized and combined funds from eight different state agencies and shifted them to local collaboratives to spend according to their desired results – the state budget for this Caring Communities initiative was approximately $22 million in FY 2000.
Make room for varied strategies, including no- and low-cost ones. When deciding what strategies should be supported to accomplish the desired results, it is important to make room for different ideas. Too often, state and local groups think that they need to endorse only one strategy; forcing this decision causes more controversy than necessary. But a variety of places – from Tillamook County, Oregon, to Mecklenberg County, North Carolina – have found that people will be most energized to help if they can pursues strategies they believe in. Rather than allocating funds for one approach to reduce teen pregnancy, the Tillamook County Commission on Children and Families endorsed several complementary strategies organizations could support, including abstinence, sex education, after-school programs, mentoring and youth employment programs. Some of them didn’t cost anything, but all of them addressed an established factor in reducing teen pregnancy, and county leaders believe all of them played a part in their dramatic improvements in that result.
Show auditing, budgeting and finance people the advantages of aligning resources with results. As described above, while program staff often see the immediate benefits of a focus on results and an increase in front-line flexibility, finance staff often do not. They are asked to change a fundamental orientation, increase their workload, and increase their personal and professional risk – without an immediate motivation to change. Giving them a reason to change can help increase their willingness to take the necessary risks.
Form a core group that can start the movement. Many places have found it helpful to start with a core group that can start the shift to results-based decisionmaking. This can be especially helpful in the budgeting arena. Iowa has created a core group of legislators from the key budget and finance committees who are willing to take the time to learn about results-based decisionmaking and consider how it can impact the legislative process. In Washington state, staff at the Family Policy Council found state auditing staff who were enthusiastic about a results-focused approach and would take the time to work with local collaboratives to fit their work with auditing rules.
As important as planning, aligning resources and making cultural changes are to using results, the bottom line comes down to, are organizations and individuals using performance data to improve the efforts to improve the lives of children and families? Do they complete the cycle of planning, service provision and data collection, to actually use data to make policy and program changes? Relatively few places have actually gone through this complete cycle, but there is enough experience and wisdom to offer some insights:
Recognize that there needs to be an explicit set of steps between collecting data on performance, and taking actions because of that performance. There needs to be an analytical process between learning about the achievement levels of a program and administering the consequences of that performance. A poor level of achievement may be the result of: (1) the wrong strategy; (2) the right strategy, poorly implemented; (3) the right strategy, well-implemented but inadequately funded; (4) outside forces that no one anticipated; (5) a population with more challenges than anticipated; or even (6) faulty data that misrepresented the result. The participants in this analytical process need to examine both results measures and process measures to determine the reason for the level of performance before deciding on an indicated course of action to improve.
There also needs to be a separate process to determine what level of performance is “good” or “bad.” Relatively few accountability systems take into account the demographic and other factors that can make it harder for one group to improve. One way to address this is to examine the performance among groups or organizations with similar characteristics. A key piece of the Montgomery County (MD) School System accountability approach is to examine performance over time of schools with similar demographics – this was a deciding factor in the re-assignment of a principal whose student test scores were persistently lower than those of schools with similar student populations. A second way to address this is to use more sophisticated modeling programs that take these factors into account (as does the Kentucky school system when determining how well schools should perform and as does the Oklahoma Department of Vocational Rehabilitation when deciding how much it should cost to serve people with different disabilities). A set of guidelines for results accountability is included in appendix 3.
Recognize that there needs to be a variety of responses to good or poor performance. Too often, one hears “collect the data and then cut the budget or fire somebody” in the same breath, as though “cutting the budget and firing somebody” are the only and automatic responses to poor performance. This creates a number of problems. First, it rightly sets off alarm bells within the population vulnerable to criticism and throws up a firewall of resistance.
Second, it will often only exacerbate the problem. As Robert Behn notes in “The Wrong Way to Motivate,” “[I]t makes little sense to use budget cuts or increases to punish or reward public agencies for their performance. If a public agency is performing badly, it needs not less money but a change in leadership – or at least some improvement in its leadership. Keeping the same management at the agency while giving it fewer resources will hardly improve performance…if the fire department is doing a poor job, don’t cut its budget. Find out what the problem is and fix it.”
Third, if a system only has one major consequence, such as reduced funding or a job action, it will be reluctant to use it at all, until the situation has deteriorated to an unacceptable level.
On the other hand, performance measurement with no (good or bad) consequences is a toothless tiger. There is so much cynicism about the effectiveness of public organizations that an accountability system that does not include more hard-edged consequences is likely to met with considerable skepticism.
The key is to develop a series of consequences that is appropriate for each setting. The tone and exact procedures that work in Vermont may not work for the New York City Police Department. But the basic approach is the same. Below is a framework for a full range of consequences, based on ones states and localities are currently using. These consequences can be applied at the individual and/or organizational levels – for examples, performance bonuses can be awarded to organizations and/or individual workers. Few, if any, places use all of these, and they are not neatly sequential. But they are described generally in increasing order of severity.
Private reward and pressure, by supervisors and peers: Once an agency, organization, program, collaborative or individual has reached a certain level of performance, the first stage of using that data to improve is often private conversations between the “results broker” and “results producer,” or within a group of “results peers”. This may take place between a teacher and principal, police precinct commanders and the police chief, a community collaborative and its authorizing organization, an agency director and governor. This conversation can be used to share information about ways to address problems, to give praise for good performance, and to develop improvement plans.
There are ways to structure this interchange to encourage a positive environment for change. For example, it is important in these settings that the “results producers” be invited to share their own views on why a certain level of performance was reached, and either what the success can teach others, and/or what the person needs to do or have to improve.
It is often particularly effective to set up a peer network – peers can often reinforce each other more frequently than can supervisors and at least one study maintains that peers can exert more pressure on each other than can supervisors. However, they do need a structure that encourages positive peer cooperation rather than destructive competition. As with students in a class, if everyone is rewarded who performs at a certain level, then peers are encouraged to help each other. If only the top percentage or number are rewarded, peers are discouraged from collaborating or even sharing information.
Public reward and pressure: At a slightly more intense level of pressure is publicizing the performance data and allowing public praise or approbation to take effect. Vermont officials have found this to be a very effective approach. Simply putting each community’s data on the web, without explicit rewards or sanctions, has resulted in community members constantly asking their legislators and agency officials in public meetings about their community’s status, seeking good ideas from better-performing communities, and mobilizing to take their own actions to improve. Illinois state officials have used public pressure in a more intense way – they created a State Government Accountability Council, whose members include prominent businesspeople. The members can visit agencies, review their accountability work and provide suggestions. This puts a great deal of pressure on agencies to perform well.
Many organizations and systems have drawn the line here, resisting pressure to move to more tangible rewards and sanctions. However, others feel there need to be ways to provide more specific consequences, if peer and public pressure are insufficient.
Tangible rewards for success. More specific than public approbation is tangible rewards for good performance – money, equipment, etc. This includes more funding for programs and systems that do well, to spread their success, as well as individual rewards for people as a motivator and expression of appreciation. Many school systems have followed this route, providing cash awards either to individual teachers or to schools to distribute at their discretion, for good student performance. A version of this consequence is allowing organizations and individuals that generate savings to retain some of those funds. This approach has been especially helpful in eliciting union support – it is a step beyond simply publicizing the data but does not yet involve any penalties. Yet, some do not support the use of this form of consequences, viewing it as paying people for what they should be doing in the normal course of their work.
Increased autonomy: Organizations and individuals that demonstrate their competence can also be given more freedom from certain rules or requirements, such as administrative paperwork. Minnesota’s state refugee service program staff can renew funding for contractors with good performance without requiring organizations to re-bid the contract.
Increased assistance: Organizations and individuals that do not perform well can also be provided increased assistance, either voluntary or mandatory. The assistance can be unrestricted, such as additional resources, or restricted in some way, such as required staff training or the institution of lead workers or managers to boost flagging performance.
Reduced autonomy/increased oversight: Poor performance can also result in decreased freedom of operations and the institution of new or strengthened oversight procedures. Many school and hospital systems use this process, providing additional administrators to poorly performing schools or health care institutions. An extreme version is the state or local governments taking over schools or school systems, or courts appointing special receivers or monitors to oversee child protection agencies. The Florida Department of Environment Protection puts agencies into three groups based on performance: “good,” “watch,” and “monitor” – the third group is required to provide action plans for improvement.
Reduced or transferred funding: A more drastic step is to reduce funding for programs or organizations that are not performing well. Of course, this step is perhaps the most controversial, since it runs the risk of taking away resources from those organizations and individuals who need it most. In Florida, students at poorly performing schools can use a voucher system to attend a different school, which is a form of budget cutting for the student’s former school. It is important to note that this step generally falls at the end of a succession of other interventions.
Changing or terminating employment. The ultimate consequence is to change or terminate the employment of people who have not been able to produce satisfactory results. While many people feel this needs to be included in a complete set of options, the overwhelming advice again is never to lead with this threat but rather to try other strategies before resorting to this one. Having given an individual support, pressure and increased oversight does help in justifying and knowing when this ultimate step is necessary.
Again, leaders who are instituting results-based decisionmaking systems need to strike a balance in setting up their system of consequences – they need to show participants and the public that they are serious about accountability but also recognize that there can be a number of steps to take before reaching this point. Each place needs to decide which series of consequences is right for them and how to work with them.
Apply consequences (good and bad) to both “brokers” and “producers.” As mentioned in the beginning of the guide, Jolie Bain Pillsbury points out that in order for results-based decisionmaking to succeed, both parties in a “performance contract” must have something to gain – and something at risk. Too often, only the “producers” in any paired relationship are at risk (this can mean the agency director in a governor-agency relationship, as well as the community worker in a supervisor-worker relationship). If the state agency that is working with a community collaborative promises that it will do some work as part of the accountability relationship (such as developing a data system), and it does not, it needs to be accountable for that action or inaction.
Ensure that all parties in a performance contract understand and can carry out their responsibilities. Assigning and assuming accountability requires that participants clearly spell out the terms of the performance contract—who is responsible for what level of performance, over what period of time, with what resources, and with what consequences. Establishing and agreeing to these ground rules also requires new knowledge and skills. In order for partnerships, service providers and others to participate equitably, they need to understand indicators and performance measures, including how they should be chosen, what level of performance and what timeline is reasonable, and how consequences are assessed.
Realize that setting performance targets is an art, not a science. Part of accountability is deciding what is an acceptable level of performance, and what is not. Two points are key. The first is to use baselines – knowing how an individual, population or program has performed in the past is essential to setting reasonable expectations and knowing whether the current data indicate a satisfactory or unsatisfactory performance. Second is ensuring that performance expectations are commensurate with time and resources. One of the key mistakes often made in using results-based decisionmaking is expecting changes in the client population of one program to translate into changes in the status of larger populations. It is quite common for performance measures among clients of a particular program to show improvement, while indicators among the population at large continue to worsen. For example, no entity should expect to change the rate of school readiness for all children if they only have funds to serve 3% of the children who need help—or if they are only providing one element when the children need much more. But they can be expected to be held accountable for smaller changes, or changes within a smaller population.
Changing Management and Culture
Often changing the culture and management processes of an organization to support results-based decisionmaking is the hardest part of this entire process. Much has been written about the concerns of people who are both the “results brokers” as well as the “results producers.” These concerns – mostly that they will be held unfairly accountable for results over which they have insufficient control – are logical and need to be addressed. The results-based decisionmaking system needs to include safeguards and processes that take into account the many external variables that affect performance on results. However, the hard truth is that many people outside government are held responsible for results over which they have little or no control, and this will not be sufficient to stem the shift to results-based decisionmaking. So, the key is to build a process that moves inexorably forward while building in safeguards along the way. Some lessons learned here are:
on Changing Management and Culture for Results
1. Recognize that first impressions are important
2. Change everyday interactions
3. Provide the data, and training on using the data, necessary to improve performance
4. Support people in each of their roles
5. Help managers realize that either RBD will be imposed from the outside, or they can participate in the process
6. Provide periodic training, but back it up with management changes
7. Provide sufficient stafftime, from the state to the community levels, to get the work done
8. Create a safe support group
9. Recognize the burden on middle managers
10. Start with volunteers
11. Praise people incessantly
12. Enable people to work across agency and system boundaries
13. Recognize that implementing a results-based decisionmaking system creates a riskier and less controlled environment
14. Keep pushing ‘til it gives
Recognize that first impressions are important. As with other aspects of life, first impressions are important for setting the tone of the whole initiative to use results. If the initiative is touted as a way to “do more with less” or catch wrong-doers, or force people to work harder, then it will encounter immediate resistance from those it implicitly or explicitly criticizes. A virtually universal piece of advice is to couch the initiative as a shared effort at reaching shared aims – as a way to help workers achieve their own goals of helping families, and as a system that helps workers accomplish together what they cannot always affect as individuals. In many cases, people don’t resist change so much as they resist being changed. Using this as a way to help them work smarter and better helps reduce resistance.
Change everyday interactions. As with the budgeting process, instead of immediately tackling formal procedures and rules, try starting with changing everyday interactions and conversations. Posting results, and constantly asking both managers and front-line workers how they are contributing to those results, can have an impact.
Provide the data, and training on using the data, necessary to improve performance. One of the most challenging yet important aspects of moving to results-based decisionmaking is providing the data to measure performance, to know why a certain performance level was reached, and what to do to improve. Specific aspects of this work include:
Timeliness: Organizations that collect these data need to develop systems to provide feedback to “results brokers” and especially “results producers” on a timely basis so that they can see progress and make adjustments. Giving data a year after performance isn’t fast enough for Wal-Mart to keep inventory current, and it isn’t fast enough for workers to see the impact of their work. There needs to be interim measures (such as customer satisfaction) that help people know if their work is on track and how to make corrections. Christina Linville, the Contra Costa County (CA) deputy administrator, emphasized that timely feedback is a means to build front-line support for RBD: “When people got regular reports that showed the outcomes of their work, they got excited and saw the usefulness of the data. But it was important that data were used to help them do their jobs better, rather than just to criticize them.”
Balance: It is also important to choose a variety of measures that reflect different aspects of the work, and that enable people to see not only raw performance but other factors that give clues as to the reasons for a particular level of performance. The “Balanced Scorecard” approach is gaining popularity among private and public sectors; it recommends using a set of related measures to make decisions. If state or local leaders develop a logic model that reflects all of the pieces that need to be in place to succeed, they can track the implementation of those pieces as early signs of their likelihood of achieving results. These signs can be “leading” indicators of future performance, while waiting for the “lagging” indicators of actual result. The Montgomery County (MD) public school system uses a set of four indices to get a complete picture of how a school is performing:
Proficiency index: performance relative to a standard
Productivity index: performance over time
Equity index: performance of different groups of students
Quality index: measures of the quality of school life in addition to test scores
It is also important to make these data available to front-line
staff and community members. Vermont
posts community data on the internet, so community members can work with it
Training: It may not
be clear to front-line workers how to use the data to improve performance.
They need training on how to interpret the data, what it means for
their own way of work, and what it means for how they work with colleagues.
Support people in each of their roles. Each of the three roles described in the section on strategic planning requires a different set of skills and has within it its own set of tensions and responsibilities. Leaders of initiatives to implement results-based decisionmaking need to consider how to support people in each of these roles. For example:
People who have only been responsible for measuring process, or
who have not had responsibility for managing the performance of others, need
support to learn how to successfully perform a new role of results producer,
broker or peer. They also need to
learn how to manage using results data at the local level.
People who are accustomed to working only within their own
organizations and not to sharing responsibility or contributing resources
outside their organization will need support to become partners with other
organizations working jointly towards improved results.
People who are not accustomed to working with communities or
using their input need to learn how to do that effectively.
People who are
employed by an agency but are now expected to support the work of
community groups that can be critical of the agency may feel torn and may
not know precisely where to place their allegiance.
They often wonder if this change work they are being asked to do will
truly help – or ultimately hurt – their careers.
They need both help and reassurance to deal with these concerns.
People who are held
too tightly accountable for performance in their “producer” role will
be tempted to hold others rigidly accountable in their broker role. If
they are not given freedom to do what is necessary to produce changes, or
if they are held responsible for changes truly outside their scope, or if
there is no process to learn why they did not produce results before
implementing perhaps inappropriate consequences, they will tend to pass
those mistakes on to the people with whom they are working.
Help managers realize that either RBD will be imposed from the outside, or they can participate in the process. In many environments, the choice for managers is not whether to be accountable for results; rather, it is whether they will participate in choosing the results, and the rules, or have it done to them. Senior officials can choose the results for which they are accountable, or have the public do it for them. Sharon Lynn Kagan, of the Yale Bush Center on Child Development has used this same argument in convincing early childhood leaders to participate in the national debate about assessing young children’s readiness for school: “We as early educators must stand tall and take responsibility for designing the kind of assessment system that we want…because of the risk of being left behind if we do not [and]…because we have the knowledge to create assessment systems that will be a benefit and not a detriment to young children’s development.” In the same way, public managers may be more willing to participate if they recognize that doing so will not prevent the inevitable but rather enable them to have some input into what measures are chosen and how they are held accountable.
Provide periodic training, but back it up with management changes. People need to learn the skills that will help them carry out their new responsibilities, and they need to be able to apply them immediately and in an atmosphere of safety. Training courses for employees need to be followed immediately by opportunities to apply what they just learned, along with changes in management practices that support them in this new way of work. Telling public staff or community collaborative members that they have more responsibility for performance, along with more discretion, is useless if they are not then supported – with new procedures; data that provide useful, timely feedback; and managers that can help them use discretion wisely.
Provide sufficient stafftime, from the state to the community levels, to get the work done. . . Whether the overall approach to RBD emphasizes agency staff performance or the work of community collaboratives, this work requires paid staff dedicated to the work. Many agencies make the mistake of assigning this on top of existing staff workloads, and many state and local leaders have made the mistake of thinking that the community work can be done by volunteers. It is too time-intensive, technical, sophisticated and controversial to be pursued in spare minutes, or by volunteers. Often community collaboratives especially are wary of being criticized for “adding another layer of bureaucracy” and need explicit permission from public leaders to hire the staff necessary to do the work.
Create a safe support group. Cheryl Mitchell of the Vermont Agency of Human Services suggests creating a small management group where agency staff can discuss their concerns and obstacles they face without fear of exposure and can get peer help. Others suggest that leaders should be groomed for this work in groups of two. Everyone, including agency executives and front-line workers should have a “buddy” who is also implementing these changes, so that they can support each other.
Recognize the burden on middle managers. . . Often the most difficult group to convince to support results-based decisionmaking is middle managers. Senior government officials may be appointed based on or held directly accountable for their commitment to this process. Front-line workers can often see the benefit immediately. But mid-level managers are often stuck with much of the new and unglamorous administrative work associated with this process, while assuming a level of exposure they have never experienced before, and while not seeing the immediate benefit of this new way of work. No wonder they are reluctant. Any system that requires their commitment (and this one does) needs to recognize the burdens on them, and take steps to spread the work and increase the benefit to them.
Start with volunteers. Several states and localities have taken the tack of starting with people who are more willing to adopt results-based decisionmaking. This helps build support, generate quick wins and gain experience before trying to spread it to others. Vermont used the volunteer method with individual Agency of Human Service managers; the Oklahoma Department of Vocational Rehabilitation used it with contractors; and Florida used this approach with the first two agencies adopting its results accountability system. There is some controversy about this tactic, with some people advocating for making the adoption of RBD an across-the-board requirement from the beginning (whether that means all employees in one group or a whole state), but sometimes a phased-in approach works best.
Praise people incessantly. This old piece of advice is extremely valuable in any change process, and this one is no different. When asking people to increase their risk, it is important that they perceive some “return” as well. Con Hogan points out that public managers rarely have the opportunity to bask in the limelight of some job well done, while they are often asked to shoulder the unpleasant and unrewarding administrative work. Sharing credit – early and often – is essential to building support.
Enable people to work across agency and system boundaries. . . A major problem in changing culture to support RBD is breaking down the barriers between different programs, agencies and systems, so that they can support each other. The key to success here seems to be a deliberate strategy to identify areas of joint concern, as well as permission to pursue actions that pay off for a variety of partners. There are numerous success stories, and good advice from those stories.
Jake Jacobsen, Director of Social Services for Mecklenburg County, North Carolina, emphasizes identifying the pressures on the various systems and how one can help the others. The county’s social services department wanted access to the school’s student records in order to know which students in their caseload were in school. The school system was having trouble identifying children eligible for free and reduced price lunches (as most systems do). Jacobsen proposed to the school system that the social services department would use the two agencies’ pooled data to help both of them – and indeed they found 1,000 more kids who qualified. This joint work has promoted more collaboration between the two systems.
Beverly Godwin of the National Partnership for Reinventing Government tells of the Redland, California, police chief who used Geographic Information System data to determine the target areas and populations for crime. He found that crime among the elderly was a big problem, and that it was a concern of the city’s elderly services division as well. So one step city officials took to focus on the problem was to house the local elder affairs staff within the police department, to make it easier to coordinate the two agencies’ work. The data highlighted a joint problem, which made it easier to find a joint solution that fit the two agencies and helped both achieve their goals.
Another example comes from Washington state, where the Skagit County Community Network (a local collaborative) was charged with improving child abuse and neglect. The collaborative approached the local Child Protective Services office to ask what issues were most pressing. When the staff replied “reducing head lice,” the collaborative staff were a bit daunted, since that was not what they had in mind. But when they pulled together a group to work on that problem, they found that it was greatly troubling the school system as well. The group found a solution, and their willingness in help out with that unglamorous problem has opened the door to work on additional issues.
Recognize that implementing a results-based decisionmaking system creates a riskier and less controlled environment. Moving to results means giving up easily countable numbers in easily ruled relationships. Managers and community members alike who participate in this process need to be comfortable with giving up this tidy environment for one that is sometimes chaotic, often unpredictable and inevitably a bit vague. Of course, similar evolutions are happening in the business world, and it may help to use tools developed for that environment. Jolie Bain Pillsbury points out that while one cannot remove all of the risk associated with the shift to RBD, leaders can take steps to remove the unnecessary fear associated with it.
Keep pushing ‘til it gives. When senior leaders are asked how to overcome institutional obstacles to results, their answers sound remarkably similar. “Keep pushing ‘til it gives, “ “recognize that this takes time,” “keep finding ways to move forward,” are all common themes. One of the most important pieces of advice is to recognize that change will not happen at once, that this is sometimes a cyclical process that requires patience, the ability to look for and then take advantage of opportunities to move forward.
Results-based decisionmaking has the power to transform formal agencies, the role of communities and the lives of children and families. It can rebuild public faith in the ability of government to partner with communities to support families. It can energize tired workers and advocates who can now see progress. And it can catalyze needed changes among those who at last are rewarded not only for following the rules but for using their creativity and energy to create change.
But as with any change, there are risks. This is still a learning process, a huge experiment, albeit one that resonates deeply with many who have struggled for decades to improve the lives of children and families. The next stage will be watching and working with states and localities as they go through full cycles of accountability. It will be important to see how consequences are administered, if results improve and whether there are unintended effects. Equally important will be to explore the different approaches to results-based decisionmaking – agency-focused, community-collaborative- focused, etc. – and see if and how they move towards each other. The advice and ongoing experiences of people working in and across sites will help communities, states and other nations find better ways of using financial and human resources to achieve better lives for children and families.
Bardach, Eugene. Getting Agencies to Work Together: The Practice and Theory of Managerial Craftsmanship. Washington, DC: Brookings Institution Press, 1998.
Behn, Robert. Rethinking
Democratic Accountability. Washington, DC: Brookings Institution Press,
Robert and Peter Kant. “Strategies for Avoiding the Pitfalls of Performance
Productivity & Management Review, vol.
22, no. 4, June 1999.
Robert. “The Wrong Way to Motivate,” Governing,
Behn, Robert, Leadership
Counts: Lessons for Public Managers from the Massachusetts Welfare, Training
and Employment Program. Cambridge,
MA: Harvard University Press, 1991.
Center for the Study of
Social Policy. Beyond Lists: Moving to
Results-Based Accountability. Washington,
DC: Center for the Study of Social Policy, August 1996 (draft).
Child Trends. Children
and Welfare Reform: A Guide to Evaluating the Effects of State Welfare
Policies on Children. Washington, DC: author, 1999.
Connell, James and Anne
Kubisch. "Applying a Theory of Change Approach to the Evaluation of
Comprehensive Community Initiatives: Progress, Prospects and Problems,"
in Karen Fulbright-Anderson, Anne Kubisch and James Connell, eds., New
Approaches to Evaluating Community Initiatives, Volume 2: Theory, Measurement
and Analysis. Washington, DC: The Aspen Institute, 1998.
Friedman, Mark. A
Strategy Map for Results-Based Budgeting. Washington, DC: The Finance
Project, September 1996.
A Guide to Developing and Using Performance Measures in Results-Based
Budgeting. Washington, DC: The Finance Project, 1997.
Trading Outcome Accountability for Fund Flexibility. Washington, DC:
Center for the Study of Social Policy, no date.
Sid. “Moving Toward Outcomes: An Overview of the State of the Art and Key
Lessons for Agencies,” Fullerton, CA: Center for Collaboration for Children,
Gardner, Sid. Beyond
Collaboration to Results: Hard Choices in the Future of Services for Children
and Families. Fullerton, CA:
Center for Collaboration for Children, January 1996.
Gladwell, Malcolm. The
Tipping Point: How Little Things Can Make a Big Difference. New York:
Little, Brown and Company, 2000.
Performance Measurement: Getting
Results. Washington, DC: Urban Institute Press, 1999.
Cornelius and David Murphey. Towards an “Economics of Prevention”: Illustrations from Vermont’s
Experience. Washington, DC:
The Finance Project, 2000.
“Making Assessment Count…What Matters?” Young
Children, March 2000.
Kaplan, Robert and David
Norton. The Balanced Scorecard:
Translating Strategy Into Action. Boston: Harvard Business School Press,
Catherine Coles, and James Q. Wilson. Fixing
Broken Windows: Restoring Order and Reducing Crime in Our Communities. New
York: Free Press, 1998.
Liner, Blaine and Elisa
Vinson. Will States Meet the Challenge?
Washington, DC: The Urban Institute, June 1999.
Melaville, Atelia. A
Guide to Selecting Results and Indicators. Washington, DC: The Finance
Project, May 1997.
Montgomery County Council
Family and Children First Council, 1999
Report to the Community on Outcomes and Indicators. Dayton, Ohio: author,
Pratt, Clara et al. Building
Results: From Wellness Goals to Positive Outcomes for Oregon's Children, Youth
and Families. Salem, OR: Oregon Commission on Children and Families, 1997.
Improving the Well-Being of Children
and Families: A Results Toolkit for State Legislators (draft). Denver, CO:
National Conference of State Legislatures, forthcoming.
Morris. Working Without a Net: How to
Survive & Thrive In Today’s High Risk Business World. Englewood
Cliffs, NJ: Prentice Hall, 1996.
Schorr, Lisbeth, with
Frank Farrow, David Hornbeck and Sara Watson. The
Case for Shifting to Results-Based Accountability, Washington, DC: Center
for the Study of Social Policy, 1995.
The Finance Project. Building
Strong Communities: Crafting a Legislative Foundation. Washington, DC: The
Finance Project, 1996.
Trott, Charles and John
Baj. Building State Systems Based on
Performance: The Workforce Development Experience, A Guide for States.
Washington, D.C.: National Governors' Association, 1996.
Vinson, Elisa. Performance
Contracting in Six State Human Service Agencies. Washington, D.C.: The
Urban Institute, September 1999.
"Nothing As Practical As Good Theory: Exploring Theory-Based Evaluation
for Comprehensive Community Initiatives for Children and Families," in
James Connell, Anne Kubisch, Lisbeth Schorr and Carol Weiss, eds.,
New Approaches to Evaluating Community Initiatives: Concepts, Methods and
Contexts. Washington, DC: The Aspen Institute, 1995.
Finance Project: www.financeproject.org.
Information and materials on reform of family and children’s services,
including results-based decisionmaking.
Policy Studies Institute www.resultsaccountability.org.
Information and materials on results-based budgeting and accountability.
and Performance Accountability Guide www.raguide.org.
Online guide to results and performance accountability.
Accounting Standards Board: www.seagov.org.
Initiative to change government accounting standards to support managing for
Partnership for Reinventing Government: www.npr.gov.
Vice President Gore’s initiative to reform the federal government, including
using results-based decisionmaking.
North Carolina: www.charmeck.nc.us
Costa County, California Children’s Report Card: www.cccoe.k12.ca.us
Office of Program Policy Analysis and Government Accountability: www.oppaga.state.fl.us/government/
Budgeting for Results initiative: www.state.ia.us/government/dom/results_main_page.html
Performance Accountability System (LaPAS): www.doa.state.la.us/opb/lapas/lapas.html
state government: www.hhss.state.ne.org
Milestone (Results Based Contracting) Payment System: www.milestonemanagement.com.
Progress Board: www.econ.state.or.us/opb/index.htm
- Multnomah (OR) Progress Board: http://www.p-m-benchmarks.org/tblcnts.html
Performance-Based Budgeting System:
Appendices (not available from the on-line download)
Missouri’s System Reform Initiative: Core Results and Show Me Results
Montgomery County (MD) Public School accountability measures
Appendix 3: Sample guidelines for accountability
 Other terms that are commonly used to convey this spectrum of activities include “performance management,” “managing for results,” and “results and performance accountability” (the latter term coined by Mark Friedman). To avoid repetition, to convey that there are many different approaches to using results, and to use terms that may be familiar to other audiences, the guide occasionally uses these other phrases with similar intent.
 See for example, the series of state case studies produced by the Harvard Family Research Project, Reaching Results, Cambridge, MA: Harvard University, various dates.
 When a person is cited in this guide without a footnote, the information came from these personal interviews.
 See Mark Friedman, A Guide to Developing and Using Performance Measures in Results-Based Budgeting. Washington, DC: The Finance Project, 1997; Atelia Melaville, A Guide to Selecting Results and Indicators. Washington, DC: The Finance Project, May 1997; and Sara Watson, Using Results to Improve the Lives of Children and Families: A Guide for Public-Private Child Care Partnerships. Washington, DC: The Child Care Partnership Project, 2000.
 While the most popular term now seems to be “accountability” some experts are moving to the term “responsibility” as it seems to imply a less punitive approach. This guide will use both terms.
 Jonathan Walters, Measuring Up: Governing’s Guide to Performance Measurement for Geniuses (and other Public Managers). Washington, DC: Governing Books, 1997.
 Sid Gardner, “Moving Toward Outcomes: An Overview of the State of the Art and Key Lessons for Agencies” Fullerton, CA: Center for Collaboration for Children, California State University, 1996.
 Malcolm Gladwell, The Tipping Point: How Little Things Can Make a Big Difference. New York: Little, Brown and Company, 2000.
 Katherine Barrett and Richard Greene, speech, Managing for Results conference, April 28, 2000, Austin, Texas.
 Federal programs, such as the Workforce Investment Act, that use performance measures that cut across state agency domains (e.g., entry into unsubsidized employment) can help facilitate this cross-agency collaboration.
 See also Building Strong Communities: Crafting a Legislative Foundation. Washington, DC: The Finance Project, 1996.
 Robert Behn, Democratic Accountability. Washington, DC: The Brookings Institution, forthcoming.
 Robert Behn, ibid.
 Lisbeth Schorr, with Frank Farrow, David Hornbeck and Sara Watson, The Case for Shifting to Results-Based Accountability. Washington, DC: Center for the Study of Social Policy, 1995.
 Joe Dear, speech, Managing for Results conference, April 26, 2000, Austin, Texas.
 George Kelling Catherine Coles, and James Q. Wilson, Fixing Broken Windows: Restoring Order and Reducing Crime in Our Communities. New York: Free Press, 1998.
 Malcolm Gladwell. The Tipping Point: How Little Things Can Make a Big Difference, New York; Little, Brown and Co. , 2000
 Daniel O’Brien, speech, Managing for Results conference, Austin, Texas, April 26, 2000.
 Montgomery Council Family and Children First Council, 1999 Report to the Community on Outcomes and Indicators. Dayton, Ohio: author, 1999.
 Jerry Luke LeBlanc, speech, Managing for Results conference, April 27, 2000, Austin, Texas.
 Jerry Luke LeBlanc, ibid.
 Wilson Campbell, speech, Managing for Results conference, April 28, 2000, Austin, Texas.
Harry Hatry, speech, Managing for Results conference, April 28, 2000,
Rep. Jerry Luke LeBlanc,
Managing for Results conference, April 28, 2000, Austin, Texas.
 Philip Scheps, speech, Managing for Results conference, Austin, Texas, April 26, 2000
 Mark Simon, personal communication, October 14, 2000.
See for example, Atelia Melaville, A
Guide to Results and Indicators, Washington, DC: The Finance Project,
 Sara Watson, Using Results to Improve the Lives of Children and Families: A Guide for Public-Private Child Care Partnerships. Washington, DC: The Finance Project, 2000.
 The labels for the first two roles are from Jessie Rasmussen, secretary of the Iowa Department of Human Services.
 Mark Friedman, A Guide to Developing and Using Performance Measures in Results-Based Budgeting. Washington, DC: The Finance Project, 1997.
Hatry, speech, Managing for Results conference, Austin, Texas, April 29,
Children and Welfare Reform: A Guide to Evaluating the Effects of State
Welfare Policies on Children. Washington, DC: author, 1999.
Robert Behn, Leadership Counts:
Lessons for Public Managers from the Massachusetts Welfare, Training and
Employment Program. Cambridge,
MA: Harvard University Press, 1991.
Ron Snell, speech, Managing for Results conference, April 28, 2000,
Mark Friedman and Anna Danegger, A
Guide to Developing and Using Family and Children’s Budgets.
Washington, DC: The Finance Project, 1998.
 Cornelius Hogan and David Murphey, Towards an “Economics of Prevention”: Illustrations from Vermont’s Experience. Washington, DC: The Finance Project, 2000.
 In this guide, “consequence” encompasses both positive and negative effects, which is consistent with the technical definition.
 Dan O’Brien, Oklahoma Department of Vocational Rehabilitation, personal communication, April 27, 2000.
Eugene Bardach, Getting Agencies to Work
Together: The Practice and Theory of Managerial Craftsmanship. Washington,
DC: Brookings Institution Press, 1998.
In this guide “performance contract” means some type of formal
agreement between two parties, one of which is in the “results broker”
role and the other is in the “results producer” role.
It may be but is not always a financial agreement.
 Mark Friedman, Trading Outcome Accountability for Fund Flexibility. Washington, DC: Center for the Study of Social Policy, no date.
Lisbeth Schorr, with Frank Farrow, David Hornbeck and Sara Watson, The
Case for Shifting to Results-Based Accountability.
Washington, DC: Center for the Study of Social Policy, 1996.
Robert Kaplan and David Norton, The
Balanced Scorecard: Translating Strategy Into Action.
Boston, MA: Harvard Business School Press, 1996.
 Sharon L. Kagan, “Making Assessment Count…What Matters?” Young Children, March 2000.
Eugene Bardach, Getting Agencies to
Work Together: The Practice and Theory of Managerial Craftsmanship.
Washington, DC: Brookings Institution Press, 1998.
See for example, Morris Schechtman, Working
Without a Net: How to Survive and Thrive in Today’s High Risk Business
World. Englewood Cliffs,
NJ: Prentice Hall, 1994.