QUICK SUMMARY

Chief evaluation offices serve to coordinate and promote the use of evidence (program evaluations, data, research, etc.) for decision making within agencies, while also working to help build evidence where it is needed and lacking. More broadly, these offices work to strengthen a culture of continuous learning and improvement.

 VIDEO OVERVIEW

STRATEGY DETAILS

Q1. What is the role of a chief evaluation office?

A chief evaluation office, which in practice can go by many names, creates capacity within an agency to help use and build evidence -- and to help offices and programs within the agency to do that. It's often led by a chief evaluation officer, or similar title, and staffed by a team that has experience in topics such as program evaluation, research, data analytics. In most cases, these offices don't conduct program evaluations or other types of research themselves, but rather contract for that research or work with external research partners. Instead, the main role of a chief evaluation office tends to include:

  • Creating a learning agenda for the agency, including by supporting offices or programs within the agency to create learning agendas that are then rolled up into an agency-wide version
  • Serving as a resource for offices and programs that want to do more evidence building, from quick and low cost approaches such as data analytics or rapid experimentation (including applying behavioral insights) to longer-term program evaluation.
  • Helping allocate evaluation funds, if the agency has them, to the most important evidence-building priorities (drawing on the learning agenda).
  • Developing an evaluation plan that details how learning agenda questions will be answered using internal or external resources
  • Developing an evaluation policy that communicates the values the agency demonstrates when it conducts or contracts for program evaluations or other types of building evidence.

Q2. What are examples of a chief evaluation office?

Most of our leading examples come from federal agencies, but the same model could be applied to any level of government. Some well-known examples include:

  • The Chief Evaluation Office at the U.S. Department of Labor, which represents an office that supports the entire department.
  • The Office of Planning, Research and Evaluation (OPRE) within the Administration for Children and Families, which represents a chief evaluation office that supports a sub-agency within HHS.
  • The Institute for Education Sciences within the U.S. Department of Education, which represents a semi-independent arm of a department that both supports evidence building within the field (of education in this case) by awarding research grants, while also being an evidence advisor to the department's leadership and programs.

Q3. Can you say more about one of those examples? How does the office work?

Sure. Let's take the case of the chief evaluation office at the DOL during the Obama administration, although many of these details have continued to be true today:

  • Overall role: The role of the office -- and the Chief Evaluation Officer (CEO) who leads it -- is to coordinate, encourage and build the capacity and understanding around evaluation throughout the department. The CEO’s budget includes set-aside funds for evaluation, which she can then allocate to advance agencies’ learning agendas. The CEO role is not designed to direct or centrally control all evaluation at DOL, but rather to encourage good evaluation.
  • Relationships with bureaus: Moreover, the CEO and her office work to create strong and productive relationships with the operating agencies -- particularly in terms of showing that evaluation can be useful to those agencies -- in at least two ways. First, they have framed their office’s work around a spirit of customer service, rather than emphasizing requirements (although learning agendas are required). Second, the focus of the CEO, just like the learning agendas, is primarily on learning and performance improvement, not an “up or down” verdict on particular programs. That has helped agencies to reduce their skittishness about evaluation and to see the CEO’s office as useful to them.
  • Connecting performance and evaluation efforts: Finally, another useful role of the CEO is to help connect evidence and performance management efforts, which (at least at the federal level) can be surprisingly siloed. For example, the CEO sits in the quarterly performance reviews with agencies run by the Deputy Secretary. In most meetings, there is often some discussion of evaluations underway. The CEO also provides input and assistance about existing or proposed performance measures, including identifying ways to add outcome measures or build more knowledge about whether an agency’s (or program’s) performance measures are correlated with impacts. That, in turn, can lead to updates to the agency’s learning agenda.

ADDITIONAL RESOURCES

CUSTOMIZED ASSISTANCE

Please contact us if your agency or organization needs help in developing a chief evaluation office.

PDF VERSION OF THIS PAGE (click here)