QUICK SUMMARY

Adding evidence incentives to grant programs is a way for agencies to encourage or require grant applicants to build or use credible evidence if they become grantees. The goal is to ensure that grant dollars go to approaches that work or, where evidence is lacking, that evidence is built about what works.

  STRATEGY DETAILS

Q1. What does it mean to add evidence incentives to grant programs?

Federal, state or local grant programs that are competitive (rather than formula) mean that applicants need to apply for those grants -- and grantees are selected based on selection criteria. For example, say those criteria add up to 100 possible points, called competitive preference points. Adding evidence incentives to those programs means designating some of those competitive preference points  for evidence. For instance, an applicant could earn five points if they cite a published evaluation or study backing the intervention they plan to run. Or maybe they could earn five point if they agree to be part of a national evaluation in order to build evidence. In Q2, below, we delve into these options further.

Note, by the way, that while competitive preference points can be used to create incentives for evidence, another option for grant programs is to simply require the use of evidence. For example, it could require applicants to select a program model from a list of evidence-based programs.

Q2. What are the main ways grant programs can integrate evidence of effectiveness into their program?

One way is to have applicants use evidence to design their projects: “Entry” evidence requires or creates incentives for applicants to demonstrate that their proposed interventions are supported by evidence. That evidence can range from having a logic model to citing rigorous research on effectiveness, depending on the level of evidence you’re asking applicants to demonstrate.

The other way is to have grantees build evidence about the effectiveness of their projects: “Exit” evidence requires or creates incentives for grantees to undertake a rigorous study to determine whether, and to what extent, their interventions are effective at improving relevant outcomes. Requirements or incentives can also be created around the type and quality of applicants’ proposed evaluations.

Q3. Why is adding evidence incentives useful?

By using evidence where it makes sense to do so for your program, you can help catalyze progress for the field and strengthen your program’s outcomes. In particular, entry evidence can help focusing grantees’ interventions on what works, while exit evaluation can encourage grantees to generate important new learning – for your program and the field – about what works.

Q4. When you add evidence incentives, what level of evidence can you specify?

Deciding to add evidence to a grant program also involves deciding what level of evidence to encourage or require – whether entry evidence (the level of evidence applicants need to submit in support of their proposed interventions) or as exit evidence (the level of evidence grantees need to aim to produce through their project evaluations). For example, the U.S. Department of Education's regulations identify four levels of evidence: 1) Strong evidence of effectiveness; 2) Moderate evidence of effectiveness; 3) Evidence of promise; and 4) Strong theory.

In terms of entry evidence, all programs should be candidates for the entry evidence level of strong theory, which requires simply a logic model. Ideally programs would focuses on higher levels of entry evidence. For strong evidence, for example, a program could require a randomized controlled trial (RCT) carried out on a large and multi-site sample or, alternatively, two quasi-experimental design studies (QEDs). Lower levels of evidence could require less rigorous studies, such as one QED or a correlational study.

Q5. How does a program decide whether to create incentives for entry evidence or exit evidence? (Or both?)

A series of questions can help a grant program make that decision. For example, to incentivize or require entry evidence: Is there a research evidence base around that program area, so that there are credible studies for applicants to cite? Questions about incentivizing or requiring exit evidence include: Are average grant award sizes large enough to make evaluation financially feasible? Is the sample size per grant large enough to make evaluation methodologically feasible? And is there sufficient program or contracted staff capacity to ensure that grantee evaluation dollars are well-spent?

ADDITIONAL RESOURCES

CUSTOMIZED ASSISTANCE

Please contact us if your organization needs help in adding evidence incentives to grant programs.

PDF VERSION OF THIS PAGE (click here)