Two-year follow up shows that effects of an OES anchoring intervention persist

Evaluation and Evidence Training Series

Open to federal executive branch employees only. Register on MAX.gov.

Building Logic Models for Evaluation

February 18, 2021 from 3:00 to 5:00 pm EST

Hands-on practice in developing and using a logic model to inform evaluation activities.

Evidence Act – Lessons from the First Two Years of Implementation

March 10, 2021 from 3:00 to 4:30 pm EST

Discuss agencies’ experiences and lessons learned from the first two years of implementation of the Evidence Act.

What did it cost to deliver OES behaviorally informed interventions between 2015-2018?

A recent review of behaviorally informed OES interventions designed between 2015 and 2018 found that the interventions had a small but significant impact. This exciting result left us wondering: what did it cost to deliver these interventions? And how do these costs help us interpret the impact findings?

To supplement the large-scale review, OES analyzed the cost of delivering the interventions included in the review. A vast majority of interventions analyzed — 92% — were of no, very low or low ongoing marginal cost to deliver.

Behavioral insights were most often applied through creation of a new message distribution; for example, a mailing campaign was added or a reminder text was introduced. Read more and stay tuned for an upcoming OES workshop on cost analysis in March!

Two-year follow up shows that effects of an OES anchoring intervention persist

A 2020 paper in the Journal of Public Economics highlights successful outcomes generated by OES over a two-year period. In 2016, OES collaborated with the Department of Defense to evaluate the effect of email messages with a suggested contribution rate on military Servicemembers’ enrollment in the Thrift Savings Plan (TSP).  The authors used two years of payroll data to see whether the initial increase in enrollment lasted among the 300,000 participants who were active-duty Army Servicemembers. The group that received an email with a specific contribution rate  had an increase of 0.61 percentage points when compared to the control group. The emails with a specific contribution rate created more than $9 million of additional retirement savings over the 2-year period. Read more.

New Project Resources

• Our vaccine portfolio page shares recent learnings from several OES behaviorally-informed direct communications to improve vaccination uptake. 
• Together, results of these evaluations suggest that behaviorally-informed direct communications can increase vaccination rates at scale but may have smaller, less reliable effects than the published literature suggests. 
• A new academic publication by OES and academic collaborators provides new evidence relevant to the National Flood Insurance Program administered by FEMA.
  • A simple form of broad bracketing—presenting the cumulative probability of loss over a longer time horizon—has the potential to alleviate barriers to risk perception and increase protective actions.

New Analysis Plans

• To ensure that our results mean what they are supposed to mean, we commit to a detailed Analysis Plan before we analyze outcome data. 
• It’s tax season and an ongoing OES evaluation will look at increasing take-up of the American Opportunity Tax Credit.
• An ongoing OES evaluation with USAID will look at increasing utilization of family planning services in Mozambique through an SMS intervention.
• OES will examine the effect of the San Diego Small Business Relief Fund funding on business resilience during the COVID-19 pandemic.