Promising Practices in Scale-Up Monitoring, Learning, & Evaluation: A Compendium of Resources

Although a wealth of tested, practical methods, tools, and guidelines exist for planning, monitoring, and evaluating project-based sexual and reproductive health (SRH) interventions, there is less guidance for best practices to support scale-up efforts. While donors and governments are increasingly demanding the scale-up of evidence-based practices, products, and approaches, the reality is that many interventions that have been taken to scale do not sustain their effects once project funding ends. There is growing realization that inadequate attention has been paid to systems issues that support successful and sustainable scale-up. Organizations undertaking systems-oriented approaches to scale-up will find few tested, practical tools and guidelines to monitor and evaluate scale-up of an innovation within a complex health system. In the context of FP2020, which strives to reach 120 million women with lifesaving family planning (FP) information, services, and supplies in by 2020, a more complete road map for learning from monitoring and evaluating the introduction and scale-up of new  FP methods and approaches is critical.

Georgetown University’s Institute for Reproductive Health (IRH), through its global FAM Project (Fertility Awareness-based Methods Project, 2007-2013), worked with partners in five countries to design and guide scale-up of a new FP method – the Standard Days Method® (SDM) – into FP programs and health systems and to use rigorous Monitoring, Learning, and Evaluation (MLE) to guide scale-up, from the design phase through implementation and evaluation phases.  Guided and informed by the systems-oriented approach espoused in the ExpandNet scale-up framework, this compendium represents a distillation of knowledge and tools for scale-up MLE developed and used by IRH to provide stakeholders timely monitoring information to inform scale-up decisions, learn throughout the process, and  evaluate achievements. We have developed this compendium to share our experiences and contribute to the growing evidence base on MLE of the scale-up process.

Tools highlighted in the compendium are organized into three sections according to their primary use—planning, monitoring and supervision, and evaluation. Accompanying each tool is a description of its purpose, how it was developed and used, whether/how the tool was used to collect information on values of the innovation, and lessons learned about what worked and what IRH would do differently going forward.

Download:

Suggested Steps: Scale-up MLE Planning & Implementation

This section contains tools that assist in planning the MLE of scale-up with key partners involved in scale-up. Planning leads to a clear definition of the innovation – it is not just the product, practice or approach – and the set of interventions that will support the scale-up process and expected end results. An important planning activity is assessing the scalability of an innovation and making needed adjustments to maximize success going to scale.

Download Full Section

Tool 1: Logic Model for Scale-Up

Tool 2: Benchmark Setting Worksheet

Tool 3: Defining the Innovation Worksheet

Tool 4: Scalability Assessment Table

This section provides samples of tools used for scale-up monitoring. Several tools assess quality in order to monitor whether the innovation is offered correctly by providers as it is going to scale and to assess whether users are correctly using the innovation and are satisfied with it. The key events timeline serves to collect information on factors outside of the immediate program that may be influencing the scale-up process, such as changes in policies or laws, political leadership, and/or donor project focus.

Download Full Section

Tool 1: Benchmark Tables

Tool 2: Focus Group Discussion Guide

Tool 3: Key Events Timeline

Tool 4: Quality Assurance Tools

This section contains evaluation tools and reinforces the importance of using mixed methods for evaluation of scale-up.  Most of the tools will be familiar to MLE experts, yet the way in which these tools are used in scale-up MLE changes. For example, household surveys are usually applied at base and endline to measure changes in community awareness and attitudes towards an innovation.  For scale-up evaluation, though, a more important impact measure endline may be how well an innovation is known relative to a similar product, practice, or approach (in IRH’s case, comparing community awareness and attitudes towards SDM to another FP method), since by the end of scale-up a new innovation has to be as well-known as others to increase its probability of being sustained. In addition, household surveys may need to be applied more frequently to provide timely information that allows decision-makers to take remedial actions if needed. Open-ended tools that allow collection of unexpected results can provide critical information not collected in other tools.  IRH used the Most Significant Change methodology, but other evaluation methodologies exist that are designed to capture unexpected results or perceptions by those offering or using the innovation or value of the innovation offered at scale. The tool selection and use of findings reinforce the importance of evaluation tools that serve program decision-makers, implementers and participants equally well.

Download Full Section

Tool 1: Household Survey

Tool 2: Facility Assessment Tool

Tool 3: Provider Interview Guide

Tool 4: Stakeholder In-Depth Interview Guide

Tool 5: Most Significant Change Story Collection Form

Tool 6: Organizational Capacity Assessment Tool for Innovation

Technical Consultation Report

A Systems Approach to the M&E of Scale-Up [PDF]

Briefing Paper

Theory and Practice: Monitoring & Evaluating Scale-Up for Health System Innovations [PDF]