Monitoring and evaluation framework for WHO NTDs Roadmap

‘What gets measured gets done’. The recently published Monitoring and evaluation framework is another of the companion documents that goes alongside the new WHO 2021-2030 NTDs Roadmap. It is written not only for countries but also for implementing partners, like ILEP members. The intent is that the burden of all NTDs is reported for greater accountability and action, starting at the country level. Leprosy-endemic countries will be urged to rethink their monitoring and reporting processes so it is valuable for ILEP members and partners to have an understanding of what the framework proposes.

Quantitative measures for monitoring progress

The quantitative measures proposed in the framework include a small number of cross-cutting indicators accompanied by disease-specific indicators. These quantitative indicators are focused primarily on assessing impact. Some of the cross-cutting issues are familiar ones from the paradigm changes in the Roadmap. For example:

  • Share of countries including NTD interventions in their package of essential services and budgeting for them.
  • Number of countries that adopt and implement integrated skin NTD strategies
  • Share of countries with NTDs integrated in national health strategies/plans
  • Share of countries with guidelines for management of NTD-related disabilities within national health systems

The quantitative measures specifically for leprosy are the over-arching impact targets seen in the Global Leprosy Strategy:

  • Number of countries with zero new autochthonous leprosy cases
  • Annual number of new leprosy cases detected
  • Rate (per million population) of new cases with grade 2 disability
  • Rate (per million child population) of new paediatric cases with leprosy

One of the key expectations is that NTD disease data at country level will be collected and stored on integrated data platforms that are mainstreamed into national health information systems. There should not be separate, stand-alone data collection systems just for leprosy. The Framework recognises that this may not be achievable in all countries, but any national NTD-specific systems should at least interface with the national health information system. In many of the countries where ILEP members work, health information systems are weak, and so investment will be needed to strengthen information systems, as well as to strengthen health systems as a whole.

Qualitative measures for monitoring progress

The qualitative measures are in the form of gap assessment conducted for each NTD including leprosy, analysed through a heat map. This is expected to look similar to the heat map outlined on page 26 of the NTD Roadmap, where each disease was assessed against 11 indicators and categorised from red (severe hindrances to achieving results, needing critical action) through orange, yellow and green (no hindrances). The results for leprosy were red (1), orange (7) and yellow (3), and these would be seen as a baseline: the first re-assessment will take place in 2023. The 11 indicators covered Technical (scientific understanding, diagnostics, effective interventions), strategy and service delivery (formal operational guidance, planning and programme management, monitoring and evaluation, access and logistics, and infrastructure and workforce) and enablers (advocacy and funding, collaboration and capacity and awareness building).

Evaluation

Monitoring impacts is important, but it needs to be accompanied by evaluation. As the chart below shows, impact data (from the monitoring framework) needs to be analysed along with data on programme implementation to create a comprehensive evidence base. From that point, policy dialogue and debate should lead to agreement on actions to be taken, for example on priority-setting, resource allocation and course correction.

This evaluation activity will take at country level, and also at global level, where quantitative and qualitative data will be combined to give a picture of progress for each NTD and against the Roadmap as a whole. Leprosy data will continue to be published annually in the Weekly Epidemiological Record.  

The main M&E issues for leprosy

In the leprosy heatmap undertaken for the NTD Roadmap (mentioned above), monitoring and evaluation are rated orange (significant gap). Referring to the current status, the report said that:

  • Roll-out digitalized case-based data management system is ongoing
  • Mapping of cases is being introduced
  • Integrated programme reviews are occurring, with focus on reviewing progress in reaching the leprosy programme targets
  • Periodic monitoring for reactions is weak

The actions required were:

  • Utilize mapping tools and strong surveillance system to ensure detection of sporadic and hidden cases and to monitor progress
  • Improve notification systems
  • Develop mechanisms to monitor adverse events
  • Expand monitoring of antimicrobial resistance

Readers of the ILEP Updates will know that several initiatives are currently under way in terms of the actions required.