Module 5 – Basic Evaluation Methods and Monitoring Innovation

 

This module introduces the principles of Monitoring and Evaluation (M&E). Policymakers increasingly recognize the value of M&E to better understand what innovation-related investments and interventions are successful, which are less so, and the reasons why. Monitoring the performance of programs provides increased accountability and transparency in how scarce resources are used, contributes to policy learning and heightens the legitimacy of public action. 

M&E is a relatively young discipline, the boundaries of which are fluid and changing and, in recent years, understanding of M&E has been transformed. Traditional approaches to M&E focused on implementation and whether public actors mobilized the inputs and performed the activities as they were requested, in order to achieve the agreed outputs. Newer approaches have departed from this compliance, asking the “so what” question about the fact that outputs have been generated. They are motivated by whether interventions lead to valuable outcomes.  

This module explores in detail the sequence of steps involved in building a results-based M&E system from assessing needs, selecting indicators and setting targets to identifying appropriate data collection instruments and procedures. Each step presents policymakers with a series of trade-offs with respect to cost, precision, credibility and timeliness, and requires them to make decisions in light of local priorities and the current capacity of a government to sustain a results-based M&E system. 

The module then turns to evaluation which sheds light on whether outcomes can be attributed to (innovation) policy. Establishing that a given intervention caused rather than contributed to an outcome is complicated by a number of factors, including time lags, indirect effects, multiple and complex objectives, and the potential for public action to crowd out private initiatives. Various strategies have been conceived to address these issues. The module considers the role of randomized experiments, an evaluative method that has attracted considerable interest in the development community and has come to occupy a special place in the hierarchy of evidence. 

To work effectively, M&E regimes require political commitment and should be a consultative and participatory process. Frequently, however, the needs of end users are only vaguely understood by those responsible for M&E. Likewise, insufficient attention is paid to the organizational, political and cultural factors that impinge on the viability and sustainability of M&E. Demand for and ownership of M&E is critical to the introduction of any new system, but is often absent in developing countries. Such demand can be stimulated in a number of ways, though there is no single silver bullet for success. 

The module concludes by looking at some of the indicators and indexes that have been conventionally used to benchmark the innovation of firms and countries. It cautions that some of the traditional data collected in OECD countries have limited applicability to developing countries in light of their technological mix. It highlights alternative priorities and measures from the importance of investments in information and communication technology and linkages to proxies for the adoption and diffusion of technology that appear to have greater relevance in developing countries.

By the end of this module you will:

  • Be able to explain the role of M&E and its importance to the innovation system
  • Have discussed the various components of an effective M&E system through different stages of the design and implementation process
  • Have been exposed to the difficulty of making causal statements about the impact and effectiveness of innovation policies
  • Be able to list the political and organizational challenges to building a M&E system
  • Have discussed the strengths and weaknesses of conventional and unconventional indicators for benchmarking innovation
  • Have been exposed to real cases, such as the Finish system, which puts evaluation at the heart of policy formulation and implementation

Topics

Filter 
There are no topics to list.