170networks of people designing and executing policies, and those that evaluate the same policies, is therefore a pertinent area of study.

5.3

Policy Recommendations

Despite evaluation being a scientific field and higher education curriculum subject in economics, public policy, political sciences, psychology, and the educational sciences, there is no specific education or public certification for evaluators of public policy programs. Yet, evaluations of public policies proliferate, and today represent a large industry within and across countries. We have yet to discover if any specific common practice or ethos is present among evaluators but currently, few such indications have been found. An increased focus upon creating such an education or ensuring a common framework of evaluative practice could be an important step toward ensuring different types of evaluations are used and interpreted according to their separate purposes. Another policy change to enhance evaluative practice could be to limit the type of evaluation allowed to be conducted (and financed) by executive agencies. It is important that such agencies are allowed to learn from and improve their implementation processes but to also assign the agencies the responsibility to evaluate their own policy efficiency or effectiveness is to create a system with distorted incentives. To solve this dilemma, such evaluations should be tasked to independent agencies and, in the case that private consultants are to be procured, such procurement should Evaluating Evaluations of Innovation Policy: Exploring Reliability,. . .

171involve criteria of both appropriate methods and independent practices. Such independence could potentially also be improved through some type of single-blinded system in which the agencies evaluated are unaware of who evaluated their policies.

6

Conclusion

In this chapter, we have explored evaluations of innovation policy. We add an important piece to the puzzle of innovation policy by studying a large sample of evaluations and looking for patterns across the data. Our results show that the overwhelming majority of evaluations are positive or neutral and that very few evaluations are negative. While this is the case across all categories of evaluators, we note that consulting firms stand out as particularly inclined to provide positive evaluations. The absence of negative or critical reports can be related to the fact that most of the studies do not rely upon methods that make it possible to discuss effects.

This discrepancy between so many positive evaluations on the one hand and comparatively weak evaluation methods, on the other hand, leads us to suspect that evaluators are not sufficiently independent. Consultants and scholars that are funded