169to other policy areas. However, one area that is similar to innovation policy in this respect is foreign aid policy.

5.2

Future Research

Our novel approach to study a larger quantity of evaluations simultaneously has proven useful in exposing systematic differences between evaluators and could hence be beneficially utilized for similar future tasks. Future research could investigate incentives motivating evaluators, their relationships to the evaluated agencies, and their general evaluative competences.

One important thing to point out is that not every evaluation aims to measure both the effectiveness and efficiency of any single policy studied. Yet, a conceptual confusion exists between these concepts in the evaluations scrutinized. There are of course valid reasons to evaluate both of these concepts, by on the one hand evaluating the degree to which a policy is effective (i.e., whether it succeeds in meeting its goals), and on the other hand, evaluating its degree of efficiency (e.g., cost-effective vs expensive, simple vs cumbersome). Focusing on, for example, experiences of beneficiaries or the viewpoints of bureaucrats executing the policies provides valuable information that can help improve policy efficiency. However, and importantly: The latter type of more process-oriented evaluative methods should not, as is often done, be used to indicate whether or not a policy is truly effective, i.e., accomplishing its designated results.

170E. Collin et al.

From this perspective, it can be concluded that assessments of effectiveness seem to be almost completely absent among the studied evaluations, yet, many of the evaluations still contain phrases that can be interpreted as gauging policy effectiveness (i.e., goal accomplishment), even if that is not the explicit intention of the evaluation in question or if the methods employed do not enable assessment of policy effectiveness. It is one thing to evaluate whether or not one has achieved the expected goals but another to investigate whether one could have achieved the same with fewer resources, or achieved better effects with the same resources. If evaluations are to work as a safeguard of a society’s common resources, such a perspective is truly warranted. Moving from what can be said based on the study presented, an additional feature of the evaluative system operating close to the innovative policy field deserves to be mentioned: There are not that many agencies, firms, and people working with innovation policy (and likely other policy fields) and the evaluation of such policies in any smaller country. It is not uncommon that people start their career within an executive agency and then move to work for the evaluative branch of the complex, maintaining relationships with previous coworkers and the agency in question. In our pre-study, we came across examples of consultants winning procurements partly through such relationships or inside knowledge. While such relationships can of course provide good insight into how to evaluate, in quite a critical and efficient way they also demonstrate risks of—possibly unintentional—corruption. Studying the