162used in their evaluations.

4.1

Evaluators of Innovation Policy

Moving on to the different types of actors responsible for the evaluations of growth and innovation policies in Sweden, we see that most evaluations are carried out by consultants, either by self-employed consultants, larger firms, or several firms in constellation. Overall, slightly more than half (56 of 110) of all evaluations in our dataset are made by consultants. The second most frequent actor is evaluative agencies (31 out of 110 evaluations), followed by researchers or research groups (15 out of 110 evaluations). Public agencies evaluate themselves in the form of selfevaluations but such self-evaluations make up only 8 of the observed evaluations. In a few of the evaluations carried out by consultants and evaluative agencies, researchers have been invited to comment on the results, inform the evaluators about the evaluated field, or to carry out quantitative evaluations. In these cases, however, the researchers are not regarded as the evaluating actor because they only contribute to a small part of the work. Figure 2 shows the frequencies of different actors among the studied evaluations. The fact that so many evaluations are carried out by consultants aligns with the general public administration trend pointing to a large and increasing use of consultants in public administration (van den Berg et al., 2019; Ylönen & Kuusela, 2019).

163Evaluating Evaluations of Innovation Policy: Exploring Reliability,. . .

50

40 s n oita 31 ula 30 v e f o 20 re 15 b m u N 8 10

0 Evaluative Authority Consultants Self-evaluation Researcher

Fig. 2 Evaluative actors conducting evaluation of Swedish growth and innovation policies

4.2

Evaluation Methods and Data Sources

Regarding the methods used by evaluators, a few initial notes should be made: Evaluative practice could of course be studied and classified in different ways. One might, for instance, distinguish between methods focusing solely on goal accomplishment or on goal accomplishment as well as potential side effects. One could also focus on opinions from users or consumers of a certain policy, from the