I have always been wary of rankings in policy analysis. Benchmarking and histograms seem to over-simplify reality, trivialize discussions and encourage “me-too” policy competition.

This is why our policy dashboards (see here, here and here) do not produce rankings, but visual summaries which emphasize the diversity and richness of information.

So when we decided to elaborate, together with the European Digital Forum (i.e. the Lisbon Council and Nesta), a report presenting the results of our startup manifesto policy tracker, our first version contained few rankings and lots of qualitative information: it was designed as a “policy map”. After several iterations and discussions, we accepted to pivot it towards a “policy scoreboard” with plenty of rankings: which countries do more to support startups?

The methodology, previously discussed on this blog, was finally mutuated from the OECD Going for Growth: a simple percentage of implementation of the recommendations contained in the original Startup Manifesto. Every recommendation has the same weight, even if some are crucial (e.g. legislation on second chance for entrepreneurs) and some are of dubious effectiveness (e.g. having a national champion).

I am pleased to communicate that the report has just been published and presented to Commissioner Oettinger and widely discussed on Twitter.

I have to admit that the online discussions always started by an assessment of how a country has scored. The rankings where the single “point of entry” into the discussions, that then evolved into insightful quali-quantitative reflections.

For all their limitations, rankings were clearly necessary for communication and, most importantly, in order to kickstart meaningful discussions.

In conclusions, rankings are nor good neither bad. They are just an important tool in policy analysis, that has to be used appropriately.

 

 

Advertisements