Qualitative policy indicators cc @bettermeasured

One of our main activities at Open Evidence is to build policy scoreboards, such as this this and this.

I like this visual representation because it is not a ranking and it illustrates the full complexity of the issue. You can see who’s good and bad on a specific issue, but you don’t have a ranking which encourages “me too” behaviour and institutional isomorphism.

Screen Shot 2015-12-01 at 16.18.44.jpg
Screencap of the Startup Manifesto Policy Tracker

However, it is clear that rankings are an effective way to channel the results of an analysis. If you want to be heard, you need to tell a story. And the large dashboard tells many, not one story. I could bore you to death with the danger of a single story, but in my job, it helps to communicate.

So how can we build a qualitative policy indicator, that we could use for the startup manifesto?

The most interesting example so far is the OECD Reform Responsiveness index, included in the Going for Growth analysis (full details in the 2010 edition, page 79) and used in the Lisbon Council Euro Plus Monitor.

But still, it’s a simple percentage of reforms carried out over reform recommended. Not so sophisticated after all and without any weighting.

Do you know other good examples?

Qualitative policy indicators cc @bettermeasured

Accountability and moving targets: the importance of version control

At Open Evidence, one of our main activities is tools that allow tracking of policy progress, for instance of the Digital Agenda for Europe, eGovernment Action Plan, the Startup Manifesto, the Innovation Union, the Grand Coalition for Digital Jobs. Here, people in charge of each action are allowed to update data on its progress.

One problem we experienced, when you giver users full control, is that there are many changes to the original plan so that it becomes impossible to actually detect delays.

This is not an exception, but the rule in bureaucratic organisations. For instance, I just detected a beautiful graph from Linders, D., & Wilson, S. C. (2011). What is open government?: one year after the directive. 12th Annual International Conference on Digital Government Research (dg.o 2011), 262–271. doi:10.1145/2037556.2037599.

It compares the performance of the Open Government plans before and after the update of the action plan.

Screen Shot 2015-10-16 at 12.25.53

This shows clearly that updating the plan levelled out the differences between “good” and “bad” performers.

We should provide tools that monitor not only the performance against plan, but the changes in the plan itself across time.

Accountability and moving targets: the importance of version control

The rethorics of left wing reaction

One of my favourite books ever is Hirschman, A.O., 1991. The Rhetoric of Reaction: Perversity, Futility, Jeopardy.

It shows that the criticisms against progress have been the same for civil, political and social rights. Reactionary argument always stated that any such change would: make real conditions worse; not have an impact at all; hinder the progress already made so far.

I would suggest a similar analysis, today, on criticism towards free trade agreements such as TTIP.

First let me state that I am against the dispositions of the treaty dealing with special tribunals for disputes between companies and governments; and that I am generally in favour of free trade and against protectionism (see Ricardo and Krugman).

I see a common pattern of criticisms towards free trade agreements:

  • they create looser regulation (e.g. on environmental standards) by removing non-tariff barriers;
  • they favour multinationals;
  • they favour outsourcing jobs towards third countries hence leading to job losses.

For instance, the criticism towards TTIP are similar to those made towards the TPP here.

I think it would be nice to study if previous free trade agreements which are today considered as “given”, including the Single European Act of 1986, encountered similar criticisms to those listed above.

This would help putting the debate in perspective and eliminate criticisms defeated by history – hence giving more weight to well-founded criticisms.

The rethorics of left wing reaction

The image that makes the case for #openscience better than a thousand words


This is the coding mistake in Reinhart-Rogoff 2010 paper that was uncovered in 2013 by a student who finally obtained the original datasets. It’s easy to see the missing cells which include Belgium, a country with high debt and high growth.

Three years passed between the publication of the report and the uncovering of the mistakes. During this time, the report had been quoted extensively by scholars and politicians to justify austerity.

If releasing data were the default option in any such paper, such mistake would have been easily detected .

The image that makes the case for #openscience better than a thousand words

Podemos in power / 2

Just noticed today in the press that the former mayor envisages a possible alternative to Ada Colau, through a “saint alliance” of basically all the other parties.

What is interesting in this is that the single point of substance mentioned by the mayor (“what most worries me”) is precisely the Mobile World Congress in Barcelona. He says that many cities “would fight for such an event and that we need to handle it with care”.

UPDATE: on may 29th, Colau validated the offer from the previous alcalde for the MWC. Apparently her party “has always been in favour of MWC”. However, in her first press conference, she said she was in favour “but that its benefits should spread all over the city, the jobs should not be temporary, and all the social benefits of the the apps should be exploited”. A bit of bipartisanship and diplomacy emerging here.

Podemos in power / 2

Correlation is not causation. Is non-correlation non-causation? #offthetopofmyhead

Caveat: this is another “off the top of my mind” posts.

“Correlation is not causation” is a concept well familiar to all analysts – and to policy analysts in particular. It is so mainstream that it has its own wikipedia entry. The web is full of examples of weird correlations.

However, can we say that the absence of correlation indicates the absence of causation?

Certainly not: there might be other factors in place that affect a phenomenon and thereby “hide” the correlation.

At the same time, the absence of correlation is a much more reliable sign of the absence of causation, than its presence. It is much more likely that no correlation is confirmed (after more in depth analysis) as no causation, than correlation is confirmed as causation.

I would even say (as a rule of thumb) that the majority of non-correlations turns out to be non-causation, while the minority of correlations turns out to be causations.

Correlation is not causation. Is non-correlation non-causation? #offthetopofmyhead

Podemos takes the power: notes from Barcelona

In the elections this sunday, Ada Colau’s party  (affiliated to Podemos) was the most voted.

It is likely that she will be the new major of Barcelona, my current city.

This will be VERY interesting. Today, Barcelona is very much startup and business oriented, focussing on design and innovation, and the Mobile World Congress is probably its foremost symbol. Just as a sign of this culture, our previous building was recently renamed “Barcelona Growth Center”.

Obviously, we expect this to change quickly. It is a unique opportunity to see to what extent a city can change, in case of a power shift from right to radical left.

From today onwards, I will start taking notes to document this change.

The first sign appeared today in El Pais: it appears that the Mobile World Congress is less sure to remain in Barcelona, as the new “alcaldesa” warns that is should “provide benefits to the whole city”.

Podemos takes the power: notes from Barcelona