Quantifying the quantified self


I finally bought a smart wristband, a device that measures how much you sleep and how much you walk.

After one month, I like it more then ever. It’s not very accurate, but it gives me a rough idea of how much I walk and rest. It nudges me in doing the right thing: if I see that I did not walk much, I’m more likely to choose not to take the bus back and walk instead.

What matters is not the actual number of steps, but the capacity to be aware of myself. I can easily see if I am more or less active compared to my normal rate. I have trend data, and that’s all I care. I don’t care much for comparing against other people. But its great to be able to detect whether I’m having a lazy or active day.

And that’s probably true for big data in general. Despite the hype, it’s very very difficult to make sense of many large datasets. Cross analyzing data, identifying patterns and correlation is harder than we expected [1,2]. Basically, the problem is to make sense of big data.

But big data means also that as many things get measured, you will soon start having trend data for everything. You will uncover anomalies much earlier because everything will be measured.

In other words, while it remains difficult to cross analyze a huge amount of large datasets to uncover correlation, it will become much easier to simply uncover anomalies by comparing new data with old data. This suggests that big data will become much more important in the data: we’re today deploying sensors and struggling to make sense of the data, but in the future trend data from these sensors will become available, and simply detecting an anomaly will raise attention on potential problems.

Speaking of which, I am currently looking for data on the market for “quantified self”. How many people have smart wristbands? how many have fitness or health apps? Where can I find key data points?

Quantifying the quantified self

Despite the rethorics, planning is more important than ever

This is a “off the top of my mind” post. Don’t expect anything useful.

There’s lots of talk about the need for emergent structures as opposed to ex-ante planning. About the death of traditional planning, the illusion of control, unpredictability and the role of black swans.

But in my daily worklife, I see planning becoming more, not less important.

Professional project management practices is becoming more, not less, important. It is percolating into more and more areas. Project management is a more important skill in the workforce, across sectors.

Great, modern startups and organisations are super professional in the way they manage projects. They always use templates and structured processes.

True, management is changing, becoming more flexible and less imposing (read: from MS Office to Asana). But scale matters more than ever, and you can’t scale without structured processes.

To sum up: it’s true that we moved from MS Project to Asana. Despite or because of that, project management is more pervasive and mainstreamed than ever.

Despite the rethorics, planning is more important than ever

Open Data in real life: some reasons to keep it closed

I have just registered my kid for school in Barcelona, it was an exhausting experience. Basically, if you have a two years old and live in Barcelona, chances are you’ve devoted the last three months mostly to this.

By the end of the process, I learnt what data are most used for decision:

  • number of teachers per pupil
  • number of external activities per trimestre
  • non-usage of textbook in primary school (indicator of openness/modernity)
  • socio-economic standing of the families
  • facilities (garden, sport, etc) but even basic things such as windows in rooms (some schools don’t have it)
  • own kitchen
  • additional services (e.g. logopedy)
  • average results of kids in secondary school test

On top of this basic data, most used it the feedback of other parents – provided think like you.

But because of the procedure used for subscription, the fundamental datum is oversubscription rate. You should choose a school that is not oversubscribed, and therefore the oversubscription rate is fundamental.

Obviously, these data are often not available and you have to chase them. I expect soon some app becoming available gathering such data.

But the fundamental point is that these data are not being published in easily readable format on purpose. The argument is that if data, and in particular the average results of pupils, were made available, parents will flock to the best performing schools and enhance the inequalities of the system. 

In other words, transparency would create a “rich getting richer” effect. The difficulty in getting the data ensures that only those who are really interested are getting this information.

Is this argument sound? Is it applicable to other domains? What are the main counterarguments? For sure this can be applied to mortality rates of the hospitals.

It reminds me of another apparently sound argument for limiting the openness, i.e. the embargo period for scientific publication coming from specific datasets in order to allow the author of the dataset to write his conclusions. This is also recognized as a valid limitation, but is it really so?

I wrote in the past on the need for “sharing literacy” rather than sharing culture , because one should know when and where to be open.

Can we identify a set of valid arguments to limit openness across domains?

Open Data in real life: some reasons to keep it closed

Case Studies in e-Government 2.0: Changing the citizen relationship

The great Marijn Janssen kindly invited me last year to write the preface of the book he edited, which has now been published.

You can download a pdf of my foreword here, but the gist is:

These factors led to an organic, rather than rationally planned, adoption of government 2.0 across governments. This was probably inevitable in view of the very nature of “2.0” technologies and should not be considered as negative in itself. However, this unstructured, bottom-up adoption led to a reality of many fragmented and improvised 2.0 initiatives. Decision-makers were put in a position where 2.0 initiative were suddenly a “must”, without being equipped with the intellectual tools to design, run and evaluate them. This is probably the reason why it is fair to say that while adoption of government 2.0 is almost universal, its impact is far from being demonstrated. There were a lot of “wheels” being reinvented, and disparate projects were launched in very different fields (from service delivery to political campaigning) without integration.

Case Studies in e-Government 2.0: Changing the citizen relationship

A single interface for Google and Twitter and Kindle? #canwehaveitplease

Today I was looking for evidence on the impact of open data on corruption. I tried google and google scholar, without much success. Next steps will be Quora and Twitter, maybe Diigo. Once I find something, I read it and look for links/references.

Can we have an integrated interface or workflow to do this? So that automatically what I type in Google becomes a question in Quora or Twitter, then highlights the same words in a text I am reading.

More broadly, it’s a pity that we split these information requests. If it was integrated, the flow that I follow would then become a path that could help further searches.

A single interface for Google and Twitter and Kindle? #canwehaveitplease

we need an internet of things for documents

The IoT is expected to bridge the digital and real world.

Unfortunately we still miss the tools to bridge the web with the documents world.

Some text are good for the web: they need to be short and snappy; others are good for reading as a document: they are typically long and deep.

There’s little in between. If I have a good paper or book, there is no way to interact with it; to re-publish; to discuss.

I am a big fan of in-line commentable documents (see this), but we have to be honest: it just doesn’t work, does not get traction.

The solution lies probably in e-books and reader, but they are still far from achieving this.

we need an internet of things for documents