[DISCUSSION] How can we measure innovation in a future open innovation system?

In Europe (for example), patent output is a measure with which the EU ranks member states on innovation.
See here: http://ec.europa.eu/research/innovation-union/index_en.cfm?pg=output

What indicators would Europe use in an innovation system that is underpinned by open source?

This is a systemic barrier to what we are trying to achieve and perhaps conceiving/speculating on an alternative set of measures, suited to our work, would be an interesting starting point for future policy-related activities.

3 Likes

Itā€™s definitely a tricky question. I imagine you would need a variety of different indicators in order to arrive at a fuzzy level of ā€˜innovationā€™. Thereā€™s a freakonomics post about the question of measuring innovation more broadly which is pretty interesting.

One kind of parallel you could look at would be something like impact factor in the academic world, except instead of citations, you could measure a kind of ā€˜usefulness factorā€™ where a project whose code/design elements are used in other open source projects gets a higher rating.

However we donā€™t tend to track usage in the open source world. For example in software, most F/LOSS projects have no idea how many users they have, let alone how many other projects are using their code. You can estimate it based on downloads, page views, activity on the code repository or community forum, but thereā€™s no helpful spyware telling you what people are doing with the software.

There are a lot of new plans afoot for being able to track and attribute digital assets using the :sparkles:BLOCKCHAIN :sparkles: which could be interesting if people are using open source licenses, but it could also lead to some kind of corporate-controlled dystopia where Big Brother knows all and you need to sign a EULA before you make a cup of tea.

Anyway, although you canā€™t easily measure use, one advantage of open source projects is that you can measure development, because the plans are open and they are (you would hope) regularly updated. Again, using software as an example, Openhub tracks the git repositories of most major F/LOSS projects - this the video editor that I use: Kdenlive.
Here you can see the whole history of the projectā€™s code, how many people have contributed to the codebase over the years. You can also see the worrying period between June and December 2013 when the main developer burned out and went AWOL, and there were no commits. Not such an innovative time. (heā€™s back now, and things are going really well for the project, if you were wondering :slight_smile: )

1 Like

I wonder what is done with the ranking and which effect it has e.g. on policies. It already is insufficient today, so I wonder what relies on this metric (taken from somewhere: Ā»All metrics are flawed, but some are more usefully flawed than othersā€¦Ā«)

1 Like

i would think that measures should include: classification of the type of innovation, its stage, the level of integration ( is the innovation used solely by its ā€œcreatorsā€ or has it been transferred to other groups), finally impact(s).

1 Like