The problem with big data and the arts

‘Big data’ describes the increasing volume and detail of information available to organisations, companies and individuals. In the context of arts organisations, think of the amount of data generated by or contained within web analytics, CRM systems, social media and digitised archives. There’s a lot of talk going round about how this data may be captured, stored, analysed and visualised.

The aim of all of this is to gain insights (ideally meaningful and/or profitable ones) that would otherwise be unattainable. For instance, think about how marketing might be improved if organisations understood more about their patrons’ behaviours and preferences. Or look at how data analysis is changing research and creating the growing area of digital humanities.

However, there’s a suggestion that this might prove to be a tricky wave for the cultural sector to surf. Let’s have a look at a diagram…

The chart above shows the ‘big data potential value index’. It comes from a report called Big data: The next frontier for innovation, competition, and productivity that was published last year by the McKinsey Global Institute. Stick with me…

Can you see the bubble (more like a dot) that represents arts and entertainment? It’s tiny, showing that the sector makes a relatively small contribution to GDP in the USA. What’s more, its position in relation to all the other bubbles shows that, in terms of potential value, pretty much everyone other sector stands to benefit more.

That’s part of the problem.

The other part is that all this data manipulation requires people with a particular set of skills and those people aren’t abundant in the marketplace. I was at an event not so long ago where someone observed that recruiters for search marketing jobs are going after junior actuaries because they have the necessary mathematical and statistical skills. There’s a similar problem here. The report says:

There will be a shortage of talent necessary for organizations to take advantage of big data. By 2018, the United States alone could face a shortage of 140,000 to 190,000 people with deep analytical skills as well as 1.5 million managers and analysts with the know-how to use the analysis of big data to make effective decisions

How are arts organisations (even in the aggregate) going to fare in the competition for these people? Not well, I’d guess. They’re going to find themselves outgunned by everyone else.

In conclusion:

• There will be too few people with the necessary data analysis skills; and
• Large industries with a strong profit incentive will hoover up the scarce (and likely to be expensive) talent

So what exactly are they saying about the arts and big data?

There’s a table in the executive summary (p10) that scores each sector against several criteria. Overall, the arts and entertainment sector finds itself in the bottom quintile in terms of being able to capture the value potential of big data.

The report points to a lack of relevant skills in the sector and the absence of a pervading, data-driven mindset. However, at the same time, it’s said to be one of the most IT-intensive sectors with an average amount of data available.

In other words, there’s data around and the sector’s got the tools. There are just too few people who know what to do with it all.