A look at Let's Get Real 2: 'A journey towards understanding and measuring digital engagement'

Culture24 released their Let’s Get Real 2 report at a conference in Brighton the other week. I wasn’t able to go this time but I’ve been having a read through the report.

It’s another good piece of work and has moved on in a few ways since the first one (which I discussed here).

Recommendations

I really do recommend you download the report and read it. However, I know that too few people will bother with that so, in the interests of this information being spread a little more widely, here are some of the key recommendations:

Understanding web behaviours

• Keep asking yourself – what are the most important goals for my organisation that my website helps people accomplish?
• Don’t just ask what your audience does on your website or how they do it, but also why
• Learn to love segments in analytic tools
• Prioritise your design and content decisions based on the patterns of audience behaviour you want to have on your website
• Keep reviewing your audiences’ web behaviours to better understand and act on changes
• Make it a priority to be able to access analytics data from any third party ticketing websites or systems you may use

That first one’s interesting. There’s always a sense that the website must serve every possible department in an organisation, where actually a website could be a lot more effective if it concentrated on knocking a few key interactions out of the park. Good luck changing that attitude/approach though.

The last one hints at quite a meaty issue. In an ideal world cultural organisations would be able to afford excellent systems that provided them with all the information they needed. Experience suggests that ideal world is a way off.

Understanding mobile behaviours

• If 20% or more of your online audience is visiting your site via mobile devices and you don’t have a mobile-friendly site, you need to address this as a priority
• Track visits from mobile devices, separating out mobile phones and tablets and keep a close on eye on differences, and how those differences are changing over time
• Consider the tools that you may need to better understand mobile and tablet users
• Focus on micro conversions as well as macro conversions when seeking to understand mobile behaviours

I’ve discussed the first of these a couple of times below. I’ve got a ‘yes, and…’ that relates to that recommendation.

Separating out mobile and tablet users is important – the use cases can be very different. The final recommendation here surely applies to all users, not just mobile ones.

Understanding social media behaviours

• Use the Culture24 ‘Social Media Evaluation Framework’ to help you meaningfully interrogate your audiences’ existing social media behaviours against your organisational goals
• Consider if driving social media behaviours is the best way to achieve your organisational goals
• Remember social media is not free
• Use interaction and virality rates as a way of evaluating which social media posts users like to engage with
• Using imagery in your social media posts will drive audience comments and response
• Use qualitative analysis to interrogate more deeply the quality and sentiment of conversations taking place on your social media channels
• If your primary objective for using social media to engage audiences is to drive them to your website – think again!
• Remember that audience engagement with your organisation also occurs outside your own social media channels

Lots of good stuff there, although I totally disagree with the second to last one as an absolute statement (and have more than enough stats to back myself up there). The Social Media Evaluation Framework is included on p37 of the report.

There’s also a set of recommendations on conducting digital engagement research experiments. I’m not going to regurgitate those though because that wouldn’t quite do that section of the report justice. It’s interesting so please go read it.

Some of my thoughts

These are just some of the things I jotted down while reading through the report – notable bits, bits that made me say ‘yes, and…’ and bits that made me say ‘I’m not so sure about that’.

On p5:

For cultural activities, where success criteria (or performance indicators) are often not financial, this becomes a search to measure value. Within this search for value we can usefully use data analysis to drive better decision-making, internal change and ultimately to improve impact. Measuring value is subjective and must always be personal.

To better understand digital engagement, cultural organisations need to explore what and who they value, as well as understanding what their audiences value, before exploring how these can be enhanced through digital channels.

Yes to this. I’m not sure I quite grasp what the ‘must always be personal’ bit means but that’s by the by.

On p6:

If 20% or more of your online audience is visiting your site via mobile devices and you don’t have a mobile-friendly site, you need to address this as a priority

This is a decent rule of thumb but, where possible, I prefer to think of this in terms of cash money. If your mobile revenue isn’t keeping pace with/increasing in relation to desktop revenue then you should invest asap – you should be even be able to calculate how not going this is costing (whether you can afford the investment’s another issue).

On another note:

If your primary objective for using social media to engage audiences is to drive them to your website – think again!

I strongly disagree with this one, but I think that’s down to the difference between the data samples available to Culture24 and myself. From my point of view, plenty of cultural organisations’ websites are doing well out of social referrals – providing decent traffic and good amounts of revenue.

I’ve been going through some client’s stats recently and social referrals are outstripping organic searches in some cases – especially around on-sales. Besides, just because it’s a small percentage doesn’t mean it’s not a valuable one. Not that website referrals are the sole point of social media, yadda yadda, etc.

Remember that audience engagement with your organisation also occurs outside your own social media channels

This is important and so often overlooked. Discussed more on p62 of the report.

On p14:
Interesting that ‘consume digital content’ wasn’t one of the motivation/entrance categories. Not explicitly, anyway. I say interesting, because the Arts Council are prodding their portfolio organisations towards providing much more of this. Not that I necessarily agree that it’s a great idea – not for everyone, anyway (this is a conversation for another time).

Also, there’s a hint of disappointment here that it wasn’t possible to use exit surveys on the websites in the study. I’m pretty glad that’s not the case – those things are intrusive and evil.

On p16:
Good to see some of the research challenges and limitations discussed. This whole area is so rarely simple and clean-cut.

On pp17-21:
Oh no! Everything leading up to the stats was really good but why use stacked charts? I can’t make sense of the information at all. All I can do is compare the first and last items and be sure that everything adds up to 100%. And I know the colours are on-brand, but to the detriment of visibility?

On p22:

Planning a visit was the most popular category

As they say, unsurprising this, as websites for most cultural venues exist to support the venue. Strikes me that there’s room for organisations to develop more of a pureplay ‘digital’ offering. There’s a big digital land-grab going on out there at the moment.

On p23:

Cultural organisations should become better at attracting and supporting casual visits

I’m not quite clear what a ‘casual visit’ is and, come to think of it, this feels a bit at odds with some of the other advice in the report. What’s the point of these casual visits and how does pouring effort into them further the organisation’s mission. Is it a call for more cat videos? Is the ‘Engage in casual browsing’ segment intended to include visits from people who followed a link to an interesting thing from Twitter? Questions, questions.

On p24:
Nice bunch of questions for interrogating data.

On p31:
I wondered about the exclusion of BMAG, the Mead Gallery and the Photographer’s Gallery from this chart before I twigged that this is probably down to their not being any data for the first part of 2012. I know the Mead Gallery site didn’t launch until July 2012.

The velocity of change in mobile web visits indicates that even if organisations are not quite at the 20% tipping point, they will be by the end of the year. This inevitability exacerbates the need for all cultural organisations to examine their mobile and tablet visit statistics as a matter of urgency and to respond rapidly to the needs of these mobile audiences.

Yes. Although see above for my thoughts on this. To elaborate, I think I’ve come to the conclusion that visits aren’t really the issue. The question is can a user complete their goals on a given device? Those motivational segments are good for this. If people are just coming to your site to get information about how to visit then chances are you’re fine, especially as far as tablets are concerned. Quite a lot of desktop sites look alright on iPads if they’ve been built sensibly. Even on mobiles text is usually fine to read.

It’s transactions (and other key interactions) you want to worry about most. For instance, there are still a lot of flash-based purchase pathways out there. If your number of transactions aren’t at least keeping pace with whatever’s happening on desktops then you want to worry. The good thing is you can quantify how much it’s costing you and tell the person who holds the purse strings what needs to happen.

On p39:

Conducting individual research experiments

Good to see this. Nice step up from the first version of this project.

On p63:

As not all the organisations had ecommerce activities or the analytics settings to track traffic across subdomains, this part of the analysis was based on the few responses given on this aspect. Social media was not a significant direct source for sales since it represented a small percentage compared to other sources.

Just because it’s a small percentage compared to other sources doesn’t mean it’s not significant. See above for what I said about social referrals to cultural organisations’ websites – some do very well out of this and I suspect the results here are maybe just down to the organisations involved in the study and the ‘few responses’ given.

From what I’ve seen (and I’ve got access to analytics for quite a lot of organisations whose websites lean towards the transactional), social media drives more than enough sales to be notable. In fact, right now I’m looking at stats for a couple of organisations that have each taken more than £100k via social in the past year alone.