The Social Media Metrics Debate – “Meaningless and Spurious Metrics?”

In 2014, social media represents a massive and growing opportunity for marketing/communications strategies, using big data and analytics. There is massive and growing disagreement in the ICT industry on how to measure the effectiveness of those strategies.

This is not new. Over the past few years, advice from industry marketing gurus has ranged from  “100 ways to measure social media” by David Berkowitz (http://www.marketersstudio.com/2009/11/100-ways-to-measure-social-media-.html) to

“Why Your Social Media Metrics Are a Waste of Time” from  Ivory Madison, founder and CEO of Red Room (http://blogs.hbr.org/2012/12/why-your-social-media-metrics/.)

Now in a January 22, 2014 Future Buzz post, The No-Nonsense Guide to ROI in Social Media, (http://thefuturebuzz.com/2014/01/22/the-no-nonsense-guide-to-roi-in-social-media/) guest author Muhammad Saleem tries to set the record straight. The author notes that “As a digital marketing strategist it confounds me that even as 2014 is in full swing, marketers, publishers and startups are sadly (still) debating the issue of return on investment (ROI) from social media…Yet none of this is new and for the sophisticated marketer social measurement is a mature part of their programs.”

In this case, Muhammad’s frustration has been sparked by “a flurry of articles written on the topic recently driven in large part by an E-commerce Holiday Trends and Benchmarks study from Custora declaring search and email marketing as clear winners over social media when it comes to driving Ecommerce purchases. What is particularly surprising is the marketers in this case ignore the very basics of the sales process and the role social media plays in guiding customers through the funnel.

The only way to reach this flawed conclusion – there is no ROI in social media – is to completely ignore how customers enter and flow through the sales funnel, and focus solely on the point of sale. -”

He goes on to give a graphic explanation of the Sales Funnel and the Metrics for Each Stage of the Funnel.

To be fair, there is now growing agreement on the need for a new approach to ICT industry social media metrics but still a wide rift on how that should be done. However, a couple of recent papers by PR industry gurus suggest that the ICT debate is just the tip of the iceberg on a debate about a PR metric that has been around for 80 years, the  Advertising Value Equivalency (AVE).

An  Oct 28, 2013 impact blog, What do leading PR/ measurement Industry practitioners say about AVEs? (http://www.impactmeasurement.co.in/the-impact-blog/What-do-leading-PR-measurement-Industry-practitioners-say-about-AVEs-.html), notes that “Globally, we have seen an increasing number of clients and PR practitioners reject AVEs as a measure for PR effectiveness. I am sharing reference articles below. These are by reputed institutes and practitioners in the field of PR measurement. You may like to read these:”

Quotes from the reference articles include:

A new paper by Professor Tom Watson presents the history of Advertising Value Equivalency, the discredited-but-still-common public relations measurement technique that the PR world is now in the process of rejecting. Read a summary of the paper, “Advertising Value Equivalence – PR’s orphan metric”( http://kdpaine.blogs.com/themeasurementstandard/aves/)

The main reason that AVEs have still not gone away from our industry is down to a lack of education.  In the spirit of playing my part to counter this, here is a list of reasons on why you should ignore this spurious and meaningless metric… Sixteen reasons AVEs do not measure the value of PR” (http://www.gorkana.com/measurement-matters/measurement-matters/pr-measurement-media-evaluation/aves-dont-measure-pr-heres-why/)

What is surprising is that there is still debate on AVEs despite the Barcelona Declaration of Research Principles Barcelona Principles (Barcelona Principles), a set of seven voluntary guidelines established by the public relations (PR) industry to measure the efficacy of PR campaigns. The Barcelona Principles were agreed upon by PR practitioners from 33 countries who met in Barcelona, Spain in 2010 for a summit convened by the International Association for Measurement and Evaluation of Communication (AMEC). The Barcelona Principles identify the need for outcome, instead of output, based measurement of PR campaigns, calls for the exclusion of ad value equivalency metrics, and recognizes the communications value of social media. (http://en.wikipedia.org/wiki/Barcelona_Principles)

Some insights on the cultural natur of resistance to the Barcelona Principles are given in a  Nov 05, 2013 AMEC post Can Barcelona Principles be implemented in India? – a measurement practitioner’s perspective (http://amecorg.com/2013/11/can-barcelona-principles-be-implemented-in-india/). Here Aseem Sood, CEO of AMEC member company, Impact Research, talks of his experience in countering objections to the Barcelona Principles. He gives three reasons why the Barcelona Principles can be implemented in India, citing the questions raised. These reasons could equally apply to Australia or any other country. The following is an extract:

1. Because they (Barcelona Principles) were created by some trade body outside India and they may not understand Indian environment. This is a fair point as long as you do not know that PR and measurement practitioners from over 30 countries participated in the creation of Barcelona principles. India was also represented in this mix via Impact Research & Measurement. I was personally present there and I can promise you that the challenges we face in India are not very different from the ones faced by people in the other 30 countries…

2. Because they refute using advertising value equivalents (AVEs) or false multipliers but thats what we use and our management team only understands $ metrics. In the past, it was a common practice to use AVEs that equate the value of public relations with the cost of advertising and that earned media is always at least thrice as valuable as paid media. But, we all know that its not true. But some of us are still using the AVEs. At the Barcelona summit, when Barcelona Principles were being debated, there was near-total agreement on this principle (92%) amongst the practitioners present there….

(On the question of what are the right metric to use, the author suggests the  AMEC’s Valid Metrics Framework.)

3. Because they don’t tell me how to measure PR. They don’t de?ne any metrics or detail the technique to be used for quantitative or qualitative measurement. You are right. Barcelona Principles are not meant to be a guide to formulas or metrics or methodology for output measurement. They are bigger than that. They are standards which attempt to bring all practitioners and practice of measurement on a common platform. They create a base that the industry can build-on further. In fact, they are very, very simple and basic (and that’s also another criticism that I have heard) . But following them requires clarity of thought, knowledge of fundamentals and passion for PR.

From an ICT metrics perspective, it looks as if we could resolve some of our disputes, e.g. on ROI,  by looking at what our PR colleagues have been doing and in particular at the Barcelona Agreement.  However, a May 2013 research paperby Professor Jim Macnamara outlines a new approach which he has used successfully on a recent high profile Government engagement, see  ‘Toe bone to the head bone’ logic model in this issue.