The debate over how best to measure the effectiveness of media relations has encompassed multiple streams of thought, moving from saying “it’s impossible,” all the way to saying, “it’s quantifiable.” Unfortunately, advertising value equivalency (AVE) became a popular means of applying dollar figures to unpaid media. You take the number of column inches in print, time of mention in broadcast, or space on a Web page occupied by the mention of the company or organization in question, and ask, “How much would we have had to pay to take out an ad of equivalent size/time?”
The AVE practice has been under attack by some of us, poorly understood by others, but more widely used in PR agencies than many would like to think. It even was formally condemned by the Institute for PR Measurement Commission this fall.
AVE has major flaws — measurement experts (including one notable, even famous one) have decried the practice and detailed why frequently. I’ll not repeat the argument here. This paper provides those details in part. Instead, I’ll merely say that even with substantial adjustments to methodology, it never represented a business outcome, was based on an assumption of equivalent understanding on the part of the receiver, and was wholly unsuited to describing success in social media. That alone was a huge problem for me.
The thing is, there is substantive research that supports the idea that editorial content about a product and an ad are perceived similarly by receivers. A paper by Dr. Don Stacks and Dr. David Michaelson (albeit based on one experiment) found ads and editorial to be equally effective in generating interest in a new product. If that’s so, evaluating the PR placement in comparison to ad cost makes sense. PR costs orders of magnitude less than advertising.
Two papers by Angela Jeffrey, Dr. Stacks and Dr. Michaelson explored the linkages between volume of media coverage and share of media coverage and business outcomes (such as unit sales, tickets sold, etc.) and included media cost data in calculations. This set the stage for a controversial finding: Media costs improved correlations, significantly.
Now, Jeffrey, vice president of research for VMS, and Dr. Brad Rawlins, Brigham Young University, and Bruce Jeffries-Fox of Jeffries-Fox Associates, have written a brilliant paper further detailing the relationship between cost and outcomes, with four case studies. The “Weighted Media Cost” has a strong effect. From the paper:
…if we’re getting better results with costs for purchasing media space and time data, should we…set new parameters for its proper use?”
Emphatically, yes. The paper, written in a very approachable and intuitive style, makes a compelling case.
Read the paper if you care at all about measurement in our profession.
Tags: communication, effective communication, evaluation, measurement, Media Relations, PR measurement, reputation management, Research, Social Media
Sean – this is a terrific summary of what we’ve written, and I can’t thank you enough. If I never hear the term AVE again, it’ll be too soon. But Bruce Jeffries-Fox’s quote about not throwing the baby out with the bathwater is right on the money. The data is useful, but must be used in an entirely new manner – as only an index score against which to measure outcomes. In that sense, it performs extremely well. Anyway, thanks for the kudos!
Angie, thanks for stopping by. I realize the subject is controversial, but your data really speaks for itself. Congratulations again to you, Bruce and Brad.
[...] This post was mentioned on Twitter by Sean Williams, Sean Williams. Sean Williams said: With AVEs now officially condemned, does ad cost have anything to add to the #measurement discussion. Yes. http://bit.ly/70Oxmc [...]
Social comments and analytics for this post…
This post was mentioned on Twitter by CommAMMO: With AVEs now officially condemned, does ad cost have anything to add to the #measurement discussion. Yes. http://bit.ly/70Oxmc…
Sean -
Compellingly written, as always. And by way of full disclosure, I am on excellent terms with the “notable, even famous” expert you refer to (at least, whom I think you refer to), familiar with most of the IPR brigade, and extremely fond of Angie; and while I don’t know Brad well, respect him immensely.
Here’s my problem with “weighted media cost,” and granted, I’m nowhere near as brilliant as Angie, Katie and the rest of the IPR brigade.
First, it is written in SO much jargon it is really difficult for “common folk” to understand. So do us all a favor & dumb it down, wontcha?
Second, it still seems to me that, at the end of the day, WMC is still about measuring outputs, and not outcomes.
We all know that outcomes are what we should be correlating our communication work to. So WTF are the leaders in PR measurement doing still focusing on outcomes, when plebians like myself, the HSUS, Scott Henderson et al have shown time and time again that we can actually correlate our work to organizational KPIs and come out on top?
I care a great deal about smart measurement, as you know, but it’s going to take a lot more to convince me WMC is the way to go when the work I’ve done in the past has won an award (in conjunction with Katie) from said IPR for excellence in measurement, that focuses on correlations to outcomes, not just outputs.
Please, Angie, prove me wrong!
I’d be interested to know, firstly, if there has been research done on whether media consumers are aware of how much media has been generated by the efforts of PR folk and, secondly, whether this knowledge has influenced the credibility/influence of media.
Is media editorial the new advertising?
[...] This post was mentioned on Twitter by Craig Pearce, Craig Pearce. Craig Pearce said: AVE for media placement taken to task and valuable discussion @CommAMMO: http://bit.ly/5Iw4Rx [...]
Let me try my hand at a simplified summary of the paper, per Shonali’s request. The paper is providing the argument that if you want to show the correlation between your output and your outcomes (the cases used in the paper correlate output with outcomes, similar to the paper written about tying PR efforts to business outcomes written by Jeffrey, Paine, and Draper), including a weighted media cost increases the correlation.
It’s really a simple idea. Advertisers are willing to pay more for their space and time in media that reach more people and are more influential. When earned media messages appear in the same space, they are more likely to reach more people through more influential channels. Therefore, these messages should be more effective than if published in media that reaches fewer people and with less influence. So, when this data is included as part of the measurement of the output, it predicts outcomes better than other data such as simple placements or OTS.
The paper isn’t arguing that all output is equally successful for reaching outcome objectives, but including the WMC increases the correlation in all cases we studies.
I hope that clears up some of the misunderstanding.
Brad, Craig and Shonali, appreciate your comments.
Shonali, I think Brad’s comment helps out here in terms of clarity. My view is that the research warrants discussion. It’s true that the AVE question has been asked and answered. As of last fall, definitively, in fact, when the IPR Measurement Commission condemned the practice.
However, as I see it, that was more about equating our activities to advertising and condemning the often sloppy way that AVE were calculated. Off-the-shelf rates, failure to net out expenses — all of that was addressed in the paper.
Unfortunately, any discussion of media cost seems to be a third rail of PR — touch it and you die! I read the paper and appreciated the thinking and scholarship. I am also aware that you can’t always convince people that they can do without the comparison to advertising — it’s a difficult situation.
Brad – thanks so much for offering this very clear, jargon-free summary. I find it very helpful and hope that others will too. Most of all, I hope that everyone will read the paper closely!
Cheers all.
Even if you can convince me that this is a good idea, will someone please tell me how or where the average non-profit PR person, with an under $100K a year budget is going to find the WMC is for their local blogger who has been laid off from the local newspaper, has started his/her own blog and is now the best read outlet in the region? Where is this data going to come from and how much will it cost?
Katie, thanks so much for weighing in. Your questions point out the macro problem with PR measurement in general during a time of continuing and dramatic change. We will need different tools to measure success.
But we are not there yet — media is not yet dead. For your straw horse non-profit PR person, WMC is irrelevant. For organizations which are still seeing MSM as an important channel, it appears from this research that it IS relevant. This is not a zero sum game, and I’m not sure it ever will be.
I’ve got no skin in this game at all — I like to to advance the cause on broad channels.
Thanks again for commenting here. I feel honored.