Posts Tagged ‘measurement’

Internal Communications at its Best

Tuesday, July 6th, 2010

The UK’s Liam FitzPatrick wrote a post decrying the tendency of internal comms people complaining about manager communication incompetence.  FitzPatrick says: “I believe we get the internal clients we deserve.  If senior managers are used to a diet of crap communications support, that is all they’ll ever understand.”

He’s right, and he’s wrong.

The challenge always is whether to keep fighting or just give managers what they want.  FitzPatrick relates a story about a senior manager who wants “intelligence” about what employees are saying and thinking from her internal comms support.  There are a lot of things a skilled internal communicator can do to gather that intelligence, but much of the budgetary process is more output-focused than outcome-focused (echoing the same tendency elsewhere in corporate communications.)

The key for any of us is research (he said self-servingly — my practice includes research services, just sayin;.)

The research doesn’t even have to be quantitative, though tying qualitative assessment to intranet traffic, for example, can shed a lot of light on the effectiveness of our internal comms activities. We don’t have to do formal surveys, which can be very expensive and time consuming, if all we’re looking for is a snapshot to share for planning and strategy.

At Goodyear, we used an intranet poll to get just that sort of intelligence — it was a great window into what at least some employees were thinking, and it gave us a source of content, too.

But, there is no replacement for more formal measurement — even with qualification of our poll results, we still got management questions about the reach of opinion, which is a valid criticism. The old ROPE method (Research, Objective, Programming, Evaluation) still holds truth.

Meanwhile, read FitzPatrick’s piece. It’s worth reading (and commenting — no comments on his blog, so I wrote this post!)

  • Share/Bookmark

Another IABC International Conference…

Wednesday, June 16th, 2010

I recognize that if I’m not a speaker at the big IABC soiree, I’m probably not the target audience for it. I’m not surprised, therefore, that my first blush reaction to the Toronto gathering wasn’t particularly positive.  My goal for attending this year was to meet some new people and make contact with some who I haven’t seen in a while. I hope to eventually get some business from it, but really just need to expand the network.

The programming and format are nearly identical to my first International, in 1995, also in Toronto. That one was a revelation — I was just 4 years or so into the profession, and everything was new.  Every session offered fascinating insights or enhanced skills.  I met scores of people and hung out with many, enjoying my first trip to Toronto and my first extended business trip in several years.

In 1997, L.A. was a different experience. Many of the speakers were the same as two years earlier, and in 2002 at Chicago, there were just a few sessions that really caught my eye. So I took a vacation from the big show until this year.

Things that impressed me:

Erin Dick from Pratt & Whitney — a social media case study that wasn’t from a Silicon Valley firm… Her use of blogs, Twitter, YouTube and Flickr to help support P&W’s client (the U.S.Government) on the selection of an engine for the Joint Strike Force fighter was off the charts — brilliant. And it had a fairly strong measurement component. I decided to Tweet the session instead of trying to take notes. The benefit was that I had a great summary, though my thumbs threatened to lock up from BlackBerry-itis…

William Amurgis from American Electric Power — Looking for use of social media in internal communications? Amurgis delivered. AEP’s blogs, discussion boards, employee-uploaded photos, etc., set a high standard of participation. The company’s intranet philosophy? Enhance employee productivity, reinforce corporate messages and provide a place to meet for all employees. Everything has to pass through that frame, or it doesn’t happen. And, rather than buy software solutions, AEP makes their own. Amurgis has a designer and a developer on his staff.

The UnConference — OK, it was a bit different than other UnConferences (usually low-or-no-cost, open to anyone; you had to buy the day (at least) for the IABC Conference to get in, and it wasn’t cheap) — but the method of operation was different and fun. There was no pre-set program, just a list of ideas posted on the TorontoTalks website (that a few people did discuss first), and three 5-minute “keynotes” — very informally delivered.  The three-hour session on Sunday afternoon was comprised of four 25-minute blocks of time with six possible topics (being held at six tables). We wrote on sticky notes our question or suggested topic, then stuck it on a flip chart in an empty time slot. The writer could lead the discussion, or someone else could.  I talked measurement (what a shock!) with seven other folks and it was fascinating. We didn’t solve the ROI question in full, nor did we get into other facets of communication, but it still was valuable and fun.

The thing is, the (nice) venue, formal structure and overwhelming size of the show made it hard to connect with people. Even the formal networking session (the big one held on the floor of the exhibit show) was just an hour long — not near enough time to connect. (I also didn’t attend Monday’s sessions — none particularly grabbed me. That might have inhibited my networking activities, so shame on me!)

The cost was pretty high for a new entrepreneur, not only in travel but in the conference fee. I’ll be considering very carefully before jumping on again soon. But, if I wind up as a speaker…

{FYI, I’m speaking in November at IABC’s Research and Measurement Conference in Seattle, as well as at the PRSA National conference in DC in October.  I’m also willing to come to chapter lunches, etc., and can make a deal for my PRSA/IABC fellow members!}

  • Share/Bookmark

Crisis Analysis, SocMed Use, Get Globe/Mail Attention

Friday, June 11th, 2010

Canada’s outstanding The Globe and Mail has two stories today worth noting.  Vancouver, B.C., retailer Lululemon is using Twitter to gather intel from its customers about what sizes and colors to stock; British Petroleum gets second-guessed in its crisis communication strategy under the headline, “Lessons in Leadership Spill from BP.”

BP’s feckless communication strategy, especially demonstrated by company CEO Tony Hayward’s frequent gaffes when speaking off the cuff, deserves to be pilloried. Hayward and company were obviously led by lawyers in this regard, minimizing the potential impact of the disastrous gusher, appearing too rarely in public and pointing blame to subcontractors. Hayward’s “I’d like my life back” rang especially tone-deaf in the wake of 11 deaths and the potential for catastrophic wildlife impact (not to mention the economic peril for the gulf fishing industry.) Several communication experts get quoted in Wallace Immen’s excellent piece, including Michael Stern (Michael Stern Associates), Prof. Julian Barling (Queen’s University School of Business), and Guy Beaudin, (RHR International).

Lululemon sells athletic ware, and by all accounts does a bang-up job of it. Some of the success, according to CEO Christine Day, is due to its use of social media — Twitter and Facebook.  Reporter Marina Strauss quotes Day: “We learn more about [which items are in demand] on Facebook and social media: what are the guests really screaming for, and so we use [the feedback] to get a little bit more indication.”

Keeping an eye on its 127,000 Facebook fans and 32,000 Twitter followers gets Day and company a faster view than its store performance metrics (and offers perspectives from people who are just thinking about going to the store, rather than having bought something there — that’s an interesting view on potential demand, the pipeline, some call it.)

The social media use has two purposes, according to the article — to gather information, and to drive traffic to the company website. When we’re looking for ways to measure the effectiveness of social media, website traffic is more often cited than the research value, which is a pity.  Going back to the ROPE method of communication planning (Research, Objectives, Programming, Evaluation), you don’t have anything without the research.

If social media served no other purpose than market intelligence, it’d still be worth the investment, no?

{P.s., my Canadian sojourn is nearly complete – back to a more regular schedule next week.)

  • Share/Bookmark

Measurement Crucial to PR’s Business Value

Tuesday, May 18th, 2010

My learned Australian colleague Geoff Barbaro waxes rant in a post from 17 May (US time), where he inveighs against measurement.  Perhaps not the concept, as much as the practice. He asks:

Do you measure how you look after your family? Do you count the meals, the trips to school, the time spent with children to evaluate effectiveness? When you buy that great new dress or suit that you love, did you then sit down and work through complex metrics to measure what you did?

So why do you think it’s different in business? I’ll tell you why, it’s because you don’t trust people to do the job you employed them to do. You don’t believe they are motivated and care about their work, so you can only make sure they are working by measuring what they do, and then argue that this is the motivational tool. Measuring because “we do what we measure” is a failure of leadership, a failure of motivation, a failure of selection, a failure to define values, a failure of engagement and a failure of communication.

Sorry, Geoff, but this is fuzzy-headed thinking about a vital enhancement to the profession of Public Relations.

I started a comment on Geoff’s blog (a fine and interesting read, btw), but found that it was all too likely that I’d hijack it. And that’s not right. So, here is my reply to Geoff’s shot across the bow. Man the torpedos!

========================

Oh, my. Nothing like an existential rant to get one’s blood up, eh Geoff?

Let’s start by differentiating terms. Measurement isn’t gotcha. It’s not “check-up-on-the-poor-employees.” Neither is it merely about outputs or activities, at least not when it’s strategic.

We in PR have long been the only department in a firm that can say to the C-suite, “trust me” and get away with it. The question on the CEO (and CFO, especially) mind these days, however, is, “What business value do I get for my investment in PR?”

We can take a SWAG (stupid, wild-assed guess) at the answer, but then we sound like witless weasels (um, we build reputation and protect…uh, no, uh, we get media coverage…no, uh, we help the organization communicate effectively, wait, ummmm.)

The fact is that most of us don’t have a clue what the quantifiable business value of PR is, and that’s why PRSA has commissioned a task force to work on that very question. It’s also one of the driving forces in modern PR. It’s created an industry specialty that people are finding value in, even though there is much sophistry and bad measurement out there.

In modern business, every department must contribute to the bottom line. So, direct sales and the support for sales is a winner, as is direct effort to improve efficiency, save money, etc. There’s also credible research about the effect on brand awareness, attitude and disposition of various PR activity. On the internal side, engagement metrics, and employee knowledge and behavioral metrics lend credence to a communicator’s value.

The trick is to a) Measure what matters; and b) Link communication outputs to business outcomes. This is, indeed, a hairy process, filled with risks — bad math the most prevalent, if you ask me.  Correlation is not causation, but frequently it’s a pretty good stand-in for it, if your math is good.  We mustn’t give up on the goal of establishing impact metrics and ROI just because it’s so much easier if we don’t!

I don’t know, Geoff, if I agree that “what gets measured gets done,” but I’m sure that if you can’t measure it you can’t manage it.

Cheers,

Sean

@commammo

  • Share/Bookmark

Theater of the Absurd in Social Media Metrics

Tuesday, May 11th, 2010

As we PR people feel our way along in social media, the marketers are declaring the End of Times for everything else. Anecdotal evidence shows that big companies are pulling big money out of traditional advertising and funneling it into social media, and that bears examination.  But as I’ve said, I’m not ready to write obits for mass marketing/advertising in favor of “marketing to a segment of one” right this very minute.

I first heard that phrase (Marketing to a segment of one) from the lips of Steve Cone, legendary marketer and then-CMO with KeyCorp. He was the architect of dropping the “Corp” and/or “Bank” from the company name in favor of the symbol you see at right.

That made Key one of just three companies in the US bearing an eponymous symbol for its name. Shell and Apple are the other two.

Key made a strategy of getting people to see the Key logo and associate it with “bank,” as in, “I need to stop by the Key on the way home.”  The idea, Cone claimed, was to stop thinking of mass marketing — with all of its efficiency and logical, numbers-driven strategy, and think of “marketing to segments, eventually to a segment of one.” So then came emerging affluents, wealth management, small business, middle market, large corporate — all of those categories based on grouping customers in some logical way, then changing strategy to target them.

This requires information about customers and prospects. When it comes to social media, that information is scattered to the four winds, unless you’re on Facebook.  Twitter’s foray into geo-location, Foursquare, and many other social media firms are trying to gather as much data about YOU as possible to facilitate what is a pretty old marketing model.

Just as at the onset of the Web Age you had hundreds of companies popping up to “help” companies enter the Internet realm, now at the onset of the Social Media age you have companies popping up to “help” companies enter this realm. The part that twists my noodle is when companies purport to know how to measure social media come up with yowlers — like the Vitrue Facebook fan value imbroglio, the Altimeter study on correlations between social media activity and stock appreciation, and now Vitrue’s assertion that frequency of mention in social media is somehow a reflection of its social media reputation.

Vitrue offers a chance to compare brands in a handy Flash gobo that produces a cool pie chart. Just for fun, I compared Ford (which Vitrue pronounces its winner) with a couple of random words — sure enough, pop “the” in there, and you find upteen thousands (OK, 134,000) ‘somethings’ and the aforementioned cool pie chart. Ooh, and there’s a bar chart too! So kewl.  W00t!

I could go on for 1,500 words, but won’t. It’s another cow pie pretending to be a metric.  Resist this assault on rational thinking.

  • Share/Bookmark

Why Vitrue’s Facebook Fan Value is Poppycock

Wednesday, April 14th, 2010

Vitrue, a social media marketing firm founded in 2006, snagged an AdWeek article this week when it announced that it had caculated the value of Facebook fans. It’s $3.60 per fan.  What’s behind the valuation? A rash of assumptions, according to a piece on the company’s Web site.

Why do I think this is wrong? Let me count the ways:

  1. Their data is proprietary. The company says it manages 45 million fans and drew the data from a sampling across industries, but they don’t specify the amount of the sample, the specific firms involved or any other information that might provide clarity as to the methodology. No one can cross-check the data.
  2. They make several assumptions: They say they looked at the ratio between wall posts and number of fans — asking how many fans have the potential to see a post. This is similar to using circulation in a print pub. Fine. But unlike circulation (audited) or even Nielsen Ratings, we’re assuming that all fans have an equal opportunity to see, and we’re assuming that a wall post is equivalent to an ad. Then, they say that multiple posts have equivalent impressions — two per day totals 60 million impressions on a one million fan page.
  3. They then say that these impressions are free, “similar to earned media.” But we know earned media is not free — someone had to do some work to make it happen. This is one of the insidious problems with ad value equivalency — there certainly are costs associated with generating earned media, and they must be accounted for.
  4. Next assumption, cost per thousand impressions. They settle on $5 CPM, based on nothing — wouldn’t this number depend on the specific outlet?  How about some science instead of conjecture? Multiply it out using their figures and it totals $300,000 in monthly value for the two post-a-day million fan page.  They show it like this:

1M impressions x 2 posts x 30 days = 60M impressions >>> 60M impressions / 1000 x $5 CPM = $300,000

But what I believe is most egregious is the idea that engagement on Facebook is really just a game of increasing advertising impressions. This is totally contrary to how social media is designed to work. It’s push-focused instead of relationship-focused. It’s shouting from the rooftops instead of talking to your neighbors.

Look, everyone has to make a living — advertisers are pretty comfortable in their “metrics every marketer is familiar with,” as Vitrue’s article says.  But marketers need to wake up — measure something meaningful!  I don’t know, but perhaps the fans are actually doing something that increases their intent to purchase? That improves their understanding of the product? That makes them have a more favorable attitude toward the company? That they bought something?

Surely any of those is a better metric than one based on made-up numbers, bad methodology, weak assumptions and false equivalencies.

Harrumph.

  • Share/Bookmark

Survey: Internal Comm Effectiveness ‘Important Concern,’ But…

Thursday, April 1st, 2010

Researchers Dr. Juan Meng of the University of Dayton (Ohio) and Dr. Bruce K. Berger of the University of Alabama cut to the chase in their research presentation at the Institute for PR International PR Research Conference. Their first finding? “Though communication effectiveness has been an important concern for organizational leaders, the assessment of communication effectiveness has not been widely applied by using business outcome metrics in organizations.” Sigh.

Meng and Berger used both the results from the 2007-2008 IABC Research Foundation/Watson Wyatt international survey of senior communicators, and a series of in-depth interviews with 13 IABC Gold Quill winners to look for process links between internal communication effectiveness and organizational financial performance.

For me, this represents a sort of Holy Grail: we internal comms experts know that our work is impactful, but have lacked the hard evidence of causality that we perceive the C-suite respects and demands. I was disappointed, yet again, though that first finding is by no means the only one.  In brief, the other five are:

  • Measuring internal comm effectiveness should be standard operating practice.
  • There’s lots of measurement going on, evaluating awareness/understanding; engagement; job performance; employee behavior, and improvement in overall business performance.
  • Everyone has good reasons why measurement isn’t as robust as it should be, and they’re the usual culprits — lack of time/money/staff and the pain of finding actual cause-and-effect toward business results.
  • The measurement approaches used are employee surveys, employee participation in communication activities and manager surveys.
  • Four valuable purposes for internal communication: Explaining/Promoting programs and policies; educating about culture and values; providing information about performance and financial objectives, and helping employees understand the business.

At Goodyear, we made great progress toward true outcome measurement for internal communications, but didn’t quite get there. We did establish a strong link between employee knowledge/comprehension, intranet use and managerial behavior, but never got the chance to take everything to the organizational performance level.

At National City Corporation (the regional bank), our focus from the first day I arrived was on external measurement, for a variety of reasons. But the internal side wasn’t ignored — we were a Gallup Q12 company, and despite the wretched economic conditions and horrific, calamitous financial performance of the company, we still topped 94% participation in the Q12.  Right until the last moment, we were using Q12 results in our planning process, as well as beginning to use editorial content more strategically. But, again, we weren’t reaching the business outcomes level of measurement.

Here’s a quote from one of Meng & Berger’s in-depth interviews:

I think the biggest challenge in measurement continues to be convincing clients to spend, not so much the money, but to spend the time. As the industry develops, I don’t have a hard time in convincing them about the validity of measurement, but they are reluctant to actually take the time away from business to actually administer surveys or focus groups or some other measurement tools.

Looks like we have to continue making those tools easier to use and more valuable, even as we continue to scale the mountain tops for the Holy Grail.

  • Share/Bookmark

Hooked on PR Research

Monday, March 22nd, 2010

One of the great professional pleasures of my life involves an academic conference filled to the brim with fascinating public relations research. It’s the International PR Research Conference put on by the Institute for Public Relations, and I’ve attended four of the past five years. That it’s held early in March in Miami, Fla., has NOTHING to do with it!

OK, well, it has a little bit to do with it.  The tropical breezes feel especially fine in the icy wake of February in Cleveland, and there is terrific food, shopping, pleasant walks and an excellent pool. But, other than that, it’s all business for three days.

I’ve had the good fortune to present at IPRRC twice; the first time, 2008, I presented a paper with my research pal Dr. Julie O’Neil of Texas Christian University that covered one large company’s internal communication program, focusing especially on the measurement of the work. It won an award, of which I am very proud indeed – the Jackson-Sharpe Award for research by an academic and a practitioner (I’m not the academic, or wasn’t…).

This year, I presented a work in progress, an exploratory study of corporate blogs and Twitter activities, with an eye on whether they’re demonstrating James Grunig’s Excellence Theory – are they conversations? – or other PR theories.

The idea is to see what actually IS in this space for 18 companies – the work is ongoing (frantically; the final paper is due May 1), and I was able to share a few key findings.

  1. There’s a lot of using social media as a broadcasting tool – no two-way, no evidence of symmetry (mutual change) – and persuasion, therefore, still rules.
  2. There are a couple of firms that are doing yeoman’s work and engaging in conversations – there are also seven or eight companies who’ve abandoned their blogs since December 2009.
  3. One industrial giant, interestingly, has subject matter experts blog and then engage engineers and customers in a discussion about improving the product – this is a rarity.
  4. Twitter as link-bait is quite in evidence.
  5. The Cluetrain may have left the station, but it’s creeping along a siding, not hurtling on a MagLev track.

This is hardly conclusive or particularly scientific – that’s why we call the paper exploratory. Dr. O’Neil and I have more work to do this coming month, but this paper is intended to be the first of three. Next step is a qualitative discussion with some of the people behind social media at our subject companies, followed (we hope) by a quantitative survey of users of corporate blogs and their associated Twitterverse (we’ll see; that’s going to take some cash…).

In the meanwhile, stay tuned over the next few days as I recount some of the work that impressed me the most at IPRRC this year.  Once our paper is done, we’ll share.

  • Share/Bookmark

Communicators too often out the door in hard times

Monday, March 8th, 2010

I’ve been a regular reader of the New York Times’ Corner Office series, where they interview a senior executive, usually a CEO, about how they hire, how they lead and manage and what they do.  There is a central theme that runs through literally all of these interviews: communication excellence.

Every exec talks about how important communication is to their work — both by them and with them. Of course, there is a bit of the “usual” about that kind of statement. No one would ever say, “communication is not important at all in my work.”  But what I find striking about this observation is that ostensibly, we public relations people are the experts in communication, but get too easily dismissed from the leadership table.  We are welcome during times of crisis (the one time, it seems, that the mahogany row types understand our value) but aren’t regularly consulted about the communication dynamics of decisions.

Additionally, financial pressures often lead to cuts in the communication staff that could help the company deal with the financial issues, particularly internal communication.  It’s a fact that internal communication pros earn less than their media relations colleagues and tend to be lightly regarded. I’ve encountered this in my own career. I feel reasonably comfortable saying that generally, businesses don’t care about internal communication very much at all. They care about “getting the word out” internally, or “making sure we’re all rowing in the same direction.”  But honest improvement in the process of communication within the enterprise? Not so much.

Why?

Some of the issue emerges from a distinct lack of communication curriculum in colleges of business. Marketing, management theory, finance and operations, but no communication theory that could help leaders understand their workers (and customers) better.  More of the blame goes to convenient financial thinking. It’s easier to impose across-the-board cuts than dig into an income statement and excise the real waste, duplication and nonessentials.

One company had a largely decentralized communication structure that permitted significant duplication in communication infrastructure. Some such duplication is inevitable in a multi-national company, but why have 40-something different intranets, run on multifarious platforms? Why not unify newsletter design under a singular brand?  This company really has no idea what it spends on communication, because the units are autonomous — the financials are opaque.

Canada’s Bombardier (a few years ago) published a paper newsletter that permitted regions to wrap their own around the corporate version. All the design elements reinforced the brand. You knew it was a Bombardier newsletter whether you were in Montreal or Dublin, Ireland. That kind of consistency is economical.

How about automating manual processes on the intranet? Sounds like it should be an easy sell, but companies look at the up-front costs and decide to forget it.

I fault us.  As the putative experts, we should have a deep understanding of how moving communications levers will create value for the business.

But too many of us don’t have a clue, and that means, when the going gets tough, we’re the ones who get told to get going…out the door.

  • Share/Bookmark

AVE is Dead. But Ad Cost Improves Correlations

Sunday, January 17th, 2010

The debate over how best to measure the effectiveness of media relations has encompassed multiple streams of thought, moving from saying “it’s impossible,” all the way to saying, “it’s quantifiable.” Unfortunately, advertising value equivalency (AVE) became a popular means of applying dollar figures to unpaid media. You take the number of column inches in print, time of mention in broadcast, or space on a Web page occupied by the mention of the company or organization in question, and ask, “How much would we have had to pay to take out an ad of equivalent size/time?”

The AVE practice has been under attack by some of us, poorly understood by others, but more widely used in PR agencies than many would like to think. It even was formally condemned by the Institute for PR Measurement Commission this fall.

AVE has major flaws — measurement experts (including one notable, even famous one) have decried the practice and detailed why frequently. I’ll not repeat the argument here. This paper provides those details in part. Instead, I’ll merely say that even with substantial adjustments to methodology, it never represented a business outcome, was based on an assumption of equivalent understanding on the part of the receiver, and was wholly unsuited to describing success in social media. That alone was a huge problem for me.

The thing is, there is substantive research that supports the idea that editorial content about a product and an ad are perceived similarly by receivers.  A paper by Dr. Don Stacks and Dr. David Michaelson (albeit based on one experiment) found ads and editorial to be equally effective in generating interest in a new product. If that’s so, evaluating the PR placement in comparison to ad cost makes sense. PR costs orders of magnitude less than advertising.

Two papers by Angela Jeffrey, Dr. Stacks and Dr. Michaelson explored the linkages between volume of media coverage and share of media coverage and business outcomes (such as unit sales, tickets sold, etc.) and included media cost data in calculations.  This set the stage for a controversial finding: Media costs improved correlations, significantly.

Now, Jeffrey, vice president of research for VMS, and Dr. Brad Rawlins, Brigham Young University, and Bruce Jeffries-Fox of Jeffries-Fox Associates, have written a brilliant paper further detailing the relationship between cost and outcomes, with four case studies.  The “Weighted Media Cost” has a strong effect.  From the paper:

…if we’re getting better results with costs for purchasing media space and time data, should we…set new parameters for its proper use?”

Emphatically, yes. The paper, written in a very approachable and intuitive style, makes a compelling case.

Read the paper if you care at all about measurement in our profession.

  • Share/Bookmark