He’s right, and he’s wrong.

The challenge always is whether to keep fighting or just give managers what they want.  FitzPatrick relates a story about a senior manager who wants “intelligence” about what employees are saying and thinking from her internal comms support.  There are a lot of things a skilled internal communicator can do to gather that intelligence, but much of the budgetary process is more output-focused than outcome-focused (echoing the same tendency elsewhere in corporate communications.)

The key for any of us is research (he said self-servingly — my practice includes research services, just sayin;.)

The research doesn’t even have to be quantitative, though tying qualitative assessment to intranet traffic, for example, can shed a lot of light on the effectiveness of our internal comms activities. We don’t have to do formal surveys, which can be very expensive and time consuming, if all we’re looking for is a snapshot to share for planning and strategy.

At Goodyear, we used an intranet poll to get just that sort of intelligence — it was a great window into what at least some employees were thinking, and it gave us a source of content, too.

But, there is no replacement for more formal measurement — even with qualification of our poll results, we still got management questions about the reach of opinion, which is a valid criticism. The old ROPE method (Research, Objective, Programming, Evaluation) still holds truth.

Meanwhile, read FitzPatrick’s piece. It’s worth reading (and commenting — no comments on his blog, so I wrote this post!)

The programming and format are nearly identical to my first International, in 1995, also in Toronto. That one was a revelation — I was just 4 years or so into the profession, and everything was new.  Every session offered fascinating insights or enhanced skills.  I met scores of people and hung out with many, enjoying my first trip to Toronto and my first extended business trip in several years.

In 1997, L.A. was a different experience. Many of the speakers were the same as two years earlier, and in 2002 at Chicago, there were just a few sessions that really caught my eye. So I took a vacation from the big show until this year.

Things that impressed me:

Erin Dick from Pratt & Whitney — a social media case study that wasn’t from a Silicon Valley firm… Her use of blogs, Twitter, YouTube and Flickr to help support P&W’s client (the U.S.Government) on the selection of an engine for the Joint Strike Force fighter was off the charts — brilliant. And it had a fairly strong measurement component. I decided to Tweet the session instead of trying to take notes. The benefit was that I had a great summary, though my thumbs threatened to lock up from BlackBerry-itis…

William Amurgis from American Electric Power — Looking for use of social media in internal communications? Amurgis delivered. AEP’s blogs, discussion boards, employee-uploaded photos, etc., set a high standard of participation. The company’s intranet philosophy? Enhance employee productivity, reinforce corporate messages and provide a place to meet for all employees. Everything has to pass through that frame, or it doesn’t happen. And, rather than buy software solutions, AEP makes their own. Amurgis has a designer and a developer on his staff.

The UnConference — OK, it was a bit different than other UnConferences (usually low-or-no-cost, open to anyone; you had to buy the day (at least) for the IABC Conference to get in, and it wasn’t cheap) — but the method of operation was different and fun. There was no pre-set program, just a list of ideas posted on the TorontoTalks website (that a few people did discuss first), and three 5-minute “keynotes” — very informally delivered.  The three-hour session on Sunday afternoon was comprised of four 25-minute blocks of time with six possible topics (being held at six tables). We wrote on sticky notes our question or suggested topic, then stuck it on a flip chart in an empty time slot. The writer could lead the discussion, or someone else could.  I talked measurement (what a shock!) with seven other folks and it was fascinating. We didn’t solve the ROI question in full, nor did we get into other facets of communication, but it still was valuable and fun.

The thing is, the (nice) venue, formal structure and overwhelming size of the show made it hard to connect with people. Even the formal networking session (the big one held on the floor of the exhibit show) was just an hour long — not near enough time to connect. (I also didn’t attend Monday’s sessions — none particularly grabbed me. That might have inhibited my networking activities, so shame on me!)

The cost was pretty high for a new entrepreneur, not only in travel but in the conference fee. I’ll be considering very carefully before jumping on again soon. But, if I wind up as a speaker…

{FYI, I’m speaking in November at IABC’s Research and Measurement Conference in Seattle, as well as at the PRSA National conference in DC in October.  I’m also willing to come to chapter lunches, etc., and can make a deal for my PRSA/IABC fellow members!}

BP’s feckless communication strategy, especially demonstrated by company CEO Tony Hayward’s frequent gaffes when speaking off the cuff, deserves to be pilloried. Hayward and company were obviously led by lawyers in this regard, minimizing the potential impact of the disastrous gusher, appearing too rarely in public and pointing blame to subcontractors. Hayward’s “I’d like my life back” rang especially tone-deaf in the wake of 11 deaths and the potential for catastrophic wildlife impact (not to mention the economic peril for the gulf fishing industry.) Several communication experts get quoted in Wallace Immen’s excellent piece, including Michael Stern (Michael Stern Associates), Prof. Julian Barling (Queen’s University School of Business), and Guy Beaudin, (RHR International).

Lululemon sells athletic ware, and by all accounts does a bang-up job of it. Some of the success, according to CEO Christine Day, is due to its use of social media — Twitter and Facebook.  Reporter Marina Strauss quotes Day: “We learn more about [which items are in demand] on Facebook and social media: what are the guests really screaming for, and so we use [the feedback] to get a little bit more indication.”

Keeping an eye on its 127,000 Facebook fans and 32,000 Twitter followers gets Day and company a faster view than its store performance metrics (and offers perspectives from people who are just thinking about going to the store, rather than having bought something there — that’s an interesting view on potential demand, the pipeline, some call it.)

The social media use has two purposes, according to the article — to gather information, and to drive traffic to the company website. When we’re looking for ways to measure the effectiveness of social media, website traffic is more often cited than the research value, which is a pity.  Going back to the ROPE method of communication planning (Research, Objectives, Programming, Evaluation), you don’t have anything without the research.

If social media served no other purpose than market intelligence, it’d still be worth the investment, no?

{P.s., my Canadian sojourn is nearly complete – back to a more regular schedule next week.)

Do you measure how you look after your family? Do you count the meals, the trips to school, the time spent with children to evaluate effectiveness? When you buy that great new dress or suit that you love, did you then sit down and work through complex metrics to measure what you did?

So why do you think it’s different in business? I’ll tell you why, it’s because you don’t trust people to do the job you employed them to do. You don’t believe they are motivated and care about their work, so you can only make sure they are working by measuring what they do, and then argue that this is the motivational tool. Measuring because “we do what we measure” is a failure of leadership, a failure of motivation, a failure of selection, a failure to define values, a failure of engagement and a failure of communication.

Sorry, Geoff, but this is fuzzy-headed thinking about a vital enhancement to the profession of Public Relations.

I started a comment on Geoff’s blog (a fine and interesting read, btw), but found that it was all too likely that I’d hijack it. And that’s not right. So, here is my reply to Geoff’s shot across the bow. Man the torpedos!

========================

Oh, my. Nothing like an existential rant to get one’s blood up, eh Geoff?

Let’s start by differentiating terms. Measurement isn’t gotcha. It’s not “check-up-on-the-poor-employees.” Neither is it merely about outputs or activities, at least not when it’s strategic.

We in PR have long been the only department in a firm that can say to the C-suite, “trust me” and get away with it. The question on the CEO (and CFO, especially) mind these days, however, is, “What business value do I get for my investment in PR?”

We can take a SWAG (stupid, wild-assed guess) at the answer, but then we sound like witless weasels (um, we build reputation and protect…uh, no, uh, we get media coverage…no, uh, we help the organization communicate effectively, wait, ummmm.)

The fact is that most of us don’t have a clue what the quantifiable business value of PR is, and that’s why PRSA has commissioned a task force to work on that very question. It’s also one of the driving forces in modern PR. It’s created an industry specialty that people are finding value in, even though there is much sophistry and bad measurement out there.

In modern business, every department must contribute to the bottom line. So, direct sales and the support for sales is a winner, as is direct effort to improve efficiency, save money, etc. There’s also credible research about the effect on brand awareness, attitude and disposition of various PR activity. On the internal side, engagement metrics, and employee knowledge and behavioral metrics lend credence to a communicator’s value.

The trick is to a) Measure what matters; and b) Link communication outputs to business outcomes. This is, indeed, a hairy process, filled with risks — bad math the most prevalent, if you ask me.  Correlation is not causation, but frequently it’s a pretty good stand-in for it, if your math is good.  We mustn’t give up on the goal of establishing impact metrics and ROI just because it’s so much easier if we don’t!

I don’t know, Geoff, if I agree that “what gets measured gets done,” but I’m sure that if you can’t measure it you can’t manage it.

Cheers,

Sean

@commammo

I first heard that phrase (Marketing to a segment of one) from the lips of Steve Cone, legendary marketer and then-CMO with KeyCorp. He was the architect of dropping the “Corp” and/or “Bank” from the company name in favor of the symbol you see at right.

That made Key one of just three companies in the US bearing an eponymous symbol for its name. Shell and Apple are the other two.

Key made a strategy of getting people to see the Key logo and associate it with “bank,” as in, “I need to stop by the Key on the way home.”  The idea, Cone claimed, was to stop thinking of mass marketing — with all of its efficiency and logical, numbers-driven strategy, and think of “marketing to segments, eventually to a segment of one.” So then came emerging affluents, wealth management, small business, middle market, large corporate — all of those categories based on grouping customers in some logical way, then changing strategy to target them.

This requires information about customers and prospects. When it comes to social media, that information is scattered to the four winds, unless you’re on Facebook.  Twitter’s foray into geo-location, Foursquare, and many other social media firms are trying to gather as much data about YOU as possible to facilitate what is a pretty old marketing model.

Just as at the onset of the Web Age you had hundreds of companies popping up to “help” companies enter the Internet realm, now at the onset of the Social Media age you have companies popping up to “help” companies enter this realm. The part that twists my noodle is when companies purport to know how to measure social media come up with yowlers — like the Vitrue Facebook fan value imbroglio, the Altimeter study on correlations between social media activity and stock appreciation, and now Vitrue’s assertion that frequency of mention in social media is somehow a reflection of its social media reputation.

Vitrue offers a chance to compare brands in a handy Flash gobo that produces a cool pie chart. Just for fun, I compared Ford (which Vitrue pronounces its winner) with a couple of random words — sure enough, pop “the” in there, and you find upteen thousands (OK, 134,000) ‘somethings’ and the aforementioned cool pie chart. Ooh, and there’s a bar chart too! So kewl.  W00t!

I could go on for 1,500 words, but won’t. It’s another cow pie pretending to be a metric.  Resist this assault on rational thinking.

Meng and Berger used both the results from the 2007-2008 IABC Research Foundation/Watson Wyatt international survey of senior communicators, and a series of in-depth interviews with 13 IABC Gold Quill winners to look for process links between internal communication effectiveness and organizational financial performance.

For me, this represents a sort of Holy Grail: we internal comms experts know that our work is impactful, but have lacked the hard evidence of causality that we perceive the C-suite respects and demands. I was disappointed, yet again, though that first finding is by no means the only one.  In brief, the other five are:

• Measuring internal comm effectiveness should be standard operating practice.
• There’s lots of measurement going on, evaluating awareness/understanding; engagement; job performance; employee behavior, and improvement in overall business performance.
• Everyone has good reasons why measurement isn’t as robust as it should be, and they’re the usual culprits — lack of time/money/staff and the pain of finding actual cause-and-effect toward business results.
• The measurement approaches used are employee surveys, employee participation in communication activities and manager surveys.
• Four valuable purposes for internal communication: Explaining/Promoting programs and policies; educating about culture and values; providing information about performance and financial objectives, and helping employees understand the business.

At Goodyear, we made great progress toward true outcome measurement for internal communications, but didn’t quite get there. We did establish a strong link between employee knowledge/comprehension, intranet use and managerial behavior, but never got the chance to take everything to the organizational performance level.

At National City Corporation (the regional bank), our focus from the first day I arrived was on external measurement, for a variety of reasons. But the internal side wasn’t ignored — we were a Gallup Q12 company, and despite the wretched economic conditions and horrific, calamitous financial performance of the company, we still topped 94% participation in the Q12.  Right until the last moment, we were using Q12 results in our planning process, as well as beginning to use editorial content more strategically. But, again, we weren’t reaching the business outcomes level of measurement.

Here’s a quote from one of Meng & Berger’s in-depth interviews:

I think the biggest challenge in measurement continues to be convincing clients to spend, not so much the money, but to spend the time. As the industry develops, I don’t have a hard time in convincing them about the validity of measurement, but they are reluctant to actually take the time away from business to actually administer surveys or focus groups or some other measurement tools.

Looks like we have to continue making those tools easier to use and more valuable, even as we continue to scale the mountain tops for the Holy Grail.

The Institute for PR Measurement Commission condemned AVE last fall, AMEC (the professional organization for media evaluation firms) has declared its intent to find a logical replacement, and a recent paper offered Weighted Media Cost as an element worthy of inclusion in measurement programming. Where does this leave us?

I have no stake in this game. My personal belief is that AVEs are bad science, but I’m also sensitive to the need to help clients. AVE is easy for a client to grasp — “if we paid for the space our story ran in, it would have cost us X.”  Katie points out that doctors won’t prescribe a medicine if it’s not right for the patient. AVE isn’t life and death — but what do we do after we’ve explained the drawbacks and negatives and the client still wants it?

I can’t help but put myself in that situation — young company, trying to latch on with a client. Do I tell the client “No. I won’t do AVE” and risk having him/her say, “Well then, I’ll go find someone who will!” ?

#MeasurePR had much more great content than this AVE nonsense, and I really do wish we could collectively move on. I’m done writing about the debate, at least for now.

Looking for a quick way to improve measurement?

Start setting objectives and measuring your attainment of them. Stop worrying about generating lots of eyeballs and do some audience research to reach the right ones. Start looking for correlations between your various communication outputs (and outtakes) and business metrics, such as revenue, cost savings, cost avoidance, time saved, help desk traffic, speed of benefits enrollment, travel system savings, expense systems savings, etc…

The AVE practice has been under attack by some of us, poorly understood by others, but more widely used in PR agencies than many would like to think. It even was formally condemned by the Institute for PR Measurement Commission this fall.

AVE has major flaws — measurement experts (including one notable, even famous one) have decried the practice and detailed why frequently. I’ll not repeat the argument here. This paper provides those details in part. Instead, I’ll merely say that even with substantial adjustments to methodology, it never represented a business outcome, was based on an assumption of equivalent understanding on the part of the receiver, and was wholly unsuited to describing success in social media. That alone was a huge problem for me.

The thing is, there is substantive research that supports the idea that editorial content about a product and an ad are perceived similarly by receivers.  A paper by Dr. Don Stacks and Dr. David Michaelson (albeit based on one experiment) found ads and editorial to be equally effective in generating interest in a new product. If that’s so, evaluating the PR placement in comparison to ad cost makes sense. PR costs orders of magnitude less than advertising.

Two papers by Angela Jeffrey, Dr. Stacks and Dr. Michaelson explored the linkages between volume of media coverage and share of media coverage and business outcomes (such as unit sales, tickets sold, etc.) and included media cost data in calculations.  This set the stage for a controversial finding: Media costs improved correlations, significantly.

Now, Jeffrey, vice president of research for VMS, and Dr. Brad Rawlins, Brigham Young University, and Bruce Jeffries-Fox of Jeffries-Fox Associates, have written a brilliant paper further detailing the relationship between cost and outcomes, with four case studies.  The “Weighted Media Cost” has a strong effect.  From the paper:

…if we’re getting better results with costs for purchasing media space and time data, should we…set new parameters for its proper use?”

Emphatically, yes. The paper, written in a very approachable and intuitive style, makes a compelling case.

Read the paper if you care at all about measurement in our profession.

I’m grateful that Amanda included me in her list of “critical thinkers” along with Kent State prof Bill Sledzik, Ike Pigott, Joel Postman and Mark; that’s high praise from an important voice.

Look, I’m a committed capitalist, so I don’t begrudge anyone from making money, in particular, people who are early adopters and make the personal investment needed to stay just ahead of the crest of a wave. A bunch of people have done so, and are making a terrific living at it.

Some of those people don’t have anything but an expertise at sales and a gift for jargon to qualify them, and that’s a big problem in social media. Consider that we don’t even have licensing for mainstream PR and marketing — and think about how much really bad advice organizations get from those professions.”

At least in PR and Marketing there are longstanding professional associations with codes of ethics, increasingly strong academic and theoretical foundations, and a body of research-based knowledge (Cutlip, Center, Broom, 10th ed., p 120) that qualify us as members of a profession. This is despite our many weaknesses, including the presence of our own charletons.

Social media isn’t even there yet, and it needs to get there soon in order to separate the wheat from the chaff. Despite worthy efforts from Institute for PR Measurement Commission colleagues Katie Paine, Don Bartholomew and a few others, we’re still working on how best to measure social media effectiveness beyond output metrics.

We need Amanda to continue to call out snake oil salespeople, foggy logic, asinine commentary and the real danger of a lost of authoritative, professional conduct in such a fast growing area of communication practice. That she does so with wit, style and occasional vulgarity keeps the stew from being too bland.

So, count on me not only to declare Amanda relevant, but for vote #3 for the return of Strumpette — 140 characters at a time isn’t enough space.

As for “her” anonymity — I have been of two minds about it, both “yea” and “nay,” especially following my rather “eventful” introduction to Amanda last year. But in the end, I don’t think it affects credibility at all and it offers the freedom to focus on the message rather than its sender.

Finally, skepticism is not negativity, as I asserted last June. We surely are not lemmings, powerless in the thrall of the “wisdom” of the crowds, are we?