Archive for the ‘Measurement’ Category

Measuring Influence ‘Might’ Use Relationship Metrics

Monday, February 28th, 2011

 

Creative Commons, by Brian Hillegas

I’ve been thinking again. Last time, I tossed out the idea that measuring influence might be gleaned from Grunig and Hon’s work on measuring relationships.  Usually, you need to get people to fill out a questionnaire to determine the quality of the relationship, but maybe looking at the public evidence is enough.  Here are three of the six elements, with comment following about the potential for adapting to qualitative influence measurement:

Control Mutuality — The degree to which parties agree on who has the rightful power to influence one another. Although some imbalance is natural, stable relationships require that organizations and publics each have some control over the other.

Think about that in context of online influence – being “Facebook friends” might imply a mutual influence, but being friends with an organization if one’s not a customer or other stakeholder wouldn’t seem to greet the same implication.

Still, the idea that an organization would change its behavior as a consequence of interaction with its stakeholders is the essence of Grunig’s Excellence Theory (two-way, symmetrical communication.) Retweeting on Twitter, and a content analysis of the @reply sequence (actual conversations) might lead to an index by topic – it could demonstrate the extent of control mutuality as a surrogate for mutual influence. The question is whether there’s enough in the stream to properly analyze.

Trust – One party’s level of confidence in and willingness to open oneself to the other party. There are three dimensions to trust: integrity: the belief that an organization is fair and just … dependability: the belief that an organization will do what it says it will do … and, competence: the belief that an organization has the ability to do what it says it will do.

This, too, could be accomplished by content analysis, substituting individual for organization. Establishing the extent of trust could also indicate the opportunity for influential behavior, which could be apparent from the stream. We’d need to define the language trusted people use, but that doesn’t seem much different from a normal content analysis.

Satisfaction — The extent to which each party feels favorably toward the other because positive expectations about the relationship are reinforced. A satisfying relationship is one in which the benefits outweigh the costs.

This one’s tough – the nature of the relationship plays in to the analysis of satisfaction. Celebrities may make general comment about loving their fans, but is that a sincere platform for mutual satisfaction? Also, if the expectations are very low (as in celebrity culture, where the connection is, um, tenuous in reality but provides a simulation of a close relationship), does that negate the influence string?  My putative 14 year-old son may get his hair in a Beiber, demand I buy Beiber music and Beiber-esq purple garments, but is that influence or a phase? Or merely effective marketing?

Next post: the remaining three elements.

Share

Could Measures for Relationships Work for Influence?

Thursday, February 24th, 2011

What IS the formula for calculating influence?

This isn’t a stab at Klout. After reading a number of recent blog posts on influence, and participating in several of the associated discussions, I’m just weary of the chase.

My gut tells me that measuring influence is situational and specific.  You simply cannot look at tweet streams, numbers of followers, frequency of @ replies or retweets, number of Facebook friends, etc., and draw conclusions about someone’s influence, and there’s research that supports that idea.

Trey Pennington, Justin Goldsborough, Shonali Burke and Mark W. Schaefer are in the fray (and I’ve commented in a couple of cases), and I wrote my own post on the topic.  It’s been an interesting conversation split between the “Klout is useless” – “Klout is making a good attempt” and my fringe element rantings that we need better research to figure out how to measure influence.

The deal is that there are few independently researched efforts to investigate the claims of well-intentioned entrepreneurs.  There’s inevitably a black box that contains the algorithms and secret formulas, and no one wants to subject their potential cash cow to measurement that might render it an Edsel.

James Grunig and Linda Hon wrote a seminal paper about measuring relationships that might hold a key to figuring out how to measure influence.  To determine strength of relationships, they write, focus on six components: Control Mutuality, Trust, Satisfaction, Commitment, Exchange Relationship and Communal Relationship.  Coming up next week, a look at each element and how they may or may not apply to measuring influence.

BTW, I found out recently that my Technorati Authority score is 406. My Klout score is 46.  I have no idea what that means.  But I want to better understand influence, so I’m going to run this down for a while.

Share

Measuring Influence: 4 Learnings

Wednesday, February 2nd, 2011

Measurement isn't just bells and whistles

Measurement for its own sake is a waste of everyone’s time and money. It’s got to be in service of a strategy.

You might say that the foregoing statement is a canard; no one is beating down our doors asking us to just measure something, anything.  But there remain a feisty few, particularly on the social media side of the equation, who keep offering up horsepuckey in the guise of gold bullion.

Witness “4 Ways to Measure Social Media…,” a well-intentioned piece from last summer on Social Media Examiner. Author Nichole Kelly subheads the article with “exposure,” “engagement,” “influence” and “lead generation” — the “4 ways.”  Kelly’s on firm ground about exposure, pointing out the difficulty of a) getting good data and b) ensuring you’re counting only once, though equating reach to awareness is a colossal mistake.  Engagement,  too, is solid (if output-based), covering @replies, DMs, links clicked, comments and subscriptions. Good stuff.

Influence is listed third and lead generation fourth, showing exposure, engagement and influence as the top of the funnel leading to conversion.

The section on influence is underdone, and erroneously says tone (positive, negative, neutral) IS influence.  In fact, according to Yahoo!’s Duncan Watts, Winter Mason, and Jake Hofman, and the University of Michgan’s Etyan Bakshy, influence can’t be credibly determined from content analysis. Read all about it.

I heard Watts speak on this topic during the snowy last week of January at a meeting of the Institute for PR Commission on research, measurement and evaluation, of which I’m a member. Influence is a huge question, and Watts, et.al.’s work made me recall the somewhat hoary idea that understanding your specific audience (whether final audience or intermediary) is a lot more important than trying to calculate the exact number of impressions represented by friends of friends and retweet followers.

I pick on influence because it’s the biggest question in social media.  In fact, it’s been a big question in communication in general since the days of Lazarsfeld, Berelson and Gaudet and the two-step flow. Who are the “opinion leaders” and how do we calculate their effectiveness?

Here are four questions that hold promise when considering how to measure influence:

  1. Does the opinion leader “play” in the right sandbox for our intended audience/stakeholder?  Chris Brogan and Brian Solis have lots of followers, tribes that hang on their every tweet. Are their tribes our tribes?  They’ve got awesome scale by sheer numbers, but it’s anyone’s guess how involved they are or whether their followers in turn reach people we care about. We could get Brogan or Solis to talk about our service, product, leader or whatever, but to what end if their followers aren’t the right fit for us?
  2. Can we create a solid chain of links from the opinion leader’s actions to our desired actions?  If we’re working on building corporate reputation, retweets, Facebook “likes” and blog comments should have a relationship to opinions voiced by our final target audience. Simply passing along a leader’s statement (tweet, post, comment, etc.) shouldn’t be construed as adoption! Here’s where content analysis shows promise, especially in blogs and perhaps during Twitter chats. The opinion leader’s output should have some effect if he/she is truly influencing others. Note that this is a qualitative effort and suffers from lack of scale.
  3. Are we mistaking popularity for influence?  Celebrities routinely land atop the Twitter rankings, and there are brands on Facebook with upteen hundreds of thousands of “friends.” But having a lot of friends/followers just makes you popular. See #2 above.  We’ve long wondered about how to judge the effectiveness of influence in conventional relationships, but I don’t think many of us think the most popular student in high school was necessarily the most influential.
  4. Are we inappropriately drawing general conclusions from narrow findings?  Influence is personal and specific.  We make assumptions about readers of newspapers, TV viewers, etc., and have a body of research to back those assumptions up.  In social media, the appearance of influence may be mere output, or outtake at best. Outcomes outside of e-commerce are tough to come by, though clear objectives can solve this problem quickly.

The best measurement starts with research up front, which informs our strategy and objective-setting, followed by more research to determine effectiveness and progress toward objectives.  It’s not just tactical measurement designed to cover our butts or justify our budgets, especially when it’s trying to measure influence.

Share

Measurement Musing: Questions…No Answers…

Monday, December 20th, 2010

Public Relations measurement is so essential, yet so poorly understood. I’m sure it’s my bad that after gaining one client upon launching my business that focused on measurement, I haven’t had a second.  I’ve done some strategy work, some writing, and now am working on a  long-term project for a client that once again, has no measurement component to it.

I do wonder whether I’m just not the right person to help organizations measure — there are other, longer standing, better educated folks out there. Maybe it’s my destiny to stick in the internal communications space, rather than the measurement angle.  Of course, I try never to make life decisions any time from December through February — the seasonal affective disorder reaches its nadir (or zenith) as the winter solstice arrives and lingers through the cold, gray months.

I introduced measurement to my PR Tactics class at Kent State this fall — just a brief tablespoonful — I’m hoping that they remember it as they enter the profession. One thing’s for sure — they certainly had better measurement components in their final projects! Whoo-hoo!

This spring, I’m teaching a course in PR Metrics — so perhaps this is how I can drive measurement into our profession: give it to the kids who’ll replace the dinosaurs in a few years…

Speaking of which, I’ll be sourcing case studies in measurement — and entertaining guest speakers (either in person or by Skype) — so if you can help me out, let me know!

More questions:

Why do so many companies still see news media and social media as mutually exclusive instead of related?

When thinking about measuring social media, why we want to categorize it in the same way we do news media?

Why do we think reaching the most people is better than reaching the right ones?

Why do we want to define influence as only occurring through social media, and why do we so narrowly define it IN social media?

How come we can’t come up with a better means of determining appropriate scope and scale?

Why do people think the only marker of PR intelligence and value relates to external communication?

Why do companies fail to measure employee communication outcomes? Matter of fact, why do we still count clips?

Why am I thinking these hairy thoughts ?

Are these rhetorical questions? (No…)

BTW, don’t miss the Twitter chat, #MeasurePR with @Shonali Burke, Tuesday at 12 noon.

Share

Random Reflections on IABC’s 2010 Research and Measurement Conference

Saturday, November 20th, 2010
Working on the post

Sean and Shonali toiling in the service of communication

What happens when you get a roomful of communicators listening to a speaker on measurement? It’s not what you think. In this joint post, Shonali Burke and I sat atop the ivory tower after Day 1 of the Conference – and issued what Shonali’s husband would call “grand pronouncements.”

Shonali: Coming down in the elevator, I chanced upon a conversation between a gentleman attending an event hosted by The Gates Foundation, and an attendee of “our” conference. She said, “[Your conference] sounds so much more interesting. I doubt mine will be as riveting as yours.”

On being asked, she said, deprecatingly, that it was a communications conference. At this point, I couldn’t resist. I said, “You mean you’re not overwhelmed with excitement over the IABC Research and Measurement Conference?” She looked at me as if I was crazy. Just before she found out I was a speaker.

Was I mean? I don’t think so. Naughty, perhaps. Not mean. Heck, if you’re going to say whatever you like in an elevator, so can I.

Sean: Several people seemed quite taken by the morning sessions, though one person I encountered less so. She hemmed and hawed when I asked what she thought of the conference so far, never a particularly good sign. But in the end, she didn’t seem to have a clear set of objectives for attending the conference.

This is a huge theme in my teaching: Objectives are everything. If you don’t know what you’re hoping to achieve, you don’t have much of a shot at achieving it.

Shonali: A common editorial comment I keep hearing from attendees at measurement conferences (or presentations related to measurement) is: “It doesn’t seem like the basics have changed… so what do I take away from this?” It drives me a little crazy. No, the basics haven’t changed. That’s because they’re the basics.

How can you not grasp the importance of measuring numbers that matter instead of numbers that make you look good? What part of, “measure [what] has an impact as opposed to simply focusing on the tools,” isn’t easy to understand?

Sean: Angela Sinickas is a treasure trove of case studies. I have to remind myself to call her for research fodder. I saw Angela at PRSA’s 2010 International Conference, and suddenly realized I’d seen her presentation before. Some of that, no doubt, is that she boasts 23 of the Forbes worldwide list as clients. Maybe it’s rank envy! I love the fact that she represents for measurement, and I wonder what she might do with Dr. Don Stacks and Dr. Don Wright nipping at her heels on projects.

Shonali: What was really interesting about this conference was that it wasn’t the usual [measurement expert] suspects presenting.

Well, not all the usual suspects.

Well, not two-thirds of the usual suspects.

Well…

Sean: Shel Holtz said you have to measure something, and it doesn’t have to be complicated. I always say that getting your objectives right is the single best start to a measurement program. You’ve got to measure something, and starting with progress on attaining objectives is a great place to start.

I also loved that Patti Phillips went 100 percent professor on the crowd, demanding us to calculate.

Shonali: Represent. Ruminate. Calculate. Especially when it’s way after hours.

What else is a conference for?

Share

Web Tools Expanding (Slowly) into Internal Communication

Thursday, September 9th, 2010

A small but eager group of professional communicators met 7 September to explore the current state of Web tools in internal communication and found a slow, but steady increase in their use.  Twitter-based #ICChat totaled more than 200 tweets in a fast-paced, hour-long online discussion from 2-3 p.m. North American Eastern Time.

The most basic tools — e-mail and intranets and RSS — are expectedly common, but social tools — blogging, varieties of microblogging (such as Twitter) and shared document management are seen as near-term priorities.

@irosen: Q1: There is an increased demand on “basics” found on the internet such as RSS feeds, microblogging and embedded media #icchat

@Wedge: Slowly reducing our reliance on emailing Word documents around; moving to intranet web pages and shared space on intranet for docs #icchat

Microblogging, including Yammer, offers the potential for collaboration and instant access, according to the company website. The tool could lower data processing costs by redirecting communication away from e-mail, particularly for the short, direct sort of questions-and-answers an employee might need on the spur of the moment. I made the same argument for RSS and other tools

@CommAMMO: One angle @csledzik is to quantify amt of email traffic – if you banned Word doc attch’s in fav of Google docs/Sharep methods #icchat

@CommAMMO: @wheati @irosen Less email through the wall means lower data proc costs – RSS is seemless, no? #icchat

But reducing direct cost wasn’t a prime driver in one person’s opinion:

@wheati: @CommAMMO @irosen @Wedge Weren’t concerned with data cost. Interested in ease, security of doc access. And “one stop” shop for info. #icchat

Trying to reduce e-mail — for the sake of employees’ productivity — is a critical factor, in my opinion. Aggregating nonessential (but still important) material is a decidedly old-school response, though social tools offer an advantage beyond financial impact.

@jpchurch: We’re about to launch a complete intranet re-do, and introduce more targeted info & collaborative tools. Still far too many emails. #icchat

@jgombita: Q1 If staff, clients R spread out (geog’phy), working with wiki (or Google docs) is effective and inexpensive #icchat

@csledzik: So theres 2 objtvs: 1) reducing data proc. $ & incr’g knowledge sharing. Soft goal is key, but not the driver. #icchat

There currently is no organization I know of which has gotten knowledge management particularly right — though many have made progress: Ernst & Young’s Center for Business Knowledge predates the Web, using Lotus Notes databases to gather info from employees and make it available. Kind of early crowdsourcing

Microsoft’s Sharepoint suite — with its Wiki-Blogging-Discussions, etc. — came up a fair amount as a means of supporting knowledge sharing, with one participant looking for guidance on initial deployment.

@tnerko: Most excited about #SharePoint for wiki features as most in my company on a 3 year rotation and knowledge leaves often #icchat

The embrace of web tools within the workplace (particularly social media) is a referendum in organizational trust, transparency, according to one participant:

@csledzik: .@CommAMMO I see mgmt thatNot comf. w/ trust or transparency. Don’t understand benefits of sharing inter/externally. #icchat

The latter part of that tweet is pretty close to the truth: Internal communication, generally, isn’t as highly regarded in the workplace as is media relations. We shouldn’t be surprised that internal audiences are subject to fantasies of tight control — one senior leader told me that internal communication was, “a warm-fuzzy for employees” who don’t really care about the business.  It was some years ago, so I’m hopeful that opinions have moderated. But the advent of social media has shaken business leaders to their very boots in fear of loss of control. Control, by the way, that they haven’t had in 50 years, at least.

@wheati: A concern is also about company reputation. Exes want to control and package it, but SM is about neither. #icchat

@jgombita @pointsoftrue that’s why the key is guidelines #icchat

Indeed, guidelines are critical. The trick is to convince leaders that their employees can be trusted to follow them. This is a huge issue in regulated industries, such as securities firms, banks, medical.

The too-fast conversation wrapped up talking about how these web tools — in particular intranets — are measured.

@Wedge: To Q4: behaviour change. Impact, rather than ‘hits’ (although ‘hits’ are a baseline to indicate use / usefulness. #icchat #intranet

@wheati: Loosely…% of front line adopting RSS was one measure. #icchat

@CommAMMO: @wheati Tying the stats back to outcomes, even just simple correls is helpful – language of C-suite. #icchat

@tnerko: Word of mouth and feedback links for now, looking forward to commenting in sharepoint and will run focus groups as well #icchat

Next #ICChat is 21 September, 2-3 pm Eastern (North America), and we’re open to suggestions as to topic and potential guests. Hope to see you then.

What would you add to this? How can we make #ICChat better? Use the comments, or send me an email.

Sean Williams can help you: Consulting, Strategic Planning, Measurement, Training, Writing/Editing.

CommAMMO: One angle @csledzik is to quantify amt of email traffic – if you banned Word doc attch’s in fav of Google docs/Sharep methods #icchat
6:16 pm CommAMMO: @wheati @irosen Less email through the wall means lower data proc costs – RSS is seemless, no? #icchat
Share

Don’t Cry Over Measurement

Friday, August 6th, 2010

Listening to communicators carry on about how hard measurement is makes me crazy.  Look, I’m no quant-geek, high-foreheaded, pointy-headed type of guy, I’m a lowly scribe, an ink-stained (and pixel-illuminated) wretch.  That’s why I wrote an article for IABC’s CW Bulletin called “Measurement Without Crying.” Read it, and stop weeping.

Don't Cry.

You’ll also find pieces by my Institute for PR Measurement Commission colleagues Katie Paine and Mark Weiner — in this all-measurement issue.

Honest – it’s not all math all the time, and it’s not scary once you learn something about it. This fall, my students at Kent State University (PR Tactics) will find out, because I’m going to give them a window into measurement as part of the coursework!

Share

Useful Discussion on Measuring Social Media Influence

Tuesday, July 13th, 2010

Creative CommonsLynne d Johnson is working on a means of measuring social media influence, and is asking good questions about current tools and models. She rightly says that the core issue is a lack of a good definition of influence, and covers a couple of methods – Razorfish’s Social Influence Marketing Score and Altimeter’s Social Marketing Analytics — while calling for a deeper definition.

I always am wary about anything smacking of “calculators” in social media and PR, particularly those advanced by companies with an interest in selling social media as a revolution.  But Johnson’s role as SVP of the Advertising Research Foundation lends a serious imprint to the task. The ARF is working with the Word of Mouth Marketing Association (WOMMA) to create a set of social media measurement guidelines for the industry, she wrote.  My only concern is that the effort — being driven by marketers — will continue the marketing-centric, impression-oriented, reach-focused, quantity over quality mentality we’ve seen so far — or that it will be full of, well, BS metrics and methods.

Johnson writes of her similar concern, “I don’t think we’re talking about a wrong way of looking at influence, but we could be looking at only one side of the equation. In measuring social media, we have to listen, observe, and study to understand who the real influencers are. Perhaps an influencer’s influence isn’t driven online, but offline. Here’s where Razorfish’s SIM Score (or perhaps Altimeter’s Social Marketing Framework) can help us capture–along with the aid of engagement in a private community, an interview or survey–the offline component.”

Read the piece — it’s worth it.

Share

Another IABC International Conference…

Wednesday, June 16th, 2010

I recognize that if I’m not a speaker at the big IABC soiree, I’m probably not the target audience for it. I’m not surprised, therefore, that my first blush reaction to the Toronto gathering wasn’t particularly positive.  My goal for attending this year was to meet some new people and make contact with some who I haven’t seen in a while. I hope to eventually get some business from it, but really just need to expand the network.

The programming and format are nearly identical to my first International, in 1995, also in Toronto. That one was a revelation — I was just 4 years or so into the profession, and everything was new.  Every session offered fascinating insights or enhanced skills.  I met scores of people and hung out with many, enjoying my first trip to Toronto and my first extended business trip in several years.

In 1997, L.A. was a different experience. Many of the speakers were the same as two years earlier, and in 2002 at Chicago, there were just a few sessions that really caught my eye. So I took a vacation from the big show until this year.

Things that impressed me:

Erin Dick from Pratt & Whitney — a social media case study that wasn’t from a Silicon Valley firm… Her use of blogs, Twitter, YouTube and Flickr to help support P&W’s client (the U.S.Government) on the selection of an engine for the Joint Strike Force fighter was off the charts — brilliant. And it had a fairly strong measurement component. I decided to Tweet the session instead of trying to take notes. The benefit was that I had a great summary, though my thumbs threatened to lock up from BlackBerry-itis…

William Amurgis from American Electric Power — Looking for use of social media in internal communications? Amurgis delivered. AEP’s blogs, discussion boards, employee-uploaded photos, etc., set a high standard of participation. The company’s intranet philosophy? Enhance employee productivity, reinforce corporate messages and provide a place to meet for all employees. Everything has to pass through that frame, or it doesn’t happen. And, rather than buy software solutions, AEP makes their own. Amurgis has a designer and a developer on his staff.

The UnConference — OK, it was a bit different than other UnConferences (usually low-or-no-cost, open to anyone; you had to buy the day (at least) for the IABC Conference to get in, and it wasn’t cheap) — but the method of operation was different and fun. There was no pre-set program, just a list of ideas posted on the TorontoTalks website (that a few people did discuss first), and three 5-minute “keynotes” — very informally delivered.  The three-hour session on Sunday afternoon was comprised of four 25-minute blocks of time with six possible topics (being held at six tables). We wrote on sticky notes our question or suggested topic, then stuck it on a flip chart in an empty time slot. The writer could lead the discussion, or someone else could.  I talked measurement (what a shock!) with seven other folks and it was fascinating. We didn’t solve the ROI question in full, nor did we get into other facets of communication, but it still was valuable and fun.

The thing is, the (nice) venue, formal structure and overwhelming size of the show made it hard to connect with people. Even the formal networking session (the big one held on the floor of the exhibit show) was just an hour long — not near enough time to connect. (I also didn’t attend Monday’s sessions — none particularly grabbed me. That might have inhibited my networking activities, so shame on me!)

The cost was pretty high for a new entrepreneur, not only in travel but in the conference fee. I’ll be considering very carefully before jumping on again soon. But, if I wind up as a speaker…

{FYI, I’m speaking in November at IABC’s Research and Measurement Conference in Seattle, as well as at the PRSA National conference in DC in October.  I’m also willing to come to chapter lunches, etc., and can make a deal for my PRSA/IABC fellow members!}

Share

Crisis Analysis, SocMed Use, Get Globe/Mail Attention

Friday, June 11th, 2010

Canada’s outstanding The Globe and Mail has two stories today worth noting.  Vancouver, B.C., retailer Lululemon is using Twitter to gather intel from its customers about what sizes and colors to stock; British Petroleum gets second-guessed in its crisis communication strategy under the headline, “Lessons in Leadership Spill from BP.”

BP’s feckless communication strategy, especially demonstrated by company CEO Tony Hayward’s frequent gaffes when speaking off the cuff, deserves to be pilloried. Hayward and company were obviously led by lawyers in this regard, minimizing the potential impact of the disastrous gusher, appearing too rarely in public and pointing blame to subcontractors. Hayward’s “I’d like my life back” rang especially tone-deaf in the wake of 11 deaths and the potential for catastrophic wildlife impact (not to mention the economic peril for the gulf fishing industry.) Several communication experts get quoted in Wallace Immen’s excellent piece, including Michael Stern (Michael Stern Associates), Prof. Julian Barling (Queen’s University School of Business), and Guy Beaudin, (RHR International).

Lululemon sells athletic ware, and by all accounts does a bang-up job of it. Some of the success, according to CEO Christine Day, is due to its use of social media — Twitter and Facebook.  Reporter Marina Strauss quotes Day: “We learn more about [which items are in demand] on Facebook and social media: what are the guests really screaming for, and so we use [the feedback] to get a little bit more indication.”

Keeping an eye on its 127,000 Facebook fans and 32,000 Twitter followers gets Day and company a faster view than its store performance metrics (and offers perspectives from people who are just thinking about going to the store, rather than having bought something there — that’s an interesting view on potential demand, the pipeline, some call it.)

The social media use has two purposes, according to the article — to gather information, and to drive traffic to the company website. When we’re looking for ways to measure the effectiveness of social media, website traffic is more often cited than the research value, which is a pity.  Going back to the ROPE method of communication planning (Research, Objectives, Programming, Evaluation), you don’t have anything without the research.

If social media served no other purpose than market intelligence, it’d still be worth the investment, no?

{P.s., my Canadian sojourn is nearly complete – back to a more regular schedule next week.)

Share