360DigitalInfluence

Ogilvy Public Relations Worldwide
Apr 22

It’s Time for Measurement of Social Media

John Bell

by John Bell
Category: Measurement

We need to apply some ‘pretty good’ measurement model to the programs we are implementing in social media here at Ogilvy. For over two years we have been creating blog outreach “campaigns”, building Web 2.0 applications for clients and monitoring what is being said about brands and issues out there. We have always used some type of metric to report results. Still, many clients have been in the ‘try-and-learn’ mode, willing to deploy programs with some simple measurement models in place - web traffic, video views, even number of blog mentions.

Now our programs are much more ambitious. We have proven the potential of social media in an influencer marketing strategy and as an engagement strategy. We can do more and we can apply more budget (still a fraction of media buys) to what we call digital influence programs.  

Thanks to Charlene Li at Forrester Research, we have a starting point. At least for blogs. (We constantly remind ourselves that social media  is not defined by ‘blogs’. That is merely one expression.)

We have simplified her model and expanded it beyond blogs. Her approach set out to define the ROI of blogging. As such, she had a formula that divided the benefits by costs and then factored in the risks to a figure that represented the ROI.

Our goal is to define a measure of success. How will we know if our program is working? Which parts are working and which are not? We need to be able to report progress but also make course corrections in what could be an enduring effort to build sustained word of mouth.

Ultimately, we do want to know the return on investment. We need to understand how specific programs have paid off. And how they compare to more traditional marketing disciplines and channels. This is where it gets a bit squirrelly. We are driven to compare output performance in social media with output performance in advertising. After all, many clients require ad-equivalency measures in traditional PR programs in order to compare apples to oranges (yes, I meant to say that). Charlene takes this into account by suggesting everything from paid advertising comparisons to search engine optimization programs to Bzzagent equivalencies.

Three challenges with this approach:

  • Many of the ultimate measures (e.g. BzzAgent equivalency, the  ’standard’ cost of SEO, etc…)  are not well understood by all marketing personnel within a client company and require significant explanation (making it hard for them to “explain up” in their organization)
  • Some are just based upon sand to begin with. Set aside the collective disbelief that ad measurement is grounded in any meaningful measure. Just consider  judging a social media-based word of mouth program where the unit of comparison is the value of one BzzAgent. I love BzzAgent but they are just establishing themselves amongst the media buying set. Abstracting the cost of a single agent to compare to some other WOM-generating campaign just doesn’t hold up to the board room test (could you explain it to the board of directors with a straight face?)
  • WOM Units (an actual term used by the Word of Mouth Marketing Association to which we belong) are more valuable than ad impressions. That is what excites us all, no? You recommend a hotel to me and that means more than the ad I see in the New York Times Magazine.  But how much more valuable?  

Simplify  

We are trying to simplify things. First, we took the risks out of the equation. No, not in that magical sense. The risks still exist. I would prefer to counsel clients on any potentail risks outside the value equation. Most of our clients over-anticipate risks from getting involved in blogs or social networks. The last thing I want to do is suggest that they need to calculate an actual number to represent that fear.

Here are  some starting points:

If the program is designed to generate digital word of mouth including blog posts, comments, message board posts, then we need to benchmark existing discussion before the program and then measure what our efforts generated. That’s called counting. I know, pretty simple. But then we have to apply a value. What’s that magic multiple? 5, 10, more? Positive word of mouth mentions are gold. Ed Keller says  2 is the multiple. I think that is way, way low.    I am thinking that positive WOM mentions ar eat least 5 times more powerful than standrad ad impressions. I need to find some credible research to inform this multiple. Got any?

If the program is meant to create more product or service recommendations from reviews to   blog or message board posts, than count the number of new reviews and multiply times a billion. Okay, maybe a billion is a little much, but, again, positive customer reviews are priceless. If you are not already using the Net Promoter Score, you may want to look into it. In the meantime, mutliply each review by 10x a premium online cpm ($35?).

If the program is simply meant to raise awareness and not necessarily drive relevant word of mouth, like a viral video or game program, then mutliply WOM units by 2x a premium online cpm. Chances are you are spending more time with your audience. That should be worth something more than a standard impression. Still, many viral programs don’t have a tremendous amount of brand relevance.

If the program intends to raise or add search engine results on the first two results pages of Google, AOL Search, Ask.com, Yahoo and MSN, then we need a measurement. I know, I know. I am not saying anything here. We cannot simply compare the cost of a standard SEO program (as if that existed) to a social media program’s impact on results.  By engaging with social media and people on blogs, message boards and review sites, we can generate 3rd party content that supports our position in those search results. Most people approach SEO as getting a client’s Website higher on the page. That’s not good enough. We need to connect with others and motivate them to become our champions. People want 3rd party corroboration that something is good. We become suspicious if we see content all from one source.  We need  a metric for the percentage of positive or neutral mentions in these 100 slots (10 results per page x 2 pages x 5 search engines) weighted heavily, of course, for Google over Ask.com.

We are on to something. Perhaps we don’t have all of the  answers   in hand  in a way that can be generally applied across projects, but we do have enough information to gauge the effectiveness of a program. Soon, we will have a generally accepted model for ROI.

But I need your help to push this forward.  Tell me about your measures of success, your ROI, your sense that we are all onto something big with word of mouth…

This entry was posted on Sunday, April 22nd, 2007 at 5:22 pm and is filed under Measurement. You can follow any responses to this entry through the RSS 2.0. You can leave a response, or trackback from your own site.


13 Responses to “It’s Time for Measurement of Social Media”

  1. Jim Dowling Says:
    April 22nd, 2007 at 10:53 pm

    Step 1.
    Sit down, look the client in the eye, and be convinced that they know EXACTLY what it is we are going to do for them. I never underestimate this - there is too much mystique or misunderstanding around what PR does. Throw in digital, things start getting really hazy. Using language like “x recommends a hotel to me, that’s got to be worth more than an ad in the NY Times,” sounds simple, but works.

    Step 2.
    So I think you’re saying John, that we would be executing up to four tasks:

    a) word of mouth (a link, post or comments that extends a conversation)
    b) awareness (let’s call this noise)
    c) recommendations (a call to action: eg ‘click here to buy’ or simply an expressed phrase eg ‘that new Nick Cave album, you just gotta get it.’
    d) traffic / sales figures (number of people clicking on a specific URL referred there from our work).

    Regarding effective SEO and gathering up champions - I think that could be measured within the ‘recommendations’ category. SEO is a tool to get you more of those champions.

    Step 3.
    Applying a multiplier
    Maybe we should be looking at an industry standard multiplier, based on some hard-nosed quantitative research. If some research was conducted, paid for by a WOMMA-type organisation, we could attach a multiplier to each of those 4 comms tasks which was accepted and respected.

    We need a proven multiplier. If we can prove (or have authoritative research that backs it up) that you are 10.7 times more likely to buy a product because a friend says so, than you multiply a recommendation by 10.7.

    If we can prove that you are 2.3 times more likely to buy a product because you are aware of it, then we have a multiplier figure for the ‘awareness’ category, and so on.

    And then click-throughs to a specific purchasing URL, from one of our referring URLs - well, that’s pretty concrete.

    Finally, until we get inside people’s brains and can read their minds, I don’t see how we can ever prove 100% that PR, digital influence, advertising or any form of marketing can be attributed to a specific sale. We can gather the evidence to measure the likely, probably, almost definite effectiveness of our campaign. But who’s to say all customers aren’t standing in front of shelf pointing at products saying “eeny-meeny-miney-moh!”

    Of course they’re not - but measurement requires a leap of faith for all concerned. As I said at the top, look at our clients squarely in the eye, and challenge them. “Do you believe your customer will buy your product if someone they trusts recommends it to them?” So do we - and here is a very sensible method of measuring whether the work we’ll do for you, works or not.

  2. Qui Diaz Says:
    April 23rd, 2007 at 1:19 pm

    Timely post, John. Identifying and crystallizing these measurements will lead to scrutiny, a la IAB’s recent call for an audit of comScore and Nielson/Net Ratings. See “IAB Declares War On Web Measurement Firms” for more > http://publications.mediapost.com/index.cfm?fuseaction=Articles.san&s=59082&Nid=29632&p=370277.

    (Hat tip to KD Paine, who posted about this earlier today: http://kdpaine.blogs.com/kdpaines_pr_m/2007/04/this_is_huge.html).

    So, if we nail down these metrics (or should I say, “when”), that WOMMA approval/buy-in from our fellow socialoguers comes immediately prior to sharing the metrics far and wide. And we gotta be able to take’em to the mat. Obiviously.

    Viva la revolution!

  3. Rad Tollett Says:
    April 23rd, 2007 at 2:27 pm

    Per your post: “If the program is designed to generate digital word of mouth including blog posts, comments, message board posts, then we need to benchmark existing discussion before the program and then measure what our efforts generated…What’s that magic multiple? 5, 10, more?…I am thinking that positive WOM mentions are at least 5 times more powerful than standard ad impressions. I need to find some credible research to inform this multiple. Got any?”

    Yes. Sort of.

    I’m assuming your comment about Ed Keller and the multiple of 2 is from the WOMMA deck he presented in Orlando in January. I found where Ed Keller got his “twice as much” claim from.

    He didn’t directly source it in the deck, but this claim is from a slide that Roper put together in 2005 for their WOM report. In it, it has the following data points:

    PERCENT SAYING THE FOLLOWING INSPIRED THEM TO TRY SOMETHING NEW THE LAST TIME

    Messenger Percent

    Yourself 71
    Friends or relatives 49
    A professional 43
    Spouse or partner 41
    Magazine articles 38
    Your child 34
    Movies, TV programs, or music 31
    Colleagues at work 26
    TV or radio commercials 25
    Websites/Blogs 25
    Magazine Ads 21
    Infomercials 11

    What Roper did was pool these data points by WOM and by traditional media. In this way, they calculated – and it says on the slide – “Word-of-mouth is valued twice as much by consumers as traditional media.” Again, I would imagine this is where Ed found his definitive view on the multiple being 2 rather than something higher.

    Now let’s break this out data by FOUR groups:

    Group ONE will be “core network” (a term Pew uses – one that I really like). This includes yourself, friends or relatives, and spouse/partner, and kids.

    Group TWO will be “extended network” (another Pew term). This includes professionals, colleagues at work, and websites/blogs (Roper – I think rightly – puts websites/blogs into this category. After all, a person going to a website or blog is looking for advice from a professional).

    Group THREE will be media “content.” This includes magazines articles, movies, TV programs, music.

    Group FOUR will be traditional advertising. This includes TV or radio commercials, magazine ads, and infomercials.

    Pooling and averaging, we get the following:

    PERCENT SAYING THE FOLLOWING INSPIRED THEM TO TRY SOMETHING NEW THE LAST TIME

    Messenger Percent

    Core/Significant Network 49
    Extended Network 31
    Media Content 35
    Traditional Advertising 19

    A couple things stand out here. First off, if you compare the power of WOM deriving from your core/significant ties then you can conclude that it is 2.5 (not 2) times more effective than traditional advertising.

    Another interesting thing to note is that WOM deriving from your extended network – professionals, coworkers, and bloggers – is as inspirational as traditional media content. Nevertheless, WOM from an individual in one’s extended network is still 1.6 times more effective than traditional advertising.

    So there you have it. Using the same data charts and cutting the data in a more precise way, at best the 2 should bump to 2.5.

    Another question I can’t answer but find intriguing is what you have already asked. Can we break this down by positive and negative WOM vs. positive and negative media/advertising vehicles? Will we discover that the 2.5 remains constant for positive WOM vs. tradition while discovering that the 2.5 bumps to a variance of 5 for negative experiences/inspiration?

    These questions need answering, and Roper’s line of questions could use some amending (in my opinion). Still, this is what we have to work with today.

  4. jerrysjuicebar.com » Blog Archive » Size Matters? Says:
    April 23rd, 2007 at 8:07 pm

    [...] John Bell has an interesting discussion on his blog about measurement criteria and measurement formulas. [...]

  5. John Bell Says:
    April 24th, 2007 at 7:24 am

    Jim - I couldn’t agree more that the look-them-in-the-eye conversation is important. That’s where we have tried to be up until now. Yet as our budgets grow, clients need something more. We are starting to unleash some sizaeble social media programs. The budgets are not coming from PR but elsewhere in the organization often taken from traditional ad budgets offline and online. To justify this, clients want some trustworthy metric. (Some clients will never be satisfied with this shift as they are invested in the mechanism of advertising)

    Reagarding the multiplier - yup, that’s what we need. There’s another elusive multiplier out there - how many people am I likely to recommend Nick Cave’s Grinderman CD? BzzAgent developed a formal way to promote this spiderweb of conversations. But amongst us civilians, how many people are we likely to reccommend a product to? There are so many variables that my head hurts again.

    But did I tell you just how fabulous the Grinderman CD was?

  6. Jim Dowling Says:
    April 24th, 2007 at 6:43 pm

    You didn’t tell me.

    Amongst us civilians, that would depend on how many friends you had to talk to (access/exposure), who was prepared to listen to you (influence), and how deeply care you about the subject anyway.

    This takes us into an area of social grouping / social DNA. There’s compare me and you and the example of Nick Cave.

    John - Tier 1
    Blogs about Nick Cave
    Tags stuff on Nick Cave
    Is in various groups, has bought several albums off i-Tunes, selected him on Pandora etc…

    Jim - Tier 2
    Blogs occasionally about alternative music
    Tags stuff on all sorts of bands which from time to time may include NC & Bad seeds.
    Buys music from the same category as Nick Cave

    We can assume…or come up with a calculation which assumes…how many times I’m likely to talk about Nick, and how many times you are? And would this not simply be placing me and you in our beloved Ogilvy funnel…(where you are a ‘loyal’ Nick Cave fan whilst I am ‘interested / aware?’)

    I don’t think we’re far off. Is there a mathematician anywhere?

  7. Social media - en allt viktigare kanal - där mätverktyg saknas « PR 2.0 Says:
    April 25th, 2007 at 3:47 am

    [...] Däremot sÃ¥ sprang jag pÃ¥ det utmärkta blogginlägget: “It’s Time for Measurement of Social Media” av John Bell (Oglivy PR). Inte heller han kan rekommendera nÃ¥gra riktigt bra universala mätverktyg för denna spretiga värld. Men har belyser sÃ¥väl problem som möjligheter pÃ¥ ett mycket bra sätt. [...]

  8. David Johnson Says:
    April 27th, 2007 at 10:15 am

    rafat and the paidcontent crew just put on a confab for the economics of social media. it would most likely be of interest to this group: http://www.econsm.com/

  9. links for 2007-05-15 at Framtider.net Says:
    May 15th, 2007 at 5:22 pm

    [...] 360 Digital Influence: It’s Time for Measurement of Social Media John Bell pÃ¥ Ogilvy om hur annonsörer kan räkna pÃ¥ effekten av social marknadsföring. (tags: annonser marknadsföring uppmärksamhet john_bell ogilvy bloggosfären den_sociala_webben samtalet ekonomi statistik) [...]

  10. 360 Digital Influence » Blog Archive » Building a solid foundation Says:
    May 31st, 2007 at 1:04 pm

    [...] Our fearless leader, John Bell, who is co-moderating today, has been preaching loudly about the need for legit metrics for the work that all of us in the social media sphere have our hands in. A few of his recent thoughtful and warranted diatribes can be read here, here and here. [...]

  11. The Smoking Foot » Blog Archive » Web Analytics Says:
    July 17th, 2007 at 8:04 am

    [...] Social Media Measurement [...]

  12. exsmoker Says:
    September 30th, 2007 at 10:34 pm

    Hello All,

    I was reading around some of the posts here and I found interesting things that you guys talk about, I just made a blog about quitting smoking resources and ideas that you might want to check out.
    If someone is interested in this topic just go to; http://endthehabitnow.blogspot.com and let me know what you think. Your honest feedback would be greatly appreciated.
    Thanks in advance.

  13. What are the results? « Kendall Wadsworth Says:
    April 2nd, 2009 at 10:59 am

    [...] time to talk about what are some of the results that come out of these companies and tools. Both John Bell of Ogilvy and Charlene Li of Forrester Research discuss two different ways of understanding the [...]

Post Your Comment

 

dailyinfluencepromo1

Blog Network

CATEGORIES

TAGS

RECENT POSTS

RECENT COMMENTS

OTHER BLOGS

The WPP Reading Room

Public Relations
Advertising
Branding
and more...

Join the Ogilvy PR Worldwide/ 360° Digital Influence group on LinkedIn
Join the Ogilvy PR Worldwide/ 360° Digital Influence group on Facebook

NETWORK FEED

Join the Ogilvy PR Worldwide/ 360° Digital Influence group on LinkedIn
    Ogilvy TripodPRSA-Chicago Senior Leaders Award Ceremony Honoring Marcia Silverman, December 15, 2009PRSA-Chicago Senior Leaders Award Ceremony Honoring Marcia Silverman, December 15, 2009PRSA-Chicago Senior Leaders Award Ceremony Honoring Marcia Silverman, December 15, 2009PRSA-Chicago Senior Leaders Award Ceremony Honoring Marcia Silverman, December 15, 2009PRSA-Chicago Senior Leaders Award Ceremony Honoring Marcia Silverman, December 15, 2009Taiwan Social Media SeminarOgilvy 360 DI Asia on WSJ

  • Compassion in Hong Kong

 
Ogilvy Public Relations Worldwide

© 2010 Ogilvy PR Worldwide

Privacy Policy