Five Social Media Research Warning Signs

Mar 15

Five Social Media Research Warning Signs

by John Stauffer
Category: Digital Influence, How-To, Influencers, Measurement, Research & Insights

Word of mouth practitioners often find themselves in uncharted territory. The majority of the digitally led programs being developed or put into the marketplace simply could not have existed two years ago. Sometimes not even two months ago.

As a result, planners and strategists in our industry rely on piles of research to gain insights into what’s likely to work for a particular audience. Some of that research comes from inside our own walls, sometimes we consult outside experts and publishers to make sure we know as much as humanly possible about a particular topic. The latter can come from peer-reviewed academic journals or independent publishers like eMarketer or Forrester.

Beyond those sources, though, there’s been a sharp uptick in research from a new universe of resources. Some doing great work. Some not.

Here are a few warning signs that suggest you could be relying on some shaky data…

Ghost Towns

Look out for research that relies on a low number of participants. This number is sometimes expressed by the variable (n = ) in the methodology section. There’s no hard and fast rule about the minimum number of participants in a valid survey but use the audience segment as a guide. A 100 person survey charting the online behavior of commercial architects? That’s probably okay. A 100 person survey asking moms about their favorite websites? Alarm bell central.

Rookies

Whenever I land on a new piece of research, I immediately go to the about section, especially if the publisher isn’t one I’ve come across before. New publisher doesn’t always mean bad research, but be extra cautious with research from an unfamiliar source and ask yourself “is there really no way to get this same data point from the likes of Pew or eMarketer?

Fish Hooks

Don’t bite on a campaign that’s cloaked as research. There’s nothing to prevent an organization from publishing data that’s not representative of a comprehensive audience segment. Selective sampling creates catchy headlines (e.g. 9 out of 10 doctors…) but you’ll get burned if you rely on this sort of thing for a communications program. To avoid getting hooked, simply ask yourself: Who conducted the poll and who paid for it?

Fuzzy Math

$fuzzy-math$

Before getting too deep in the data, flip to the methodology to make sure all the vital signs of a healthy research document are present. These include a date range, number of participants (see #1) and margin of error (indicative of sample size).

Also look for the confidence interval (CI) which tells you the reliability of the data set. A margin of error of 2.5% and a confidence interval of 95% simply means that if you ran the same survey 100 times, 95 of those would land within 2.5% of the original results. Think of this as a stress test for the research.

Conclusion Leaps

Simply put, watch out for leaps between the data and interpretation of data. I tend to focus on the raw results and let my specific project, client, and audience determine what it means in a particular instance. Though often times, the raw results are intermingled along with pieces of editorializing. Make sure you know the difference.

What other guidelines or tips do you use in your own research?

[Hat tip to Ogilvy's Dr. Jennifer Scott, Managing Director of Research & Insights, for her counsel in this post and general research assessments]

This entry was posted on Monday, March 15th, 2010 at 5:54 pm and is filed under Digital Influence, How-To, Influencers, Measurement, Research & Insights. You can follow any responses to this entry through the RSS 2.0. You can leave a response, or trackback from your own site.

16 COMMENTS
POST A COMMENT

TAGS: Tags: guidelines, social meda research

16 Responses to “Five Social Media Research Warning Signs”

Research In Link Popularity : Effortless Article Writing? Which One of Instant Article Wizard Pro or Answer Analyst should you use? Says:
March 15th, 2010 at 7:58 pm
[...] Five Social Media Research Warning Signs [...]
bianca Says:
March 16th, 2010 at 6:32 am
I am a pr student and i never heard of these 5 signs. Thank you for making public these “secrets”.
Mary Fletcher Jones Says:
March 16th, 2010 at 9:18 pm
Not naming any names but a large, reputable company has been guilty of this. When several of their blog subscribers called them on their shaky data collection and questionable findings, they back-pedaled, saying they felt their results would have been the same, even if their sample were not selective. (!?) There was data from other resources refuting their conclusions, so it just made it worse. They had a point they wanted to prove and fell in love with an idea, but they just didn’t have the numbers to back it up. No real harm done, except it was difficult to take their subsequent “reports” seriously after that.

I recently wrote a blog post profiling http://fletcherprince.wordpress.com/2010/02/12/online-marketing-trends 5 of my favorite, reliable social media research resources and what I like best about each.
John Stauffer Says:
March 17th, 2010 at 9:41 am
Mary, yes, it is certainly is best to be cautious when sourcing third party research. Enjoyed your post on the top sources for social media data.
Fantasy Baseball Draft Says:
March 18th, 2010 at 10:52 am
I love the Jump to Conclusions Map. Social Media is something you need to embrace, but you have to be wary of getting hooked.
KaiLee Viehland Says:
March 18th, 2010 at 2:16 pm
I am a student majoring in public relations at Western Kentucky University in Bowling Green, Ky. I am in a research in ad & pr class, and my assignment is to contact bloggers or experts in my field and ask a few questions about my research topic. I know you’re very busy but I was hoping you would be able to answer a few questions that I’ve posted below.

1. What do you think about social media?
2. Where do you think the future of advertising and public relations are headed?
3. How you do think the shift to online media will influence/change the way the industry works?
4. This is a transitional time for the industy so to speak, is it hard adpating to these changes since you’re already in the field? Do you think it would be easier for those just graduating to transition into this new digital age?
5. What is your advice to college students and those getting ready to enter the workforce in reagards to social media, word of mouth, new research methods being used, etc.?
6. What are some of the things you do when researching?

Any feedback would be greatly appreciated! Feel free to add on anything you think is important or relevant that I didn’t ask!
John Stauffer Says:
March 19th, 2010 at 9:01 am
@KaiLee - shoot me an email: john.stauffer at ogilvypr.com and we can connect.
Hopscotch Weekly » 5 pièges à éviter dans les études concernant les médias sociaux Says:
March 25th, 2010 at 2:03 am
[...] > Pour en savoir plus [...]
Stephanie Says:
July 26th, 2010 at 10:08 pm
I love these five signs. There is so much crap out there online and so much just made up research.
James the Marketing Consultant Says:
July 27th, 2010 at 2:27 am
The 5 warning signs are uniquely described here in your blog. By using simple terms and striking images, anyone can easily understand what you want to discuss about. Very interesting points.
Confidence Says:
October 6th, 2010 at 12:21 pm
Know maths and you can twist any research to any degree you want.To an unaware consumer , it is not less than a trap.
Tim Bonderud Internet Marketing Trends For 2011 Says:
December 19th, 2010 at 5:44 pm
I absolutely agree that more people have to use social media to grow there businesses, over the past few years the human element has been removed from alot of the marketing techniques and now people are realizing that social media is huge, remember people by from people not companies.
fast cash approval Says:
April 3rd, 2011 at 1:47 am
This is very useful suggestions. I have to say I really like encountering this alot. It aids me to become better grasp about the subject. It is very well produced. I am going to definitely search for this specific content pretty fascinating. Hopefully it is possible to supply more sooner or later.
Vaporizers Says:
April 15th, 2011 at 1:55 am
The very root of your writing while appearing reasonable at first, did not work very well with me after some time. Somewhere within the paragraphs you actually managed to make me a believer unfortunately only for a while. I still have a problem with your jumps in assumptions and you might do nicely to fill in all those gaps. In the event that you actually can accomplish that, I could certainly be impressed.
Etha Fennessy Says:
June 29th, 2011 at 9:34 am
Another thing I have really noticed is that for many people, bad credit is the results of circumstances outside of their control. By way of example they may are already saddled with illness so that they have excessive bills for collections. It might be due to a work loss or perhaps the inability to go to work. Sometimes divorce proceedings can send the financial circumstances in the undesired direction. Many thanks sharing your thinking on this website.
escorte romania Says:
August 17th, 2011 at 5:43 pm
Currently, wherever escort services confused the culture somebody or a merchant who has an escort agency desires to knowledge escort directory wordpress and escort agency wordpress providers with all the ever growing technological innovation it should wants that you as being a merchant to adopt the substantial technology.