BLOG
Social Media Monitoring Gone Wrong
May 3, 2010 | Neal | 3 commentsI grew up in Philadelphia as a huge sports fans and if you know anything about Philly sports fans then you’ll know that we don’t tend to like New York teams very much. If you’re particularly up-to-date, you’ll know that in baseball my Phillies lost the World Series to the Yankees last season. So, despite the fact that my little league team growing up was called the Yankees, I dislike them more than usual these days. When I saw a headline late last week on ESPN.com indicating that the Yankees WEREN’T the most hated team in baseball, you can imagine it got my attention. They are certainly the most hated team in MY world and I felt like I knew enough about the sports world to know that no team in baseball engenders as much passion as the Yankees. My first thought was “Are the Phillies now more hated? They have been pretty good for the last couple years and non-Philadelphians generally don’t like Philly sports teams.” My second thought was “Have people finally turned on the Red Sox after years of lovable loser status followed by a few years of being happy that they were finally winning?” What I found once I read the article was even more interesting and it unexpectedly brought together two of my passions, social media and sports.
.
With an already skeptical attitude based entirely on the headline, I started reading this article that had originally appeared in the Wall Street Journal. When I read the following…
Contrary to popular belief, the Yankees are only the fifth-most despised team in the majors, according to an Internet algorithm built by Nielsen Co. that analyzes how people feel about certain things.
I started thinking about all the things that could have gone wrong that got the reporter from the common sense conclusion that the Yankees are the most hated team in baseball to the stunning conclusion that people hate Cleveland Indians, Boston Red Sox, Houston Astros, and Cincinnati Reds more than the Yankees. Outside of the Red Sox, there isn’t even a team in that group that inflames national passions other than perhaps pity for the Indians and their fans. So what went wrong?
W
What went wrong here was what goes wrong so frequently with social media monitoring efforts in corporate America – too much reliance on the tools, too little knowledge of the strengths and weaknesses of the tools being used, and inexperienced analysts being trusted to bridge that gap. I don’t know Mr. Biderman (the WSJ reporter who wrote the story) but I think it is safe to say that he isn’t an expert in Nielsen BuzzMetrics or any other social media monitoring tool. When he saw the sentiment ranking for the Yankees wasn’t as negative as those of lower profile teams, he saw what he wanted to see which was a headline that would grab attention. That the Yankees wouldn’t be the most hated team would indeed be big news. What he didn’t realize, and what later had to be corrected was that sentiment analysis of this sort a) isn’t historical in nature and b) that the measurement of sentiment isn’t as precise as hatred and love.
What the combination of the reporter, Nielsen and the WSJ apparently failed to realize is that sentiment is hard to measure. The analyst involved really has to know the tools they have at their disposal and dig in well beyond the surface findings that social media monitoring tools provide to get to real meaning. As I’m sure you can probably guess, those teams that ranked lower in terms of sentiment than the Yankees weren’t ranked that way because people hate them. A quick glance at the current standings will show you that those teams are struggling and people are likely disappointed with their performance so far. There’s a fine line between hate (“I hate the Yankees”) and disappointment (“The Red Sox suck so far this season”) and the Nielsen BuzzMetrics tool isn’t capable of distinguishing the difference.
.
5 Take-Aways
- Don’t Skimp on Analysis - Many organizations place the monitoring of social media via tools like Radian6 and BuzzMetrics in the hands of junior analysts because they believe the tools themselves contain the value. This example shows how in the wrong hands, the output of the tools can be misinterpreted.
- Be Skeptical, Especially on Sentiment – Of all the things that social media monitoring tools can do, sentiment analysis is their biggest flaw. Picking up the nuances of the written word is extremely difficult. Was “Product X is just great” meant at face value or was it laced with sarcasm? What was the nature of the sentiment expressed? Was it 25 different issues expressed occasionally or was it one major theme repeated over and over again?
- Hidden Danger? – When a mistake like this is made publicly in an article, there is limited downside. People will call you on it in comments section. Editors will get to the bottom of it. A modified article will be posted. In the real world, there may not be such checks and balances. If you’re going to entrust your social media analysis to a relatively junior analyst, there is a strong chance that you will be adding flawed conclusions to your discussion of what the “voice of the customer” is saying and how you should react to it.
- Hard Work – To get to the bottom of what is being said and summarized in all of those pretty charts and graphs, someone is likely going to at least have to skim the actual posts to make sure that any analysis being done is an accurate portrayal of what is going on. The tools are great for getting the feedback together and giving you analysis about keywords and who has the most influence but if you really want to know what’s going on, there’s no substitute for reading.
- The Yankees ARE Still the Most Hated – Let’s not kid ourselves, the only people who root for the Yankees are New Yorkers, displaced New Yorkers, and front-runners who have sold their souls to associate themselves with the Evil Empire.
Tags: Baseball, BuzzMetrics, Digital, ESPN, MLB, New York Yankees, Nielsen, Radian6, Social Media Monitoring, Wall Street Journal
Leave a Comment
 |
SubscribeSubscribe to a feed |
||||||||
|
|||||||||
| |
|
 |
Tweet Links |
Comments
[...] This post was mentioned on Twitter by Integrasco, astensby. astensby said: Why you should never trust entirely on a #SocialMedia monitoring tool: http://ow.ly/1GnpH – I couldn't agree more! (via @Integrasco) [...]
Great site. A lot of useful information here. I’m sending it to some friends!
Really decent post… I love it. Keep ‘em coming…