Research Study: @sockington is more influential than @chrisbrogan

This Webecology research report has been making the rounds on Twitter. I haven’t had time to read it until now, here are my reading notes:

The Webecology team uses large scale data mining to identify patterns indicative of online culture and community. Wish I’d do this, too – and will, as soon as I find a research partner to help with the data mining part.

For this project, the authors set out to create a more accurate measure of influence on Twitter that goes beyond either:

  1. number of followers; or
  2. followers/friends ratio

The authors defined influence on Twitter as:

influence on Twitter = the potential of an action of a user to initiate a further action by another user

Specifically, influence means the potential of a tweet to generate replies, mentions (conversational behaviors), RTs, and attributions (content-pushing behaviors).

This is an atheoretical, operational definition of influence (the study’s Achille’s heel).

As far as I understand, all 4 actions were weighed equally. So, a RT factors the same as an @reply in determining influence.

They selected 12 Twitter accounts to study. The selection was based on this criterion: the 12 accounts were  “widely perceived to be among the more influential users on Twitter.” It is not clear who did the perceiving, and what definition or measure of influence they used in the process of perception. IMO, the arbitrary selection of the sample is another major weakness – but in this case, I can live with it, because the purpose is not to derive conclusions about Twitter culture as much as it is to demonstrate how the methodology can be used.

Then, the 12 users were grouped into 3 categories. Here is a table with the accounts they analyzed, and their number of tweets over 10 days, as well as the number of followers and friends at the end of the 10 days:

Celebrities Username Tweets Followers Followees
Ashton Kutcher aplusk 3,205 3,407,385 209
Shaquille O’Neil THE_REAL_SHAQ 2,072 2,092,541 562
Stanley Kirk Burrell MCHammer 6,016 1,331,797 31,202
Sockington sockington 5,711 1,089,984 380
Justine Ezarik ijustine 7,718 605,441 3,039
News Outlets Username Tweets Followers Followees
CNN Breaking News cnnbrk 1,096 2,712,530 18
BarackObama.com BarackObama 330 2,018,016 761,851
Mashable.com mashable 17,914 1,363,510 1,925
CNN cnn 11,607 193,625 50
Social Media Analysts Username Tweets Followers Followees
Gary Vaynerchuk garyvee 7,532 862,790 9,683
Chris Brogan chrisbrogan 48,341 94,715 88,431
Robert Scoble Scobleizer 23,112 94,295 2,423

The data that they mined was as collected over 10 days, in August 2009. The data included:

  • The 2143 tweets generated by the 12 users
  • The 90,130 actions (responses, RTs) triggered by the original 2143 tweets
  • All the tweets generated in connection with the 12 users (by their followers and friends;a total of 134, 654 tweets, 15,866,629 followers, and 899,773 friends/followees)

The authors produced 2 types of influence reports, based on the type of action that was triggered:

  1. conversational action (people replied, or mentioned the user – e.g. “meeting @stockington for catnip”)
  2. content-pushing action (people retweeted, or gave attribution – e.g. “via@username”)

Please note that a mention may or may not be a response to a tweet. If they were not responses to a tweet, they fall outside the authors’ definition of Twitter influence, and they should have been excluded from the analysis.

Here we go, on to the findings:

Conversational action

This graph shows you the amount of conversational activity (@replies and mentions) each user got in response to one (average) tweet.

Content action

This graph shows you how much content action (retweets and attributions) each user got for each (average) tweet:

So here we see that, per tweet, @sockington did get more retweets than @chrisbrogan.

The authors claim that these graphs of influence/tweet are the most accurate measure of Twitter influence so far. Therefore:

@sockington IS more influential on Twitter than @chrisbrogan,

because the fake cat gets more retweets. (sorry, @sockington, I do love you!!!)

I know exactly what you’re thinking, it starts with B and ends with T.

That’s because here we have a problem of construct validity. The measures do not actually measure influence. I wish the authors had read some research in communication & persuasion about the concept of influence, then worked their way from a conceptual to an operational definition.

Obviously, @sockington gets more retweets because he’s cuter & funnier than @chrisbrogan (sorry, Chris!). We don’t know why people reply or retweet. This study ignores a very important aspect of human relations: meaning. There is meaning in tweets, and meaning in why people retweet. But that is not captured in this study.

That being said, the report shows what can be done with data mining – it’s awesome! With a bit of help from people who know how to study meaning (hint, hint!), this type of research will be extremely valuable.

If anything, let this be an argument for computers & communication people working together, across disciplines.

In a future post, I will review conceptual and operational definitions of influence.

Advertisements

Tags: , ,

6 responses to “Research Study: @sockington is more influential than @chrisbrogan”

  1. Sockington says :

    HEY THAT’S WHAT I SAID FLEET GUY ooo dark pants purrrrrrrrr

  2. Trey Pennington says :

    Thank you for your analysis. I retweeted the study myself this morning but noted that I hadn’t drawn any conclusions. The other thing I noted, it is a relief to see that there is a movement towards rigorous academic research for social media.

    The hype is deafening. Those hungry for solid understanding consume all the free $197 webinars/handbooks/manuals and go away feeling the pangs more acutely.

    We miss you in the Upstate!

  3. Dave Fleet says :

    Part of the problem here is that you have two different contexts thrown together. There’s business advice and there’s amusement. Both have value, but to compare influence via the volume of retweets etc in two different contexts leaves room for misinterpretation.

    As far as the rest of the survey goes, it’s hardly surprising that CNN has more influence than individual marketers, and it’s hardly surprising that celebrities generated chatter.

    Bottom line: data means little without context.

  4. Mihaela says :

    Ellen: thank you for the kind words. To produce meaningful research, we need both the processing tools & methodologies (I think this is what the report is about) AND the construct validity that comes from those fields that have been studying people for a while.

    Sockington: mrrrow, meow – wow!

  5. SOCKINGTON says :

    TOTALLY DEMANDING A RECOUNT HERE LADY since when does cat reality tv have more or less validity than social media litterbox doofufery ALSO NOTE SHINY COAT

    as for retweeting please note that helpful socks missives considered useful as humor capsules exploding in brains IN OTHER WORDS EQUAL MEANING TO ANYTHING SOCIAL MEDIA PEOPLE ARE SAYING think clipped comics next to clippings about brouhaha THAT IS ALL THANKS meow

  6. Ellen says :

    Excellent analysis. You’ve singled out the weaknesses in this particular try, yes; but you still offer the explanations as to why they are weaknesses and, thus, still keep this work educational.

    I’m glad Webcology offered the the report and you offered the commentary. Thanks.

%d bloggers like this: