Accessing data from social networks, such as LinkedIn, Facebook, or Twitter, used to be much simpler and easier several years ago. Now, most of the APIs have restrictions. Also, the accessing methods are a little bit more involved. First, one has to get authentication (which used to be the case even earlier) and then use methods that access either friends or connections. We have only chosen Twitter for demonstrating the analysis of social network data here, but you can also find other social media data in a similar way.
In order to access Twitter data, as we noticed from the previous chapters (when we discussed word clouds), you have to get authentication keys to access their APIs. There are four keys: CONSUMER_KEY, CONSUMER_SECRET, ACCESS_TOKEN_KEYS, and ACCESS_TOKEN_SECRET. Once these credentials are verified via Python successfully, you can call GetFriends() and GetFollowers() to get the list of friends and followers. There are many packages available in Python to access Twitter data. So, it is very confusing which ones to use. We have used tweepy in past examples. Here, in the following code, we will use Python-Twitter because it has convenient modules to get data, summarize it, store it in cPickle, and then visualize it.
import cPickle
import os
import twitter # https://github.com/ianozsvald/python-twitter
# Usage:
# $ # setup CONSUMER_KEY, CONSUMER_SECRET, ACCESS_TOKEN_KEY, ACCESS_TOKEN_SECRET
# as environment variables
# $ python get_data.py # downloads friend and follower data to ./data
# Errors seen at runtime:
# raise URLError(err)
# urllib2.URLError: <urlopen error [Errno 104] Connection reset by peer>
DATA_DIR = "data" # storage directory for friend/follower data
# list of screen names that we'll want to analyze
screen_names = [ 'KirthiRaman', 'Lebron' ]
def get_filenames(screen_name):
"""Build the friends and followers filenames"""
return os.path.join(DATA_DIR, "%s.friends.pickle" % (screen_name)), os.path.join(DATA_DIR, "%s.followers.pickle" % (screen_name))
if __name__ == "__main__":
# deliberately stripped my keys
t = twitter.Api(consumer_key='k7atkBNgoGrioMS...',
consumer_secret='eBOx1ikHMkFc...',
access_token_key='8959...',
access_token_secret='O7it0...');
print t.VerifyCredentials()
for screen_name in screen_names:
fr_filename, fo_filename = get_filenames(screen_name)
print "Checking for:", fr_filename, fo_filename
if not os.path.exists(fr_filename):
print "Getting friends for", screen_name
fr = t.GetFriends(screen_name=screen_name)
cPickle.dump(fr, open(fr_filename, "w"), protocol=2)
if not os.path.exists(fo_filename):
print "Getting followers for", screen_name
fo = t.GetFollowers(screen_name=screen_name)
cPickle.dump(fo, open(fo_filename, "w"), protocol=2)The friends and followers information is dumped in cPickle. By running the following commands (as explained in https://github.com/ianozsvald/python-twitter), you can run the following code:
python get_data.py python summarise_data.py python draw_network.py
