In the previous section, we implemented the streaming server. The server application allows other applications to send words that you can listen to from the statuses/sample
endpoint from Twitter. When the server receives geotagged tweets containing the words tracked by a client, it will deliver the client a simplified version of the tweet. The server doesn't enforce what the client applications do with the tweets; the client application could just count the tweets, visualize the frequency of each term in time with a heat map, or create a network chart showing the co-occurrence of the terms.
In this section, we will implement a client application. The application will display a map showing the location of the tweets that match each term in a world map, and will display a bar chart that will display the count for each term. As we did in Chapter 6, Interaction between Charts, we will use Backbone and D3 to structure our application. A screenshot of the application is shown as follows:
The code of the application is available in the chapter12/chirp-client
folder. We will begin by describing the project structure, and then implement the project components.
As mentioned earlier, we will use Backbone to structure the application's components. As we want to visualize the places where different topics are mentioned in the world at any given time, we will define a topic as the main component of our application. A topic will contain a word (the string that we want to track in Twitter), a color to visualize it, and a list of matching tweets. We will create a Topic
model containing these attributes.
We will create a collection for the topics. The collection will be in charge of creating the topic instances when the user adds words in the input element and appending the tweets to the corresponding topic instance as they arrive. We will provide our collection with three views: the world map view, the barchart view, and the input element, where the user can add a new topic.
The code for the application components will be in the src/app
directory in the project folder. We will have separate directories for the models, collections, and views of our application; we will have the app.js
and setup.js
files to define the application namespace and to set up and launch our application:
app/ app.js models/ collections/ views/ setup.js
We will encapsulate the components of our application under the App
variable, which is defined in the app/app.js
file. We will add attributes for the collections, models, and views to the App
object, as follows:
// Define the application namespace var App = { Collections: {}, Models: {}, Views: {} };
To run the application, run the server application and go to the chirp-client
directory and start a static server. The Gruntfile
of the project contains a task to run a static server in the development mode. Install the Node modules with the following command:
$ npm install
After installing the project dependencies, run grunt serve
. This will serve the files in the directory as static assets and open the browser in the correct port. We will review the application models and views.
The main component of our application will be the Topic
model. A Topic
instance will have a word
, color
, and an array of simplified tweets matching the topic's word. We will define our model in the app/models/topic.js
file. The Topic
model will be created by extending the Backbone.Model
object:
// Topic Model App.Models.Topic = Backbone.Model.extend({ // The 'word' attribute will uniquely identify our topic idAttribute: 'word', // Default model values defaults: function() { return { word: 'topic', color: '#555', tweets: [] }; }, // addTweet method... });
The word
string will uniquely identify our models in a collection. The topics will be given a color to identify them in the bar chart and the map views. Each topic instance will contain an array of tweets that match the word
of the topic. We will add a method to add tweets to the array; in this method, we will add the topic's color. Array attributes (such as the tweets
property) are treated as pointers; mutating the array won't trigger the change
event. Note that we could also create a model and collection for the tweets, but we will use an array for simplicity.
We will trigger this event explicitly to notify potential observers that the array has changed:
// Adds a tweet to the 'tweets' array. addTweet: function(tweet) { // Adds the color of the topic tweet.color = this.get('color'), // Append the tweet to the tweets array this.get('tweets').push(tweet); // We trigger the event explicitly this.trigger('change:tweets'), }
Note that it is not necessary to define the default values for our model, but we will add them so that we remember the names of the attributes when describing the example.
We will also create a collection for our topics. The Topics
collection will manage the creation of the Topic
instances and add tweets to the corresponding collection as they arrive from the server. We will make the socket
endpoint accessible to the Topics collection, passing it as an option when creating the Topic
instance. We will bind the on
socket event to a function that will add the tweet to the corresponding topic. We also set a callback for the add
event to set the color for a topic when a new instance is created, as shown in the following code snippet:
// Topics Collection App.Collections.Topics = Backbone.Collection.extend({ // The collection model model: App.Models.Topic, // Collection Initialization initialize: function(models, options) { this.socket = options.socket; // Store the current 'this' context var self = this; this.socket.on('tweet', function(tweet) { self.addTweet(tweet); }); this.on('add', function(topic) { topic.set('color', App.Colors[this.length - 1]); this.socket.emit('add', {word: topic.get('word')}); }); }, // Add tweet method... });
The addTweet
method of the collection will find the topic that matches the tweet's word
attribute and append the tweet using the addTweet
method of the corresponding topic instance:
addTweet: function(tweet) { // Gets the corresponding model instance. var topic = this.get(tweet.word); // Push the tweet object to the tweets array. if (topic) { topic.addTweet(tweet); } }
In our application, we only need views for the Topics
collection and for the application itself. We will have a view associated with the bar chart, a view for the map of tweets, and a view for the input element, which will be used to create new topic instances. The code for each of these views is in the src/app/views
directory in the topics-barchart.js
, topics-map.js
, and topics-input.js
files, respectively.
The input element will allow the user to add a new topic. The user can write a new topic in the input box and click on the Send button to send it to the server and add it to the list of watched topics. The input element is shown in the following screenshot:
To render this view, we will create a template with the markup of the form. We will add the template under the body tag in the index.html
document:
<!-- Input Element Template --> <script type='text/template' id='topics-template'> <form role="form" class="form-horizontal form-inline" id="topic-form"> <div class="form-group"> <label for="msgToServer" class="sr-only">Message</label> <input type="text" class="form-control input-sm" id="new-topic" placeholder="Add a new topic"> <button type="submit" class="btn btn-default btn-sm">Send</button> </div> </form> </script>
We will implement this view by extending the Backbone.View
object. The template
attribute will contain the compiled template. In this case, the template doesn't have placeholder text to be replaced when rendering, but we will keep _.template()
to allow the use of template variables in the future:
// Topic Input App.Views.TopicsInput = Backbone.View.extend({ // Compile the view template template: _.template($('#topics-template').html()), // DOM Events events: { 'submit #topic-form': 'addOnSubmit', }, initialize: function (options) { // The input element will be disabled if the collection has five or more items this.listenTo(this.collection, 'add', this.disableInput); }, render: function () { // Renders the input element in the view element this.$el.html(this.template(this.collection.toJSON())); return this; }, disableInput: function() { // Disable the input element if the collection has five or more items }, addOnSubmit: function(e) { // adds a topic when the user press the send button } });
We will allow up to five topics by a user to avoid having too many similar colors in the map. We have limited space for the bar chart as well, and having an unlimited number of topics per user could have an impact on the performance of the server. When a new topic instance is created in the collection, we will invoke the disableInput
method, which will disable the input element if our collection contains five or more elements:
disableInput: function() { // Disable the input element if the collection has five or more items if (this.collection.length >= 5) { this.$('input').attr('disabled', true); } },
To add a new topic, the user can type the terms in the input element as soon as the Send button is pressed. When the user clicks on the Send button, the submit event is triggered, and the event is passed as an argument to the addOnSubmit
method. In this method, the default action of the form is prevented as this would cause the page to reload, the topic is added to the topics collection, and the input element is cleared:
addOnSubmit: function(e) { // Prevents the page from reloading e.preventDefault(); // Content of the input element var word = this.$('input').val().trim(); // Adds the topic to the collection and cleans the input if (word) { this.collection.add({word: word}); this.$('input').val(''), } }
The bar chart view will encapsulate a reusable bar chart made in D3. We will describe the view and then comment on the implementation of the chart. For now, we will just need to know about the interface of the chart. The code for the bar chart view is available in the src/app/views/topics-barchart.js
file. A bar chart showing the tweet count for a set of topics is shown in the following screenshot:
We begin by extending and customizing the Backbone.View
object. We add the chart
attribute, which will contain a configured instance of the charts.barChart
reusable chart. The bar chart will receive an array of objects that will be used to create the bars. The label
attribute allows you to set a function to compute the label for each bar. The value
attribute will allow you to configure the value that will be mapped to the bar's length, and the color
attribute will allow you to configure the color of each bar. We set functions for each one of these attributes, assuming that our array will contain elements with the word
, count
, and color
properties:
// Bar Chart View App.Views.TopicsBarchart = Backbone.View.extend({ // Create and configure the bar chart chart: charts.barChart() .label(function(d) { return d.word; }) .value(function(d) { return d.count; }) .color(function(d) { return d.color; }), initialize: function () { // Initialize the view }, render: function () { // Updates the chart } });
In the initialize
method, we add a callback for the change:tweets
event in the collection. Note that the change:tweets
event is triggered by the topic instances; the collection just echoes the events. We will also render the view when a new topic is added to the collection:
initialize: function () { // Render the view when a tweet arrives and when a new topic is added this.listenTo(this.collection, 'change:tweets', this.render); this.listenTo(this.collection, 'add', this.render); },
The render
method will construct the data array in the format required by the chart, computing the count
attribute for each topic; this method will select the container element and update the chart. Note that the toJSON
collection method returns a JavaScript object, not a string representation of an object in the JSON format:
render: function () { // Transform the collection to a plain JSON object var data = this.collection.toJSON(); // Compute the tweet count for each topic data.forEach(function(item) { item.count = item.tweets.length; }); // Compute the container div width and height var div = d3.select(this.el), width = parseInt(div.style('width'), 10), height = parseInt(div.style('height'), 10); // Adjust the chart width and height this.chart.width(width).height(height); // Select the container element and update the chart div.data([data]).call(this.chart); return this; }
In the topics map view, the tweets from all the topics will be drawn as points in a map, with each tweet colored as per the corresponding topic. The code of this view is in the src/app/topics-map.js
file. Tweets for each topic in a world map is shown in the following screenshot:
To create this view, we will use the charts.map
reusable chart, which will render an array of GeoJSON features and a GeoJSON object base. This chart uses the equirectangular projection; this is important to set the width and height in a ratio of 2:1. In the initialize
method, we set the base GeoJSON object provided in the options object. In this case, the options.geojson
object is a feature collection with countries from Natural Earth. We will render the view only when a tweet arrives:
// Topics Map View App.Views.TopicsMap = Backbone.View.extend({ // Create and configure the map chart chart: charts.map() .feature(function(d) { return d.coordinates; }) .color(function(d) { return d.color; }), initialize: function (options) { // Sets the GeoJSON object with the world map this.chart.geojson(options.geojson); // Render the view when a new tweet arrives this.listenTo(this.collection, 'change:tweets', this.render); }, render: function () { // Gather the tweets for all the topics in one array var tweets = _.flatten(_.pluck(this.collection.toJSON(), 'tweets')); // Select the container element var div = d3.select(this.el), width = parseInt(div.style('width'), 10); // Update the chart width, height and scale this.chart .width(width) .height(width / 2) .scale(width / (2 * Math.PI)); // Update the chart div.data([tweets]).call(this.chart); return this; } });
In the render
method, we gather the tweets for all the topics in one array, select the container element, and update the chart. The geotagged tweets will be drawn as small points on the world map.
We will create a view for the application itself. This is not really necessary, but we will do it to keep things organized. We will need a template that contains the markup for the application components. In this case, we will have a row for the header, which will contain the page title, lead paragraph, input element, and bar chart. Under the header, we will reserve a space for the map with the tweets:
<!-- Application Template --> <script type='text/template' id='application-template'> <div class="row header"> <!-- Title and about --> <div class="col-md-6"> <h1 class="title">chirp explorer</h1> <p class="lead">Exploring geotagged tweets in real-time.</p> </div> <!-- Barchart --> <div class="col-md-6"> <div id="topics-form"></div> <div id="topics-barchart" class="barchart-block"></div> </div> </div> <div class="row"> <div class="col-md-12"> <div id="topics-map"></div> </div> </div> </script>
The application view implementation is in the src/app/views/application.js
file. In this case, we don't need to compile a template for the view, but we will compile it if we add template variables as the application name or lead text:
// Application View App.Views.Application = Backbone.View.extend({ // Compile the applicaiton template template: _.template($('#application-template').html()), // Render the application template in the container render: function() { this.$el.html(this.template()); return this; } });
In the render
method, we will just insert the contents of the template in the container element. This will put the markup of the template in the container defined when instantiating the view.
With our models, collections, and views ready, we can proceed to create the instances and wire things up. The initialization of the application is in the src/app/setup.js
file. We begin by creating the app
variable to hold the instances of collections and views. When the DOM is ready, we will invoke the function that will create the instances of our views and collections:
// Container for the application instances var app = {}; // Invoke the function when the document is ready $(function() { // Create application instances... });
We begin by creating an instance of the application view and setting the container element to the div with the application-container
ID. The application view doesn't have an associated model or collection; we can render the view immediately as shown in the following code:
// Create the application view and renders it app.applicationView = new App.Views.Application({ el: '#application-container' }); app.applicationView.render();
We can create an instance of the Topics
collection. At the beginning, the collection will be empty, waiting for the user to create new topics. We will create a connection to the streaming server and pass a reference to the server endpoint as well as to the collection of topics. Remember that in the initialization method of the Topics
collection, we add a callback for the tweet
event of the socket, which adds the tweet to the corresponding collection. Remember to change localhost
to an accessible URL if you want to use the application on another device:
// Creates the topics collection, passing the socket instance app.topicList = new App.Collections.Topics([], { socket: io.connect('http://localhost:9720') });
As we have an instance of the Topics
collection, we can proceed to create instances for the topic views. We create an instance of the TopicsInput
view, the TopicsBarchart
view, and the TopicsMap
view:
// Input View app.topicsInputView = new App.Views.TopicsInput({ el: '#topics-form', collection: app.topicList }); // Bar Chart View app.topicsBarchartView = new App.Views.TopicsBarchart({ el: '#topics-barchart', collection: app.topicList }); // Map View app.topicsMapView = new App.Views.TopicsMap({ el: '#topics-map', collection: app.topicList });
In the map chart of the TopicsMap
view, we need a GeoJSON object with the feature or feature collection to show as the background. We use d3.json
to load the TopoJSON
file containing the world's countries and convert it to the equivalent GeoJSON object using the TopoJSON
library. We use this GeoJSON
object to update the map chart's geojson
attribute and render the view. This will display the map that shows the world's countries:
// Loads the TopoJSON countries file d3.json('dist/data/countries.json', function(error, geodata) { if (error) { // Handles errors getting or parsing the file console.error('Error getting or parsing the TopoJSON file'), throw error; } // Transform from TopoJSON to GeoJSON var geojson = topojson.feature(geodata, geodata.objects.countries); // Update the map chart and render the map view app.topicsMapView.chart.geojson(geojson); app.topicsMapView.render(); });
Finally, we render the views for the topicList
collection:
// Render the Topic Views app.topicsInputView.render(); app.topicsBarchartView.render(); app.topicsMapView.render();
At this point, we will have the input element, an empty bar chart, and the world map without tweet points. As soon as the user adds a topic, the server will begin to deliver tweets, which will appear in the views. The application and rendered topic views can be seen in the following screenshot:
Let's recapitulate how the client application works. Once the views are rendered, the user can add topics by typing words in the input box. When the Enter key is pressed, the contents of the input element will be added as the word attribute of a new topic instance. The collection will send the word to the streaming server, which will add the topic and a reference to the client in the topics list. The server will be connected to the Twitter-streaming API. Each time the server receives a geotagged tweet, it will compare the tweet text with the topics in the list; if a match is found, a simplified version of the tweet will be sent to the client. In the client, the tweet will be added to the tweets
array of the topic that matches the tweet text. This will trigger the views to render, updating the bar chart and the map. Here, we can guess where people are having their breakfast and where they are having dinner, as shown in the following screenshot:
The streaming server can be used with any application that can send the add
event and receive the tweets when the server emits the tweet
event. We chose to create a client to visualize the geographic distribution of tweets, but we could have implemented a different representation of the same data. If you want to experiment, use the streaming server as a base component to create your own client visualization. Here are some suggestions: