So far, we have covered different chart types that Highcharts offers as well as their combinations. We have also covered Highcharts APIs and its events that allow us to program interactive charts dynamically. In this chapter, we will cover different techniques to work with data that are essential when developing apps or websites with Highcharts.
When working with a large amount of data, it's not feasible to put all of it on the page itself. Instead, the data is retrieved from an external source such as a file, database, or an API. Before using this data with Highcharts, it's crucial to process it so that it can be plotted correctly. This process of fetching the data and formatting it correctly before it can be plotted is called preprocessing.
In the current section, we will take a look at preprocessing data from a number of file formats, including CSV, XML, and JSON. We will also look at JavaScript libraries that make working with these formats easier and also provide a handful of methods to achieve different tasks rather quickly.
Comma Separated Values or Character Separated Values (CSV) is a common format to store tabular data in plain text. It contains records, each in its own row. These records contain fields separated by a character, usually a comma. It is supported by a number of online and offline applications and is extremely useful when transferring tabular data between programs.
In the following example, we will plot the climate data of Chicago for July, 2014. This data will be presented in a CSV file with three header fields, that is, maximum, minimum, and average temperatures. Each record will correspond to a day in a month except the header record.
Since the parsing of CSV files is not natively supported in JavaScript, we will use an external library Papa Parse for this purpose. It's a wonderful library with lots of ground-breaking features such as multi-threaded processing, streaming support, support to parse a string or a file, and JSON-to-CSV conversion. It comes with a handful of useful methods and configuration options. You can find more about Papa Parse and its documentation at http://papaparse.com/.
You will need to download Papa Parse and include the papaparse.min.js
file from the downloaded package into your webpage.
The structure of our webpage stays the same as our previous example, except that we will include an input field of type file to load the CSV file. As soon as the file loads, our callback function will begin parsing the CSV file with Papa Parse and will pass the preprocessed data to the Highcharts configuration object.
The file input field is as follows:
<input type="file" id="csv_file">
The bare-bone chart configuration object is as follows:
var chartConfigOptions = { title: { text: 'Climate Data for Chicago - July 2014' }, subtitle: { text: '<a href="http://www.ncdc.noaa.gov" target="_blank">NCDC</a>', useHTML: true }, chart: { type: 'line' }, tooltip: { valueSuffix: '°F', useHTML: true }, plotOptions: { line: { pointStart: Date.UTC(2014, 06, 01, 00, 00, 00), pointInterval: 3600 * 1000 * 24 } }, xAxis: { title: { text: 'Date' }, type: 'datetime' }, yAxis: { title: { text: 'Temperature in °F', useHTML: true } }, series: [] };
This is a line
chart with xAxis
of type datetime
to support days of the month. The xAxis
component starts at July 1, 2014 and has pointInterval
of one day. The series
array has been left blank since it will be filled during data preprocessing.
Now, we capture the change
event on the file input field and bind a callback function to parse the data in the CSV file:
var file = ''; $( '#csv_file' ).on( 'change', function( e ) { file = e.target.files[0]; if ( $( '#climate_data' ).highcharts() ) { $( '#climate_data' ).highcharts().destroy(); } Papa.parse( file, { header: true, complete: function( results ) { for ( var i in results.meta.fields ) { var name = results.meta.fields[i]; chartConfigOptions.series[i] = {}; chartConfigOptions.series[i].name = name; chartConfigOptions.series[i].data = []; for ( var j in results.data ) { var currentDataPoint = results.data[j][name]; currentDataPoint = parseInt( currentDataPoint ); chartConfigOptions.series[i].data.push( currentDataPoint ); } } $( '#climate_data' ).highcharts( chartConfigOptions ); } }); });
We first initialize an empty variable file
for the file DOM object and then bind an event handler to the change
event on the file input field. Inside the event handler function, the file DOM object is captured with the help of event object e
.
In the next step, we check whether Highcharts has already been initialized for the same container so the previous chart is destroyed using the chart.destroy()
method.
In the Papa.parse()
method, we pass the file DOM object as the first argument and a configuration object containing the callback function as the second argument. Note the use of the header
property in the configuration object; this will treat the first row of the data that has been passed as field names. Now, each row will be an object of data, keyed by field names. The following are the first three rows of the CSV file:
Maximum,Minimum,Average 80,67,74 70,56,63 76,56,66
The following screenshot illustrates it as an object in the console:
In the next step, the results.meta.fields
array is looped through, each series is given its name, and an empty array for its data
is initialized. In an inner loop, the results.data
array is iterated and fields with the same header field are held into a currentDataPoint
variable. The currentDataPoint
variable holds the data point that is currently being iterated. The JavaScript method parseInt()
is called on the current data point to make sure that it's typecasted as an integer. Finally, the currentDataPoint
variable is pushed into the data
array of the current series. This process is repeated for each header field until all the data in the CSV file is processed. Finally, Highcharts is initialized on the container. The following chart is produced with the provided CSV file:
eXtensible Markup Language (XML) is another popular format to save data. It has a similar structure to HTML; in fact, HTML is based on XML. Due to this similar syntax and DOM structure, jQuery can be used to parse XML files.
In the following example, we will plot the same data as we used in the previous example. However, this time the data is saved in XML format. The XML file has data in the following format:
<chart> <series> <name>Maximum</name> <points>80,70,76,79,...</points> </series> <series> ... </series> <series> ... </series> </chart>
As there are no predefined tags in XML, we have defined our own tags to define the markup of our document. The chart
tag is the top-level tag that contains multiple series
tags. Each series
tag has a name
and points
tag. The name
tag contains the name of the series and the points
tag contains all the data points separated by a comma (,
).
The chartConfigOptions
object used to hold chart configuration options is the same as the previous example. The code to retrieve the XML file and then parsing it is as follows:
$.get( 'climate-july-14.xml', function( data ) { var xml = $( data ); xml.find( 'series' ).each(function( i ) { var $this = $( this ); chartConfigOptions.series[i] = {}; chartConfigOptions.series[i].data = []; chartConfigOptions.series[i].name = $this.find( 'name' ).text(); var points = $this.find( 'points' )[0].innerHTML.split( ',' ); for ( var j in points ) { chartConfigOptions.series[i].data.push( parseInt( points[j] ) ); } }); if ( $( '#climate_data' ).highcharts() ) { $( '#climate_data' ).highcharts().destroy(); } $( '#climate_data' ).highcharts( chartConfigOptions ); });
We first retrieve the XML file using the jQuery's $.get()
method. The file contents are passed as an argument to the callback function that we wrap inside the jQuery object and assign to a variable named xml
. We then iterate through each series element inside and create a new empty series object at index i
of the chartConfigOptions.series
array. At the same time, we also assign an empty data
array to this object and give a value to the name
property of the series.
In the next step, we find the points
element and split its value using the comma (,
) delimiter. This splitting results in an array that we call points
. We then iterate over the values of points
and push them into the data
array of the current series.
Before loading the chart, we check and destroy any previously loaded chart into the container. Finally, we load the current chart with new data into the container.
Since the data is the same for the current and the previous example, the charts produced are also identical.
Parsing a JavaScript Object Notation (JSON) file in JavaScript is relatively easy since it's natively a JavaScript object. We can iterate through its properties just as we iterate through a normal JavaScript object.
In the following example, we will be using the same data as the previous two examples and plotting a line chart by preprocessing data from a JSON file.
The JSON file has data in the following format:
{ "series": [ { "name": "Maximum", "data": [80,70,76,79,...] }, { ... }, { ... } ] }
The series
array is wrapped inside a top-level object and it contains multiple objects with name
and data
properties. The name
property is a string, while the data
property is an array that contains data points.
The code to retrieve the JSON file and then plot the chart is as follows:
$.getJSON( 'climate-july-14.json', function( data ) { chartConfigOptions.series = data.series; if ( $( '#climate_data' ).highcharts() ) { $( '#climate_data' ).highcharts().destroy(); } $( '#climate_data' ).highcharts( chartConfigOptions ); });
We first get the JSON file using the jQuery's $.getJSON()
method that accepts the path of JSON file as the first argument. The parsed data is then passed to a callback function as an argument. Since the structure of data.series
is the same as required for series
in Highcharts' configuration object, we assigned data.series
to chartConfigOptions.series
.
In the next step, we destroyed any previously loaded chart in the container and initialized a new chart by passing the chartConfigOptions
object.
Referring to the preceding steps in which we initialized a chart by loading the data from a JSON file, it becomes clear that the JSON format requires the most minimal steps for data preprocessing.
In this section, we learned to preprocess data contained in static files. In the next section, we will learn to retrieve and preprocess data from a database using a server-side programming language.