XML is a plaintext, human-readable, structured markup language upon which many document formats have been based. It employs a tagging structure in some ways similar to HTML, but is far stricter about formatting. For this reason, it is a popular online format to store structured datasets.

The XML package by Duncan Temple Lang provides a suite of R functionality based on the popular C-based libxml2 parser to read and write XML documents. It is the grandfather of XML parsing packages in R and is still widely used.

Recently, the xml2 package by Hadley Wickham has surfaced as an easier and more R-like interface to the libxml2 library. The rvest package, which was covered earlier in this chapter, utilizes xml2 behind the scenes to parse HTML. Moreover, rvest can be used to parse XML as well.

Because parsing XML is so closely related to parsing HTML, the exact syntax is not covered here. Please refer to these packages' documentation for examples.

Online applications communicate with one another using web-accessible functions known as Application Programming Interfaces (APIs). These interfaces act much like a typical website; they receive a request from a client via a particular URL and return a response. The difference is that a normal website returns HTML meant for display in a web browser, while an API typically returns data in a structured form meant for processing by a machine.

Though it is not uncommon to find XML-based APIs, perhaps the most common API data structure today is JavaScript Object Notation (JSON). Like XML, it is a standard, plaintext format, most often used for data structures and objects on the Web. The format has become popular recently due to its roots in browser-based JavaScript applications, but despite the pedigree, its utility is not limited to the Web. The ease in which JSON data structures can be understood by humans and parsed by machines makes it an appealing data structure for many types of projects.

JSON is based on a simple {key: value} format. The { } brackets denote a JSON object, and the key and value parameters denote a property of the object and the status of the property. An object can have any number of properties and the properties themselves may be objects. For example, a JSON object for this book might look something like this:

{
  "title": "Machine Learning with R",
  "author": "Brett Lantz",
  "publisher": {
     "name": "Packt Publishing",
     "url": "https://www.packtpub.com"
  },
  "topics": ["R", "machine learning", "data mining"],
  "MSRP": 54.99
}

This example illustrates the data types available to JSON: numeric, character, array (surrounded by [ and ] characters), and object. Not shown are the null and Boolean (true or false) values. The transmission of these types of objects from application to application and application to web browser, is what powers many of the most popular websites.

Public-facing APIs allow programs like R to systematically query websites to retrieve results in the JSON format, using packages like RCurl and httr. Though a full tutorial on using web APIs is worthy of a separate book, the basic process relies on only a couple of steps—it's the details that are tricky.

Suppose we wanted to query the Google Maps API to locate the latitude and longitude of the Eiffel Tower in France. We first need to review the Google Maps API documentation to determine the URL and parameters needed to make this query. We then supply this information to the httr package's GET() function, adding a list of query parameters in order to apply the search address:

> library(httr)
> map_search <-
    GET("https://maps.googleapis.com/maps/api/geocode/json",
          query = list(address = "Eiffel Tower"))

By typing the name of the resulting object, we can see some details about the request:

> map_search
Response [https://maps.googleapis.com/maps/api/geocode/json?address=Eiffel%20T ower]
  Status: 200
  Content-Type: application/json; charset=UTF-8
  Size: 2.34 kB
{
   "results" : [
      {
         "address_components" : [
            {
               "long_name" : "Eiffel Tower",
               "short_name" : "Eiffel Tower",
               "types" : [ "point_of_interest", "establishment" ]
            },
            {
...

To access the resulting JSON, which the httr package parsed automatically, we use the content() function. For brevity, only a handful of lines are shown here:

> content(map_search)
$results[[1]]$formatted_address
[1] "Eiffel Tower, Champ de Mars, 5 Avenue Anatole France, 75007 Paris, France"

$results[[1]]$geometry
$results[[1]]$geometry$location
$results[[1]]$geometry$location$lat
[1] 48.85837

$results[[1]]$geometry$location$lng
[1] 2.294481

To access these contents individually, simply refer to them using list syntax. The names are based on the JSON objects returned by the Google API. For instance, the entire set of results is in an object appropriately named results and each result is numbered. In this case, we will access the formatted address property of the first result, as well as the latitude and longitude:

> content(map_search)$results[[1]]$formatted_address
[1] "Eiffel Tower, Champ de Mars, 5 Avenue Anatole France, 75007 Paris, France"

> content(map_search)$results[[1]]$geometry$location$lat
[1] 48.85837

> content(map_search)$results[[1]]$geometry$location$lng
[1] 2.294481

These data elements could then be used in an R program as desired.

On the other hand, if you would like to do a conversion to and from the JSON format outside the httr package, there are a number of packages that add this functionality.

The rjson package by Alex Couture-Beil was one of the earliest packages to allow R data structures to be converted back and forth from the JSON format. The syntax is simple. After installing the rjson package, to convert from an R object to a JSON string, we use the toJSON() function. Notice that the quote characters have escaped using the " notation:

> library(rjson)
> ml_book <- list(book_title = "Machine Learning with R",
                  author = "Brett Lantz")
> toJSON(ml_book)
[1] "{"book_title":"Machine Learning with R",
"author":"Brett Lantz"}"

To convert a JSON string into an R object, use the fromJSON() function. Quotation marks in the string need to be escaped, as shown:

> ml_book_json <- "{
  "title": "Machine Learning with R",
  "author": "Brett Lantz",
  "publisher": {
    "name": "Packt Publishing",
    "url": "https://www.packtpub.com"
  },
  "topics": ["R", "machine learning", "data mining"],
  "MSRP": 54.99
}"

> ml_book_r <- fromJSON(ml_book_json)

This results in a list structure in a form much like the original JSON:

> str(ml_book_r)
List of 5
 $ title    : chr "Machine Learning with R"
 $ author   : chr "Brett Lantz"
 $ publisher:List of 2
  ..$ name: chr "Packt Publishing"
  ..$ url : chr "https://www.packtpub.com"
 $ topics   : chr [1:3] "R" "machine learning" "data mining"
 $ MSRP     : num 55

Recently, two new JSON packages have arrived on the scene. The first, RJSONIO, by Duncan Temple Lang was intended to be a faster and more extensible version of the rjson package, though they are now virtually identical. A second package, jsonlite, by Jeroen Ooms has quickly gained prominence as it creates data structures that are more consistent and R-like, especially while using data from web APIs. Which of these packages you use is a matter of preference; all three are virtually identical in practice as they each implement a fromJSON() and toJSON() function.