When a client issues requests to the server and the downstream application, we might potentially have a major design problem: how do we know in advance what kind of requests we might receive? If we had to re-implement a new set of standard requests every time we developed a web application, it would be difficult to reuse code and write generic services that other programs could call, since their requests would potentially have to change for every web application a client might interact with.
This is the problem solved by the HTTP standard, which describes a standard language and format in which requests are sent between servers and clients, allowing us to rely upon a common command syntax, which could be consumed by many different applications. While we could, in theory, issue some of these commands to our prediction service by pasting a URL into the address bar of our browser (such as GET, described below), this will only cover a subset of the kinds of requests we want to issue. The standard sorts of requests we typically implement in a web application are:
The GET requests only retrieve information, which will then be rendered in our web browser depending upon the kind of response. We could receive back an actual html page, or simply a piece of text. In order to specify what information we want to receive, a GET request will include variables in the URL in the form url?key1=value1&key2=value2. URL is the web address given to the prediction service, which in our example will just be the local machine, but could also be any valid IP address or URL. This URL is separated by a question mark (?) from the (key, value) pairs that define the parameters of our request for information. Multiple parameters may be specified: for example, we could indicate a pair of parameters for a user and item dataset using the string userid=12894&itemid=93819, with each key, value pair separated by the ampersand symbol (&).
We can directly issue a GET request by pasting the URL format described previously into the address bar of a browser or by issuing a curl command to the same address by typing the following into a terminal:
> curl <address>
We can also use the Python requests library (http://docs.python-requests.org/en/master/), which allows us to not worry about the details of formatting the URL. Using this library, the same GET request is called in the following way:
>>> r = requests.get(url,params)
Here, params is a dictionary of key-value pairs that we would have passed in the URL. The requests library performs this formatting for us, as we can see by printing the resulting URL:
>>> print(r.url)
Once we have issued the request, we can check the result using either of the following two commands:
>>> r.json() >>> r.text
We can also check the status code of the response to see if there was an error or not (see aside on standard response codes):
>>> r.status_code
Tip
Aside: HTTP Status Codes
When we issue a request to a web application using the methods discussed in this chapter, one way to check the success of the request is to examine the response code, which gives a standard number corresponding to the response of the web application to the request. You may have even seen these codes before without realizing it, such as the 404 error that is returned when a webpage cannot be displayed in your browser. The standard codes to be aware of are:
200: success, we usually check this value to make sure we received a correct response.
404: not found, indicating that the web application could not find the resource we requested.
500: server error, which we will often receive if the code run by our web application runs into problems.
For a more comprehensive list please, see (Nottingham, Mark, and Roy Fielding. "Additional HTTP Status Codes." (2012); Berners-Lee, Tim, Roy Fielding, and Henrik Frystyk. Hypertext transfer protocol--HTTP/1.0. No. RFC 1945. 1996).
Unlike the GET command, the POST request does not use data contained in the URL, but rather transmits information separate from the URL. If you have ever entered your credit card information in an online store, this information is probably transmitted using a POST request, which is fortunate since it then remains hidden. However, the fact that the information for the request is not contained in the URL means that we cannot simply paste the request into the address bar of our web browser: we would need a form on the webpage that issues the POST request or make the request programmatically ourselves. Without an actual form on a webpage, we can use a curl command to issue a POST request using the following syntax:
> curl –x POST -d <data> <url>
We can also use the Python requests library:
>>> r = requests.post(url,data)
In the preceding code, data is a Python dictionary of information that the web application can access in fulfilling the POST request.
Like the GET request, HEAD retrieves information, but instead of the body of the response (such as a webpage or JSON), it only retrieves metadata about the response (such as the encoding). We can issue a HEAD request using the following:
> curl –i –X HEAD <url>
Note that we have added the –i flag to this request; normally, the curl command will not print header information without this option. Using the Python requests library we would use the command:
>>> requests.head(url)
In cases where our web application has access to a database system, we issue PUT commands in order to store new information. Using curl, we make this request using the following:
> curl -X PUT -d key1=value1 -d key2=value2 <url>
We can also make this request using the requests library:
>>> r = requests.put(url,data)
Here, data is a dictionary of the arguments we wish to place in the applications storage system.
The opposite of the PUT command, DELETE requests are issued to remove a piece of data from the application's storage system. The curl command is as follows:
> curl -X DELETE -d key1=value1 -d key2=value2 <url>
While the same request using the requests library is as follows:
>>> r = requests.delete(url,data)
Here, data is a dictionary of the arguments we wish to remove from the applications storage system.
While there are other requests types available, we will not cover them in this discussion; for more details please see (Berners-Lee, Tim, Roy Fielding, and Henrik Frystyk. Hypertext transfer protocol--HTTP/1.0. No. RFC 1945. 1996). Note that since we can issue these requests using the Python request library, we can actually test our web application in the Python notebooks we have been using in the exercises in this volume.
For our purposes, the client will be the Jupyter notebook itself or the command line of the terminal; however, we could imagine other cases where the client is actually another web application that issues these commands and acts on the response. Again, since the server only needs to guarantee a particular message format rather than the details of the sender, either option is interchangeable.
Now that we know how to issue HTTP requests to our service, let us look at the server.