CHAPTER

5

Web Remoting Techniques—
the A in Ajax

Chapter Objectives

• Discuss Web Remoting

• Introduce the XMLHttpRequest object and how to make asynchronous requests

• Discuss the hidden IFrame technique and give an example

• Describe the HTTP Streaming technique

5.1 Web Remoting

 

Web Remoting is a term used to categorize the technique of using JavaScript to directly make an HTTP request to the server and process the response. By tradition, web browsers initiate HTTP requests, not scripts. The classical function of a web browser is based on a synchronous request-response model where HTTP requests are made in response to the user clicking links, submitting a form, or typing in a URL (Figure 5.1.1). The browser processes the request by discarding the current page (including any running scripts), downloading a new page from the server, and then loading the new page in the browser for the user to view. This is a time-consuming process, especially when you do not need to display a whole new page. Often you need to send only a small amount of information to the server or to update a portion of the current page with a little data from the server.

To work around the limitations of this page-driven paradigm, web developers have developed techniques to exploit URL-supporting HTML tags and CSS properties in nontraditional ways. Every tag with an src or href attribute that can be used without reloading the entire page is a candidate, including <img>, <script>, <link>, <frame>, and <iframe>. When a script sets the src or href property to a URL, the browser performs an HTTP GET request to download the content of the URL, without reloading the page. You can exploit this fact to send information to the server by encoding that information in the URL query string. The server must respond with content expected by the tag to avoid generating an error, like an image or style sheet, but the content can be unused, hidden, or insignificant such that it does not noticeably change the page (like a transparent 1 × 1 pixel image). If you cannot use the response content type, you can also have the server set a cookie in the response with information you need.¹

Figure 5.1.1

Classical Synchronous HTTP Request-Response Model

One of the most popular techniques (used by Google Maps) is to use an <iframe> or <frame> tag that is hidden using CSS. You encode information to send to the server in a URL and set that URL in the frame's src attribute. The server responds with a full HTML page, which can contain much information for you to use on the main visible page, including scripts that perform some task. Using the <script> tag is also popular because the response, being JavaScript, is automatically executed by the browser. Dynamically creating <script> tags for web remoting is commonly called On-Demand JavaScript. This approach is used in some JavaScript toolkits for lazily loading scripts.²

Although these techniques that use HTML tags (also called remote scripting) are often useful, they are, strictly speaking, a misuse of their intended design and can therefore sometimes be complicated to get working properly and uniformly across browsers. Fortunately, another technique exists that was designed with Web Remoting in mind. The XMLHttpRequest object provides full support for making asynchronous HTTP requests, including HEAD, POST, and GET, and can, despite its name, handle responses in plain text, XML, or HTML. XMLHttpRequest and a few of the alternative techniques are discussed in more detail in the rest of this chapter.

5.2 XMLHttpRequest

 

The real driver behind Ajax fame is the XMLHttpRequest object. XMLHttpRequest allows making background asynchronous requests to the server and receiving responses. This allows the web application to process the user's input and respond with data from the server without having to make the user wait for an entire page to load. Proper use of XMLHttpRequest makes web applications feel much more responsive than traditional web applications—more like desktop applications. Let us assume that you are developing a web application that allows users to search for a specific model of car being sold in their area. You might have a page with a form like Figure 5.2.1 that allows users to select the year, make, and model of the car that they are looking for, as well as enter a ZIP code to narrow their search.

Figure 5.2.1

HTML Car Search Form

You do not want to have to embed all the possible makes and models on the page because doing so would make the page much larger and thus slower to load. You would also like to check that the year and ZIP code are valid before the user is allowed to submit the form, but you would like to centralize the validation logic on the server and you would prefer to validate each value after it is entered instead of waiting until the entire form is submitted. You could write JavaScript code to validate the user input directly in the browser. However, best practices for web development dictate that if you validate user input on the client, you should also validate that data on the server to be certain that the data was not changed. This means that you would have to write the validation logic in two different languages: JavaScript for the client and the language you are using on your server. Having two copies of validation logic written in two different languages increases your development time and maintenance.

Figure 5.2.2

Form Processing without XMLHttpRequest

In essence, you would like your form to feel more like a desktop application without having to put all the data and logic on the client. To accomplish this, you need to attach an event handler to each field that is triggered after the user populates and leaves the field. Before XMLHttpRequest, the event handler would have to submit the entire form, which would be followed by a reload of the entire page on response. Figure 5.2.2 illustrates how the sequence of events might look. In step 1 the user enters the URL of your website into the browser, and the browser loads the page with the form for the first time. In step 2 the user selects a make from the list, and the event handler is triggered to submit the form to the server to look up associated models. The server responds with a whole new page including the form, the selected make, and the associated models. In step 3 the user has populated the ZIP code and left that field, which triggered an event handler that submitted the form for validation. The server determines that the ZIP is invalid and responds with an entire new page including the form, the selected and populated fields, and a validation error message. With this model, the user must wait for the entire page to be reloaded after every field is populated. Unless the page is really small and the network is really fast, the user is likely to get tired of waiting on your site and leave.

Figure 5.2.3

Form Processing with XML-HttpRequest

You do not want the user to leave your site without finding a car to buy, so let us consider the alternatives. Before XMLHttpRequest, the typical hack was to use a hidden <iframe> to make the request and grab data from the response page that loaded in the <iframe> to update the main page. With XMLHttpRequest, a better alternative exists. Figure 5.2.3 illustrates how the sequence of events might look using XMLHttpRe-quest. Step 1 is the same as before—the user navigates to your website. In step 2, after the user selects a make, the event handler uses an XML-HttpRequest to send only the selected make to the server, which responds with only the list of associated models, which are then used to update the display. In step 3, after the user populates the ZIP code field, the event handler uses an XMLHttpRequest to send only the entered ZIP to the server, which responds with only a validation error message that is, in turn, displayed to the user. The response is much faster because the client and server are sending very little information back and forth, and the browser does not have to rerender the entire page just to show things like an enabled <select> with a list of models or a short validation message. Also, because the request/response processing is handled in a background thread, the user interface (UI) does not lock, allowing the user to continue populating the form (or whatever else) without having to wait.

Microsoft was the first to implement this functionality in Internet Explorer version 5.0, released in March 1999, in response to a desire by developers of Web Access 2000. Microsoft first implemented the object as XMLHTTP, an ActiveX component in an XML library called MSXML. XMLHTTP quickly became popular among IE developers, and in response, Mozilla ported the idea to their Mozilla 1.0 browser, released in May 2002. Mozilla decided to call their object XMLHttpRequest³ and to make it native to the browser's JavaScript interpreter. Apple followed later with a native implementation in Safari 1.2 (February 2004), and Opera added a native implementation in version 8.0 (April 2005), both naming their objects XMLHttpRequest. Eventually, with Internet Explorer 7 (October 2006), Microsoft implemented a native replacement to XMLHTTP, which they also called XMLHttpRequest.⁴

XMLHttpRequest has not yet become a W3C standard; however, W3C has published a working draft of an XMLHttpRequest Specification⁵ to document the commonalities among the browser vendor implementations.Also, the W3C has applied some of the ideas behind XMLHttpRequest to the DOM Level 3 Load and Save Specification,⁶ which has since become a W3C Recommendation. Overall, modern browsers have little to no support for DOM Level 3 Load and Save. On the contrary, the implementation of XML-HttpRequest in modern browsers is mostly universal—Firefox, Netscape, Opera, iCab, Safari, Konqueror, and Internet Explorer all have support.

Using XMLHttpRequest is essentially a three-step process.

 

1. Create an XMLHttpRequest object.

2. Configure and submit your HTTP request to the server.

3. Process the server's response.

The following sections describe these steps.

5.2.1 Creating an XMLHttpRequest Object

To use XMLHttpRequest, you must first create an instance of the object. Because Microsoft's implementation before IE 7 was an ActiveX object, you need to use some conditional logic. You first check whether the native XMLHttpRequest object exists and instantiate it; otherwise, you assume that you are in an IE browser before IE 7 and instantiate an ActiveX object. If you are unsuccessful in creating either object, then you assume that you are in a browser that does not support XMLHttpRequest, and you either disable the Ajax functionality or you use another technique, like a hidden <iframe> discussed in Section 5.3. Listing 5.2.1 provides a cross-browser object that you can use to instantiate an XMLHttpRequest object. We will add more useful functions to this object in the rest of this chapter.

In Listing 5.2.1 we define and execute a function in one statement to create and return an object that we call RequestHelper. The returned object defines the public functions for RequestHelper, whereas all the logic before the return statement defines private data that can be referenced only by the returned object. The private section initializes RequestHelper by determining if we can use the native XMLHttpRequest object or we are in an IE browser before version 7 and must use the XMLHTTP ActiveX object.

Because several versions of Microsoft's MSXML library are in existence and multiple versions can be installed on the same computer, we must determine which version is available. The msXmlVersions array contains ProgIDs that identify an MSXML version. This ProgID must be passed to the ActiveXObject constructor. If the browser does not support the ProgID, the ActiveXObject constructor will throw an error. The only way to find out which version is supported is to try each one, from the most desired to the least desired. The only versions of MSXML that Microsoft suggests using are 3.0 and 6.0.⁷ Version 6.0 is preferred, and 3.0 is a fallback for older systems. All other versions have problems with security, stability, or availability. Windows Vista ships with MSXML 6.0, which can also be installed on Windows 2000, Windows Server 2003, and Windows XP.⁸

Listing 5.2.1

RequestHelper. createXHR() Method

5.2.2 Sending an HTTP Request

Fortunately, once you have created an instance of XMLHttpRequest, the API for using it is the same in all browsers. First let us outline the properties and methods of XMLHttpRequest and then we will dive into a code example.

5.2.2.1 XMLHttpRequest methods

void abort()

Aborts the request. This can be used to time out requests that take too long.

DOMString getAllResponseHeaders()

Returns all the HTTP response headers in one string. The header name-value pairs are separated by a colon (“:”), and each header is separated by a carriage return line feed (“ ”).

DOMString getResponseHeader(DOMString header)

Returns the HTTP response header value for the given header name.

void open(DOMString method, DOMString url, boolean async, DOMString user, DOMString password)

Initializes the request, using two required parameters and three optional ones, which are described next.

method (required): A string that defines the HTTP method⁹ of the request, which in essence describes the operation to be performed on the resource identified by the URL of the request. Typical HTTP methods are HEAD, GET, and POST. The HEAD method tells the web server to return just the HTTP headers associated with the URL, so you could, for instance, check the last modified date of a document without downloading it. Most requests use the GET method to download the content of the URL. The POST method is used mostly by HTML forms to submit data to the web server as name-value pairs in the HTTP request's entity body. To improve cross-browser compatibility, the method string should be all-capital letters (i.e., “GET” instead of “get”).

url (required): A string defining the target URL of the request.

async (optional): A Boolean value indicating whether the request should be made asynchronously. By default the value is true, which processes the request asynchronously. There is practically no reason to set this value to false to process the request synchronously. Doing so will block the browser UI until the response is returned and processed.

user (optional): A string identifying a user for a web server that requires authentication.

password (optional): A string identifying the user's password for a web server that requires authentication.

void send(Object data)

Sends the request to the server. If the request was configured to be asynchronous (async parameter of the open() method was set to true) then this method returns immediately; otherwise, it blocks until the response is received. The optional parameter is sent as part of the request body. The parameter can be a DOM Document object, an input stream, or a string. The parameter is usually used only when sending a POST request with form data. The form name-value pairs are concatenated into the form of a query string and passed to the send() method. The behavior of this method is somewhat unusual for JavaScript in that if you choose not to use the optional parameter, you must explicitly pass null.

void setRequestHeader(DOMString header, DOMString value)

Sets the specified HTTP request header to the given value. Probably the most common use of this method is to set the Content-Type header to the value “application/x-www-form-urlencoded” when sending a form with a POST request, which is required in that case.

5.2.2.2 XMLHttpRequest properties

onreadystatechange

An event handler that is invoked as the request changes state. An XMLHttpRequest object progresses through five states as the request is being processed, which are described next.

readyState

An integer value that is set to the current state of the request. The browser changes the readyState of an XMLHttpRequest object as the request is being processed. When this value changes the onreadystatechange event handler is called, allowing you to react to the change in state. The possible values are listed. Usually, the only state you will care about is 4 Completed. Unfortunately, browsers handle the readyState change differently. Some browsers do not set the 1 or 2 state at all, some repeatedly set the 3 state while the response is being downloaded, and some set the 3 state only once. Consequently, the best cross-platform use of readyState is to wait for 4 Completed and ignore all the other states.

0 Uninitialized: The open() method has not been called yet.

1 Loading: The open() method has been called, but the send() method has not.

2 Loaded: The send() method has been called, but the server has not responded yet.

3 Interactive: A partial response from the server has been received.

4 Completed: The complete response has been received and the connection is closed.

responseText

The body of the server's response as a string of text. The server can respond to the request with plain text, HTML, XML, or any other format. As long as the request was successful, this value will be set to the response data in string form no matter what format came back.

responseXML

If the response is an XML document (and the response Content-Type header is set to “text/xml”), then the XML will be parsed into a DOM Document object and assigned to this property. This saves you time because you do not have to parse the XML yourself.

status

An integer value containing the HTTP status code of the response. Some of the most common status codes are 200 OK (meaning that the request was successful), 302 Found (indicates that the resource has moved to another URL, called a redirect), 304 Not Modified (usually means that the response was pulled from the browser cache instead of being downloaded from the server), 404 Not Found (indicates that the resource was not found), and 500 Internal Server Error.

statusText

A string value containing the text description of the status code (“OK” for 200, “Found” for 302, etc.).

Using an XMLHttpRequest object is a multistep process. You first initialize the object to specify what URL you want to request and how you want to use the object; then you call the send() method to actually make the request; and finally, you process the response. Initialization of the XML-HttpRequest involves the following steps, which can occur in any order.

 

• Call the open() method to specify the URL and whether you want to make a synchronous or asynchronous request.

• Call the setRequestHeader() method to specify any HTTP headers to be sent with the request.

• Set the onreadystatechange property to a function that will be called as the state of the request changes (and process the response when the readyState is 4 Completed).

To demonstrate some code, we build on our Car Search example illustrated by the form in Figure 5.2.1. We already discussed how we want to attach an event handler to the car make drop-down list that downloads the associated models from the server and assigns them to the car model drop-down list. Listing 5.2.2 demonstrates how you might use XMLHttpRequest to handle fetching the car models from the server after the user selects a car make.

In Listing 5.2.2 we define an event handler called fetchModels that takes a car make and sets up an XMLHttpRequest to download the associated car models from the server. The fetchModels function starts by creating an XMLHttpRequest object by using the RequestHelper that was defined in Listing 5.2.1. Then the onreadystatechange callback function is defined. The onreadystatechange callback function is called whenever the readyState of the XMLHttpRequest changes. No parameters are passed to the callback function, but you need to use the XMLHttpRequest object to get the response data, so you must maintain a reference to the XMLHttpRequest object or use a closure, as we have done here. Inside the callback function we must check to see if the state is 4 Completed before we process the response because the response may not yet be complete. We must also check that we got a successful HTTP status header because the server may have encountered an error, in which case the responseText property would be empty. After we are certain that the response is complete and successful, we process the response. In this example we show how you might handle a response that consists of one string of HTML <option> tags. Here response-Text returns a string of car model options like this.

We can apply these options directly to the body of the model <select> tag thanks to the innerHTML property.

All major browsers provide a nonstandard property on DOM objects called innerHTML. The string that you assign to innerHTML is parsed into DOM objects and used to replace the body content of the tag. The change is immediately reflected in the page display. Although technically proprietary, and therefore not recommended by everyone, the innerHTML property is compatible across major browsers, is easy to use, and is generally processed faster than creating DOM objects manually by using the standard methods like createElement() and createTextNode().¹⁰ We will cover processing responses in more detail later. For now let us continue dissecting Listing 5.2.2.

Listing 5.2.2

Fetching Car Models from the Server with XMLHttpRequest

The last part of the fetchModels function in Listing 5.2.2 calls the XML-HttpRequest open() method to declare the URL that we want to request. We pass the HTTP method, the URL, and true to say that we want an asynchronous request.

Finally, we call send() to actually make the request. The send() method returns immediately because we configured XMLHttpRequest to send an asynchronous request. The next step is to wait until the onreadystatechange callback function is invoked to notify that the request is complete. If we had passed a false to the open() method for the asynchronous parameter (the third parameter), then the send() method would have blocked until the request was complete. When making synchronous requests using XML-HttpRequest you do not have to specify an onreadystatechange event handler because you know that the response is complete when the send() method returns. However, there is not much reason to make a synchronous request because setting up an asynchronous request is simple, and making asynchronous requests improves the responsiveness of the application.

request.send(null);

The last part of Listing 5.2.2 handles setting up an event handler to respond to the user's selecting a car make from the drop-down list. The init() method assigns an onchange event handler to the car make <select> tag, which, upon being invoked, gets the currently selected car make and passes it to the fetchModels() method. In turn, the fetchModels() method makes the XMLHttpRequest to download the associated models and update the display. If the user deselects a car make, the onchange event handler resets the car model <select> tag to no selection and disables it. The final line of Listing 5.2.2 assigns the init() method as the onload event handler of the window object. Therefore, immediately after the page is finished loading (before the user has a chance to use the form), the init() method is called to set up the onchange event handler. The interaction for this example is illustrated in Figure 5.2.4.

The figure on page 192 illustrates what the car model drop-down list might look like after the user selects “Jeep” from the car make drop-down list and the server returns the associated models. The full HTML, CSS, and server-side code for an example is given at the end of this chapter. For now we want to focus only on the client-side pieces that interact with XMLHttpRequest.

Figure 5.2.4

Typical Ajax Interaction

 

5.2.2.3 Sending a POST Request

You must pass null to the send() method if you have no data to send. However, if you want to send some data to the server in the body of the request, you can pass it to the send() method. For example, suppose that you want to send a POST request with the fields from the car search form. The first thing that you must do is concatenate the form field name-value pairs into a URL-encoded string that you will pass to the send() method. URL encoding (also called percent encoding¹¹.is the process of replacing characters that are not allowed in a URI with a code that consists of a percent symbol (%) followed by a pair of hexadecimal digits. For example, a dollar sign ($) is encoded as %24. The following is the list of reserved characters and their encoded values.

In addition to the preceding list of reserved characters, spaces are encoded as %20. When data in a form is sent to the server, it must be encoded in the format defined by the application/x-www-form-urlencoded MIME (Multipurpose Internet Mail Extensions) type. This format is the same as URL encoding except that spaces are encoded as + instead of %20. To handle this encoding, we add a method to our RequestHelper that takes a form object and returns its field's name-value pairs in an encoded string (Listing 5.2.3).

Now we can use RequestHelper.encodeForm() to send the entire car search form to the server by using XMLHttpRequest. We just have to set up the request for a POST, set the proper MIME type header, encode the form data, and pass it to the XMLHttpRequest send() method.

Listing 5.2.3

RequestHelper.encode Form() Method

5.2.3 Processing the Response

The type of data sent by the server in the HTTP response is identified by the Content-Type header, which specifies a MIME type.¹² In theory, you can respond to an XMLHttpRequest in any format that the browser supports, but the XMLHttpRequest object has special handling only for XML data. The browser will automatically parse a response that is well-formed XML with the Content-Type header set to “text/xml”, “application/xml”, or any other MIME type that ends with “+xml” (like “application/xhtml+xml”). If the XML data is not well formed or the Content-Type header is not set properly, then the XMLHttpRequest responseXML property will not contain a DOM Document object.

If the response does not contain XML, then the browser assigns the response body to the XMLHttpRequest responseText property and you must parse the data, if needed. The most common response formats used in Ajax development are XML (no surprise there), HTML (“text/html”), plain text (“text/plain”), JavaScript (“text/javascript”), and JavaScript Object Notation (“application/json”). We cover how to process these different response formats in the following sections.

5.2.3.1 HTML Response

An example using HTML as the response format was already given in Section 5.2.2, where we used the nonstandard innerHTML property. In that example, the response from the server was a string of HTML <option> tags. Using innerHTML made processing that response easy because all we had to do was assign the string of options to the innerHTML property of the <select> tag and the browser did the work of parsing the string into DOM objects.

The innerHTML property has the advantages of being easy to use with fast processing by the browser. One disadvantage to using innerHTML is that it is not a standard property (yet—it is being added to the HTML 5 Specifica-tion¹³). If you follow best practices, you typically try to avoid nonstandard browser features because nonstandard features are less likely to be supported by, and have uniform behavior across, all browsers. One could argue that another drawback to using innerHTML is that the user is allowed to assign HTML that is not well formed, which can mess up the DOM tree. For instance, your HTML string may have a missing closing tag.

Even though innerHTML has some drawbacks, its use is appealing. Using HTML as the response format is natural for web developers and easy for most to understand. If you want to use HTML as the response format but do not want to use innerHTML to update the DOM tree, your work becomes more difficult. You must parse the string that you get from responseText yourself and use the DOM methods to create Nodes and append them to the DOM tree. For an example, we will use the same context that we used in the innerHTML example in Section 5.2.2. You have set up an onchange event handler for the car make drop-down list to create an XMLHttpRequest object and use it to send a GET request to download the car models for the selected make. Now you have a string of <option> tags from the server's response, and you need to use them to update the car model drop-down list. You need to convert the string of HTML tags to DOM nodes. So the first thing that you must do is write code to parse the string. In Listing 5.2.4 we have added a method to RequestHelper called parseHTML() that will parse a given HTML string into DOM nodes, add them to a DocumentFragment,¹⁴ and return the DocumentFragment. When a DocumentFragment is added to a Node, the children of the DocumentFragment are appended to the Node instead of the DocumentFragment itself. Thus, DocumentFragment works as a convenient container of the Nodes we create from the tags in the HTML string. This code is missing some error handling that would be needed to make it robust enough to be used on a production website. The code is fairly complex as it is, and we want to keep it as simple as we can for this example. Also, this example uses regular expressions to parse the string. This is not a book on regular expressions, so they will not be explained, but the comments describe what each one does. You can find some good resources onlin¹⁴ if you would like to learn more about regular expressions.

Now that you have a way of parsing the text, you need to add code to take the result and apply it to the car model select tag. To accomplish this, you would have to use the code on page 199 in place of the code that applies the response string to innerHTML in the onreadystatechange event handler of Listing 5.2.2.

Listing 5.2.4

RequestHelper.parse-HTML() Method

5.2.3.2 XML Response

The X in the acronym Ajax stands for XML, and “XML” is in the name of the client-side object that made Ajax so popular, XMLHttpRequest. So, naturally, most people think that Ajax development must involve the use of XML. As we have already demonstrated, that is not true. However, XML is a versatile and well-supported choice for your response format. Practically all server-side languages have support for processing XML, and the browser will automatically parse an XML response into DOM objects and make the data available via XMLHttpRequest's responseXML property. Using XML, you have precise control over how your response data looks. On the client side, you can use various parts of your XML response to update various parts of the HTML. For example, using our car search form example, when the user selects a car make, the server could also validate the year field and respond with both a validation error message and a list of car models. We do not have the option to update two separate DOM elements when we use HTML as the response format and we set the innerHTML property. Technically, we could parse the HTML response ourselves and use the result to update multiple DOM objects, but we have already seen how difficult the parsing logic is to write in JavaScript.

The main drawback to using XML is that the client-side code that uses the DOM API to update parts of the loaded document can become verbose. You will see a little of that in this example, in which we will use the scenario that we just described. When the user selects the car make, the server will respond with both a validation error message for the year field and a list of associated car models. To accomplish this, we will need to modify the code of Listing 5.2.2 to send the server both the selected car make and the entered year and then use the response to put a validation error message beside the year field and update the car model selection with the list of models. Let us assume that the user enters something into the year field that does not look like a year—he has entered “200u” (a typo) in the year field and then selected “Jeep” from the car make drop-down list. The server then responds with the following HTTP message, which includes XML and the necessary Content-Type header. The browser will parse an XML response and assign it to the XMLHttpRequest responseXML property only if the Content-Type HTTP header is set to “text/xml”, “application/xml”, or a type suffixed with “+xml”. If you forget to set the correct Content-Type header on the server, then when you go to use responseXML you will find that it is empty.

To send the request and process this XML response, we make the changes in Listing 5.2.5 to Listing 5.2.2. The largest change is to the onreadystatechange event handler, which now uses the responseXML property of XMLHttpRe-quest and makes a call to a new displayModels() function. The code that used innerHTML is removed. Now we have code within displayModels() that first gets the parsed error node and assigns the content to a span tag, which it inserts after the year field to display the validation error message. Then the code gets the parsed model nodes, creates options from them, and adds the new options to the car model drop-down list. At the end of the fetch-Models() method we modified the code to add the year field value to the URL query string so that the year gets sent to the server along with the selected make.

Notice how much code is required to process the XML response and get the data into the page. Although XML is flexible, it can be verbose to use the DOM API. The figure on page 203 illustrates what the web page would look like after processing the response. As before, a complete HTML, CSS, and server-side code is left to the end of this chapter so that we can focus on how to use XMLHttpRequest.

Listing 5.2.5

Using an XML Response Format

5.2.2.3 Plain-Text Response

When you want to do something as simple as have the server validate a field and respond with an error message if the validation failed, the easiest response format is just plain text. Say that we wanted to validate the year field of our car search form immediately after the user leaves the field instead of waiting until a car make was selected. To accomplish this, we could add an onchange event handler to the year field that sent the entered year to the server to get validated. All that the server would need to send back is a pass message or a fail message. The pass message could be as simple as “OK”.

The fail message could be as simple as “*Must be a 4-digit year”.

That is it. No HTML or XML. We could add an empty <span> tag beside the year field that we stuff the fail message into when it comes back. The <span> tag just displays its contents, so when it is empty, it does not display anything. The modification to the HTML to add the span tag looks like this.

The error CSS class is for applying any style to the message, like red-colored text. The id attribute is so that we can easily look it up. Below is the JavaScript code, which is simple. We define a validateField() function that submits a request to the server to validate the year field. If the response message is not “OK”, it puts the value of responseText in the span tag. Using the init() function from earlier, we attach an onchange event handler to the year text field that calls the validateField() function.

5.2.2.4 JavaScript and JSON Responses

Another popular response format is JavaScript. The text returned from the server and made available via the responseText property of XMLHttpRe-quest can be JavaScript code. As a string, the code does not do us much good, but fortunately JavaScript provides a built-in eval() function that will take a string, interpret it as JavaScript, and return the result. The code defined in the string can reference existing variables and objects and can contain any JavaScript construct including statements, expressions, function declarations, and object declarations. For a simple example, we can modify the code in the previous section so that the response from the server contains the JavaScript code to update the <span> tag with the validation error message. If the year field validated successfully, then the server can just return an empty string. If the year field failed validation, then the server can return the following as one string.

Now we can replace the logic in the validateField() function that puts the responseText in the <span> tag with one line that evaluates the responseText.

This technique is powerful. You could return an entire JavaScript program that rewrites the entire page! But doing so would be a poor development practice. If fact, you should avoid executing statements by putting them in a string and passing it to eval(). Doing so opens up some potential security risks because the eval() function will execute anything passed to it, including malicious code from a hacker. It also makes the application more difficult to analyze and maintain. You can accomplish everything you need by loading JavaScript via <script> tags.

That being said, there is an “acceptable” use of eval() in Ajax development that has gained much popularity. Instead of passing code logic statements back to the client as a string, you use JavaScript syntax to pass data. This technique was made popular by Douglas Crockford,¹⁶ who formalized the idea into a specification called JavaScript Object Notation (JSON).¹⁷ JSON is defined as a lightweight data-interchange format that is based on a subset of the JavaScript language, namely, JavaScript object and array literals. JavaScript supports defining objects and arrays by using a simple syntax called literal notation. JavaScript literal notation was described in the last chapter. Instead of defining your data in XML, for instance, you could define it in JSON. For example, we could take the XML response given in Section 5.2.3.2 and rewrite it in JSON format as follows.

You can see from this example that JSON is much less verbose than XML, which explains why it is touted as the “fat-free alternative to XML.” An object in JSON is declared as a set of name-value pairs enclosed in braces, {}. Each name in the object is a string followed by a colon (:) and the name-value pairs are separated by a comma (,). The value can be a string, number, object, array, true, false, or null. In the preceding example we have defined three objects, one outer object that encapsulates the response, and two nested objects, one for the error message and another for the car make. An array in JSON is an ordered collection of values enclosed in brackets, []. Each value in the array is separated by a comma (,). In the preceding example, the list of models is defined as an array. We can convert this JSON string to an object by using the eval() method as follows. Assume that we have already assigned the preceding JSON string to a variable called jsonRespStr.

You must surround the JSON string with parentheses, (), or the JavaScript interpreter will not know that your curly braces represent an object. Curly braces are used in other constructs in JavaScript, like function definitions and if/else statements; therefore, the interpreter needs to know what context the pair of curly braces is being used in to determine what they represent. Normally, object literals are preceded by an equal sign (when being assigned to a variable) or a return statement, or they are defined inside an array literal, or they are surrounded by parentheses when being passed to a function. Now that you have been given the simple idea behind JSON, we can create an example to see JSON in use. We can easily rewrite our XML example in Listing 5.2.5 to use JSON, as Listing 5.2.6 illustrates. Now, instead of the response being parsed into DOM Node objects, the data is parsed into a simple JavaScript object, and we get to the data we need by referencing the object's fields.

Once the JSON string is converted to an object, we can use it like we do any other object in JavaScript. Besides the fact that JSON is less verbose than XML and takes up less bandwidth when it is transferred across the Internet, it is also much less awkward (more natural) to work with in JavaScript than using the DOM API. As you can see, the code that references the JSON object is clean, succinct, and easy to understand. However, a drawback to JSON compared with XML is that the syntax can be more difficult for some to read. The XML syntax is verbose because it is designed to be easy to read and clearly identify the data. With JSON, it is fairly easy to miss a brace or comma and create an error. However, JSON encoders have been written that can be used to automatically convert data in your code to a JSON string.

As we mentioned earlier, passing a string to eval() has some security implications. The JSON specification just deals with data, but there is nothing to stop you from defining a function in the JSON string—it will work even though it is not part of the JSON spec. To close this security hole, Douglas Crockford (and others) have written JSON parsers that recognize only JSON syntax and reject text that contains anything else, like functions and assignments. Crockford's parser is simply called JSON.¹⁸ Using the JSON parser, we could replace the line using eval() with the following. If the JSON string contains something like a function, the parser throws an error.

Listing 5.2.6

Using a JSON Response Format

Constructs like a collection of name-value pairs and an ordered list of values are present in pretty much all programming languages. Therefore, JSON can be used to send data not only from the server to the client but also from the client to the server. All you need is a processor on the server to convert your JSON data into the programming language of your choice. Fortunately, processors for many different languages already exist to convert your data both to and from JSON. Crockford has a comprehensive list on his website.¹⁹ For example, Crockford's JSON parser also has a method to convert a JavaScript object to a JSON string, which you can then send to the server. For example, we could create a JavaScript object containing the selected car make and the entered year and then use JSON to convert it to a JSON string like this.

5.2.4 Timing Out a Request

A deficiency of XMLHttpRequest is the lack of a timeout mechanism. If you make a synchronous request and the web server fails to respond, the browser will lock up, remaining blocked in the send() method. If you send an asynchronous request you will not have this problem because the send() method does not block, allowing the user to continue using your application and the browser. However, it is common to disable a widget or feature of the web application that depends on the asynchronous request's completing. For example, you may disable a form submit button until the response is processed to prevent the user from submitting the same form multiple times. If you use this technique, the form button will remain disabled until the request is finished, which may never happen. Also, it is a good practice to inform the user if something did not work, such as if the user submitted a form and never received a response.

Fortunately, there is a simple solution to this problem. The XMLHttpRe-quest object does provide the abort() method to cancel a request, so all we need to do is wait for a specified time, and if the request has not been responded to, call abort(). We do this by defining a function that calls abort() on the XMLHttpRequest object and passing the function to window.setTimeout() with a length of time to wait before executing the function. If the request completes before the time is up, you call window.clearTimeout() to cancel the timeout. Listing 5.2.7 is an example based on Listing 5.2.6, the JSON example from the previous section.

The setTimeout() function creates a timer that waits a given number of milliseconds before invoking the given function. You can pass it a function reference or a function definition, as we have done here. To handle timing out the request, we pass setTimeout() a function that simply calls abort() on the XMLHttpRequest object and displays an informing message to the user. The setTimeout() function returns an Integer that identifies the timer and can be used later to cancel it. In the onreadystatechange event handler, after we know that the request is complete, we call the clearTimeout() function to cancel the timer. The clearTimeout() function cancels the timer identified by the given Integer.

5.3 Hidden IFrame

 

Long before there was widespread browser support for, or use of, an XML-HttpRequest object, developers were using hidden frames to make requests to the server without reloading the page that the user is viewing. The <frame> and <iframe> tags became an official part of the HTML specification with version 4.0 in April 1998. A frame is essentially a window that is embedded in the browser's main window. Frames are independent of the main window in that they can load their own complete HTML page without affecting the content loaded in the rest of the browser. You can hide a frame by using CSS and use JavaScript to dynamically load content into the hidden frame without the user's knowing. A <frame> tag must be declared inside a <frameset>, which replaces the <html> tags. A <frameset> does not display any content itself; its nested frames do. In contrast, an <iframe> can be placed anywhere between the <body> tags, and so it becomes embedded in the content of the HTML page. The <iframe> tag has become more popular than the <frame> tag partially because when the code uses a <frameset> the browser's location bar points to the URL of the <frameset> instead of the URL of the content actually being viewed by the user, which prevents the user from bookmarking the page.

A limitation of the XMLHttpRequest object is that you cannot use it to upload a file. Web developers typically handle file upload by using the tag <input type=“file”/>. The browser renders an input field and a button for this tag that, when selected, displays the operating system's file selection dialog for the user to select the file to be uploaded. After the user has selected a file, all you have to do is submit the form and the browser handles sending the data to the web server for you. Unfortunately, for security reasons, browsers do not allow you to access the file system from JavaScript, and XMLHttpRequest does not have a feature to do this for you. So the only alternative is to use a hidden frame.

Listing 5.2.7

Using setTimeout() and abort() to Time Out a Request

In this section we will show an example using a hidden <iframe> to upload a file without reloading the page that the user is viewing. Suppose that we want to create another form for our web application that allows the user to register a car that she wants to sell. This registration form will allow the user to enter information about the car and upload a picture of the car, which the application will use to make a listing for the car that other users can browse. Figure 5.3.1 shows how the new registration form may look and illustrates the typical data flow with a hidden <iframe>. In step 1 the main page uses JavaScript to pass data to the page loaded in the IFrame and sets the source URL of the IFrame. When the IFrame's source URL is changed, it automatically sends a request to the web server (step 2). In step 3 the web server responds with an HTML page containing any data that the client needs. In step 4, JavaScript loaded in the IFrame page updates the main page.

The code for this example begins with the HTML page that contains the form and the hidden IFrame. We use the code that defined the car search form in Figure 5.2.1 with three additional fields—the condition that the car is in, a short description of the car, and the picture of the car to be uploaded. Listing 5.3.1 shows the HTML, with the fields from the car search form removed for brevity.

Figure 5.3.1

Typical Data Flow Using a Hidden IFrame

The car registration form defines an input field of type file for uploading the picture. Because we are uploading a file, the form method must be POST and the MIME type must be set to multipart/form-data, which is handled by the form's enctype attribute. The IFrame is defined at the bottom of the page and is hidden by CSS. The CSS is left out of this example to keep it simple, but all you have to do to hide a form (or any other element) is to set the CSS display property to “none” (display: none). To accomplish the form submission without reloading this page, the form's target attribute is set to the name of the IFrame. This will cause the response to be loaded in the IFrame instead of replacing this page. The IFrame's src attribute is initially set to about:blank²⁰ to have the frame load a blank page (we do not need the frame to load a page initially). The JavaScript code is in Listing 5.3.2.

The JavaScript code simply sets an onsubmit event handler on the form and an onload event handler on the IFrame. The onsubmit event handler disables the submit button to prevent the user from repeating the submission and sets the text of the button to a message notifying the user that the data is being uploaded. The onload event handler is invoked when the IFrame has received the response and loaded the page. This event handler sets the button back to the original text and enables it so that the user can upload another car.

Listing 5.3.1

HTML Car Registration Form

Using a hidden IFrame, you can do much more than this example illustrates. You can have the IFrame preload a form and use JavaScript to set data in that form, which it gets from a form in the main page that the user populates. You can get data back from the server by putting information in the response page that is loaded in the IFrame. You can then take that response data and update the main page display by using JavaScript. Or you can put JavaScript in the response page that performs some function. You have many possibilities.

Listing 5.3.2

upload.js

5.4 HTTP Streaming

 

The standard HTTP model is based on the web browser pulling data from the web server. The user clicks a link, the browser opens a connection to the server and sends a request, the server responds, and the browser closes the connection. But what if you want to push data to the client, without the client having to explicitly request it? For example, chat systems (AOL Instant Messenger, Windows Live Messenger, Yahoo! Messenger, Google Talk, etc.) push messages out to the clients. As soon as a user enters a message in his client, that message is sent to the server, which pushes it out to others that the user is chatting with. If chat systems relied on a pull mechanism, the clients would have to continuously poll the server to know when messages were sent. This approach would put a large burden on the server and reduce the performance of the system. You would not get that instant response that you are used to.

Achieving a push model with a standard web browser and web server is not simple. It involves using the HTTP model in a manner that it was not designed for. One of two techniques is typically used, although a combination can also be done. The first technique involves the normal request from the browser, followed by multiple responses from the server, using the same long-lived HTTP connection. This technique is sometimes called page streaming. Usually, the response from the server sends one response (HTML page) back to the client and the HTTP connection is closed. With page streaming, the initial HTML page is sent, but the connection is kept open by running a loop on the server that listens for state changes. When new data is to be sent to the browser, the server writes that data to the output stream and flushes it, without closing the connection. During this long-lived connection, the browser is technically still processing the initial page, but because browsers display content as soon as it is received, the page can continuously be updated. The data pushed to the client can include any HTML tags (which can initially be hidden) and typically includes <script> tags with logic to modify the displayed content or to position and make visible hidden content. The JavaScript is executed by the browser as soon as it is received. You could implement a stock ticker or news scroller this way. While page streaming, you could also make a background XMLHttp-Request call to the server as a way of telling the server about something different that you want pushed to the browser without affecting the long-lived initial HTTP connection. Figure 5.4.1 illustrates the communication between browser and server.

Some drawbacks to page streaming include the following: (1) the browser keeps accumulating objects, which uses up memory and could eventually cause the interface to bog down; (2) HTTP connections will inevitably fail, so you must have some plan to recover when this happens; (3) most servers are not designed to handle multiple simultaneous long-lived connections—however, many are working on changing this, and some new servers are built with this in mind, like the server called Twisted (http://twistedmatrix.com).

The second technique, which is more flexible, uses one or more long-lived XMLHttpRequest calls. In this technique, sometimes called service streaming, you can make the long-lived HTTP connection(s) anytime after the page is loaded and close and reopen the connection whenever you need, such as to recover from a failed connection. The HTTP connection is kept open and data is pushed to the client from the server in the same manner as page streaming. An advantage to service streaming, however, is that you can push just about any data that you want as long as you can process it in JavaScript (like JSON). With page streaming, you can push only HTML tags because the browser is processing the data as if it is part of the initially loaded page. If you were to push a JSON string across when page streaming, the string would most likely be written to the page for the user to view—surely not the outcome that you want. Figure 5.4.2 illustrates the communication in service streaming.

Figure 5.4.1

Page Streaming Communication

With service streaming, you have a bigger challenge in handling the data that is pushed to the browser because you must process it yourself. When you use page streaming, the browser automatically processes the HTML tags that you send across. With service streaming, the XMLHttpRequest responseText string grows with each push from the server. In other words, the data is appended to what has already been sent. You must repeatedly parse the responseText string to find out what the server sent last. One way to make this approach work is to delimit the data with a token that a regular expression can look for, like @end@, or encapsulate your data in JSON strings, which are naturally delimited by curly braces, {}. With a delimiter, the regular expression can look as simple as this: /^(.*@end@)*(.*)@end@.*$/. Or you could alternatively keep track of the index in the string where you read to last time, and next time check for data only after that index.

Figure 5.4.2

Service Streaming Communication

A drawback to service streaming is that browser support for this technique is not widespread. For example, Firefox invokes the onreadystatechange event handler with a readyState of 3 as chunks of data are received from the server, but Internet Explorer does not. IE does not call onreadystatechange until the response is complete. An alternative technique is to use a hidden IFrame that uses page streaming to receive <script> tags that update the main page.

5.5 Web Remoting Pitfalls

 

The Web has been around long enough that users are accustomed to certain conventions. For example, you expect to get a visual progress indicator from the browser when waiting for a page to load, you expect to be able to click the back button and see the last web page that you were viewing, and you expect to be able to bookmark pages so that you can return directly to them later. Ajax introduces forms of interaction that break these conventions. Most people do not react well to changes that break conventions that they have come to expect. Therefore, if you want to attract users instead of driving them away, you should do what you can to maintain these conventions, or at least provide a smooth transition to a new form of interaction.

Figure 5.5.1

Visual Progress Examples

One of the easiest problems to fix that Ajax introduces is a lack of visual feedback. With the traditional use of HTTP, when a user clicks on a hyperlink the browser provides some form of visual feedback that work is being done, like a spinning image or a progress bar. When you use the XML-HttpRequest object, the browser does not provide any indication to the user that something is happening. Without that visual feedback, the user will think that nothing happened or that the application is locked up. If a user thinks that nothing is happening, he will usually click on something else, which, depending on your design, may kill the original action. A simple fix for this obstacle is to display a text message or animated image while the XMLHttpRequest is being completed and then to hide the indicator when the request is complete. The same guideline applies when you are using another technique, like IFrames. For example, in Listing 5.3.2 we disabled the button and changed the text of the button to read “Uploading Information…” while the request was being processed and changed it back when the request was complete. Figure 5.5.1 illustrates five different examples of progress indicators that you see online.

A more difficult problem to tackle is related to how the browser tracks a user's browsing history. As a user surfs the Web, the browser stores the URLs for all the pages that the user has visited in a cache, commonly called the history. The browser then enables the Back and Forward buttons to allow the user to navigate backward and forward through the browsing history. When a request is made using XMLHttpRequest, the URL is not stored in the browser's history. So, say that we created an Ajaxified search for our online car sales application. We have a page with a search form that allows you to enter criteria, submit a search, and get back a new page with a list of cars that match your criteria. Because the results can be large, we implement a paging mechanism that allows you to click “next” and “previous” buttons on the results page to move forward and backward through the pages of results, as you can do with Google search results. Because there is other content and JavaScript libraries loaded on the page, we found that the paging works faster if we simply pull down the next page of results from the server by using XMLHttpRequest and then update only the list of results instead of reloading the entire page. When using the application, after paging forward through several pages of results, you decide that you want to go back a few pages. So you naturally click the browser's Back button (instead of the page's previous button). But instead of taking you back one page in the list of results, the browser takes you back to the search form. That is annoying.

In general, you can avoid this pitfall by designing your application such that elements that look like hyperlinks are traditional hyperlinks that do a page reload and are entered into the browser's history. Elements of your application that use XMLHttpRequest do not have to fool the user into thinking that they should behave like a traditional hyperlink. You can make them appear different, which can go a long way to making the user realize that they are not expected to work with the browser's Back and Forward buttons. For example, in the Google Maps application, the user can click on the map and drag it to scroll. Because the use of this functionality is different from using a hyperlink, the user does not expect to be able to use the Back button to scroll back to where she was.

That being said, there are times when it would be nice to be able to make an XMLHttpRequest as the result of the user's clicking on a hyperlink and still have the browser's Back button work as expected. The search result paging described earlier is a good example. Much work has been put into this area by JavaScript toolkits. The excellent Yahoo! User Interface library has a Browser History Manager²¹ that allows you to register the state of your application in the browser's history and have the Back button work as expected. But the system is not perfect. For example, it does not work in the Opera browser.

By tradition, the page that a user sees in the browser is directly related to the URL that he surfed to. With Ajax applications, this is not true. An XML-HttpRequest could be made and the response used to change a significant section of the page, but the URL in the browser's navigation bar will not change. The Web was originally designed around static pages, not around the concept of an online application with a specific state. With static pages, the URL in your browser's navigation bar can be bookmarked and you can return to that page by simply clicking on the bookmark. Therefore, users will expect this of your Ajax web application as well. You can reduce your users’ expectations of bookmarking ability by changing only content on the page that strictly has an “application” feel to it, but this approach will take you only so far. It is important that you design for the ability to bookmark the state of your web application even though you may have to provide a nontraditional way for the user to bookmark it. For example, after scrolling and zooming a map in the Google Maps application, you cannot bookmark what you see by simply activating your browser's bookmark feature. To solve this problem, Google Maps provides you a “Link to this page” hyperlink that generates a URL that you can paste into your navigation bar and then bookmark. The Yahoo! User Interface library's Browser History Manager mentioned earlier also helps with establishing a bookmarkable state.

Finally, we will discuss a security restriction that frustrates many web developers called the same-origin policy. This policy prevents JavaScript code from reading or setting the properties of windows and documents that have a different origin from that of the document containing the script. The origin of a document includes the protocol, the domain name (host), and port of the URL from which the document was loaded. Documents belong to the same origin if and only if those three values are the same. For example, all the following URLs have a different origin.

In other words, if the web page in which your JavaScript code is loaded was pulled from http://www.ajaxbook.com and you wanted to use an XML-HttpRequest to directly grab some data from an Amazon web service, you would be out of luck. The same-origin policy was implemented to prevent malicious scripts in one frame or window from accessing personal content in another frame or window.

The only current, cross-browser exception to the same-origin policy applies to windows and frames. The domain property of the Document object, by default, contains the hostname of the server from which the document was loaded. JavaScript in two different windows or frames can change their domain property to the same domain to interact with each other. However, the domain property can be set only to a valid domain suffix of the default value. For example, if the default value is “www.ajaxbook.com” then you can change the value only to “ajaxbook.com”. The domain set must have at least one dot in it, so it cannot be set to a top-level domain, like “com”. For example, if one frame loaded a document from http://carsearch.ajax-book.com and another loaded a document from http://www.ajaxbook. com, JavaScript loaded in those two frames could both set their domain property to “ajaxbook.com” and communicate with each other.

Another exception to the same-origin policy has been drafted by the W3C and implemented by Firefox 3. When this chapter was written, the W3C's Access Control for Cross-site Requests²² specification was in draft form and Firefox 3 was in beta. The access control specification provides two methods through which you can make cross-site XMLHttpRequests. The first is using an Access-Control header set by the server. For example, if a server set the following in an HTTP response, then you would be able to get this document by using XMLHttpRequest and Firefox 3 even if your document was loaded from a different server.

The second technique uses an XML processing instruction to express the same information. For example, the following at the top of an XML file that was returned as the body of an HTTP response would also work.

Because these techniques are not currently widely supported by browsers, you may be wondering what you can do now to pull in content or data from another site by using XMLHttpRequest. The common solution that web developers use is to put a proxy on the server that forwards your requests for data from another site. In this solution you (the browser) make an XMLHttpRequest to your server with the URL of another site that you want to contact. Your server sees that you want to get data from another domain, so the server makes the request for you to that URL and sends the response on to you (the browser). Therefore, the server makes the cross-site request instead of the browser.

5.6 Complete Example

 

This section provides a complete example. Up to this point we have concentrated on showing only the client-side code, which is what this book is mostly concerned with. Now we will include the server-side code so that you can set up a complete example and see the traffic between the client and the server. The example in this section uses PHP version 5.2.5 as the server-side language. PHP is a widely popular, open-source, general-purpose scripting language that can be embedded in HTML to quickly generate dynamic pages. Its syntax is similar to that of C and Java. To learn more about PHP, you can read the PHP manual online.²³

To serve up dynamic web pages written using PHP, you need a web server. The web server processes HTTP requests for static content, like CSS, images, and JavaScript files, and passes requests for PHP files over to the PHP engine. PHP generates the HTML and hands it back to the web server, which sends it on to the browser. The web server that we use in this chapter is the popular open-source Apache HTTP Server.²⁴ This book's Appendix has information on installing and configuring Apache and PHP to run the following examples. Although doing so is not required, you may want to install and use the NetBeans IDE and the NetBeans PHP plugin, which will give you features like syntax highlighting of the PHP code, automatic code completion, automatic deployment of your code to the Apache HTTP Server, and debugging. You can write the code with any text editor, and you can manually move the code into Apache's htdocs directory to deploy it, but using the NetBeans IDE will make the process easier for you. The installation of NetBeans and the PHP plugin is also described in the Appendix.

In this example, we will complete the Ajax validation for the car search form so that you can see the server side and play with some working code. We will not be creating a real car sales web application, just the Ajax validation on the car search form. The development of this application is laid out in the following steps. Follow them closely and everything should work as advertised.

1. Install the Apache Web server and PHP as described in the Appendix. Make sure that Apache and PHP are working together by creating a test PHP page and navigating to its URL with your web browser.

2. Create a NetBeans PHP project. The version of NetBeans used in this example is 6.0.1. Open NetBeans and select File → New Project from the menu. From the New Project dialog that appears, select PHP → PHP Project, as you see in the diagram, and click the Next button.

3. In the dialog that appears next, enter the project name and, optionally, a project location. Then click the Next button.

4. In the next dialog for “Configured Web Servers” select the Apache web server that you configured when setting up the PHP plugin as described in the Appendix. You also have the option here to enter a context path. The context path is used in the URL for the web application, after the server name, which here will be localhost. The Http Path field of the dialog shows you how the URL of your web application will look. After you are satisfied, click the Finish button and NetBeans will create a project for you.

5. All the files in this example will be created in the Source Files directory under the project. The following screenshot shows how the project will look when done.

6. Now to write some code. We will start with the car search form page. Right-click on the Source Files folder and select New → PHP File… as in the following screenshot.

7. Enter carSearchForm for the File Name and click the Finish button. See the following screenshot.

8. In the PHP editor that appears, replace the template PHP code with Listing 5.6.1. Most of this listing is the HTML that defines the form, the text fields, and the drop-down lists. This file contains no CSS or JavaScript code; instead, it just sources in the CSS and JavaScript files. This is a best practice. Keep your CSS and JavaScript separate from your HTML. Doing so makes the maintenance much easier. The PHP code at the top of this file is used to output the XML declaration and doctype required for an XHTML file. The statement with the $mime variable is necessary because some browsers, like Internet Explorer, do not support the correct XHMTL MIME type of application/xhtml+xml. In those cases we fall back to claiming that this document is HTML by using the text/html MIME type. The MIME type is also used later to create the meta tag. The PHP in this file that really matters is the line in the car make select tag that uses the PHP require statement to source in another PHP file that dynamically creates options for all the car makes. The carMakeOptions.php file and the CSS and JavaScript files will be defined as we follow through the rest of this example.

9. The CSS styles the HTML, so we have a gray background and a black border around the form, the labels are aligned to the right, and the form shrinks to fit its contents. To create the CSS file, right-click on the Source Files folder and select New → Cascading Style Sheet…. Enter carSearch for the File Name and click the Finish button. In the CSS editor that appears, replace the template CSS with Listing 5.6.2. A best practice used here is that the label width is set to use ems instead of pixels. This approach makes the label “elastic” such that it grows to fit the font if the font is increased using the browser's font settings. Something else interesting in this CSS is the use of the float rule. Setting the float to left or right causes the form to shrink to fit its contents. Otherwise, the form would be as wide as the page or its container.

Listing 5.6.1

carSearchForm.php

10. Next let us create the data for car models and makes. In a production system, this data would normally be contained in a database, but to make this example simple we are going to create an XML file for our data. In NetBeans, right-click on the Source Files folder and select New → XML Document…. In the dialog that appears, enter cars-min as the File Name and then click the Next button. Select the radio button for “Well-formed Document” and click the Finish button. In the XML editor that appears, replace the template XML with Listing 5.6.3. In this listing we define an XML root tag called <cars>, which contains several <make> tags that define the car makes. These <make> tags in turn contain <model> tags to define the car models. A full list of car makes and models would be large, so we narrowed the list to six models for each of the major American makes.

11. Now we can create the PHP file that uses this data to create a dynamic list of select tag options for all the car makes. Create a new PHP file called carMakeOptions.php and enter Listing 5.6.4. This PHP code uses the DOM and XPath to read cars-min.xml and output an <option> tag for each car make. DOMDocument²⁵ is PHP's implementation of the Document interface, and DOMXPath²⁶ is PHP's XPath implementation. The output from this PHP code is used in the car model <select> tag in the car search form of Listing 5.6.1.

12. Now that we have created the basics for how the car search form HTML page is created, let us add some dynamic features to it with JavaScript. We start by defining RequestHelper.js. This RequestHelper is slightly modified from what was used in the rest of this chapter to encapsulate more of the boilerplate logic that is used on every XMLHttpRequest call. In NetBeans, right-click on the Source Files folder and select New → JavaScript File…. Name the file RequestHelper and click the Finish button. In the JavaScript editor that appears, enter Listing 5.6.5. This time, we defined a namespace for the JavaScript code, which is a best practice. To do this you simply create a JavaScript object that all the code is placed in. It is not a true namespace like packages in Java, but it does the job of preventing name clashes with other JavaScript code that is loaded in the page. The namespace we used is ajaxbook. The main difference with this RequestHelper and that used in the rest of this chapter is the addition of the sendGet() function, which will make a GET XMLHttpRequest by using the given parameters. The parameters are the URL to call, a function to call when the XMLHttpRe-quest successfully completes, a function to call if the request fails, a length of time (in milliseconds) to wait before timing out the request, and an object to pass to the success and failure callback functions. RequestHelper passes the XMLHttpRequest object to the success and failure callback functions, but it is often useful to be able to pass other data to the callbacks—something that you cannot directly do in this example because we are relying on Request-Helper to call the callbacks.

Listing 5.6.2

carSearch.css

Listing 5.6.3

cars-min.xml

Listing 5.6.4

carMakeOptions.php

13. Now to use RequestHelper to make some XMLHttpRequest calls. In NetBeans, right-click on the Source Files folder and select New → JavaScript File…. Name the file carSearch and click the Finish button. In the JavaScript editor that appears, enter Listing 5.6.6. As with the RequestHelper, all the code in this file is placed under the ajaxbook namespace. This file defines a generic Validator class with a validate() function that handles making a call to the server to validate a field and updating an HTML tag with the message response from the server. In Listing 5.6.1 we defined <span> tags next to the text input fields to hold the validation messages that come back from the server. This JavaScript code creates two different instances of the Validator to handle validating the year text input field and the ZIP code text input field. Next we define a CarModelFetcher to handle getting the car models from the server for the selected car make. This code looks similar to that in Listing 5.2.5, minus the validation code, which has been moved to the validators, and now it is packaged in an object and uses the new RequestHelper.sendGet() method. Finally, at the end of this file we use the window.onload event handler to invoke a function that defines the event handlers for the form fields that use the two validators and the model fetcher. This is how you define event handlers without mixing JavaScript code in the HTML code. We have used this technique throughout this chapter, so you should be familiar with it. JavaScript toolkits like Yahoo! User Interface and Dojo provide functions that help you to bind multiple initializer functions to the window.onload event handler. Here, we only need one.

Listing 5.6.5

RequestHelper.js

14. Next we need to create the PHP files on the server that JavaScript calls to validate the text fields and fetch the car models. Start with the validateYear.php file, which validates a year submitted to it. Create this file in the Source Files folder in NetBeans and make it look like Listing 5.6.7. This listing is simple. It simply checks if it was passed a “year” parameter. If it was, it takes the year parameter and verifies that it matches a regular expression defining a four-digit year. If the validation fails, it sends back an error message. If it passes, it sends back the string “OK”. The ajaxbook.Validator in carSearch.js sets the returned message (if it is not “OK”) as the innerHTML of the <span> tag that displays the message.

carSearch.js

validateYear.php

validateZip.php

15. Create the validateZip.php file. Create this file in the Source Files folder in NetBeans and make it look like Listing 5.6.8. This file is just like validateYear.php except that it has a regular expression that defines a United States ZIP code. As with validateYear.php, the ajaxbook.Validator in carSearch.js sets the returned message (if it is not “OK”) as the innerHTML of the <span> tag that displays the message.

16. Finally, create the last PHP file, getCarModelsXML.php, and make it look like Listing 5.6.9. This code uses the PHP DOM and XPath implementations like we did in Listing 5.6.4. But this time we use XPath to find the <make> tags (Nodes) in the document with a make name matching the one submitted to this file. There should be only one match, so we just grab the first Node from the NodeList and use the convenient DOMDocument->saveXML() method to output the XML as a string. The ajaxbook.CarModelFetcher in carSearch.js creates <option> tags from the returned <model> XML tags and sets them on the car model <select> tag. The response Content-Type is set to application/xml. If it is not, then the browser will not automatically parse the XML and make it available via XMLHttp-Request's responseXML property.

getCarModelsXML.php

17. Now to deploy the code to Apache and test it. First make sure that the Apache Web server is running. Refer to the Apache documentation²⁷ for how to do this—it depends on the version that you have installed, your platform, and how you installed it. If you are not using NetBeans, then you just need to copy the source files to the htdocs directory of your Apache installation. Otherwise, from Net-Beans you can simply right-click on the name of your project and select Run Project, or you can select the green right-arrow button in the toolbar, or press the F6 key. NetBeans will copy your source code to the Apache htdocs directory for you and load the URL to your application in your default web browser. The page displayed in your browser depends on what you had selected in NetBeans. If you ran the project by right-clicking on the project name, or you had the project name selected when you ran the project by using one of the other methods, then NetBeans will point your browser to the context root of your application. On the basis of how you have Apache configured when it is pointed to the context root of your application, you will get a directory listing of the files that you have deployed to Apache. If you had one of the files open and selected in NetBeans, then NetBeans will have your browser point to that file. The following screenshots illustrate running your project from Net-Beans.

18. If you have an index page as in the preceding screenshot, then select carSearchForm.php. Otherwise, enter the correct URL in your browser to point it to carSearchForm.php. You should see the following page.

19. Enter some invalid values in the year and ZIP fields and select various makes to test the form.

20. If you are using Firefox and you have Firebug installed, you can see the traffic between the browser and the server. Just open Firebug and select the Net tab. You should see the URLs that were requested from the server. You can narrow the list to only the requests made using XMLHttpRequest by selecting the XHR button. If you open vali-dateYear.php, validateZip.php, or getCarModelsXML.php request, you can see the HTTP request that was made and the response that was received. See the following screenshot for an example.

5.7 Summary

 

Ajax has boomed in popularity because it gives you the tools to transform websites into web applications with the responsiveness and feel of desktop applications. Web remoting techniques, and in particular the XMLHttpRe-quest object, are the foundation that makes this possible. You are no longer constrained to making the user wait while you fetch and load an entire page just to send a few user input fields to the server or update the page with a few lines of new content. In fact, with the tools and techniques that you have now learned, you can write an entire application in one page! In essence, that is what Google Maps is.

This chapter covered the core techniques that define Ajax—how to send asynchronous requests to the server by using XMLHttpRequest or an IFrame and how to process the response data. Sending an XMLHttpRequest generally involves three steps.

1. Create an instance of XMLHttpRequest for the specific browser that your code is executing in. Internet Explorer 5 and 6 use an ActiveX object, whereas all other browsers implement a native XMLHttpRequest object.

2. Invoke the open() method to tell XMLHttpRequest what URL you want to request, assign the onreadystatechange event handler to be notified when the request is complete, and then invoke the send() method to actually make the request.

3. Process the response from the server—do something with the dataset in the responseXML or responseText property.

You must use HTTP to package the request to the server and the response to the client, but the kind of data that you include in the HTTP request body is up to you. This chapter covered the most popular data formats used: HTML, XML, JSON, and plain text. HTML has the benefit of being directly assignable to the innerHTML property of HTML DOM elements. XML has the advantage of flexibility, readability, and the fact that the browser will automatically parse XML data for you. JSON has the advantage of being lightweight with low bandwidth consumption and is directly executable and usable in JavaScript. Plain text has the advantage of simplicity and is ideal for small pieces of data, like single messages that can be directly assigned to the innerHTML property of DOM elements that accept TextNode children.

The future of Ajax and Ajax-like techniques is bright. It only gets better and more interesting from here. For instance, in this chapter we also discussed a relatively new technique called HTTP Streaming in which data is pushed to the client—the opposite model that HTTP is based on. Alex Russell,²⁸ a cocreator of the Dojo JavaScript toolkit, coined the term Comet to describe this evolving web-based push architecture, and under that name it has received the most attention and development. The term Comet is a play on Ajax—both Ajax and Comet are common household cleaners. Some well-known applications using Comet are the Meebo²⁹ web-based chat application and Google's chat built into Gmail.³⁰

5.8 Self-Review Questions

1. The term Ajax was originally an acronym. What did it stand for?

a. Asynchronous JavaScript and XML

b. Another Java Abstraction for X-Windows

c. Another Java and XML Library

d. Abstract JSON and XML

2. Who coined the term Ajax?

a. Douglas Crockford

b. Bill Gates

c. Jesse James Garrett

d. James Gosling

3. The traditional HTTP request/response model is

a. Synchronous

b. Asynchronous

4. innerHTML is generally faster than W3C DOM methods in all browsers.

a. True

b. False

5. innerHTML is a W3C DOM standard.

a. True

b. False

6. XMLHttpRequest is a W3C standard.

a. True

b. False

7. XMLHttpRequest is the only cross-platform client-side approach-to perform asynchronous server requests.

a. True

b. False

8. Creating an instance of the XMLHttpRequest object is the same in all browsers.

a. True

b. False

9. Which object do you use in Internet Explorer version 5 and 6 to make asynchronous requests to the server?

a. ActiveX

b. XMLHttpRequest

c. XMLHTTP

d. window

10. Which method is not a member of XMLHttpRequest?

a. open()

send()

cancel()

setRequestHeader()

11. Which XMLHttpRequest method actually makes the request?

a. getAllResponseHeaders()

b. makeRequest()

c. open()

d. send()

12. If req is a reference to an XMLHttpRequest object, which of the following is using XMLHttpRequest for a synchronous request?

a. req.open(“GET”, “http://myserver/getSomething”, false);

b. req.open(“GET”, “http://myserver/getSomething”, true);

c. req.open(“GET”, “http://myserver/getSomething”);

13. Which of the following calls to send() is invalid?

a. send()

b. send(null)

c. send(“x=1&y=2”)

14. What MIME type must be set when using XMLHttpRequest to send a POST request of a form?

a. application/octet-stream

b. application/xhtml+xml

c. application/x-www-form-urlencoded

15. Which readyState and status combination should you check for to know that an asynchronous request is complete?

a. 3 and 304

b. 2 and 200

c. 4 and 200

d. 5 and 404

16. The intermediate values for readyState are unreliable across browsers because some do not set the Loading or Loaded states and/or set the Interactive state only once.

a. True

b. False

17. Which of the following MIME types will cause the web browser to automatically parse an XML response?

a. text/xml

b. text/javascript

c. text/plain

d. text/html

18. What is JSON an acronym for?

a. JavaScript Over Network

b. JavaScript Object Notation

c. Java Simple Object Notation

19. Which JavaScript method is used primarily with JSON?

a. eval()

b. test()

crun()

d. exec()

20. Which field of XMLHttpRequest will be populated with a JSON response?

a. responseXML

b. responseText

c. response

d. statusText

21.Which of the following is not valid JSON?

a. { “name”: “Bob” }

b. { “user”: { “name”: “Bob” } }

c. { “name”: “Bob”, “getName”: function() { return this.name } }

22. The XMLHttpRequest object can make both a GET and a POST request.

a. True

b. False

23. The XMLHttpRequest object can be used to upload a file.

a. True

b. False

24. The web browser displays a visual progress indicator while an XMLHttpRequest is being processed.

a. True

b. False

25. You can use XMLHttpRequest in your website to directly make a request to any other website.

a. True

b. False

Keys to the Self-Review Questions

1.a 2. c 3. a 4. a 5.b 6.b 7.b 8.b 9. c 10. c 11. d 12. a 13. a 14. c 15. c 16. a 17. a 18. b 19. a 20. b 21. c 22. a 23. b 24. b 25. b

5.9 Exercises

1. What is Web Remoting?

2. What steps has W3C taken to standardize the concepts behind XMLHttpRequest?

2. Describe the steps involved in using XMLHttpRequest.

4. Write code to instantiate an instance of XMLHttpRequest that works in all modern browsers. If the browser does not support XMLHttpRequest, then handle that condition appropriately.

5. Compare and contrast the different response formats covered in this chapter: plain text, HTML, XML, and JSON. Describe a scenario for each format in which you would choose that format over the others.

6. Write the JavaScript code to use XMLHttpRequest to make an asynchronous GET request and process the response. No server-side code is required, but comments about the data returned by the server are required. The example may be simple and show only enough of the HTML so that another can understand it.

7. Write the JavaScript code to use XMLHttpRequest to make an asynchronous POST request and process the response. No server-side code is required, but comments about the data returned by the server are required. The example may be simple and show only enough of the HTML so that another can understand it.

8. Write the JavaScript code for a simple example that uses XMLHttpRequest and the XML response format.

9. Write the JavaScript code for a simple example that uses XMLHttpRequest and the JSON response format.

10. Describe the hidden <iframe> technique and describe a situation in which you would use it instead of XMLHttpRequest.

11. Describe HTTP Streaming and explain how it differs from the traditional HTTP request/response model.

12. Summarize the pitfalls of Web Remoting covered in this chapter.

5.10 Programming Exercises

1. Write code for a complete example (including server-side code) that uses XMLHttpRequest to autoupdate a portion of a web page with news headlines (a news ticker). The news headlines can be fake.

2. Develop a complete example that displays a progress bar in an HTML page during a long-running server process and uses XMLHttpRequest to get the status from the server and update the progress until it finishes.

3. Develop a complete example that uses XMLHttpRequest to get data from the server to display in a pop-up tooltip when the user mouses over something on the web page, like an image or a link. An example of this can be seen at http://www.netflix.com/BrowseSe-lection. If you hold your mouse over the movie poster images, you will get a pop-up tooltip with more information about the movie.

4. Develop a complete example that uses XMLHttpRequest to provide an autocomplete feature for a text field, like Google Suggest does with its text field. The data for the text field can be anything you like, such as states, ZIP codes, names, and search results like Google.

5.11 References

Asleson, Ryan, and Nathaniel T. Schutta. 2005. Foundations of Ajax. Berkeley, CA: Apress.

Crockford, Douglas. JSON. Douglas Crockford. 2008. http://www.json.org/ (accessed February 28,2008).

Fielding, R., J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, and T. Berners-Lee. Hypertext Transfer Protocol—HTTP/1.1. The Internet Society, 1999. http://www.ietf.org/rfc/rfc2616.txt (accessed February 28,2008).

Flanagan, David. 2006. JavaScript: The Definitive Guide. 5th ed. Sebastopol, CA: O'Reilly Media.

Garrett, Jesse James. “Ajax: A New Approach to Web Applications.” Adaptive Path, LLC. 2008. https://www.adaptivepath.com/ideas/essays/archives/000385.php (accessed February 28,2008).

Gehtland, Justin, Ben Galbraith, and Dion Almaer. 2006. Pragmatic Ajax: A Web 2.0 Primer. Raleigh, NC: Pragmatic Bookshelf.

Keith, Jeremy. 2007. Bulletproof Ajax. Berkeley, CA: New Riders.

Koch, Peter-Paul. “Javascript-Benchmark-W3C DOM vs. innerHTML.” Peter-Paul Koch. 2008. http://www.quirksmode.org/dom/innerhtml.html (accessed February 28,2008).

Mahemoff, Michael. 2006. Ajax Design Patterns. Sebastopol, CA: O'Reilly Media.

PHP Documentation Group. PHP Manual. The PHP Group. 2008. http://us.php.net/manual/en/index.php (accessed February 28,2008).

Wikipedia. “Percent-encoding.” Wikimedia Foundation, Inc. 2008. http://en.wikipedia.org/wiki/Percent-encoding (accessed February 28,2008).

W3C Recommendation. Document Object Model (DOM) Level 3 Load and Save Specification, Version 1.0. April 7, 2004. http://www.w3.org/TR/DOM-Level-3-LS/.

W3C Recommendation. Document Object Model (DOM) Level 3 Core Specification, Version 1.0. April 7.2004. http://www.w3.org/TR/DOM-Level-3-Core/.

W3C Working Draft. Access Control for Cross-site Requests. February 14,2008. http://www.w3.org/TR/access-control/.

W3C Working Draft. The XMLHttpRequest Object. October 26, 2007. http://www.w3.org/TR/XMLHttpRequest/.

Zakas, Nicholas C., Jeremy McPeak, and Joe Fawcett. 2007. Professional Ajax, 2nd Edition. Indianapolis, IN: Wiley.

---

¹http://www.ashleyit.com/rs/rslite/.

² http://ajaxpatterns.org/On-Demand_Javascript.

³http://developer.mozilla.org/en/docs/XMLHttpRequest.

⁴http://msdn2.microsoft.com/en-us/library/ms535874.aspx.

⁵http://www.w3.org/TR/XMLHttpRequest/.

⁶http://www.w3.org/TR/DOM-Level-3-LS/.

⁷http://blogs.msdn.com/xmlteam/archive/2006/10/23/using-the-right-version-of-msxmlin-internet-explorer.aspx.

⁸http://www.microsoft.com/downloads/details.aspx?FamilyID=993c0bcf-3bcf-4009-be21-27e85e1857b1.

⁹http://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html#sec5.1.1.

¹⁰http://www.quirksmode.org/dom/innerhtml.html.

¹¹http://en.wikipedia.org/wiki/Percent-encoding.

¹²http://www.iana.org/assignments/media-types/.

¹³http://www.whatwg.org/specs/web-apps/current-work/#innerhtml0.

¹⁴http://www.w3.org/TR/DOM-Level-3-Core/core.html#ID-B63ED1A3.

¹⁵http://www.regular-expressions.info/.

¹⁶http://www.crockford.com/.

¹⁷http://www.json.org/.

¹⁸http://www.json.org/js.html.

¹⁹http://www.json.org/.

²⁰http://en.wikipedia.org/wiki/About:_URI_scheme#Common_about:_addresses.

²¹http://developer.yahoo.com/yui/history/.

²²http://www.w3.org/TR/access-control/.

²³http://us.php.net/manual.

²⁴http://httpd.apache.org/.

²⁵http://us3.php.net/manual/en/function.dom-domdocument-load.php.

²⁶http://us3.php.net/manual/en/function.dom-domxpath-query.php.

²⁷http://httpd.apache.org/docs/2.2/invoking.html.

²⁸http://alex.dojotoolkit.org/.

²⁹http://www.meebo.com/.

³⁰http://mail.google.com.