<img> tag

The simplest way to load an image is to use the HTML <img> tag. This tag requires only a single src attribute (which points to the image location), and doesn’t even need to be closed (see Example 1-1 and Figure 1-1).

Example 1-1. Simple <img> tag

<img src="book.jpg">

Figure 1-1. Result of Example 1-1

The full <img> tag supports various other attributes, most notably alt, height, and width (see Example 1-2 and Figure 1-2). The alt attribute holds a textual description of the image, to be displayed as a placeholder and used by screen readers and other accessibility tools. The height and width attributes explicitly indicate the dimensions of the image.

Example 1-2. Full <img> tag

<img src="book.jpg" alt="A Book" height="200" width="100">

Figure 1-2. Result of Example 1-2

The alt attribute has no real performance impact, though it does affect our ability to implement alternate image loading techniques, as we’ll see further on.

The height and width attributes, however, do impact performance. If these attributes were omitted, the browser would have no way of knowing how much area it should allocate for the image, until it actually downloads the image file and sees its dimensions. This browser would reserve some arbitrary (and usually small) space, and once enough of the image data arrives (i.e., enough for the browser to conclude the image’s dimensions) it would update the layout—also known as reflow. Reflows have computational cost and take time, but more importantly, they make for a very poor user experience, as page parts move around while the user is trying to read them, possibly being pushed below the visible area. Therefore, an important best practice is to always specify dimensions in your <img> tag.

Note the width and height of an image can also be specified in the CSS rules of the page. If you believe that the dimensions of an image have more to do with how it’s laid out on the page than the image itself, then CSS is a more logical place to state them. That is especially true in responsive layouts, where the image’s display dimensions may depend on the current breakpoint and are often relative to its container or the viewport. On the other hand, if the image is of fixed dimensions and those dimensions are tied to the actual visual—the contents of the image—then element attributes may be the way to go.

From a performance perspective, the source of the height and width matters very little. Specifying the dimensions in CSS means the browser won’t see them until it has downloaded and processed all the relevant CSS files, making the attribute path theoretically faster. However, the browser doesn’t perform the initial layout until all CSS is fully processed anyway, so in practice, it doesn’t help to know the dimensions earlier.

We won’t be discussing layout much, but if you’d like to learn more about how rendering is handled in the browser, check out the “Critical Rendering Path” article on Web Fundamentals.

CSS background-image

Another prevalent path to load images is the CSS background-image property (see Example 1-3 and Figure 1-3). This styling instruction was originally used as a richer alternative to a background color, but is now used for many different purposes, ranging from rounded corners to logos to rich photography positioned behind the page’s content.

Example 1-3. Simple background-image

<style>
#title {
        background-image: url("stars.jpg");
        background-size: contain;
        color: white
}
</style>
<h1 id="title" color="white">Reach for the stars!</h1>

Figure 1-3. Result of Example 1-3

Background images are designed—surprise, surprise—to be in the background, and much of their handling assumes they do not hold important content. In reality, however, background images often do hold critical content. Examples include tab or section titles, primary navigation icons, visual context for foreground content (e.g., a map, with foreground landmarks), and more.

In addition, background images are sometimes used for actual foreground imagery. This is usually done for performance reasons, such as image sprites or responsive images.

The use of background images for core content has various implications, with the primary impact being on file structure and accessibility.

Building the Document Object Model

As soon as a browser starts receiving HTML data, it will start parsing it and constructing the Document Object Model (DOM). The DOM is the programmatic representation of the page—practically everything we see or do on a page results in reading data from it or making a change to it.

As it builds the DOM, the browser encounters references to external resources, such as external JavaScript files, links to CSS, and—of course—images. Once discovered, the browser queues the resource to be downloaded, working within the network constraints we’ll discuss later.

While both the DOM and HTML are tree structures, converting the HTML into a DOM isn’t simple. HTML is a very loose language to begin with, and browsers have always been very permissive when it comes to malformed HTML. Instead of erring, browsers automatically apply fixes to the page, making changes such as closing open tags, moving elements between body and head, and even correcting common typing mistakes. For instance, most browsers today support loading an image using the nonstandard <image> tag, most often by silently converting to an <img> tag. In general, browsers are willing to jump through hoops to make pages work, even if their content is not standard and only resembles well-formed HTML when the lights are dim and the music is loud.

One especially painful complexity with building the DOM comes from JavaScript. By design, JS code is able to read and manipulate the DOM, which is the primary means to make a web page interactive. For synchronous scripts (so <script> tags without async or defer attributes), that may mean that the script is relying on the current DOM to be in a particular state.

If the script is appending new nodes to the tree (e.g., using document.body.appendChild()), they are expected to be added in a particular place in the tree. The same goes for document.write() calls, which add HTML to the HTML parser in the exact position that the script is in. Since the browser doesn’t want the page to break when these things happen, it must halt the parser whenever a synchronous script is encountered, until the script has finished downloading, parsing, and executing. On top of that, synchronous script execution can be halted on in-flight CSS files, as they may impact the result of JS execution—for example, if the script is reading styling information from the DOM.

Figure 1-4. Sequential JS downloading in IE7

The waterfall in Figure 1-4 clearly shows the delay this sequential behavior causes. A very simple test page holding only six scripts (a third of today’s average) and an image, will be painfully slow on Internet Explorer 7. Why use IE7? Because starting with IE8 (and other browsers released shortly after), browsers stopped being silly, and started using a preloader.

The Preloader

Nothing can be done to prevent synchronous scripts from blocking the building of the DOM without breaking pages. Browsers (rightfully) favor functionality over performance, and will not consider breaking pages just to speed them up. However, what can be done is to separate parsing from downloading resources—and that’s precisely what the preloader does.

The preloader, also known as the “look-ahead parser,” the “speculative parser,” or the “preparser,” is a second parser inside the browser. Just like the DOM building parser, it starts digesting the HTML as soon as it arrives, but instead of building a structure, it just tokenizes the HTML and adds subresource URLs that it encounters to the download queue. Since the preloader doesn’t provide any of the page’s functionality, it doesn’t need to stop when it sees a script, and can simply plow along and discover all the subresources referenced in the HTML.

With this added functionality, the browser can go ahead and download resources even before it’s ready to process them in full, decoupling download from execution. In addition, browsers can conceiveably do some processing on these resources—for instance, parsing a JS/CSS file or decoding an image.

As mentioned before, the first preloader was introduced in IE 8, and is possibly the single biggest web performance improvement we’ve seen in browsers. It’s been further improved over the years, and now triggers additional actions such as DNS resolutions, Transmission Control Protocol (TCP) connections, Transport Layer Security (TLS) handshakes, and more. Figures 1-5 through 1-7 show the evolution of preloader-triggered downloads on the simple six-script page from the previous section, going from no preloader in IE 7, through the first generation in IE 8, to the latest iteration in IE 11.

Figure 1-5. Fully sequential JS downloads in IE 7

Figure 1-6. Mostly parallel JS downloads in IE 8

Figure 1-7. Fully parallel JS and image downloads in IE 11

While awesome, the preloader can sometimes make mistakes. As it runs ahead of the main parser, it’s forced to make some simplifying assumptions. For instance, when the preloader sees two consecutive external scripts, it queues both for download right away. If it turns out the first script, when executed, navigated away from the page, it will render the second script’s download unnecessary.

Cases like this happen quite often—for instance, with scripts that manipulate or change pages for A/B testing purposes, or scripts that employ client-side device detection and redirect to a mobile website. Despite this limitation, browser data indicates that the preloader is undoubtebly a good way to speed up the web. As long as its predictive downloads are accurate the vast majority of the time (which they currently seem to be), we all come out ahead.

For example, consider the following code:

<script>
    document.write("<-"+"-);
</script>
<img src="a_funky_sloth.jpg">

The preloader in this example will skip over the script and start downloading the image resource, since it assumes that it will be required later on. But, since the document.write() directive starts an HTML comment, making everything that comes after it irrelevant, that download would be spurious.

Despite the preceding example, that’s not a bug, but a conscious design decision. The preloader is a heuristic optimization, and is there to make the 99% cases faster, even if some edge cases will be slower as a result.

Networking Constraints and Prioritization

Between the DOM parser and the preloader, browsers can quickly build up a long list of resources to download. You may think the next step is to simply charge ahead and download all of those resources in parallel. After all, doing more in parallel leads to faster results, right?

As usual, it’s not that simple. Downloading all resources at once can easily overwhelm home routers and servers and create network congestion along the way, as it effectively disables TCP’s congestion avoidance mechanisms. This in turn can lead to packet loss—and so to a slower web experience. To reduce that risk, browsers limit the number of simultaneous connections they open up against a single host and (to a lesser extent) in total. Most browsers allow no more than 6–8 concurrent connections per host, and no more than 10–16 parallel connections in total (across all hosts). This browser limit led to the creation of an optimization technique called domain sharding.

There are also cases where a parallel download of all resources provides an inferior user experience. For instance, assume your page has 100 nonprogressive images, each 10 KB in size, and that your bandwidth is 100 KB/s. If all files are downloaded in parallel, it will take 10 seconds until all images are downloaded. Until then, no image will be fully displayed. In contrast, if only 10 images were downloaded in parallel, you would get 10 new complete images every second. It is often considered a better user experience to provide the user with some complete content as soon as possible, especially considering that there is a good chance that many of the 100 images in our preceding example might be outside of the initial viewport.

One last (but definitely not least) reason for not downloading all files in parallel is that some resources matter more than others. For example, browsers don’t render anything on a page until all CSS files have been fully downloaded and processed, to avoid showing unstyled content. Images, on the other hand, don’t block the rendering of anything but themselves. Therefore, it makes sense to favor the download of CSS files over those of images when you’re required to make such a decision.

Browsers are constantly faced with such decisions. Given the (self-imposed) connection limit and the bandwidth concerns, prioritization often means delaying the download of certain resources until others are fully fetched. In fact, browsers assign each resource a priority, taking into account parameters, such as the resource type, whether it’s async, and whether it’s visible.

While resource prioritization is becoming increasingly dynamic, initial priorities are based on resource type in most cases. For images, that means that their initial priority is rather low (as other resource types, such as scripts and stylesheets, often have a larger impact on the page). Because of the preloader, when the browser starts downloading images, it is often unaware of their visibility in the initial viewport, their display dimensions, and so on. Later on, once these extra parameters become known (after the page’s layout takes place), visible and high-prominence resources may get their priorities upgraded.

It’s worth noting that HTTP/1.1 (and older) don’t have a built-in prioritization mechanism, so browsers can only prioritize by delaying or blocking entire resource downloads, which may underutilize the network. For example, downloading a single JS file will usually block the download of all images, even if there are available idle connections, as the browser doesn’t want any low-priority resource to contend over bandwidth with the higher-priority script. The newer SPDY and HTTP/2 protocols provide better prioritization mechanisms.

HTTP/2 Prioritization

As we’ve seen, with HTTP/1.1, the browser has very rough control when it comes to resource prioritization, where its decision is binary: “Should this resource be requested right now or not?”

With a newer version of the HTTP protocol, that is no longer the case. HTTP/2 solves a lot of networking-related deficiencies that HTTP/1.1 suffered from:

• It can multiplex multiple requests and responses on a single connection.

• It can compress HTTP headers.

• The browser can attach fine-grained priority to each request it sends the server.

The last point emphasizes the difference in prioritization from earlier versions of the protocol. In HTTP/1.1, the browser maintains a queue of resources to be fetched, and the priorities of fetched resources are maintained internally, as part of that queue. Once a low-priority resource made it to the top of the queue (for lack of higher-priority resources that the browser is aware of), that resource was requested. And once that happened, prioritization is out the window. Even if a higher-priority resource arrived at the queue a few milliseconds later, there was no practical way to give it higher network priority than the resources already requested.

As a result, some browsers preferred to hold back on requesting low-priority resources until they were sure all high-priority ones had already arrived, which led to behavior such as Chrome limiting image requests until all CSS and JavaScript components were downloaded.

With HTTP/2, the browser doesn’t need a request queue at all. It can just send all the pending requests to the server, each with its priority, and let the server do the hard work of deciding which resource should be sent down first. The multiplexing capabilities also allow the server to interrupt low-priority responses whenever a higher-priority response data becomes available. The protocol also enables re-prioritization of requests—for example, when an image becomes visible in the viewport.

So for HTTP/2-enabled sites, when it comes to image priorities, the browser can actually permit itself to offload the prioritization smarts to the server, and just make sure that it sends the right priorities.

CSSOM and Background Image Download

In previous sections we talked about the preloader and the fact that it is used by the browser for early discovery of the resources that will be required later in the page. Unfortunately, since the main way it does this is by looking at HTML tokens, that doesn’t work well for CSS-based resources, and in particular the preloader doesn’t preload background images in any browser today.

While theoretically in some cases a browser could download background images using a mix of CSS tokenization preloading and smart heuristics based on HTML tokenization, no browser actually does that. Even if it did, such heuristics run a high risk of triggering spurious downloads due to the cascading nature of CSS.

In practice that means that background images are discovered pretty late in the page’s loading process, only after all CSS resources have finished downloading and the style has been calculated. So, if you have a prominent background image, how can you make sure that it’s discovered in a relatively early stage and loaded as soon as possible?

Up until recently you could only do so using hacks such as including an equivalent invisible <img> tag in your HTML, or a new Image().src='prominent_bg_img.jpg' inline script. But nowadays, you can use the shiny new preload directive and include something like <link rel=preload href='prominent_bg_img.jpg' as=image> in your markup to tell the browser that it needs to load that resource while treating it as an image in terms of priority, requests headers, or the like.

Full coverage of preload is outside the scope of this lesson, but if you’re curious, a recent article explains it in detail.

Service Workers and Image Decoding

Another recent development in the browser world is the advent of service workers (SW). In short, service workers are browser-based network proxies that you can set up to intercept and control your site’s entire network traffic. While the use cases for them are wide and cover many aspects of the page loading process, we will examine a particular use case for them: using service workers to roll your own image format!

We have discussed the hardships of image format compatibility, the various browser-specific formats, and the need to serve specific formats to specific browsers. But service workers bring another possiblity to that mix: you can now ship SW-based image decoders and serve new and improved image formats only when native support is in place or when a SW-based polyfill is installed. In the latter case, you can “decorate” the outgoing requests by, for example, extending the Accept header, and then convert the responses to an image format that the browser recognizes.

For example, newly developed formats that may be useful for delivering images on the web. But, no browser actually supports these formats, which means they are of little practical use on the web. Or are they?

With service workers, you can convert these formats to either JPEG or BMP in the browser, saving bytes over the network but still providing the browser with a format it can properly process and display. And even more, you can do that without any changes to your HTML or your application logic. SWs run at a lower layer, and perform all the required conversions without requiring your application’s awareness.

One caveat to that approach is that JavaScript decoding implementations run a risk of being more costly than the native, highly optimized image decoders. One future browser enhancement that can help on that front is better access to low-level image decoding APIs that can speed things up: browsers could expose an API that enables decoding of video iframes or arithmetic decoding and significantly speed up decoding of new image formats that rely on these primitives.