XML comes from a rich history of text processing systems. Markup actually predates the computer. Figure 2-1 shows a marked-up manuscript that might have been submitted to a human who would compose the type for printing.

Figure 2-1. A manuscript “marked up” by hand

The first wave of automated text processing was computer typesetting. An author would type a document and include style codes to describe how the document should be formatted. The computer would read the style codes and the rest of the text and print the document with the described formatting.

The file that contains the data of the document, plus the description of the desired format, is called a rendition. The style codes in the rendition file are called formatting markup.

The system interprets the formatting markup and converts the rendition into something physically perceivable by a human being – a presentation. The presentation medium was originally paper, but eventually electronic display was added.

This scenario isn’t totally history. You and Word work with renditions today. It’s just that Word gives you a nicer interface to manipulate them.

Word’s user interface to the rendition (that is, to the .doc file with the style codes in it) is designed to look like the presentation (the finished paper product). The interface is called a What You See Is What You Get (WYSI-WYG) interface. Since a rendition describes a presentation, it is convenient to have the user interface reflect the end-product.

More than convenient. For Word it is essential, as we saw in 1.2.1, “Separating the document representation from the software”, on page 9. Word’s native binary .doc rendition file format is unfathomable to humans, and to virtually all non-Microsoft software.

For interchange (and virus avoidance) Word also offers a plain-text equivalent called Rich Text Format (RTF). While computers can handle it better than .doc, humans won’t find it much of an improvement, as a glance at Example 2-1 will demonstrate.^[1]

pardplains1ql li0
i0sb240sa60keepnwidctlparaspalpha
aspnumfaautooutlinelevel0adjustright
in0lin0itap0
pararsid13583126 f1fs32lang1033langfe1033kerning32cgrid
langnp1033langfenp1033
{insrsid13583126par }
pard s1ql li0
i0sb240sa60keepnwidctlparaspalphaaspnum
faautooutlinelevel0adjustright
in0lin0itap0pararsid5243775
{insrsid10036224 Sales Updatepar }
pardplain ql li0
i0widctlparaspalphaaspnumfaauto
adjustright
in0lin0itap0pararsid10036224 fs24lang1033
langfe1033cgridlangnp1033langfenp1033
{insrsid16529125 Doug Jones}
{insrsid10036224par February 3, 2004par }
{insrsid10036224charrsid10036224par }
pardplain s2ql li0
i0sb240sa60keepnwidctlpar
aspalphaaspnumfaautooutlinelevel1adjustright
in0lin0itap0
pararsid12923755 if1fs28lang1033langfe1033cgridlangnp1033
langfenp1033
{insrsid10036224 A great month!}
{insrsid1358312par }
pardplain ql li0
i0widctlparaspalphaaspnumfaauto
adjustright
in0lin0itap0 fs24lang1033langfe1033cgrid
langnp1033langfenp1033
{insrsid10036224par This month
quote s figures are a }
{iinsrsid10036224charrsid10036224 huge}
{insrsid10036224 improvement over this month last year. We }
{insrsid12923755 sold 1,342 widgets for a total revenue of
$14,327.}
{insrsid10036224parpar }
pardplain s2ql li0
i0sb240sa60keepnwidctlparaspalpha
aspnumfaautooutlinelevel1adjustright
in0lin0itap0
pararsid12923755 if1fs28lang1033langfe1033cgridlangnp1033
langfenp1033
{insrsid10036224 More work to dopar }
pardplain ql li0
i0widctlparaspalphaaspnumfaauto
adjustright
in0lin0itap0 fs24lang1033langfe1033cgrid
langnp1033langfenp1033
{insrsid10036224 Let
quote s not rest on}{insrsid12923755 our
past success. Let
quote s get out there and sell, sell, sell!}
{insrsid10036224par }{insrsid13583126par }

Formatting markup is sufficient if your only goal is to create a single rendition and then print it. In 1969, IBM asked a young researcher named Charles Goldfarb (the name may sound familiar) to build a system for editing, searching, managing, and publishing legal documents.

Goldfarb found that there were IBM products for each of these tasks but they could not communicate with each other. They could not share information! Each of them used different markup. They could not read each other’s files, just as you may have had trouble loading WordPerfect files into Word or vice versa.

The problem then, as now, was that to integrate these diverse products, a neutral document representation^[2] was needed for the information – one that wasn’t designed for a single product.

Goldfarb – later joined by two other IBM researchers, Ed Mosher and Ray Lorie – set out to solve this problem. The team recognized (eventually) that the solution would need to satisfy four principles:

neutral data representation (markup language)

extensible markup

rule-based markup

stylesheets

These principles are important far beyond the exchange of traditional documentation. In fact they underlie the exchange of any form of information.

The IBM team’s solution was the Generalized Markup Language (GML), which Goldfarb later drew on for his invention of the Standard Generalized Markup Language (SGML) – the parent of HTML and XML.

In the following sections, we’ll look at each of these principles in a bit more detail and see how they apply to Word.

Extensible markup

The IBM team realized that the neutral representation should be specific to legal documents while at the same time being general enough to be used for things that are completely unrelated to the law. This seems like a paradox but it is not as impossible as it sounds!

Vocabularies

This idea is a little more subtle to grasp, but vital to understanding XML. For example, lawyers and scientists both use Latin, but they do not use the same vocabulary.

Similarly, in Example 2-2 we have Doug’s newsletter article in XML, using a vocabulary that was designed for newsletter articles. In Example 2-3, however, we have the same article represented using a different vocabulary.

Example 2-2. Doug’s article, represented in article XML

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<article xmlns="http://xmlinoffice.com/article"
         type="sales" id="A123">
  <title>Sales Update</title>
  <author>Doug Jones</author>
  <date>February 3, 2004</date>
  <body>
   <section>
     <header>A great month!</header>
     <para>This month's figures are a <em>huge</em> improvement over
this month last year. We sold 1,342 widgets for a total revenue of
$14,327.</para>
   </section>
   <section>
     <header>More work to do</header>
     <para>Let's not rest on our past success. Let's get out there
and sell, sell, sell!</para>
   </section>
  </body>
</article>

Example 2-3. Doug’s article, represented in WordML XML (article WordML.xml)

<w:body>
 <wx:sect>
  <wx:sub-section>
   <w:p><w:pPr><w:pStyle w:val="Heading1"/></w:pPr></w:p>
  </wx:sub-section>
  <wx:sub-section>
   <w:p>
    <w:pPr><w:pStyle w:val="Heading1"/></w:pPr>
    <w:r><w:t>Sales Update</w:t></w:r></w:p>
   <w:p><w:r><w:t>Doug Jones</w:t></w:r></w:p>
   <w:p><w:r><w:t>February 3, 2004</w:t></w:r></w:p><w:p/>
   <wx:sub-section>
    <w:p>
     <w:pPr><w:pStyle w:val="Heading2"/></w:pPr>
     <w:r><w:t>A great month!</w:t></w:r></w:p>
    <w:p/>
    <w:p>
     <w:r><w:t>This month's figures are a </w:t></w:r>
     <w:r><w:rPr><w:i/></w:rPr><w:t>huge</w:t></w:r>
     <w:r><w:t> improvement over this month last year. We sold 1,342
widgets for </w:t></w:r><w:proofErr w:type="gramStart"/>
     <w:r><w:t>a total</w:t></w:r><w:proofErr w:type="gramEnd"/>
     <w:r><w:t> revenue of $14,327.</w:t></w:r></w:p><w:p/>
   </wx:sub-section>
   <wx:sub-section>
    <w:p>
     <w:pPr><w:pStyle w:val="Heading2"/></w:pPr>
     <w:r><w:t>More work to do</w:t></w:r></w:p>
    <w:p><w:r><w:t>Let's not rest on our past success. Let's get out
there and sell, sell, sell!</w:t></w:r></w:p><w:p/>
    <w:sectPr>
     <w:pgSz w:w="12240" w:h="15840"/>
     <w:pgMar w:top="1440" w:right="1800" w:bottom="1440"
      w:left="1800" w:header="720" w:footer="720" w:gutter="0"/>
     <w:cols w:space="720"/><w:docGrid w:line-pitch="360"/>
    </w:sectPr>
   </wx:sub-section>
  </wx:sub-section>
 </wx:sect>
</w:body>

Before we look at the differences between the two XML representations, let’s use Example 2-2 to learn the basics of XML: elements and tags, content and data, and attributes.

elements and tags

The document has one title element. It begins with a start-tag (<title>) and ends with a corresponding end-tag (</title>). The forward slash differentiates an end-tag from a start-tag. The purpose of the tags is to identify the type of element (title, in this case) and show where it starts and ends.

content and data

The text in between the tags is the content of the element. In the case of title the content is entirely data characters. However, the content of body has only markup that represents elements, and the first para has mixed content: data and an element (em). Elements that occur in the content of elements form a hierarchy. At the top is the document element (or root element): in this case, article.

attributes

An element can have properties besides its element type name and content. These are represented by name-value pairs called attributes. In the example, only article has attributes. The value of the one named id is the name of this particular article element, which is useful when a group of articles is combined in a newsletter. The xmlns attribute declares a namespace, which we explain in 2.6, “Namespaces”, on page 39.

The attribute names and element-type names, such as title, comprise a custom vocabulary specifically designed to describe newsletter articles. In contrast, Example 2-3 uses a vocabulary called the Word Markup Language (WordML).^[4]

WordML uses XML, as do thousands of other data representations. However, the WordML vocabulary is unique. It was designed by Microsoft to be the XML equivalent of .doc and RTF. As such, it describes components and properties of the Word formatting model. Even if one of its element-type names were the same as one in some other vocabulary, it would probably not identify the same type of thing. (We explain WordML in detail in Chapter 5, “Rendering and presenting XML documents”, on page 86.)

The reason that generalized markup became successful is that users can create their own vocabularies to meet their needs. But all vocabularies use the standard markup language syntax (and other language constructs), so tools can be developed to do the large amount of common processing that is necessary for all vocabularies.

Abstractions and renditions

Computers are not as smart as we are. If we want the computer to consider a piece of text to be written in a foreign language (for instance for spell-checking purposes) then we must label it explicitly foreign-phrase and not just put it in italics! The “foreign phrase” is the abstraction that we are trying to represent; italics is just a particular rendition of that abstraction for visual presentations. For audible presentations, the rendition might be a voice with an accent.

Formatting markup is specific to a particular use of the information. Search engines cannot do very useful searching on italics because they do not know why something is italicized. It could be a foreign phrase but it could also be a citation of another document.

In contrast, the search engine could do something very helpful with suitably-marked citation elements: it could return a list of those documents that are cited by other documents.

Italics are a form of markup specific to a particular application: formatting. In contrast, the citation element is markup that can be used by a variety of applications. That is why Goldfarb named this form of markup generalized markup. Generalized markup is the alternative to formatting markup and other specialized single-use coding schemes.

Caution

Generalized markup is often called structured markup and the act of using it structuring a document. Unfortunately, “structured” is the most misused term in markup languages, with at least four different meanings (see 20.1, “Structured vs. unstructured”, on page 430). This use of it implies that only abstractions have structure, which Figure 2-2 clearly refutes. It shows the Word template ProfessionalReport.dot with the Document Map and Style Area in view. The structural hierarchy is in the left pane and style codes are in the center. A Word template is not just a stylesheet, but also a guide to the structure of the rendition.

Figure 2-2. Structure of a rendition

<!ELEMENT Q-AND-A (QUESTION,ANSWER)+>
<!-- This allows: question, answer, question, answer ... -->
<!ELEMENT QUESTION (#PCDATA)>
<!-- Questions are just made up of textual data -->
<!ELEMENT ANSWER (#PCDATA)>
<!-- Answers are just made up of textual data -->

<schema xmlns='http://www.w3.org/2001/XMLSchema'
        xmlns:qa='http://www.q.and.a.com/'
        targetNamespace='http://www.q.and.a.com/'>
 <element name="Q-AND-A">
 <complexType>
  <sequence minOccurs="1" maxOccurs="unbounded">
   <element ref="qa:QUESTION"/>
   <element ref="qa:ANSWER"/>
  </sequence>
 </complexType>
 </element>
<!-- This allows: question, answer, question, answer ... -->
 <element name="QUESTION" type="string"/>
<!-- Questions are just made up of textual data -->
 <element name="ANSWER" type="string"/>
<!-- Answers are just made up of textual data -->
</schema>

Stylesheets

Of course, if you are using XML for publishing, you must still be able to generate high quality print and online renditions of the document. Your readers do not want to read XML text directly. Instead of directly inserting the formatting commands in the XML document, we usually tell the computer how to generate formatted renditions from the XML abstraction.

For example in a print presentation, we can make the content of TITLE elements bold and large, insert page breaks before the beginning of chapters, and turn emphasis, citations and foreign words into italics. These rules are specified in a file called a stylesheet. The stylesheet is where human designers can express their creativity and understanding of formatting conventions. The stylesheet allows the computer to automatically convert the document from the abstraction to a formatted rendition.

Stylesheets for XML invariably conform to the Extensible Style Language Transformations (XSLT) W3C recommendation. XSLT can do much more processing than formatting markup ever could. Often it is used for tasks that don’t involve formatting at all.

Moreover, XSLT stylesheets are normally written to apply to all documents of a given type, rather than a single document. Just as the DTD or schema sets the rules for markup, an XSLT stylesheet is a set of rules for processing, as depicted in Figure 2-3.^[6]

Figure 2-3. Rule-based processing

Most documents (for example books and magazines) can easily be broken down into components (chapters and articles). These can also be broken down into components (titles, paragraphs, figures and so forth). And those components can be broken down into components until we get to the textual data itself – words and sentences. At this point we would typically stop breaking the document into components unless we were interested in linguistic research.

It turns out that every document can be viewed this way, though some fit the model more naturally than others. In fact all information can be viewed this way...with the same caveat!

In XML, these components are called elements. Each element represents a logical component of a document. Elements can contain other elements and can also contain the words and sentences that you would usually think of as the text of the document. XML calls this text the document’s character data. This hierarchical view of XML documents is demonstrated in Figure 2-4.

Figure 2-4. Hierarchical views of documents

Markup professionals call this the tree structure of the document. The element that contains all of the others (e.g. Book, Article or Memo) is known as the root element. This name captures the fact that it is the only element that does not “hang” off of some other element. The root element is also referred to as the document element because it holds the entire logical document within it. The terms root element and document element are interchangeable.

The elements that are contained in the root are called its subelements. They may contain subelements themselves. If they do, we will call them branches. If they do not, we will call them leaves.

Thus, the Chapter and Section elements are branches (because they have subelements), but the Paragraph and Title elements are leaves (because they only contain character data).

Elements can also have extra information attached to them called attributes. Attributes describe properties of elements. For instance a CIA-record element might have a security attribute that gives the security rating for that element. A CIA database might only release certain records to certain people depending on their security rating. It is somewhat of a judgement call which aspects of a document should be represented with elements and which should be represented with attributes.

Real-world documents do not always fit this tree model perfectly. They often have non-hierarchical features such as cross-references or hypertext links from one section of the tree to another. XML can represent these structures too.

A WordML document also has an element structure. But as the WordML document type is a complex rendition with over 400 element types, the structure isn’t usually as clear as the abstraction in Figure 2-4.

Every language has rules about what is or is not correct in the language. In human languages that takes many forms: words have a particular correct pronunciation (or range of pronunciations) and they can be combined in certain ways to make valid sentences (grammar). Similarly XML has two different notions of “correct”. The first is merely that the markup is intelligible: the XML equivalent of “getting the pronunciation right”. A document with intelligible markup is called a well-formed document. One important goal of XML was that these basic rules should be simple so that they could be strictly adhered to.

The XML equivalent of “using the right words in the right place” is called validity and is related to the notion of document types. A document is valid if it declares conformance to a DTD in a document type declaration and actually conforms to that DTD.

A document could also (or exclusively) be identified as conforming to a schema. If it actually does conform, it is said to be schema valid (commonly shortened to “valid”).

When the Web began it was pretty simple. You entered a Web address into your browser and it displayed the page from that address. The address was a Uniform Resource Locator (URL) and the page was marked up in HTML.

The W3C maintains a DTD for HTML, but many HTML documents don’t conform to it. So browsers attempted to cope with errors rather than report them to the users (who couldn’t correct them in any case).

The Web has since diverged from that simple model, both in terms of the addresses and the delivered pages. The addressing model has gotten richer, and the pages mostly don’t exist until they hit your browser.

There is a problem that arises when you allow anybody to pick names as XML does. The problem is that different people in different places will invariably use the same names for different things. This makes it very difficult to build systems that work with documents from multiple independent sources because a publisher could use the element-type name PAR to mean paragraph while a military vocabulary could use an element type with the same name to mean paratrooper. A mathematician might use an element type with that name to label paradoxes!

Identifiers

Of course, this solution appears to create its own problem: How can you be sure that different XML designers will use met to refer to the same vocabulary? The short and happy answer is: They don’t have to!

That’s because within your document met – which, remember, is just a nickname – is associated with an unambiguous identifier of the vocabulary. That identifier is a URI reference and could look something like http://www.weatherworld.com. Other documents might use a different abbreviation for that vocabulary or use met: to prefix a different vocabulary.

In practice, though, vocabulary developers recommend prefixes and people tend to use the recommended ones.

Note

The URI of a namespace does not have to address a real Web page. The Web addressing system is just being used as a namespace within which all URIs – and therefore namespace names – are unique.

There are a few more XML constructs that you may need to know about, but which you will use less often – if at all – than those previously discussed. The details, including information about the nature and extent of Word’s support for them, can be found in Chapter 15, “The XML language”, on page 350.

There are two, however, that show up in the examples that illustrate Part Two: processing instructions (PIs) and comments. They are both markup that is not part of the document structure or data. They are essentially messages: to software in the case of PIs, and to people in the case of comments.

<?xml version="1.0" encoding="UTF-8"?>
<?mso-application progid="Word.Document"?>

<!-- Help, I'm a prisoner in a publishing house! -->

If you plan to use XML products other than Office, develop your own schemas, or use Office to share data with enterprise systems and Web services, you’ll want to learn more about XML. As our readers will have differing experience with XML and different requirements for its use, there seemed no sensible way to intersperse detailed XML education with the Office XML tasks that are the focus of the book.

Instead we’ve put the detailed tutorials and references on the XML language and related standards where you can easily find them when you need them. They are in Part 3, “XML Tutorials”, on page 348. The book’s Table of Contents and Index can guide you to specific subjects, and we also provide appropriate cross-references to the tutorials from the Office XML task chapters in Part Two.