The last collection type in our survey is the Python tuple. Tuples construct simple groups of objects. They work exactly like lists, except that tuples can’t be changed in-place (they’re immutable) and are usually written as a series of items in parentheses, not square brackets. Although they don’t support any method calls, tuples share most of their properties with lists. Tuples are:

Table 7-1 highlights common tuple operations. Tuples are written as a series of objects (really, expressions that generate objects), separated by commas, and enclosed in parentheses. An empty tuple is just a parentheses pair with nothing inside.

Notice that tuples have no methods (e.g., an append call won’t work here), but do support the usual sequence operations that we saw for strings and lists:

>>> (1, 2) + (3, 4)             # Concatenation
(1, 2, 3, 4)

>>> (1, 2) * 4                  # Repitition
(1, 2, 1, 2, 1, 2, 1, 2)

>>> T = (1, 2, 3, 4)            # Indexing, slicing
>>> T[0], T[1:3]
(1, (2, 3))

The second and fourth entries in Table 7-1 merit a bit more explanation. Because parentheses can also enclose expressions (see Section 4.3), you need to do something special to tell Python when a single object in parentheses is a tuple object and not a simple expression. If you really want a single-item tuple, simply add a trailing comma after the single item and before the closing parenthesis:

>>> x = (40)         # An integer
>>> x
40
>>> y = (40,)        # A tuple containing an integer
>>> y
(40,)

As a special case, Python also allows you to omit the opening and closing parentheses for a tuple in contexts where it isn’t syntactically ambiguous to do so. For instance, the fourth line of the table simply listed four items, separated by commas. In the context of an assignment statement, Python recognizes this as a tuple, even though it didn’t have parentheses. For beginners, the best advice is that it’s probably easier to use parentheses than it is to figure out when they’re optional. Many programmers also find that parenthesis tend to aid script readability.

Apart from literal syntax differences, tuple operations (the last three rows in Table 7-1) are identical to strings and lists. The only differences worth noting are that the +, *, and slicing operations return new tuples when applied to tuples, and tuples don’t provide the methods you saw for strings, lists, and dictionaries. If you want to sort a tuple, for example, you’ll usually have to first convert it to a list to gain access to a sorting method call, and make it a mutable object:

>>> T = ('cc', 'aa', 'dd', 'bb')
>>> tmp = list(T)
>>> tmp.sort(  )
>>> tmp
['aa', 'bb', 'cc', 'dd']
>>> T = tuple(tmp)
>>> T
('aa', 'bb', 'cc', 'dd')

Here, the list and tuple built-in functions were used to convert to a list, and then back to a tuple; really, both calls make new objects, but the net effect is like a conversion. Also note that the rule about tuple immutability only applies to the top-level of the tuple itself, not to its contents; a list inside a tuple, for instance, can be changed as usual:

>>> T = (1, [2, 3], 4)
>>> T[1][0] = 'spam'                  # Works
>>> T
(1, ['spam', 3], 4)
>>> T[1] = 'spam'                     # Fails
TypeError: object doesn't support item assignment

Most readers are probably familiar with the notion of files—named storage compartments on your computer that are managed by your operating system. This last built-in object type provides a way to access those files inside Python programs. The built-in open function creates a Python file object, which serves as a link to a file residing on your machine. After calling open, you can read and write the associated external file, by calling file object methods. The built-in name file is a synonym for open, and files may be opened by calling either name.

Compared to the types you’ve seen so far, file objects are somewhat unusual. They’re not numbers, sequences, or mappings; instead, they export methods only for common file processing tasks.

Table 7-2 summarizes common file operations. To open a file, a program calls the open function, with the external name first, followed by a processing mode ('r' to open for input—the default; 'w' to create and open for output; 'a' to open for appending to the end; and others we’ll omit here). Both arguments must be Python strings. The external file name argument may include a platform-specific and absolute or relative directory path prefix; without a path, the file is assumed to exist in the current working directory (i.e., where the script runs).

Once you have a file object, call its methods to read from or write to the external file. In all cases, file text takes the form of strings in Python programs; reading a file returns its text in strings, and text is passed to the write methods as strings. Reading and writing both come in multiple flavors; Table 7-2 gives the most common.

Calling the file close method terminates your connection to the external file. In Python, an object’s memory space is automatically reclaimed as soon as the object is no longer referenced anywhere in the program. When file objects are reclaimed, Python also automatically closes the file if needed. Because of that, you don’t need to always manually close your files, especially in simple scripts that don’t run long. On the other hand, manual close calls can’t hurt and are usually a good idea in larger systems. Strictly speaking, this auto-close-on-collection feature of files is not part of the language definition, and may change over time. Because of that, manual file close method calls are a good habit to form.

>>> myfile = open('myfile', 'w')             # Open for output (creates).
>>> myfile.write('hello text file
')        # Write a line of text.
>>> myfile.close(  )

>>> myfile = open('myfile', 'r')             # Open for input.
>>> myfile.readline(  )                           # Read the line back.
'hello text file
'
>>> myfile.readline(  )                        # Empty string: end of file
''

Now that we’ve seen all of Python’s core built-in types, let’s take a look at some of the properties they share.

Table 7-3 classifies all the types we’ve seen, according to the type categories we introduced earlier. Objects share operations according to their category—for instance, strings, lists, and tuples all share sequence operations. Only mutable objects may be changed in-place. You can change lists and dictionaries in-place, but not numbers, strings, or tuples. Files only export methods, so mutability doesn’t really apply (they may be changed when written, but this isn’t the same as Python type constraints).

class MySequence:
    def __getitem__(self, index):
        # Called on self[index], others
    def __add__(self, other):
        # Called on self + other

We’ve seen a number of compound object types (collections with components). In general:

Because they support arbitrary structures, Python’s compound object types are good at representing complex information in a program. For example, values in dictionaries may be lists, which may contain tuples, which may contain dictionaries, and so on—as deeply nested as needed to model the data to be processed.

Here’s an example of nesting. The following interaction defines a tree of nested compound sequence objects, shown in Figure 7-1. To access its components, you may include as many index operations as required. Python evaluates the indexes from left to right, and fetches a reference to a more deeply nested object at each step. Figure 7-1 may be a pathologically complicated data structure, but it illustrates the syntax used to access nested objects in general:

>>> L = ['abc', [(1, 2), ([3], 4)], 5]
>>> L[1]
[(1, 2), ([3], 4)]
>>> L[1][1]
([3], 4)
>>> L[1][1][0]
[3]
>>> L[1][1][0][0]
3

Figure 7-1. A nested object tree

Section 4.6 in Chapter 4 mentioned that assignments always store references to objects, not copies. In practice, this is usually what you want. But because assignments can generate multiple references to the same object, you sometimes need to be aware that changing a mutable object in-place may affect other references to the same object elsewhere in your program. If you don’t want such behavior, you’ll need to tell Python to copy the object explicitly.

For instance, the following example creates a list assigned to X, and another assigned to L that embeds a reference back to list X. It also creates a dictionary D that contains another reference back to list X:

>>> X = [1, 2, 3]
>>> L = ['a', X, 'b']           # Embed references to X's object.
>>> D = {'x':X, 'y':2}

At this point, there are three references to the first list created: from name X, from inside the list assigned to L, and from inside the dictionary assigned to D. The situation is illustrated in Figure 7-2.

Figure 7-2. Shared object references

Since lists are mutable, changing the shared list object from any of the three references changes what the other two reference:

>>> X[1] = 'surprise'         # Changes all three references!
>>> L
['a', [1, 'surprise', 3], 'b']
>>> D
{'x': [1, 'surprise', 3], 'y': 2}

References are a higher-level analog of pointers in other languages. Although you can’t grab hold of the reference itself, it’s possible to store the same reference in more than one place: variables, lists, and so on. This is a feature—you can pass a large object around a program without generating copies of it along the way. If you really do want copies, you can request them:

For example, if you have a list and a dictionary, and don’t want their values to be changed through other variables:

>>> L = [1,2,3]
>>> D = {'a':1, 'b':2}

simply assign copies to the other variables, not references to the same objects:

>>> A = L[:]              # Instead of: A = L (or list(L))
>>> B = D.copy(  )            # Instead of: B = D

This way, changes made from other variables change the copies, not the originals:

>>> A[1] = 'Ni'
>>> B['c'] = 'spam'
>>>
>>> L, D
([1, 2, 3], {'a': 1, 'b': 2})
>>> A, B
([1, 'Ni', 3], {'a': 1, 'c': 'spam', 'b': 2})

In terms of the original example, you can avoid the reference side effects by slicing the original list, instead of simply naming it:

>>> X = [1, 2, 3]
>>> L = ['a', X[:], 'b']          # Embed copies of X's object.
>>> D = {'x':X[:], 'y':2}

This changes the picture in Figure 7-2—L and D will point to different lists than X. The net effect is that changes made through X will impact only X, not L and D; similarly, changes to L or D will not impact X.

One note on copies: empty-limit slices and the copy method of dictionaries still only make a top-level copy—they do not copy nested data structures, if any are present. If you need a complete, fully independent copy of a deeply nested data structure, use the standard copy module: import copy, and say X=copy.deepcopy(Y) to fully copy an arbitrarily nested object Y. This call recursively traverses objects to copy all their parts. This is the much more rare case, though (which is why you have to say more to make it go). References are usually the behaviour you will want; when they are not, slices and copy methods are usually as much copying as you’ll need to do.

All Python objects also respond to the comparisons: test for equality, relative magnitude, and so on. Python comparisons always inspect all parts of compound objects, until a result can be determined. In fact, when nested objects are present, Python automatically traverses data structures to apply comparisons recursively—left to right, and as deep as needed.

For instance, a comparison of list objects compares all their components automatically:

>>> L1 = [1, ('a', 3)]         # Same value, unique objects
>>> L2 = [1, ('a', 3)]
>>> L1 == L2, L1 is L2         # Equivalent? Same object?
(1, 0)

Here, L1 and L2 are assigned lists that are equivalent, but distinct objects. Because of the nature of Python references (studied in Chapter 4), there are two ways to test for equality:

In the example, L1 and L2 pass the == test (they have equivalent values because all their components are equivalent), but fail the is check (they are two different objects, and hence two different pieces of memory). Notice what happens for short strings:

>>> S1 = 'spam'
>>> S2 = 'spam'
>>> S1 == S2, S1 is S2
(1, 1)

Here, we should have two distinct objects that happen to have the same value: == should be true, and is should be false. Because Python internally caches and reuses short strings as an optimization, there really is just a single string, 'spam', in memory, shared by S1 and S2; hence, the is identity test reports a true result. To trigger the normal behavior, we need to use longer strings that fall outside the cache mechanism:

>>> S1 = 'a longer string'
>>> S2 = 'a longer string'
>>> S1 == S2, S1 is S2
(1, 0)

Because strings are immutable, the object caching mechanism is irrelevent to your code—string can’t be changed in-place, regardless of how many variables refer to them. If identity tests seem confusing, see Section 4.6 in Chapter 4 for a refresher on object reference concepts.

As a rule of thumb, the == operator is what you will want to use for almost all equality checks; is is reserved for highly specialized roles. We’ll see cases of both operators put to use later in the book.

Notice that relative magnitude comparisons are also applied recursively to nested data structures:

>>> L1 = [1, ('a', 3)]
>>> L2 = [1, ('a', 2)]
>>> L1 < L2, L1 == L2, L1 > L2     # less,equal,greater: tuple of results
(0, 0, 1)

Here, L1 is greater than L2 because the nested 3 is greater than 2. The result of the last line above is really a tuple of three objects—the results of the three expressions typed (an example of a tuple without its enclosing parentheses).

The three values in this tuple represent true and false values; in Python, an integer 0 represents false and an integer 1 represents true. Python also recognizes any empty data structure as false, and any nonempty data structure as true. More generally, the notions of true and false are intrinsic properties of every object in Python—each object is true or false, as follows:

Table 7-4 gives examples of true and false objects in Python.

Python also provides a special object called None (the last item in Table 7-4), which is always considered to be false. None is the only value of a special data type in Python; it typically serves as an empty placeholder, much like a NULL pointer in C.

For example, recall that for lists, you cannot assign to an offset unless that offset already exists (the list does not magically grow if you assign out of bounds). To preallocate a 100-item list such that you can add to any of the 100 offsets, you can fill one with None objects:

>>> L = [None] * 100
>>>
>>> L
[None, None, None, None, None, None, None, . . . ]

In general, Python compares types as follows:

In later chapters, we’ll see other object types that can change the way they get compared.

Figure 7-3 summarizes all the built-in object types available in Python and their relationships. We’ve looked at the most prominent of these; most other kinds of objects in Figure 7-3 either correspond to program units (e.g., functions and modules), or exposed interpreter internals (e.g., stack frames and compiled code).

Figure 7-3. Built-in type hierarchies

The main point to notice here is that everything is an object type in a Python system and may be processed by your Python programs. For instance, you can pass a class to a function, assign it to a variable, stuff it in a list or dictionary, and so on.

Even types are an object type in Python: a call to the built-in function type(X) returns the type object of object X. Type objects can be used for manual type comparisons in Python if statements. However, for reasons to be explained in Part IV, manual type testing is usually not the right thing to do in Python.

A note on type names: as of Python 2.2, each core type has a new built-in name added to support type subclassing: dict, list, str, tuple, int, long, float, complex, unicode, type, and file (file is a synonym for open). Calls to these names are really object constructor calls, not simply conversion functions.

The types module provides additional type names (now largely synonyms for the built-in type names), and it is possible to do type tests with the isinstance function. For example, in Version 2.2, all of the following type tests are true:

isinstance([1],list)
type([1])==list
type([1])==type([  ])
type([1])==types.ListType

Because types can be subclassed in 2.2, the isinstance technique is generally recommended. See Chapter 23 for more on subclassing built-in types in 2.2 and later.

Besides the core objects studied in this chapter, a typical Python installation has dozens of other object types available as linked-in C extensions or Python classes. You’ll see examples of a few later in the book—regular expression objects, DBM files, GUI widgets, and so on. The main difference between these extra tools and the built-in types just seen is that the built-ins provide special language creation syntax for their objects (e.g., 4 for an integer, [1,2] for a list, the open function for files). Other tools are generally exported in a built-in module that you must first import to use. See Python’s library reference for a comprehensive guide to all the tools available to Python programs.

Part II concludes with a discussion of common problems that seem to bite new users (and the occasional expert), along with their solutions.

>>> L = [1, 2, 3]
>>> M = ['X', L, 'Y']       # Embed a reference to L.
>>> M
['X', [1, 2, 3], 'Y']

>>> L[1] = 0                # Changes M too
>>> M
['X', [1, 0, 3], 'Y']

>>> L = [1, 2, 3]
>>> M = ['X', L[:], 'Y']       # Embed a copy of L.
>>> L[1] = 0                   # Changes only L, not M 
>>> L
[1, 0, 3]
>>> M
['X', [1, 2, 3], 'Y']

>>> L = [4, 5, 6]
>>> X = L * 4           # Like [4, 5, 6] + [4, 5, 6] + ...
>>> Y = [L] * 4         # [L] + [L] + ... = [L, L,...]

>>> X
[4, 5, 6, 4, 5, 6, 4, 5, 6, 4, 5, 6]
>>> Y
[[4, 5, 6], [4, 5, 6], [4, 5, 6], [4, 5, 6]]

>>> L[1] = 0            # Impacts Y but not X
>>> X
[4, 5, 6, 4, 5, 6, 4, 5, 6, 4, 5, 6]
>>> Y
[[4, 0, 6], [4, 0, 6], [4, 0, 6], [4, 0, 6]]

>>> L = ['grail']              # Append reference to same object.
>>> L.append(L)                # Generates cycle in object: [...]
>>> L
['grail', [...]]

                  T = (1, 2, 3)
                  T[2] = 4             # Error!
                  T = T[:2] + (4,)     # Okay: (1, 2, 4)

This session asks you to get your feet wet with built-in object fundamentals. As before, a few new ideas may pop up along the way, so be sure to flip to Section B.2 when you’re done (and even when you’re not). If you have limited time, we suggest starting with exercise 11 (the most practical of the bunch), and then working from first to last as time allows. This is all fundamental material, though, so try to do as many of these as you can.