At the beginning of this chapter, we saw that a function can take input parameters. Before we delve into all possible type of parameters, let's make sure you have a clear understanding of what passing a parameter to a function means. There are three key points to keep in mind:
- Argument passing is nothing more than assigning an object to a local variable name
- Assigning an object to an argument name inside a function doesn't affect the caller
- Changing a mutable object argument in a function affects the caller
Let's look at an example for each of these points.
Take a look at the following code. We declare a name x in the global scope, then we declare a function func(y) and we call it, passing x. I highlighted the call in the code.
key.points.argument.passing.py
x = 3
def func(y):
print(y)
func(x) # prints: 3When func is called with x, what happens is that within its local scope, a name y is created, and it's pointed to the same object x is pointing to. This is better clarified by the following picture:
The right part of the preceding picture depicts the state of the program when execution has reached the end, after func has returned (None). Take a look at the Frames column, and note that we have two names, x and func, in the global namespace (Global frame), pointing to an int (with a value of three) and to a function object, respectively. Right below it, in the rectangle titled func, we can see the function's local namespace, in which only one name has been defined: y. Because we have called func with x (line 5 in the left part of the picture), y is pointing to the same object that x is pointing to. This is what happens under the hood when an argument is passed to a function. If we had used the name x instead of y in the function definition, things would have been exactly the same (only maybe a bit confusing at first), there would be a local x in the function, and a global x outside, as we saw in the Scopes and name resolution section.
So, in a nutshell, what really happens is that the function creates in its local scope the names defined as arguments and, when we call it, we basically tell Python which objects those names must be pointed towards.
This is something that can be tricky to understand at first, so let's look at an example.
key.points.assignment.py
x = 3
def func(x):
x = 7 # defining a local x, not changing the global one
func(x)
print(x) # prints: 3In the preceding code, when the line x = 7 is executed, what happens is that within the local scope of the function func, the name x is pointed to an integer with value 7, leaving the global x unaltered.
This is the final point, and it's very important because Python apparently behaves differently with mutables (just apparently though). Let's look at an example:
key.points.mutable.py
x = [1, 2, 3]
def func(x):
x[1] = 42 # this affects the caller!
func(x)
print(x) # prints: [1, 42, 3]Wow, we actually changed the original object! If you think about it, there is nothing weird in this behavior. The name x in the function is set to point to the caller object by the function call and within the body of the function, we're not changing x, in that we're not changing its reference, or, in other words, we are not changing the object x is pointing to. What we're doing is accessing that object's element at position 1, and changing its value.
Remember point #2: "Assigning an object to an argument name within a function doesn't affect the caller". If that is clear to you, the following code should not be surprising.
key.points.mutable.assignment.py
x = [1, 2, 3]
def func(x):
x[1] = 42 # this changes the caller!
x = 'something else' # this points x to a new string object
func(x)
print(x) # still prints: [1, 42, 3]Take a look at the two lines I have highlighted. At first, we just access the caller object again, at position 1, and change its value to number 42. Then, we reassign x to point to the string 'something else'. This leaves the caller unaltered, according to point #2, and, in fact, the output is the same as that of the previous snippet.
Take your time to play around with this concept and experiment with prints and calls to the id function until everything is clear in your mind. This is one of the key aspects of Python and it must be very clear, otherwise you risk introducing subtle bugs into your code.
Now that we have a good understanding of input parameters and how they behave, let's see how we can specify them.
There are five different ways of specifying input parameters. Let's look at them one by one.
Positional arguments are read from left to right and they are the most common type of arguments.
arguments.positional.py
def func(a, b, c): print(a, b, c) func(1, 2, 3) # prints: 1 2 3
There is not much else to say. They can be as numerous as you want and they are assigned by position. In the function call, 1 comes first, 2 comes second and 3 comes third, therefore they are assigned to a, b and c respectively.
Keyword arguments are assigned by keyword using the name=value syntax.
arguments.keyword.py
def func(a, b, c):
print(a, b, c)
func(a=1, c=2, b=3) # prints: 1 3 2Keyword arguments act when calling the function instead of respecting the left-to-right positional assignment, k. Keyword arguments are matched by name, even when they don't respect the definition's original position (we'll see that there is a limitation to this behavior later, when we mix and match different types of arguments).
The counterpart of keyword arguments, on the definition side, is default values. The syntax is the same, name=value, and allows us to not have to provide an argument if we are happy with the given default.
arguments.default.py
def func(a, b=4, c=88):
print(a, b, c)
func(1) # prints: 1 4 88
func(b=5, a=7, c=9) # prints: 7 5 9
func(42, c=9) # prints: 42 4 9The are two things to notice, which are very important. First of all, you cannot specify a default argument on the left of a positional one. Second, note how in the examples, when an argument is passed without using the argument_name=value syntax, it must be the first one in the list,, and it is always assigned to a. Try and scramble those arguments and see what happens. Python error messages are very good at telling you what's wrong. So, for example, if you tried something like this:
func(b=1, c=2, 42) # positional argument after keyword one
You would get the following error:
SyntaxError: non-keyword arg after keyword arg
This informs you that you've called the function incorrectly.
Sometimes you may want to pass a variable number of positional arguments to a function and Python provides you with the ability to do it. Let's look at a very common use case, the minimum function. This is a function that calculates the minimum of its input values.
arguments.variable.positional.py
def minimum(*n):
# print(n) # n is a tuple
if n: # explained after the code
mn = n[0]
for value in n[1:]:
if value < mn:
mn = value
print(mn)
minimum(1, 3, -7, 9) # n = (1, 3, -7, 9) - prints: -7
minimum() # n = () - prints: nothingAs you can see, when we specify a parameter prepending a * to its name, we are telling Python that that parameter will be collecting a variable number of positional arguments, according to how the function is called. Within the function, n is a tuple. Uncomment the print(n) to see for yourself and play around with it for a bit.
Note
Have you noticed how we checked if n wasn't empty with a simple if n:? This is due to the fact that collection objects evaluate to True when non-empty, and otherwise False in Python. This is true for tuples, sets, lists, dictionaries, and so on.
One other thing to note is that we may want to throw an error when we call the function with no arguments, instead of silently doing nothing. In this context, we're not concerned about making this function robust, but in understanding variable positional arguments.
Let's make another example to show you two things that, in my experience, are confusing to those who are new to this.
arguments.variable.positional.unpacking.py
def func(*args):
print(args)
values = (1, 3, -7, 9)
func(values) # equivalent to: func((1, 3, -7, 9))
func(*values) # equivalent to: func(1, 3, -7, 9)
Take a good look at the last two lines of the preceding example. In the first one, we call func with one argument, a four elements tuple. In the second example, by using the * syntax, we're doing something called unpacking, which means that the four elements tuple is unpacked, and the function is called with four arguments: 1, 3, -7, 9.
This behavior is part of the magic Python does to allow you to do amazing things when calling functions dynamically.
Variable keyword arguments are very similar to variable positional arguments. The only difference is the syntax (** instead of *) and that they are collected in a dictionary. Collection and unpacking work in the same way, so let's look at an example:
arguments.variable.keyword.py
def func(**kwargs):
print(kwargs)
# All calls equivalent. They print: {'a': 1, 'b': 42}
func(a=1, b=42)
func(**{'a': 1, 'b': 42})
func(**dict(a=1, b=42))All the calls are equivalent in the preceding example. You can see that adding a ** in front of the parameter name in the function definition tells Python to use that name to collect a variable number of keyword parameters. On the other hand, when we call the function, we can either pass name=value arguments explicitly, or unpack a dictionary using the same ** syntax.
The reason why being able to pass a variable number of keyword parameters is so important may not be evident at the moment, so, how about a more realistic example? Let's define a function that connects to a database. We want to connect to a default database by simply calling this function with no parameters. We also want to connect to any other database by passing the function the appropriate arguments. Before you read on, spend a couple of minutes figuring out a solution by yourself.
arguments.variable.db.py
def connect(**options): conn_params = { 'host': options.get('host', '127.0.0.1'), 'port': options.get('port', 5432), 'user': options.get('user', ''), 'pwd': options.get('pwd', ''), } print(conn_params) # we then connect to the db (commented out) # db.connect(**conn_params) connect() connect(host='127.0.0.42', port=5433) connect(port=5431, user='fab', pwd='gandalf')
Note in the function we can prepare a dictionary of connection parameters (conn_params) in the function using default values as fallback, allowing them to be overwritten if they are provided in the function call. There are better ways to do this with fewer lines of code but we're not concerned with that now. Running the preceding code yields the following result:
$ python arguments.variable.db.py {'host': '127.0.0.1', 'pwd': '', 'user': '', 'port': 5432} {'host': '127.0.0.42', 'pwd': '', 'user': '', 'port': 5433} {'host': '127.0.0.1', 'pwd': 'gandalf', 'user': 'fab', 'port': 5431}
Note the correspondence between the function calls and the output. Note how default values are either there or overridden, according to what was passed to the function.
Python 3 allows for a new type of parameter: the keyword-only parameter. We are going to study them only briefly as their use cases are not that frequent. There are two ways of specifying them, either after the variable positional arguments, or after a bare *. Let's see an example of both.
arguments.keyword.only.py
def kwo(*a, c): print(a, c) kwo(1, 2, 3, c=7) # prints: (1, 2, 3) 7 kwo(c=4) # prints: () 4 # kwo(1, 2) # breaks, invalid syntax, with the following error # TypeError: kwo() missing 1 required keyword-only argument: 'c' def kwo2(a, b=42, *, c): print(a, b, c) kwo2(3, b=7, c=99) # prints: 3 7 99 kwo2(3, c=13) # prints: 3 42 13 # kwo2(3, 23) # breaks, invalid syntax, with the following error # TypeError: kwo2() missing 1 required keyword-only argument: 'c'
As anticipated, the function, kwo, takes a variable number of positional arguments (a) and a keyword-only function, c. The results of the calls are straightforward and you can uncomment the third call to see what error Python returns.
The same applies to the function, kwo2, which differs from kwo in that it takes a positional argument a, a keyword argument b, and then a keyword-only argument, c. You can uncomment the third call to see the error.
Now that you know how to specify different types of input parameters, let's see how you can combine them in function definitions.
You can combine input parameters, as long as you follow these ordering rules:
- When defining a function, normal positional arguments come first (
name), then any default arguments (name=value), then the variable positional arguments (*name, or simply*), then any keyword-only arguments (eithernameorname=valueform is good), then any variable keyword arguments (**name). - On the other hand, when calling a function, arguments must be given in the following order: positional arguments first (
value), then any combination of keyword arguments (name=value), variable positional arguments (*name), then variable keyword arguments (**name).
Since this can be a bit tricky when left hanging in the theoretical world, let's look at a couple of quick examples.
arguments.all.py
def func(a, b, c=7, *args, **kwargs):
print('a, b, c:', a, b, c)
print('args:', args)
print('kwargs:', kwargs)
func(1, 2, 3, *(5, 7, 9), **{'A': 'a', 'B': 'b'})
func(1, 2, 3, 5, 7, 9, A='a', B='b') # same as previous oneNote the order of the parameters in the function definition, and that the two calls are equivalent. In the first one, we're using the unpacking operators for iterables and dictionaries, while in the second one we're using a more explicit syntax. The execution of this yields (I printed only the result of one call):
$ python arguments.all.py a, b, c: 1 2 3 args: (5, 7, 9) kwargs: {'A': 'a', 'B': 'b'}
Let's now look at an example with keyword-only arguments.
arguments.all.kwonly.py
def func_with_kwonly(a, b=42, *args, c, d=256, **kwargs):
print('a, b:', a, b)
print('c, d:', c, d)
print('args:', args)
print('kwargs:', kwargs)
# both calls equivalent
func_with_kwonly(3, 42, c=0, d=1, *(7, 9, 11), e='E', f='F')
func_with_kwonly(3, 42, *(7, 9, 11), c=0, d=1, e='E', f='F')Note that I have highlighted the keyword-only arguments in the function declaration. They come after the variable positional argument *args, and it would be the same if they came right after a single * (in which case there wouldn't be a variable positional argument). The execution of this yields (I printed only the result of one call):
$ python arguments.all.kwonly.py a, b: 3 42 c, d: 0 1 args: (7, 9, 11) kwargs: {'f': 'F', 'e': 'E'}
One other thing to note are the names I gave to the variable positional and keyword arguments. You're free to choose differently, but be aware that args and kwargs are the conventional names given to these parameters, at least generically. Now that you know how to define a function in all possible flavors, let me show you something tricky: mutable defaults.
One thing to be very aware of with Python is that default values are created at def time, therefore, subsequent calls to the same function will possibly behave differently according to the mutability of their default values. Let's look at an example:
arguments.defaults.mutable.py
def func(a=[], b={}):
print(a)
print(b)
print('#' * 12)
a.append(len(a)) # this will affect a's default value
b[len(a)] = len(a) # and this will affect b's one
func()
func()
func()The parameters both have mutable default values. This means that, if you affect those objects, any modification will stick around in subsequent function calls. See if you can understand the output of those calls:
$ python arguments.defaults.mutable.py [] {} ############ [0] {1: 1} ############ [0, 1] {1: 1, 2: 2} ############
It's interesting, isn't it? While this behavior may seem very weird at first, it actually makes sense, and it's very handy, for example, when using memoization techniques (Google an example of that, if you're interested).
Even more interesting is what happens when, between the calls, we introduce one that doesn't use defaults, like this:
arguments.defaults.mutable.intermediate.call.py
func()
func(a=[1, 2, 3], b={'B': 1})
func()When we run this code, this is the output:
$ python arguments.defaults.mutable.intermediate.call.py [] {} ############ [1, 2, 3] {'B': 1} ############ [0] {1: 1} ############
This output shows us that the defaults are retained even if we call the function with other values. One question that comes to mind is, how do I get a fresh empty value every time? Well, the convention is the following:
arguments.defaults.mutable.no.trap.py
def func(a=None): if a is None: a = [] # do whatever you want with `a` ...
Note that, by using the preceding technique, if a isn't passed when calling the function, you always get a brand new empty list.
Okay, enough with the input, let's look at the other side of the coin, the output.