Let's now adopt a practical approach. I will show you how to test a piece of code, and we will touch on the rest of the important concepts around unit testing, within the context of this example.
We want to write an export function that does the following: it takes a list of dictionaries, each of which represents a user. It creates a CSV file, puts a header in it, and then proceeds to add all the users who are deemed valid according to some rules. The export function takes also a filename, which will be the name for the CSV in output. And, finally, it takes an indication on whether to allow an existing file with the same name to be overwritten.
As for the users, they must abide by the following: each user has at least an email, a name, and an age. There can be a fourth field representing the role, but it's optional. The user's email address needs to be valid, the name needs to be non-empty, and the age must be an integer between 18 and 65.
This is our task, so now I'm going to show you the code, and then we're going to analyze the tests I wrote for it. But, first things first, in the following code snippets, I'll be using two third-party libraries: marshmallow and pytest. They both are in the requirements of the book's source code, so make sure you have installed them with pip.
marshmallow is a wonderful library that provides us with the ability to serialize and deserialize objects and, most importantly, gives us the ability to define a schema that we can use to validate a user dictionary. pytest is one of the best pieces of software I have ever seen. It is used everywhere now, and has replaced other tools such as nose, for example. It provides us with great tools to write beautiful short tests.
But let's get to the code. I called it api.py just because it exposes a function that we can use to do things. I'll show it to you in chunks:
# api.py
import os
import csv
from copy import deepcopy
from marshmallow import Schema, fields, pre_load
from marshmallow.validate import Length, Range
class UserSchema(Schema):
"""Represent a *valid* user. """
email = fields.Email(required=True)
name = fields.String(required=True, validate=Length(min=1))
age = fields.Integer(
required=True, validate=Range(min=18, max=65)
)
role = fields.String()
@pre_load(pass_many=False)
def strip_name(self, data):
data_copy = deepcopy(data)
try:
data_copy['name'] = data_copy['name'].strip()
except (AttributeError, KeyError, TypeError):
pass
return data_copy
schema = UserSchema()
This first part is where we import all the modules we need (os and csv), and some tools from marshmallow, and then we define the schema for the users. As you can see, we inherit from marshmallow.Schema, and then we set four fields. Notice we are using two String fields, Email and Integer. These will already provide us with some validation from marshmallow. Notice there is no required=True in the role field.
We need to add a couple of custom bits of code, though. We need to add validate_age to make sure the value is within the range we want. We raise ValidationError in case it's not. And marshmallow will kindly take care of raising an error should we pass anything but an integer.
Next, we add validate_name, because the fact that a name key in the dictionary is there doesn't guarantee that the name is actually non-empty. So we take its value, we strip all leading and trailing whitespace characters, and if the result is empty, we raise ValidationError again. Notice we don't need to add a custom validator for the email field. This is because marshmallow will validate it, and a valid email cannot be empty.
We then instantiate schema, so that we can use it to validate data. So let's write the export function:
# api.py
def export(filename, users, overwrite=True):
"""Export a CSV file.
Create a CSV file and fill with valid users. If `overwrite`
is False and file already exists, raise IOError.
"""
if not overwrite and os.path.isfile(filename):
raise IOError(f"'{filename}' already exists.")
valid_users = get_valid_users(users)
write_csv(filename, valid_users)
As you see, its internals are quite straightforward. If overwrite is False and the file already exists, we raise IOError with a message saying the file already exists. Otherwise, if we can proceed, we simply get the list of valid users and feed it to write_csv, which is responsible for actually doing the job. Let's see how all these functions are defined:
# api.py
def get_valid_users(users):
"""Yield one valid user at a time from users. """
yield from filter(is_valid, users)
def is_valid(user):
"""Return whether or not the user is valid. """
return not schema.validate(user)
Turns out I coded get_valid_users as a generator, as there is no need to make a potentially big list in order to put it in a file. We can validate and save them one by one. The heart of validation is simply a delegation to schema.validate, which uses validation engine by marshmallow. The way this works is by returning a dictionary, which is empty if validation succeeded, or else it will contain error information. We don't really care about collecting the error information for this task, so we simply ignore it, and within is_valid we basically return True if the return value from schema.validate is empty, and False otherwise.
One last piece is missing; here it is:
# api.py
def write_csv(filename, users):
"""Write a CSV given a filename and a list of users.
The users are assumed to be valid for the given CSV structure.
"""
fieldnames = ['email', 'name', 'age', 'role']
with open(filename, 'x', newline='') as csvfile:
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writeheader()
for user in users:
writer.writerow(user)
Again, the logic is straightforward. We define the header in fieldnames, then we open filename for writing, and we specify newline='', which is recommended in the documentation when dealing with CSV files. When the file has been created, we get a writer object by using the csv.DictWriter class. The beauty of this tool is that it is capable of mapping the user dictionaries to the field names, so we don't need to take care of the ordering.
We write the header first, and then we loop over the users and add them one by one. Notice, this function assumes it is fed a list of valid users, and it may break if that assumption is false (with the default values, it would break if any user dictionary had extra fields).
That's the whole code you have to keep in mind. I suggest you spend a moment to go through it again. There is no need to memorize it, and the fact that I have used small helper functions with meaningful names will enable you to follow the testing along more easily.
Let's now get to the interesting part: testing our export function. Once again, I'll show you the code in chunks:
# tests/test_api.py
import os
from unittest.mock import patch, mock_open, call
import pytest
from ..api import is_valid, export, write_csv
Let's start from the imports: we need os, temporary directories (which we already saw in Chapter 7, Files and Data Persistence), then pytest, and, finally, we use a relative import to fetch the three functions that we want to actually test: is_valid, export, and write_csv.
Before we can write tests, though, we need to make a few fixtures. As you will see, a fixture is a function that is decorated with the pytest.fixture decorator. In most cases, we expect fixture to return something, so that we can use it in a test. We have some requirements for a user dictionary, so let's write a couple of users: one with minimal requirements, and one with full requirements. Both need to be valid. Here is the code:
# tests/test_api.py
@pytest.fixture
def min_user():
"""Represent a valid user with minimal data. """
return {
'email': '[email protected]',
'name': 'Primus Minimus',
'age': 18,
}
@pytest.fixture
def full_user():
"""Represent valid user with full data. """
return {
'email': '[email protected]',
'name': 'Maximus Plenus',
'age': 65,
'role': 'emperor',
}
In this example, the only difference is the presence of the role key, but it's enough to show you the point I hope. Notice that instead of simply declaring dictionaries at a module level, we actually have written two functions that return a dictionary, and we have decorated them with the pytest.fixture decorator. This is because when you declare a dictionary at module-level, which is supposed to be used in your tests, you need to make sure you copy it at the beginning of every test. If you don't, you may have a test that modifies it, and this will affect all tests that follow it, compromising their integrity.
By using these fixtures, pytest will give us a new dictionary every test run, so we don't need to go through that pain ourselves. Notice that if a fixture returns another type, instead of dict, then that is what you will get in the test. Fixtures also are composable, which means they can be used in one another, which is a very powerful feature of pytest. To show you this, let's write a fixture for a list of users, in which we put the two we already have, plus one that would fail validation because it has no age. Let's take a look at the following code:
# tests/test_api.py
@pytest.fixture
def users(min_user, full_user):
"""List of users, two valid and one invalid. """
bad_user = {
'email': '[email protected]',
'name': 'Horribilis',
}
return [min_user, bad_user, full_user]
Nice. So, now we have two users that we can use individually, but also we have a list of three users. The first round of tests will be testing how we are validating a user. We will group all the tests for this task within a class. This not only helps giving related tests a namespace, a place to be, but, as we'll see later on, it allows us to declare class-level fixtures, which are defined just for the tests belonging to the class. Take a look at this code:
# tests/test_api.py
class TestIsValid:
"""Test how code verifies whether a user is valid or not. """
def test_minimal(self, min_user):
assert is_valid(min_user)
def test_full(self, full_user):
assert is_valid(full_user)
We start very simply by making sure our fixtures are actually passing validation. This is very important, as those fixtures will be used everywhere, so we want them to be perfect. Next, we test the age. Two things to notice here: I will not repeat the class signature, so the code that follows is indented by four spaces and it's because these are all methods within the same class, okay? And, second, we're going to use parametrization quite heavily.
Parametrization is a technique that enables us to run the same test multiple times, but feeding different data to it. It is very useful, as it allows us to write the test only once with no repetition, and the result will be very intelligently handled by pytest, which will run all those tests as if they were actually separate, thus providing us with clear error messages when they fail. If you parametrize manually, you lose this feature, and believe me you won't be happy. Let's see how we test the age:
# tests/test_api.py
@pytest.mark.parametrize('age', range(18))
def test_invalid_age_too_young(self, age, min_user):
min_user['age'] = age
assert not is_valid(min_user)
Right, so we start by writing a test to check that validation fails when the user is too young. According to our rule, a user is too young when they are younger than 18. We check for every age between 0 and 17, by using range.
If you take a look at how the parametrization works, you'll see we declare the name of an object, which we then pass to the signature of the method, and then we specify which values this object will take. For each value, the test will be run once. In the case of this first test, the object's name is age, and the values are all those returned by range(18), which means all integer numbers from 0 to 17 are included. Notice how we feed age to the test method, right after self, and then we do something else, which is also very interesting. We pass this method a fixture: min_user. This has the effect of activating that fixture for the test run, so that we can use it, and can refer to it from within the test. In this case, we simply change the age within the min_user dictionary, and then we verify that the result of is_valid(min_user) is False.
We do this last bit by asserting on the fact that not False is True. In pytest, this is how you check for something. You simply assert that something is truthy. If that is the case, the test has succeeded. Should it instead be the opposite, the test would fail.
Let's proceed and add all the tests needed to make validation fail on the age:
# tests/test_api.py
@pytest.mark.parametrize('age', range(66, 100))
def test_invalid_age_too_old(self, age, min_user):
min_user['age'] = age
assert not is_valid(min_user)
@pytest.mark.parametrize('age', ['NaN', 3.1415, None])
def test_invalid_age_wrong_type(self, age, min_user):
min_user['age'] = age
assert not is_valid(min_user)
So, another two tests. One takes care of the other end of the spectrum, from 66 years of age to 99. And the second one instead makes sure that age is invalid when it's not an integer number, so we pass some values, such as a string, a float, and None, just to make sure. Notice how the structure of the test is basically always the same, but, thanks to the parametrization, we feed very different input arguments to it.
Now that we have the age-failing all sorted out, let's add a test that actually checks the age is within the valid range:
# tests/test_api.py
@pytest.mark.parametrize('age', range(18, 66))
def test_valid_age(self, age, min_user):
min_user['age'] = age
assert is_valid(min_user)
It's as easy as that. We pass the correct range, from 18 to 65, and remove the not in the assertion. Notice how all tests start with the test_ prefix, and have a different name.
We can consider the age as being taken care of. Let's move on to write tests on mandatory fields:
# tests/test_api.py
@pytest.mark.parametrize('field', ['email', 'name', 'age'])
def test_mandatory_fields(self, field, min_user):
min_user.pop(field)
assert not is_valid(min_user)
@pytest.mark.parametrize('field', ['email', 'name', 'age'])
def test_mandatory_fields_empty(self, field, min_user):
min_user[field] = ''
assert not is_valid(min_user)
def test_name_whitespace_only(self, min_user):
min_user['name'] = ' '
assert not is_valid(min_user)
The previous three tests still belong to the same class. The first one tests whether a user is invalid when one of the mandatory fields is missing. Notice that at every test run, the min_user fixture is restored, so we only have one missing field per test run, which is the appropriate way to check for mandatory fields. We simply pop the key out of the dictionary. This time the parametrization object takes the name field, and, by looking at the first test, you see all the mandatory fields in the parametrization decorator: email, name, and age.
In the second one, things are a little different. Instead of popping keys out, we simply set them (one at a time) to the empty string. Finally, in the third one, we check for the name to be made of whitespace only.
The previous tests take care of mandatory fields being there and being non-empty, and of the formatting around the name key of a user. Good. Let's now write the last two tests for this class. We want to check email validity, and type for email, name, and the role:
# tests/test_api.py
@pytest.mark.parametrize(
'email, outcome',
[
('missing_at.com', False),
('@missing_start.com', False),
('missing_end@', False),
('missing_dot@example', False),
('[email protected]', True),
('δοκιμή@παράδειγμα.δοκιμή', True),
('аджай@экзампл.рус', True),
]
)
def test_email(self, email, outcome, min_user):
min_user['email'] = email
assert is_valid(min_user) == outcome
This time, the parametrization is slightly more complex. We define two objects (email and outcome), and then we pass a list of tuples, instead of a simple list, to the decorator. What happens is that each time the test is run, one of those tuples will be unpacked so to fill the values of email and outcome, respectively. This allows us to write one test for both valid and invalid email addresses, instead of two separate ones. We define an email address, and we specify the outcome we expect from validation. The first four are invalid email addresses, but the last three are actually valid. I have used a couple of examples with Unicode, just to make sure we're not forgetting to include our friends from all over the world in the validation.
Notice how the validation is done, asserting the result of the call needs to match the outcome we have set.
Let's now write a simple test to make sure validation fails when we feed the wrong type to the fields (again, the age has been taken care of separately before):
# tests/test_api.py
@pytest.mark.parametrize(
'field, value',
[
('email', None),
('email', 3.1415),
('email', {}),
('name', None),
('name', 3.1415),
('name', {}),
('role', None),
('role', 3.1415),
('role', {}),
]
)
def test_invalid_types(self, field, value, min_user):
min_user[field] = value
assert not is_valid(min_user)
As we did before, just for fun, we pass three different values, none of which is actually a string. This test could be expanded to include more values, but, honestly, we shouldn't need to write tests such as this one. I have included it here just to show you what's possible.
Before we move to the next test class, let me talk about something we have seen when we were checking the age.