pytest is a framework written in Python that helps in creating and automating test cases. It is easy to learn and takes a lot of the work out of writing and managing test cases. In particular, only the logic for the test cases itself needs to be implemented. Unlike some other frameworks, there is no need to learn a large number of functions to set up test assertions; one is enough.
The module unittest, which is integrated in Python, is less easy to handle than pytest and therefore less common. Details about both can be found at https://knapsackpro.com/testing_frameworks/difference_between/unittest/vs/pytest. Conveniently, pytest also allows you to use a possibly existing test base, created with unittest, permitting a step-by-step migration from unittest to pytest.
A.1 Writing and Executing Tests
A.1.1 Installing pytest
Before you can use pytest, you need to install it. This can be done using the pip tool, which is simply called pip for Linux and Windows but pip3 for Mac OS.
Open a console and type the following command (in the following text, and this book in general, I always use $ to indicate input on the console, which is the terminal on MacOS or the Windows command prompt):
$ pip install -U pytest
In addition, a few plugins are quite useful, such as this one for parameterized tests
To test a module, a corresponding test module is usually written. To be recognized by pytest, it should end with the postfix test or _test, such as ex03_palindrome_test. Often, to validate important functionality, you start by testing a few key functions. This should be extended step by step. Test cases are expressed as special test functions, which must be marked with the prefix test_. Otherwise, pytest does not consider them as test cases and ignores them during test execution.
Let’s have a look at an introductory example:
def test_index():
# ARRANGE
name = "Peter"
# ACT
pos = name.index("t")
expected = 2
# ASSERT
assert pos == expected
Interestingly, there’s no dependency on pytest. In fact, the whole thing is automatically linked to pytest, and the execution standard assert is varied in such a way that pytest hooks in and produces test results.
Also worth mentioning is the three-way split with ARRANGE-ACT-ASSERT for preparing the actions, executing them, and evaluating the results. This structure helps to write clean and understandable tests. There is not always an ARRANGE part and the comments can be omitted if you are more experienced. This is described in much more detail in my book Der Weg zum Java-Profi [Ind20].
A.1.3 Executing Tests
To run the unit test with pytest, you can either use
the command line or
the IDE.
Executing Tests on the Console
Running the unit tests with pytest can be done from the console in the root directory of your project. In the following code, use python3 and the module specification with -m. This is the only way the tests always run cleanly for me.
$ python3 -m pytest
This will start all tests and log the result on the console. For this book, it is shortened as follows:
$ python3 -m pytest
================= test session starts ===================
platform darwin -- Python 3.10.1, pytest-7.1.1, pluggy-1.0.0
=============== 645 passed in 1.97s ====================
When getting started with the following parameters,
$ python3 -m pytest --html=pytest-report.html
an additional HTML report of the test results gets generated. This can be examined with the browser of your choice. An example is shown in Figure A-1.
Figure A-1
HTML representation of a test report
Executing Tests from the IDE
Alternatively, it is a bit more convenient to start test execution directly in the IDE. Before doing so, however, pytest must be configured correctly. Conveniently, pytest is integrated with the popular IDE PyCharm. Tests can be executed either via a context menu or via buttons in the GUI. This produces output similar to that shown in Figure A-2.
Figure A-2
Test run from the GUI of the IDE
A.1.4 Handling Expected Exceptions
Sometimes test cases are supposed to check for the occurrence of exceptions during processing, and an absence would constitute an error. An example is deliberately accessing a non-existent position of a string. An IndexError should be the result. To handle expected exceptions in the test case in such a way that they represent a test success and not a failure, the executing functionality must be called specifically surrounded by with pytest.raises():
In the second and third test case, you see how easy it is to access the contents of the thrown exceptions, for example to check the text or other details.
A.1.5 Parameterized Tests with pytest
Sometimes you need to test a large number of value sets. Creating a separate test function for each of them would make the test module quite bloated and confusing. To solve this more elegantly, there are several variants. All of them have their specific strengths and weaknesses.
In the following, assume that calculations are to be checked for fixed ranges of values or a selected set of inputs.1
A parameterized test allows you to do just that: write the test function and define a set of inputs and expected results. Based on this, the testing framework automatically executes the test function for all specified combinations of values.
Introduction to Parameterized Tests
With pytest, defining parameterized tests is very simple. All you need to do is apply a suitable import and then specify the desired values as follows:
In the code, you see that the parameterized test must be annotated with @pytest.mark.parametrize. The first parameter specifies the parameter names and the evaluation of the values. These values are passed as a list of tuples. For each parameterization specified as a tuple, a separate test case is created and executed.
Other Possibilities in Parameterized Tests
Ingeniously, all collection literals (i.e., tuples, lists, sets, and dictionaries) can be used when specifying test inputs and results:
result = all_combinations_with_value(digits, value)
assert result == expected
A.2 Further Reading on pytest
This appendix just provided a first introduction to testing with pytest so you can follow the examples more easily. Of course, there is much more to discover, such as various plugins. More information on how to use pytest appropriately can be found in the following books:
Python Testing with pytest: Simple, Rapid, Effective, and Scalable by Brian Okken [Okk17]
pytest Quick Start Guide: Write better Python code with simple and maintainable tests by Bruno Oliveira [Oli18]
