7. Testing
Testing in Go, as in any language, is a subject worthy of its own book. Unfortunately, covering all topics, such as benchmarking, example tests, and design patterns, is outside the scope of this book. In this chapter, we cover the fundamentals of testing in Go, including writing, running, and debugging tests. We also cover table driven tests, test helpers, and code coverage.
Testing Basics
Go ships with a powerful testing framework, and as such, there is a strong emphasis on testing in Go. Tests in Go are written in the Go language, so there is no need to learn another syntax. Because the language was designed with testing in mind, it tends to be easier to create and understand tests in Go than in some other languages.
As with any statically typed language, the compiler catches a lot of bugs for you. However, it cannot ensure your business logic is sound or bug free. In Go, there are several tools to ensure your code is bug free (such as tests), as well as tools to show you what part of your code has been tested and which has been not (code coverage).
We introduce some of the basic testing features now, and as we continue to introduce new topics later, we show the corresponding test features and approaches as they pertain to those topics and features.
Naming
Up until now, there has not been much convention to the naming or placement of files or functions in our Go code. However, with testing, naming of files and functions play an incredibly important part of how tests work.
Test File Naming
When creating the code for a test, that code has to be placed in a test file. Go identifies a file as being a test file with the test suffix. If a file has a _test.go suffix, Go knows to process that as a test.
A nice side effect of this naming convention is that all test files appear in the file list right next to the file they are usually testing, as in Listing 7.1.
This naming convention is a required naming convention. Even if you write test code in another file, if it does not have the _test.go suffix, the Go framework does not run the code as a test.
Listing 7.1 Test Files End in _test.go and Live Next to the File They’re Testing
Test Function Signature
Aside from putting each test in a _test.go file, you also need to name your test functions properly. Without the proper naming and signature, Go does not run the function as a test.
Each test function must have the signature Test<Name>(*testing.T).
In Listing 7.2, the function starts with the uppercase word Test. You can have other functions in your test files, but only the functions that start with the Test keyword run as a test. Additionally, each test must accept the (t *testing.T) argument.
Listing 7.2 A Simple Go Test
The *testing.T Type
The testing.T1 type, Listing 7.3, is used to validate and control the flow of a test. It has the following methods available to use to control the flow of a test. We cover most of the testing.T type, and its methods, in this chapter.
1. https://pkg.go.dev/testing#T
Listing 7.3 The testing.T Type
Marking Test Failures
Two of the most common calls to make while testing are testing.T.Error,2 Listing 7.4, and testing.T.Fatal,3 Listing 7.5. Both report a test failure, but testing.T.Error continues with execution of that test, whereas testing.T.Fatal ends that test.
2. https://pkg.go.dev/testing#T.Error
3. https://pkg.go.dev/testing#T.Fatal
Listing 7.4 The testing.T.Error Method
Listing 7.5 The testing.T.Fatal Method
Using t.Error
In Listing 7.6, the GetAlphabet function returns a slice of letters for each country’s alphabet. For example, if given the key US, the GetAlphabet function returns a slice of 26 characters, A-Z, and a nil to represent no error. If the requested key is not present in the map, then a nil is returned in place of the map, and the fmt.Errorf function is used to return an error.
Listing 7.6 A Function That May Return an Error
The test in Listing 7.7 for the GetAlphabet function marks test errors with the testing.T.Errorf4 function. This tells Go that the test has failed; however, the test continues to run.
4. https://pkg.go.dev/testing#T.Errorf
Listing 7.7 Using testing.T.Errorf to Indicate a Test Failure
In Listing 7.7, the key CA does not exist. The first error, could not find alphabet CA, is logged, and the test continues to run. The next check is to confirm that the thirteenth letter of the alphabet is the letter M. Because the alphabet was not found, the alpha slice is empty, so when the test tries to retrieve index 12, the test panics.
Using t.Fatal (Recommended)
Listing 7.8 uses testing.T.Fatalf5 instead of testing.T.Errorf. This time, the test fails immediately at the first error, could not find alphabet CA. Since the test stops at that first error, the alpha[12] call is never executed, and the test doesn’t panic.
5. https://pkg.go.dev/testing#T.Fatalf
Listing 7.8 Using testing.T.Fatalf to Indicate a Test Failure
In summary, when trying to determine when to use testing.T.Fatal or testing.T.Error, it usually comes down to this: If the test continues but would result in more failures due to the current condition failing, you should prevent the test from continuing by using testing.T.Fatal. If the test could continue after failing a condition but still report other errors that are valid, then use testing.T.Error.
Crafting Good Test Failure Messages
When you decide that a test has failed, you need to make sure to craft a well-written failure message that informs the person running the test what the failure was and why. This failure message should also be resilient to change in the test later.
Consider the test in Listing 7.9. The test is designed to fail if the result of the AddTen(int) method does not return 11.
Listing 7.9 A Test That Fails
While the error message in Listing 7.9 is very well-written, we have hard coded the expected value in two places. If we update one and forget to update the other, the resulting output would be very confusing. This is what we mean by all test messages should be “resilient to change” for future changes.
By using variables to hold data, Listing 7.10, like the expected value, we can easily update the code in one place and know that our error messages will be updated accordingly.
Listing 7.10 A Test That Fails but with a Better Error Message
Now if the future expected value changes, only one variable needs to be changed, which reduces the chance that an invalid error message would be rendered.
Code Coverage
One of the most important parts of writing a solid application that is robust and maintainable is to ensure that the code is well tested. This is especially important when you are working with large code bases. In this section, we cover the basics of code coverage and how to use it to help you write better tests.
Basic Code Coverage
To write the best tests, you need to know which branches of the code are covered by tests. You can do this by generating code coverage reports.
Let’s start by looking at code coverage for a package. When we run go test, Listing 7.11, the tests pass, but there is no code coverage information.
Listing 7.11 Running Tests with No Coverage Information
In Listing 7.12, we run go test with the -cover flag. We get an overall code coverage printed for the package or packages being tested.
Listing 7.12 Running Tests with Coverage Information
The -cover flag can be run both locally and in CI to help get a quick view into how well your packages are tested.
Generating a Coverage Profile
While the -cover flag gives you a quick view of how well your tests are covering your code, you can also generate a detailed coverage profile. This report gives you a breakdown of which lines of code are covered by tests and which lines are not.
In Listing 7.13, we pass the -coverprofile flag with a value of coverage.out to go test to generate a coverage profile. The tests run, and the coverage.out file is generated with the coverage information.
Listing 7.13 Generating a Coverage Profile
The file, coverage.out, as shown in Listing 7.14, generated by Listing 7.13, contains a list of all the lines of code that were covered by the tests.
Listing 7.14 Contents of the coverage.out File
The go tool cover Command
A coverage file, like the one in Listing 7.14, is not meant to be viewed directly. Instead, it’s meant to be used with the cover.cmd command, Listing 7.15.
Listing 7.15 The go tool cover Command
The go tool cover command can be used to transform the coverage profile generated by using the -coverprofile flag with go test. For example, in Listing 7.16, we use the go tool cover command to view the coverage profile generated by Listing 7.13 on a per-function basis.
Listing 7.16 Viewing the Coverage Profile on a Per-Function Basis
Generating an HTML Coverage Report
Most often, you want to generate an HTML coverage report. This report gives you a nice visual breakdown of which lines of code are covered by tests and which lines are not. You can use the -html flag with a value of coverage.out, Listing 7.17, to generate an HTML coverage report from the coverage profile generated in Listing 7.13. The result opens in your default browser.
In Listing 7.17, we are calling the go tool cover command with the -html flag and the name of the coverage file, coverage.out. When run locally, this opens the default OS browser and displays an HTML output of the code coverage, as shown in Figure 7.1.
Figure 7.1 The HTML coverage report
Listing 7.17 Generating an HTML Coverage Report
You can use the -o flag, as shown in Listing 7.18, to specify an output file instead of opening the report in your default browser.
Listing 7.18 Generating an HTML Coverage Report with an Output File
Figure 7.1 shows a copy of code coverage in a browser. The report highlights lines that are covered by the tests in green, and lines that are not covered in red. Gray lines are not tracked by coverage—for example, type definition and function declarations or comments.
Editor Support
A lot of editors, and their Go plugins, support code coverage. For example, the Go extension6 for Visual Studio Code supports displaying code coverage directly in the editor, as shown in Figure 7.2.
6. https://code.visualstudio.com/docs/languages/go
Figure 7.2 Highlighting code coverage directly in Visual Studio Code
Table Driven Testing
The code coverage demonstrated in Listing 7.19 shows that the All method has several code paths that are currently not covered by tests. Table driven tests can help us fix this.
Listing 7.19 Viewing the Coverage Profile on a Per-Function Basis
You can use table driven tests to cover a lot of ground quickly while reusing common setup and comparison code. Table driven testing is not unique to Go; however, it is very popular in the Go community.
Anatomy of a Table Driven Test
Listing 7.20 is an example of the way a table driven test is structured. We use a slice of anonymous structs. The anonymous struct defines any fields needed for each test case; then we can fill that slice with the test cases we want to test. Finally, we loop through the slice of test cases and perform the tests.
Listing 7.20 Anatomy of a Table Driven Test
Writing a Table Driven Test
Given the All method in Listing 7.21, we can use table driven tests to cover all of the error paths in the code.
Listing 7.21 The All Method
In the test defined in Listing 7.22, we first set up our common test data. For example, we can define different implementations of the Store type that have various differences, such as whether their internal data field is initialized.
After that, we create a new slice of anonymous structs that hold the test cases we want to test. In this case, we need to be able to set the Store and the error we expect to see.
Finally, we loop through the slice of test cases and perform the tests.
Listing 7.22 Table Driven Tests with No Subtests
As you can see from the test output, Listing 7.23, our tests pass, but they are all in one test.
Listing 7.23 Output of the Table Driven Test
If one of the test cases fails, Listing 7.24, we don’t know which one it is. We also don’t have the ability to run just that one test. Using subtests solves both of these problems.
Listing 7.24 Uncertainty about Which Test Case Failed
Subtests
One of the problems with our tests as they are currently written is that they are not very descriptive, and if one of those test cases fails, we don’t know which one, as we saw in Listing 7.24. This is where subtests come in. The testing.T.Run7 method, Listing 7.25, allows us to create a whole new test for each test case we define. In Go, the name of the test is derived from the name of the test function. For example, if we have a test function called TestSomething, the name of the test is TestSomething. With subtests, we can create our own names for each test case to make them as useful and descriptive as we need.
7. https://pkg.go.dev/testing#T.Run
Listing 7.25 The testing.T.Run Method
Anatomy of a Subtest
Using subtests is not much different than what we have been doing with table driven tests so far (refer to Listing 7.20). The biggest difference is in the for loop. In Listing 7.26, instead of making our assertions directly, we are, instead, calling the testing.T.Run method to create a new subtest. We give the testing.T.Run method the name of our test case and the function we want to run as a subtest.
Listing 7.26 Anatomy of a Subtest
Writing Subtests
In Listing 7.27, we have added a call to the testing.T.Run method to create a subtest for each test case. When the tests are run, each test case is displayed as its own test. Should one of the subtests fail, it is easy to identify which one it is.
Listing 7.27 Subtests Displayed as Their Own Tests
Running Tests
Understanding the different options to run tests can greatly reduce the feedback cycle during development. In this section, we cover running specific tests, verbose output, failing fast, parallel options, and more.
Running Package Tests
You can run all of the tests in the current package (folder) using the go test command without specifying any test paths, as shown in Listing 7.28.
Listing 7.28 Running All the Tests in the Current Package
Running Tests with Subpackages
Often, Go projects consist of multiple packages. To run all of these packages, you can use the ./..., Listing 7.29, identifier to tell Go to recurse through all subpackages as well as the current package.
Listing 7.29 Running All the Tests in the Current Package and Subpackages
Optionally, as in Listing 7.30, you can specify the path of one or more packages to test.
Listing 7.30 Running All the Tests in the models Package
Verbose Test Output
It can be useful to output “verbose” information when running tests, such as when you’re trying to debug a test. For example, as in Listing 7.31, you can see each test and subtest broken out individually. This allows you to see which tests were run and the status of that individual test. The -v flag turns on this verbose output.
Listing 7.31 Verbose Test Output
Logging in Tests
When using the -v flag, anything printed to the standard output is also displayed, regardless of whether the test was successful. This includes calls to fmt.Print and similar functions. You can use the testing.T.Log8 and testing.T.Logf9 methods to log information during tests. By default, any testing.T.Log and testing.T.Logf statements show up in test output only if the test fails. In Listing 7.32, you can see that when we run our test suite without the -v flag, there is no extra output.
8. https://pkg.go.dev/testing#T.Log
9. https://pkg.go.dev/testing#T.Logf
Listing 7.32 Logging in Tests
When tests are run with the -v flag, Listing 7.33, the output of testing.T.Log and testing.T.Logf statements is shown.
Listing 7.33 Enabling Logging in Tests with the -v Flag
Short Tests
Sometimes you have long running tests, or integration tests, that you might not want to run as part of your local development cycle. The testing.Short10 function shown in Listing 7.34 can be used to mark a test as being “short.”
10. https://pkg.go.dev/testing#Short
Listing 7.34 The testing.Short Function
You can pass the -short argument to the test runner and check for that in your tests with the testing.Short function, as in Listing 7.35. This is useful also if your test has outside dependencies such as a database that you might not have available for local testing.
Listing 7.35 Using the -short Flag and testing.Short to Run Specific Tests
Running Package Tests in Parallel
By default, each package is run in parallel to make testing faster. This can be changed by setting the -parallel flag to 1, as in Listing 7.36. This is usually done to prevent packages that use transactions to a database or would otherwise cause contention between package tests.
Listing 7.36 Changing the Parallelism of the Test Runner
Running Tests in Parallel
Although packages are run in different threads during testing, individual tests within those packages are not. They are run one test at a time. However, you can change this.
By using testing.t.Parallel,11 Listing 7.37, you signal that this test is to be run in parallel with (and only with) other parallel tests.
11. https://pkg.go.dev/testing#t.Parallel
Listing 7.37 The testing.T.Parallel Method
In Listing 7.38, the first thing we do inside of the Test_Example test function is to call the t.Parallel method to tell Go that this test is safe to be run concurrently with other tests.
Listing 7.38 Using testing.t.Parallel to Mark a Test as Parallel-Friendly
Running Specific Tests
The -run flag allows for the passing of a regular expression to match the names of specific tests to run. For example, in Listing 7.39, we are passing the -run flag to the test runner to run only tests that contain the string History.
Listing 7.39 Running Only Tests That Have History in Their Name
In Listing 7.40, we are passing the -run flag to the test runner to run only tests that contain the string Address.
Listing 7.40 Subtests Are Also Run with the -run Flag
Timing Out Tests
Eventually you will accidentally write an erroneous infinite for loop or other piece of code that will cause your tests to run forever. By using the -timeout flag, you can set a timeout for your tests to prevent these scenarios. In Listing 7.41, the tests stop and fail after fifty milliseconds.
Listing 7.41 Setting a Timeout for Tests with the -timeout Flag
Failing Fast
You can use the -failfast flag to stop your tests at the first test failure. This is specifically very useful when you’re doing large refactors and you want tests to stop as soon as one test fails.
In Listing 7.42, we are running the tests without the -failfast flag. As we see from the output, all the tests are run before the failures are reported. In Listing 7.43, we are running the tests with the -failfast flag, and as you see from the output, once the first test fails, the rest of the tests are not run.
Listing 7.42 Setting the -failfast Flag
Listing 7.43 Setting the -failfast Flag
Disabling Test Caching
Go automatically caches any passing tests to make subsequent test runs faster. Go breaks this cache when either the test or the code it is testing changes. This means you almost never have to deal with test caches. Occasionally there are times when you might want to disable this behavior.
One reason you may want to disable caching is if you have tests that are integration tests and test packages or systems external to the package the tests are written in. Because it’s likely the package didn’t experience changes but the outside dependencies did, you would not want to cache your tests in this scenario.
To ensure your tests don’t use any cached runs, use the -count flag, as shown in Listing 7.44, and force the tests to run at least once.
Listing 7.44 Disabling the Test Cache with the -count Flag
Test Helpers
Just like when we’re writing “real” code, our tests sometimes need helper functions. These helper functions are called “test helpers.” Test helpers might set up, tear down, or provision resources for the test. They can be used to write assertions or to mock out external dependencies.
Defining Test Helpers
Test helper functions in Go are just like any other function. The difference is that they are only defined in your tests. While not required, it is recommended that you take the testing.TB interface, Listing 7.45, as the first argument to your test helper function. This interface is the common set of functions to both the testing.T12 and testing.B,13 used for benchmarking, types.
12. https://pkg.go.dev/testing#T
13. https://pkg.go.dev/testing#B
Listing 7.45 The testing.TB14 Interface
14. https://pkg.go.dev/testing#TB
Let’s define some test helpers to help clean up tests in Listing 7.46. We create helpers to create the different Store implementations we need for the test.
Listing 7.46 The Test Function to Refactor with Helpers
First, in Listing 7.47, we create two helper functions. The first, noData(testing.TB), returns a &Store that does not have any data. The second, withData(testing.TB), returns a &Store that has its data field properly initialized.
Listing 7.47 Two Helper Functions for Initializing Store Values
In Listing 7.48, we declare the withUsers helper function needed to clean up the tests. At the moment, however, we are unable to implement the withUsers test helper, so we can call the testing.TB.Fatal15 method on the passed-in testing.TB to let Go know that we haven’t implemented this helper yet.
15. https://pkg.go.dev/testing#TB.Fatal
Listing 7.48 An Unimplemented Helper Function
Finally, in Listing 7.49, we can update our tests to use the new test helpers, passing in the testing.T argument from the test to the helper function.
Listing 7.49 The Test Function with the New Helpers
As shown in Listing 7.50, because we have yet to implement the withUsers helper, our tests will fail.
Listing 7.50 The Stack Trace Prints the Line in the Helper That Failed
When the test fails, it reported the test failure as inside the withUsers helper, Listing 7.48, and not within the test itself. This can make it difficult to debug the test failure.
Marking a Function as a Helper
To get Go to report the correct line number, inside of the test and not the helper, we need to tell Go that the withData function is a test helper. To do that, we must use the testing.TB.Helper16 method, Listing 7.51, inside of our test helpers.
16. https://pkg.go.dev/testing#TB.Helper
Listing 7.51 The testing.TB.Helper Method
Now, when the test fails, in Listing 7.52, the line number reported is the line number inside the test that failed, not the line number inside the helper.
Listing 7.52 The Stack Track Now Points to the Text Line That Failed
Cleaning Up a Helper
It is very common that a test helper, or even a test itself, needs to clean up some resources when done. To do that, you need to use the testing.TB.Cleanup17 method, Listing 7.53. With testing.TB.Cleanup, we can pass in a function that is called automatically when the test is done.
17. https://pkg.go.dev/testing#TB.Cleanup
Listing 7.53 The testing.TB.Cleanup Method
In Listing 7.54, we implement the withUsers helper function. In it, we use the testing.TB.Cleanup method to clean up the users we created.
Listing 7.54 Using testing.T.Cleanup to Clean Up Helper Resources
Clarifying Cleanup versus defer
Although the testing.TB.Cleanup method might seem like a fancy defer statement, it is actually a different concept. When a function is deferred, that function is called when its parent function returns. In the case of testing.TB.Cleanup, the testing.TB.Cleanup function is called when the test is done.
Summary
In this chapter, we covered the basics of testing in Go. We covered how to write tests, how to run tests, how to write test helpers, and how to write table driven tests. We showed you how to generate code coverage reports and how to run tests in parallel. We also covered many of the important test flags, such as -run, -v, and -timeout. Finally, we explained how to write useful test helpers and how to properly clean up after them.