Getting Started with jUnit Tests

As we noted in the introduction of this chapter, we want to test isValid, a method of theclass Accounts. Accounts is called the class under test. In jUnit 4, tests are put in a different class, the test class. By convention, the name of the test class is the name of the class under test with the suffix Test added. In this case, that would mean the name of the test class is AccountsTest. It must be a public class, but apart from that, there are no other requirements for a test class. In particular, it does not need to extend any other class. It is convenient, but not required, to place the test class in the same package as the class under test. That way, the test class has access to all members of the test class under test that have package (but not public) access.

In jUnit 4, a test itself is any method that has the @Test annotation. By convention, the name of a test method starts with test, but this is just a convention, not a requirement. To test isValid, you can use the following jUnit 4 test class:

package eu.sig.training.ch04.v1;

import static org.junit.Assert.assertFalse;
import static org.junit.Assert.assertTrue;

import org.junit.Test;

public class AccountsTest {

    @Test
    public void testIsValidNormalCases() {
        assertTrue(Accounts.isValid("123456789"));
        assertFalse(Accounts.isValid("123456788"));
    }

}

This test handles two cases:

• Bank account number 123456789: We know this is a valid bank account number (see “The 11-Check for Bank Account Numbers”), so isValid should return true. The call of assertTrue tests this.

• Bank account number 123456788: We know this is an invalid bank account number (because it differs from a valid account number by one digit), so isValid should return false. The call of assertFalse tests this.

Unit tests can be run directly in the IDE Eclipse. In addition, jUnit 4 comes with test runners to run tests from the command line. Tests can also be executed by Maven or Ant. Figure 10-1 shows the result of running the preceding test in Eclipse. The darker bar indicates that all tests have succeeded.

Figure 10-1. All tests succeeded!

The test in the preceding test class only tests normal cases: two bank account numbers of the expected format (exactly nine characters, all digits). How about corner cases? One obvious special case is the empty string. The empty string is, of course, not a valid bank account number, so we test it by calling assertFalse:

@Test
public void testEmptyString() {
    assertFalse(Accounts.isValid(""));
}

As Figure 10-2 shows, it turns out that this test fails! While the call to isValid should return false, it actually returned something else (which, of course, must be true, as there is no other option).

Figure 10-2. One test failed

The failed test points us to a flaw in isValid. In case the argument to isValid is the empty string, the for loop does not run at all. So the only lines executed are:

int sum = 0;
return sum % 11 == 0;

This indeed returns true, while it should return false. This reminds us to add code to isValid that checks the length of the bank account number.¹

The jUnit 4 runner reports this as a test failure and not as a test error. A test failure means that the test itself (the method testEmptyString) is executed perfectly, but the assertion failed. A test error means that the test method itself did not execute correctly. The following code snippet illustrates this: the showError method raises a division-by-zero exception and never even executes assertTrue:

@Test
public void showError() {
    int dummy = 1 / 0;
    // Next line is never executed because the previous one raises an
    // exception.
    // If it were executed, you'll never see the assert message because
    // the test always succeeds.
    assertTrue("You will never see this text.", true);
}

Next, we present some basic principles that will help you write good unit tests. We start with the most basic principles and then progress to more advanced ones that apply when your test efforts become more mature.

General Principles for Writing Good Unit Tests

When writing tests, it is important to keep in mind the following general principles:

Test both normal and special cases

As in the examples given in this chapter, test two kinds of cases. Write tests that confirm that a unit indeed behaves as expected on normal input (called happy flow or sunny-side testing). Also write tests that confirm that a unit behaves sensibly on non-normal input and circumstances (called unhappy flow or rainy-side testing). For instance, in jUnit it is possible to write tests to confirm that a method under test indeed throws a certain exception.

Maintain tests just like nontest (production) code

When you adjust code in the system, the changes should be reflected in the unit tests as well. This is most relevant for unit tests, though it applies to all tests. In particular, when adding new methods or enhancing the behavior of existing methods, be sure to add new test cases that cover that new code.

Write tests that are isolated: their outcomes should reflect only the behavior of the subject being tested

That is, each test should act independently of all other tests. For unit testing, this means that each test case should test only one functionality. No unit test should depend on state, such as files written by other tests. That is why a unit test that, say, causes the class under test to access the filesystem or a database server is not a good unit test.

Consequently, in unit testing you should simulate the state/input of other classes when those are needed (e.g., as arguments). Otherwise, the test is not isolated and would test more than one unit. This was easy for the test of isValid, because isValid takes a string as an argument, and it does not call other methods of our system. For other situations, you may need a technique like stubbing or mocking.

In Chapter 6, we introduced a Java interface for a simple digital camera, which is repeated here for ease of reference:

public interface SimpleDigitalCamera {
    public Image takeSnapshot();

    public void flashLightOn();

    public void flashLightOff();
}

Suppose this interface is used in an application that ensures people never forget to turn on the flash at night:

public final static int DAYLIGHT_START = 6;

public Image takePerfectPicture(int currentHour) {
    Image image;
    if (currentHour < PerfectPicture.DAYLIGHT_START) {
        camera.flashLightOn();
        image = camera.takeSnapshot();
        camera.flashLightOff();
    } else {
        image = camera.takeSnapshot();
    }
    return image;
}

Although the logic is simple (takePerfectPicture simply assumes that if the hour of the day on a 24-hour clock is lower than 6 p.m., it is night), it needs testing. For a proper unit test for takePerfectPicture to be written, taking a picture needs to be automatic and independent. That means that the normal implementation of the digital camera interface cannot be used. On a typical device, the normal implementation requires a (human) user to point the camera at something interesting and press a button. The picture taken can be any picture, so it is hard to test whether the (supposedly perfect) picture taken is the one expected.

The solution is to use an implementation of the camera interface that has been made especially for testing. This implementation is a fake object, called a test stub or simply a stub.² In this case, we want this fake object to behave in a preprogrammed (and therefore predictable) way. We write a test stub like this:

class DigitalCameraStub implements SimpleDigitalCamera {
    public Image testImage;

    public Image takeSnapshot() {
        return this.testImage;
    }

    public void flashLightOn() {}

    public void flashLightOff() {}
}

In this stub, takeSnapshot always returns the same image, which we can set simply by assigning to testImage (for reasons of simplicity, we have made testImage a public field and do not provide a setter). This stub can now be used in a test:

@Test
public void testDayPicture() throws IOException {
    BufferedImage image =
        ImageIO.read(new File("src/test/resources/VanGoghSunflowers.jpg"));
    DigitalCameraStub cameraStub = new DigitalCameraStub();
    cameraStub.testImage = image;
    PerfectPicture.camera = cameraStub;
    assertEquals(image, new PerfectPicture().takePerfectPicture(12));
}

In this test, we create a stub camera and supply it with an image to return. We then call takePerfectPicture(12) and test whether it returns the correct image. The value of the call, 12, means that takePerfectPicture assumes it is between noon and 1 p.m.

Now suppose we want to test takePerfectPicture for nighttime behavior; that is, we want to ensure that if takePerfectPicture is called with a value lower than PerfectPicture.DAYLIGHT_START, it indeed switches on the flash. So, we want to test whether takePerfectPicture indeed calls flashLightOn. However, flashLightOn does not return any value, and the SimpleDigitalCamera interface also does not provide any other way to know whether the flash has been switched on. So what to check?

The solution is to provide the fake digital camera implementation with some mechanism to record whether the method we are interested in gets called. A fake object that records whether expected calls have taken place is called a mock object. So, a mock object is a stub object with added test-specific behavior. The digital camera mock object looks like this:

class DigitalCameraMock implements SimpleDigitalCamera {
    public Image testImage;
    public int flashOnCounter = 0;

    public Image takeSnapshot() {
        return this.testImage;
    }

    public void flashLightOn() {
        this.flashOnCounter++;
    }

    public void flashLightOff() {}
}

Compared to DigitalCameraStub, DigitalCameraMock additionally keeps track of the number of times flashLightOn has been called, in a public field. DigitalCameraMock still contains preprogrammed behavior, so it is both a stub and a mock. We can check that flashLightOn is called in the unit test like so:

@Test
public void testNightPicture() throws IOException {
    BufferedImage image =
        ImageIO.read(new File("src/test/resources/VanGoghStarryNight.jpg"));
    DigitalCameraMock cameraMock = new DigitalCameraMock();
    cameraMock.testImage = image;
    PerfectPicture.camera = cameraMock;
    assertEquals(image, new PerfectPicture().takePerfectPicture(0));
    assertEquals(1, cameraMock.flashOnCounter);
}

In these examples, we have written our own stub and mock objects. This leads to a lot of code. Generally, it is most efficient to use a mocking framework such as Mockito or EasyMock. Mocking frameworks use features of the Java Virtual Machine to automatically create mock objects from normal interfaces or classes. They also provide methods to test whether methods of a mock object have been called, and with which arguments. Some mocking frameworks also provide ways to specify preprogrammed behavior of mock objects, giving them the characteristics of both stubs and mocks.

Indeed, using Mockito as an example, you can write testNightPicture without any need to write a class like DigitalCameraMock yourself:

@Test
public void testNightPictureMockito() throws IOException {
    BufferedImage image =
        ImageIO.read(new File("src/test/resources/VanGoghStarryNight.jpg"));
    SimpleDigitalCamera cameraMock = mock(SimpleDigitalCamera.class);
    PerfectPicture.camera = cameraMock;
    when(cameraMock.takeSnapshot()).thenReturn(image);
    assertEquals(image, new PerfectPicture().takePerfectPicture(0));
    verify(cameraMock).flashLightOn();
}

In this test, Mockito’s mock method is called to create cameraMock, the mock object used in this test. With Mockito’s when and thenReturn methods, the desired behavior is specified. Mockito’s verify method is used to verify whether flashLightOn has been called.

Measure Coverage to Determine Whether There Are Enough Tests

How many unit tests are needed? One way to assess whether you have written enough unit tests is to measure coverage of your unit tests. Coverage, or more precisely, line coverage, is the percentage of lines of code in your codebase that actually get executed when all unit tests are executed. As a rule of thumb, you should aim for at least 80% line coverage with a sufficient number of tests—that is, as many lines of test code as production code.

Why 80% coverage (and not 100%)? Any codebase contains fragments of trivial code that technically can be tested, but are so trivial that testing them makes little sense. Take the following typical Java getter method:

public String getName() {
    return this.name;
}

It is possible to test this getter (with something like assertEquals(myObj.getName(),"John Smith")), but with this test, you are mostly testing that the Java compiler and the Java Virtual Machine work as expected. But it is not true that you should never test getters. Take a typical class that represents postal mail addresses. It typically has two or three string fields that represent (additional) address lines. It is easy to make a mistake like this one:

public String getAddressLine3() {
    return this.addressLine2;
}

A minimum of 80% coverage alone is not enough to ensure high-quality unit tests. It is possible to get high coverage by testing just a few high-level methods (like main, the first method called by the Java Virtual Machine) and not mock out lower-level methods. That is why we advise a 1-to-1 ratio of production code versus test code.

You can measure coverage using a code coverage tool. Well-known examples are the Maven plugin Cobertura, and the Eclipse plugin EclEmma. Figure 10-3 shows coverage of the Joda codebase, an open source Java library with date and time classes that comes with many unit tests.

Figure 10-3. Joda time coverage report in Eclipse, as provided by the EclEmma plugin.

Objection: We Still Need Manual Testing

“Why should we invest in automated tests at all if we still need manual testing?”

The answer to this question is simply because test automation frees up time to manually test those things that cannot be automated.

Consider the downsides of the alternative to automated tests. Manual testing has clear limitations. It is slow, expensive, and hard to repeat in a consistent manner. In fact, technical verification of the system needs to take place anyway, since you cannot manually test code that does not work. Because manual tests are not easily repeatable, even with small code changes a full retest may be needed to be sure that the system works as intended.

Objection: I Am Not Allowed to Write Unit Tests

“I am not allowed to write unit tests because they lower productivity according to my manager.”

Writing unit tests during development actually improves productivity. It improves system code by shifting the focus from “what code should do” toward “what it should not do.” If you never take into account how the code may fail, you cannot be sure whether your code is resilient to unexpected situations.

The disadvantages of not having unit tests are mainly in uncertainty and rework. Every time a piece of code is changed, it requires painstaking review to verify whether the code does what it is supposed to do.

Objection: Why Should We Invest in Unit Tests When the Current Coverage Is Low?

“The current unit test coverage of my system is very low. Why should I invest time now in writing unit tests?”

We have elaborated on the reasons why unit tests are useful and help you develop code that works predictably. However, when a very large system has little to no unit test code, this may be a burden. After all, it would be a significant investment to start writing unit tests from scratch for an existing system because you would need to analyze all units again. Therefore, you should make a significant investment in unit tests only if the added certainty is worth the effort. This especially applies to critical, central functionality and when there is reason to believe that units are behaving in an unintended manner. Otherwise, add unit tests incrementally each time you change existing code or add new code.