12 Steps to Better Tests

Contents

Introduction

Imagine yourself implementing a feature, and after some hard work, you decide to commit and push your changes. A pipeline starts running and fails after a few minutes. A few unit tests have failed… This can be a tense moment because you changed the behavior of a legacy application, and you have no clue in what state the test class(es) are. Understanding the failed unit test(s) and maybe the dependency between them can be a challenge that makes 1 hour of work last an entire day.

I want to help you improve the overall quality of your test classes so they are more straightforward to work with. I created a 12-step test to see how much improvement a test class needs. The idea came from “The Joel Test: 12 Steps to Better Code”. It's a test to rate the quality of a software team. I liked the idea and wanted to apply it to tests. Just as with Joel's test, you don't have to do some exotic calculations with test coverages and whatnot. It's just twelve questions you can answer with Yes or No. You can give a test class one point for every Yes answer. Each test class can have a maximum of 12 points, meaning it's perfect. Anything less than 12 has the potential to become more outstanding.

These questions do not cover all the factors determining whether a test class is a joy to work with. Some questions may not apply to a test class because you don't mock anything. It could be possible to have a great test class that answers no to most questions. But if you can respond yes to most of these, then there is a big chance that your test classes are a joy to work with.

1. Are your tests independent?

You should be able to run your tests in a random order without having them depend on each other. This makes your tests more maintainable because you don't have to be afraid of breaking other tests.

2. Are your tests dependable?

You should be able to depend on your test. Nobody likes to wait for minutes on a build or pipeline that sometimes fails. Only to restart the whole process and hope it won't fail this time. Sadly the priority of fixing those tests is low, and people will waste time just restarting and waiting for a successful build.

3. Are your tests fast?

Fast feedback cycles are essential. You won't be distracted easily, keeping you in the flow. When developers wait 10 seconds on a test, they will just read something on Hacker News or Reddit. This takes you out of the flow, and getting back in will take some time. There is also the chance that people will work around the test, ignoring it or commenting it out when it takes too much time.

4. Are the @Before and @After relevant for each test?

Doing something before and after each test is a great way to reduce duplicated code. However, it can become a problem when not every test case needs this specific initial state. From that moment on, you are not only wasting CPU cycles and time, but it also makes tests harder to maintain. The @Before annotated method could create an unwanted state for test cases that don't need it.

A better way to implement the @Before and @After would be by using a parametrized test. You could also use a subclass to split the tests cases that need the @Before and @After methods from the test cases that do not require that behaviour.

A nested class separates @Before from the parent class in the following example.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
public class StepsToBetterTests {
    
    Car car;

    @Test
    void testA(){
        car = new Car();

        Assertions.assertEquals(5, car.getSpeed());
    }
    
    @Nested
    class subTests{

        @BeforeEach
        void setUp() {
            car = mock(Car.class);
            when(car.getSpeed()).thenReturn(10);
        }

        @Test
        void subtestA(){
            Assertions.assertEquals(10, car.getSpeed());
        }
    }

    class Car{

        public int getSpeed(){
            return 5;
        }

    }
}

5. Do you only test one thing in each unit test?

A unit test should only cover one test case. This makes the test a lot easier to understand. Otherwise, when a unit test fails, you have to figure out which of the cases broke; of course, those aren't nicely documented.

6. Are tests not duplicated?

The less code you have to maintain, the better. The same goes for test cases; sometimes, you must test a method with different inputs and outputs. For example, when you test a calculator, you would try a variation of numbers to see if the calculation is correct. Instead of copy-pasting a test case multiple times and only changing the numbers, check if your framework has something like parameterized test cases.

In the following example, a parameterized test is used to test the add method.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
private static Stream<Arguments> valuesAndExpectedResultForAdd() {
        return Stream.of(
        Arguments.of(1, 1, 2),
        Arguments.of(2, 1, 3),
        Arguments.of(4, 2, 6)
        );
}

@ParameterizedTest(name = "adding {0} and {1} expected result: {2}")
@MethodSource({"valuesAndExpectedResultForAdd"})
    void add_shouldAddNumbersTogether(int firstInteger, int secondInteger, int expectedResult) {
            assertEquals(add(firstInteger, secondInteger), expectedResult);
}

7. Is every mocked method and object used?

Some mocking frameworks are not very strict by default. As a result, mocks can be created without being used or setting any expectations. By being more precise and explicit about the desired behavior of mocks you improve the quality and maintainability of your tests.

Less strict tests can be harder to debug. You can end up in a situation where misconfigured mocks are used and make the test fail.

8. Is your test code logically ordered?

Like production code has an order, so does the code inside a unit test. I like to write my unit tests in a Given-When-Then style. This style clearly describes what the test case needs, what actions are performed, and when. A unit test would then consist of these three parts:

Given: Create Objects and mocks that you need for the test case;
When: The action you want to test;
Then: The result and the assertions you want to perform.

I don't mean that everybody should use this style, but every unit test in your test class or project should use a coherent style. So it's clear for everybody how to read and write test cases.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
@Test
void testA(){
    // Given
    car = mock(Car.class);

    // When
    int speed = car.getSpeed();

    // Then
    Assertions.assertEquals(5, speed);
}

9. Do you know your test framework?

Writing tests becomes easier when you know your test framework and dependencies. You don't have to know the nitty-gritty details; just know what the framework is capable of and what methods it offers for testing. So when you need those methods, you can look them up. This prevents you from reinventing the wheel and saves you time.

Use the best practices and methods the framework offers to make your tests more readable, maintainable, and much easier to write. So you can focus on how to cover the requirements with your test cases.

In my six years of being a developer, this has helped me the most.

10. Do you test the requirements instead of the implementation?

An ideal test would only need some input and verify if the output is correct. Sadly, tests in the real world are a little messy, requiring some initial state, mocks, etc.

When you write tests to see if you implemented the requirements correctly, you make the test more resistant to changes in production code. The goal is to write tests that verify the answers are correct without being tightly coupled to the production code.

11. Are your tests explicit?

You should be very explicit about the expected results when writing a test. When you test an endpoint, do not only check the body but also that the HTTP status code is what you expect it to be.

Another example would be with a test dependency like Mockito. Mockito allows you to stub a method regardless of the input parameters. This could lead to unwanted results. So, Instead of using any() and allowing any value, be explicit and state the value you expect to be passed. This will make your test more explicit and makes failures more obvious.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
@Test
void LessExplicit(){
    // Given
    car = mock(Car.class);
    
    // Here, we don't specify what number we expect but always return a correct response.
    when(car.shiftGear(anyInt())).thenReturn(10);

    // When
    int gear = car.shiftGear(10);

    // Then
    Assertions.assertEquals(10, gear);
}

@Test
void MoreExplicit(){
    // Given
    car = mock(Car.class);
    
    // Here, we make explicit what input and output we expect from the mock.
    when(car.shiftGear(10)).thenReturn(10);

    // When
    int gear = car.shiftGear(10);

    // Then
    Assertions.assertEquals(10, gear);
}

12. Are your tests simple?

Writing a good test is hard enough, so make it easy to read because it will be read a lot. Ideally, a test should be obvious and show its intent. Test code should be clean and optimized for intuitiveness and explicitness. To keep your tests understandable, you can try following the other advice in this post or looking up additional resources on the internet.

The two most important pieces of advice would be knowing what your test framework is capable of and logically ordering your tests and testing code.