12  JUnit

JUnit is a testing framework for Java programs. It was originally written by Kent Beck and Erich Gamma. It is a framework with unit-testing functionalities It is integrate as a plug-in in most IDEs, making it seamless to execute the test during the routing code development activities.

The official site is: http://www.junit.org

Unit tests are intended to verify the correctness of units of a program. During testing, units are considered in isolation. In Java, units are typically classes and packages.

Unit testing is particularly important when software requirements change frequently. Code often need be refactored to incorporate the changes. Unit tests help ensure that the refactored code continues to work.

JUnit Framework JUnit helps the programmer: Define and execute tests and test suites Formalize requirements and clarify architecture Write and debug code Integrate code and always be ready to release a working version

12.1 History

The key steps in the history of JUnit are:

  • 1997 on the plane to OOPSLA97 Kent Beck and Erich Gamma wrote JUnit
  • Junit.org – August 2000
  • Junit 3.8.1 – September 2002
  • Junit 4.0 – February 2006, the latest release: 4.13.2 - Feb 2021
  • Junit 5.0 – September 2017, the latest release: 5.12.0 – Feb 2025

12.2 JUnit at work

Each test case in JUnit is represented as a test method. A test method returns no result, it just performs operations on the code under test and checks the results. Checks are performed using a set of assert*() methods. The JUnit framework detects the anomalies and reports them.

The basic testing procedure that JUnit adopts is

  • For each test (method) Junit:
    • calls pre-test fixture Intended to acquire resources and create any objects that may be needed for testing
    • calls the test method The actual test that checks the output of the element under test
    • calls post-test fixtures Intended to release resources and remove any objects created that is no longer needed

12.2.1 Assertions

The main standard assert*() methods are;

  • assertTrue(boolean test)
  • assertFalse(boolean test)
  • assertEquals(expected, actual)
  • assertSame(Object exp, Object actual)
  • assertNotSame(Object exp, Object act)
  • assertNull(Object object)
  • assertNotNull(Object object)
  • fail()

We can test a boolean condition using assertTrue(condition). If the tested condition is

  • true => it proceeds with execution
  • false => aborts the test method execution, and prints out the optional message

We can test the opposite condition using the assertFalse(condition) that fails if condition is true.

We can test the equality of primitives and objects using the assertEquals( expected, actual) method. For floating point values we need to express a threshold assertEquals(expected,actual,err), this is due to the fact that floating point operation might produce approximantion errors, therefor a precise comparison in this case might produce the wrong result. Ex. assertEquals(1.0, Math.cos(3.14), 0.01);

All assert* method have an overload version that include also a message that is reported when the assertion failes.The method is in fact optional, but it is recommended to provided it, so that debugging a failing test is easier.

All JUnit versions up to 4 use the following method signature:

static void assertTrue( String message, boolean test)

While JUnit 5 adopted a slighly different signature with the message as the last parameter. In addition it is possible to provide a supplied instead of a precomputed message:

static void assertTrue( boolean test, String message )
static void assertTrue( boolean test, Supplier<String> messageSupplier)

12.2.2 Fixtures

The fixtures are portions of test cases that are cloned in several different tests. They are typically used for initializing or releasing resources.

There are two main types of fixtures:

  • Pre-test (set up)
  • Post-test (tear down)

JUnit allaows placing the fixtures in separate methods to avoid duplication and shorten test cases. Such fixtures are automatically executed before and after tests. Therefore they are written once and avoid the risk of forgetting some initialization.

12.2.3 Failures vs. Errors

The outcoma of a test can be

  • Success: all the code in the method has been executed and all assertions have been verified, in this case we say that the test is passed.
  • Failure: an assert*() method found the condition it checked is not verified, in practice the code under test produced an output, but it is not the expected one. During the execution of the tests an error was found (e.g., NullPointerException)
  • Error: an unexpected exception is detected during the execution of the test method, the program could not produce any output due to an error.
  • Skipped: the test has not been executed because some pre-condition was not met.

12.2.4 Skipping tests

There may be several reasons to skip a test:

  1. I wrote a test for a specific requirement, but that requirement hasn’t been implemented yet.
    It is pointless to see a failure or an error for such a test when I already know perfectly well that I haven’t done that part yet.
  2. Test execution requires a lot of time for large applications, so to speed up the excution I could skip some tests that are less critical and concentrate the attention on other more important tests.
  3. There could be conditions not met by the system where tests are executed.
    • Imagine you have an app that should perform some action using a fingerprint reader, but I’m running the test on a computer without any fingerprint reader; I know for sure that it would fail because the hardware is missing, it is better to skip the test because a failure would tell me nothing about the code.
    • I might have a test specific for MacOS, another one specific for Windows, and another specific for Linux; obviously, depending on the operating system I’m running the test on, some tests will run, others won’t.

12.2.5 Testing exceptions

In presence of methods throwing exceptions, at least two main cases shall be checked:

  • a normal behavior is expected, therefore no exception should be thrown, in this case if that exception is raised an error is reported.
  • an anomaly is expected, therefore an exception should be thrown, i n this case the tests fails if NO exception is detected.

12.2.6 Sample class: Stack

public class Stack {
    private int[] stack; 
    private int next = 0;

    public Stack(){ this(10);}
    public Stack(int size){ stack = new int[size];}
    
    public boolean isEmpty(){ return next==0; }
    
    public boolean push(int i) {
        if(next==stack.length) return false;
        stack[next++] = i;
        return true;
    }
    public int pop() throws StackException { 
        if(next==0) throw new StackException()
        return stack[--next]; 
    }
}

12.3 Junit 4 Syntax

JUnit 4 – differently from previous versions – introduced the use of java annotations to make different portion of the tests.

The adaventages are:

  • Less constraints on names
  • Easier to read/write
  • Backward compatible with JUnit 3

12.3.1 Assertions

assert*() methods assertThat() method To use the Hamcrest matchers

12.3.2 Test a Stack

public class TestStack {
@Test
public void testStack() throws StackException {
    Stack aStack = new Stack();
    assertTrue("Stack should be empty",
                     aStack.isEmpty());
    aStack.push(10);
    assertFalse("Stack should not be empty!",
                      aStack.isEmpty());
    aStack.push(-4);
    assertEquals(-4, aStack.pop());
    assertEquals(10, aStack.pop());
 }
}

12.3.3 Running a test case

The JUnit runner executes all methods that satisfy the following conditions:

  • are annotated with “@Test”
  • have public visibility
  • return void
  • accept no arguments ()

The runner ignores the rest of the class. It is common practice to include in the class attributes and helper methods provided they are not annotated or not public.

12.3.4 Fixtures

Annotate a method with @Before to make it a pre-test fixture:

  • It is executed before each test method is run
  • It is intended to initialize the objects that will be used by test methods
  • There is no limit to the setup you can do in a pre-test method
  • Helps reducing duplication of code

Annotate a method with @After to make it a post-test fixture:

  • It is executed after each test method is run
  • It is intended to release system resources (such as streams)
  • In most cases, a post-test fixture is not required
  • Before the next test is executed the pre-test fixture is run again so attribute will be re-initialized

12.3.5 Testing Exceptions

There are two alternative approaches to testing methods in a settinge where they are expected to throw an exception:

  • declare it in the annotation,
  • use a specific assert statement.

When an exception is expected from the test it can be declared with the annotation

@Test(expected=StackException.class)
public void testEmptyPop() throws StackException {
  Stack aStack = new Stack();
  aStack.pop();
}

If not declared any exception thrown will be counted as an Error

Alternatively, it is possivle to assert that an exception is expected:

assertThrows("message", 
             Exception.class, 
             ()-> { /* exception throwing code */ })

In the case of the Stack class to test that a pop operation on an empty stack produces an exception we can use the following code:

@Test
public void testEmptyErrorAssert() {
  Stack aStack = new Stack();
  assertThrows("Expected excecption on empty pop",
               StackException.class, 
               ()-> aStack.pop());
}

When an exception is not expected, it must be declared, otherwise we have an unhandled exception error:

@Test
public void testPop() throws StackException {
  Stack aStack = new Stack();
  aStack.push(1);
  aStack.pop();
}

12.3.6 TestSuite

Allows running a group of related tests as a single batch:

@RunWith(Suite.class)
@SuiteClasses({ 
    TestStack.class, AnotherTest.class 
})
public class AllTests { }

12.3.7 Skipping tests

Some tests can be intended to be skipped, e.g., not ready yet:

@Ignore("Incomplete test")

Other tests can be executed only is some condition are met, e.g., a resource is available

assumeTrue("X not available", res.isAvailable())

12.3.8 Summary of JUnit 4 Annotations

  • @Test: marks test methods
  • @Before and @After Mark pre and post fixtures
  • @Ignore Mark method or class to be skipped
  • @RunWith(Suite.class) declare a suite manager class
  • @Suite.SuiteClasses({ … }) define the classes of the suite

All classes and annotations in JUnit 4 are defined in package `org.junit.

Assertions are made available with

import static org.junit.Assert.*;
import static org.junit.Assume.*;

Annotations must be imported as

import org.junit.After;
import org.junit.Before;
import org.junit.Test;

Suites require:

import org.junit.runners.Suite;
import org.junit.runners.Suite. SuiteClasses;

12.4 Junit 5

Uses Java annotations but different from 4:

  • @Before/@After -> @BeforeEach/@AfterEach @BeforeClass/@AfterClass -> @BeforeAll/@AfterAll

Test methods not necessarily public

Java8 Lambda support, extensions, parameterized tests, etc.

Suites:

@RunWith(JUnitPlatform.class)
@SelectClasses({ … })
@SelectPackages({ … })

12.4.1 Test a Stack

public class TestStack {
    @Test
    public void testStack() throws StackException{
    Stack aStack = new Stack();
    assertTrue(aStack.isEmpty(),
                    "Stack should be empty");
    aStack.push(10);
    assertFalse(aStack.isEmpty(),
                    "Stack should not be empty!");
    aStack.push(-4);
    assertEquals(-4, aStack.pop());
    assertEquals(10, aStack.pop());
    }
}

Expected exception test

@Test
public void testSomething(){
  // e.g. method invoked with “wrong” args 
   assertThrows(PossibleException.class,()->{
    obj.method("Wrong Argument") 
 });
}

12.4.2 Skipping tests

Some tests can be intended to be skipped, e.g., not ready yet:

@Disabled("Incomplete test")

Other tests can be executed only is some condition are met, e.g., a resource is available

assumeTrue("X not available", res.isAvailable())

12.4.3 TestSuite

Indicate the JUnitPlatform runner Select classes with SelectClasses

@RunWith(JUnitPlatform.class)
@SelectClasses({ 
    TestStack.class, AnotherTest.class 
})
public class AllTests { }

12.4.4 Summary of JUnit 5

All classes and annotations are in package org.junit.jupiter.api

Assertions are made available with

import static org.junit.jupiter.api.Assertions.*;

Annotations have to be imported as

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

Suites require:

import org.junit.platform.runner.JUnitPlatform;
import org.junit.platform.suite.api.SelectClasses;

12.5 Using JUnit

12.5.1 Test-Driven Development

Specify part of a feature yet to be coded Run the test and see it fail (red bar) Write code until the test pass (green bar) Repeat until whole feature implemented Refactor while maintaining the bar green

Keep your code clean: keep the bar green

12.5.2 Regression Testing

12.5.3 Bug Reproduction

When a bug is reported, specify the expected correct outcome.

See the test fail, i.e., reproduce the bug

Modify the code and adjust it until the bug-reproducing tests pass.

Check for regressions, with the existing test suites

12.5.4 Guidelines

Test should be written before code

Test everything that can break

Run tests as often as possible

Whenever you are tempted to type something into a print statement or a debugger expression, write it as a test

M.Fowler

12.5.5 Limitations of unit testing

JUnit is designed to call methods and compare the results they return against expected results

This ignores:

  • Programs that do work in response to GUI commands
  • Methods that are used primary to produce output

Heavy use of JUnit encourages a “functional” style, where most methods are called to compute a value, rather than to have side effects. This can actually be a good thing!

Methods that just return results, without side effects (such as printing), are simpler, more general, and easier to reuse

12.6 References

  • K.Beck, E.Gamma. Test Infected:  Programmers Love Writing Tests http://members.pingnet.ch/gamma/junit.htm

  • Junit 5 home page https://junit.org

  • Junit 4 home page https://junit.org/junit4/

  • Hamcrest matchers http://hamcrest.org/JavaHamcrest/

  • AssertJ – Fluent assertions http://joel-costigliola.github.io/assertj/index.html