JaCoCo Reports With Spring Boot, Gradle and Kotlin

1. Introduction

In this blog post, I would like to walk you step by step through the process of generating test coverage reports with JaCoCo, Spring Boot and Gradle.

JaCoCo is is a free code coverage library for Java and these metrics can really help us to improve our code quality and reduce the possibility of bugs. We’ve got plenty of possibilities to incorporate it into our projects and in this tutorial we will focus on the report generation with JaCoCo Gradle plugin.

2. Prepare Example Code

Let’s start everything by adding two example Spring components:

@Component
class ComponentOne {

  fun one(): String = "example"

  fun two(arg: Int): Double =
    if (arg > 0)
      10.0
    else
      20.0
}

And the second one, called ComponentTwo:

@Component
class ComponentTwo {

  fun two(arg: Int): Double =
    if (arg > 0)
      10.0
    else
      20.0
}

As the next step, let’s create tests for ComponentOne. If you are an IntelliJ user, then you can just click on the class name, hit alt + enter and confirm tests creation:

internal class ComponentOneTest {

  private val componentOne = ComponentOne()

  @Test
  fun testNumberOne() {
    val result = componentOne.one()

    assertEquals("example", result)
  }

  @Test
  fun testNumberTwoPositive() {
    val result = componentOne.two(100)

    assertEquals(10.0, result)
  }

  @Test
  fun testNumberTwoNegative() {
    val result = componentOne.two(-100)

    assertEquals(20.0, result)
  }

}

As you might have noticed, we didn’t implement tests for the second component. This is intentional and will allow us to understand JaCoCo reports better.

It’s worth mentioning here, that these two classes has been automatically generated, when creating a Spring Boot project:

@SpringBootApplication
class JacocoApplication

fun main(args: Array<String>) {
  runApplication<JacocoApplication>(*args)
}

And related test:

@SpringBootTest
class JacocoApplicationTests {

  @Test
  fun contextLoads() {}

}

2. Add JaCoCo Plugin

With that being done, let’s navigate to the build.gradle.kts and add the JaCoCo Plugin to our Spring Boot project:

plugins {
  // Other Plugins
  id("jacoco")
}

jacoco {
  toolVersion = "0.8.7"
}

Although it’s not necessary, it’s a good practice to additionally set the version of our plugin.

3. Generate First JaCoCo Spring Boot Report

As the next step, let’s generate our first JaCoCo report with Gradle commands:

// Note #1: if we are using Windows, then we have to specify .\ instead
// Note #2: if we haven't run build before, then we have to do that to generate the report

./gradlew build  
./gradlew jacocoTestReport

The report will be generated inside the default directory:

$buildDir/reports/jacoco/test

Nextly, let’s navigate to this directory and open up the index.html file (placed inside the html folder):

As we can see, the report was generated successfully providing us with coverage information.

Please don’t worry if you find any of the above information confusing, I will explain them in the next chapters.

But for now, let’s focus on the displayed elements. The above Element names are nothing else, than our packages. If we click on the first one, we should see the following:

Unfortunately, when creating Spring Boot project with Kotlin, the JacocoApplication.kt file will result in two, separate files after compilation. Please don’t worry about this now, I will show you how to get rid of that in reports later.

As the next step, let’s get back to the previous page and check out the second package:This time, the report shows that ComponentOne is covered in 100%, which makes sense, because we’ve created 3 test cases earlier. However, the 33% percent of coverage for ComponentTwo might seem really strange, given the fact that we haven’t created any tests for this class.

Well, everything becomes clear when we check the details of this class:

Although we didn’t implement constructors in our code explicitly, they are created during the compilation process. And these default constructors are treated by JaCoCo report generator as already covered methods (and that’s the same reason behind the JacocoApplication class being displayed as covered).

4. JaCoCo Coverage Counters

With all of that being said, let’s take a while to understand the meaning of presented metrics (I highly encourage you to check out the official documentation here, to better understand these concepts).

4.1. What Excatly Counters Are?

Basically, these metrics are called the Coverage Counters. They are derived from the information contained in the Java byte code instructions and optionally embedded debug information. Nevertheless, some language constructs not always can be directly compiled to corresponding byte code, which sometimes can lead to an unexpected code coverage results. The reason I am mentioning it is pretty simple- code coverage tools are a great way to help and improve code quality and reduce the amount of bugs, but should not be treated as an oracle!

4.2. Explanation

So, let’s reveal the meaning of these counters:

• Instructions (C0 Coverage)– this one is responsible for providing information about the amount of byte code instructions that has been executed, or missed
• Branches (C1 Coverage) – this counter indicates branch coverage of all if and switch statements. For example, the simple if-else instruction makes two branches. If we write a test case, which only evaluates this if expression to true, then the branch coverage will be 50%. To achieve 100% percent branch coverage in this case, we should add a second test checking the result of the else statement.
• Cyclomatic Complexity (Cxty Column)– the cxty is the minimum number of paths that can, in (linear) combination, generate all possible paths through a method. To put it simply- it indicates the number of test cases required to fully cover given piece of code, like class or a method. For instance, a simple method containing one if-else statement has the cyclomatic complexity of 2.
• Lines– although the name could indicate the number of covered lines in our code, this is not the case here. It might seem counterintuitive, but please remember that depending on the source formatting, one line of our source code can refer to multiple classes, methods or perform multiple actions. Therefore, this counter informs us about the number of byte code instructions.
• Methods– indicates a number of non-abstract methods. Additionally, the default constructors or initializers for constants are also considered as methods. Basically, if at least one instruction from a “method” has been executed, then it is considered executed.
• Classes– finally, classes indicates whether at least one method from the given class has been executed

As I’ve mentioned earlier, I highly encourage you to take a while to understand these metrics better and validate this theory against our example report.

5. Custom Report Directory

Either way, let’s get back to the practice part of this tutorial and learn a few things, which might be useful in our real-life projects. As the first step, let’s see how to set a custom directory for our reports.

To do that, let’s edit the build.gradle.kts file:

jacoco {
  toolVersion = "0.8.7"
  reportsDirectory.set(layout.buildDirectory.dir("my-custom-dir"))
}

From now on, our reports will be generated inside the $buildDir/my-custom-dir directory.

On the other hand, we are not tightly coupled to the $buildDir, so we can easily refer to the project root, as well:

reportsDirectory.set(layout.projectDirectory.dir("my-custom-dir"))

This way, our reports will be generated inside the $projectDir/my-custom-dir.

6. Generate Reports In Different Formats

Additionally, the Jacoco plugin allows us to generate reports not only in the HTML format. This feature can be really helpful, when we want process the information with some other tool or custom application.

To declare desired output formats, let’s configure the JacocoReport task:

tasks.withType<JacocoReport> {
  reports {
    xml.required.set(true)
    csv.required.set(true)
    html.required.set(false)
  }
}

With this code, we instruct the JaCoCo plugin to disable the HTML format in favor of CSV and XML formats.

7. Generate Report After Tests

Another useful configuration might be finalizing tests invocation with report generation.

To do so, let’s configure the Test task:

tasks.withType<Test> {
  useJUnitPlatform() // Note: automatically generated when creating project
  finalizedBy(tasks.jacocoTestReport)
}

Nextly, let’s validate it by running the test command:

./gradlew test

As we can see, with this configuration we do not have to call the jacocoTestReport task explicitly anymore. It can be really helpful, if we would like to incorporate JaCoCo into our existing flows, like CI/CD for instance.

8. Exclude Files From Reports

As I’ve mentioned in chapter 3, JaCoCo generates unnecessary coverage report for the main class in Spring Boot.

However, we can get rid of any file from coverage reports with the following code:

tasks.withType<JacocoReport> {

  afterEvaluate {
    classDirectories.setFrom(files(classDirectories.files.map {
      fileTree(it).apply {
        exclude( "**/JacocoApplication**")
      }
    }))
  }

}

This way, each file matching the given pattern will be excluded from our reports.

Let’s check this theory:

./gradlew jacocoTestReport

After that, our html file will look like that:

Given the fact, that the JacocoApplication.kt file is the only one within the com.codersee.jacoco package, we cannot see it anymore in the table.

9. Exclude With Annotation

On the other hand, custom annotations are really useful, if we would like to have more control over the exclusions.

Let’s add the CoderseeGenerated annotation:

@Target(AnnotationTarget.FUNCTION, AnnotationTarget.CLASS)
@Retention(AnnotationRetention.RUNTIME)
@MustBeDocumented
annotation class CoderseeGenerated

As the next step, let’s add it to to the function one of the ComponentOne:

@CoderseeGenerated
fun one(): String = "example"

Additionally, let’s derocate our ComponentTwo class with it:

@CoderseeGenerated
@Component
class ComponentTwo

Finally, let’s re-run report generation:

./gradlew jacocoTestReport

As a result, we should see the following:

And:

The above screenshots clearly prove, that our annotation works, as expected.

10. Summary

And that would be all for this article about generating reports with JaCoCo, Spring Boot and Gradle. If you found this material useful, I will be pleasured if you would like to let me know about it (either in the comments section below, or with the contact form).

As always, if you would like to see the full source code, please check out this GitHub repository.

Previous articles, you might be interested in:

• Kotlin groupBy() and partition() Explained
• REST API With Ktor, Ktorm and PostgreSQL
• Exception Handling with @RestControllerAdvice and @ExceptionHandler