Integration Tests for WebClient with WireMock

The image is a featured image for the post about integration testing of WebClient with WireMock and consist of Spring Boot logo and WireMock logo in the foreground and a space photo in the background.

Hi guys! πŸ™‚ Today we will learn how to do integration testing for Spring WebClient that invokes external REST APIs with WireMock and JUnit 5.

As an example, we will implement a simple logic responsible for querying the GitHub API using coroutines. Nevertheless, I am pretty sure you will be able to easily adjust this article to your needs. (of course, if you are here only for the WireMock part, then I recommend skipping to the “WebClient Integration Testing With WireMock” chapter).

Lastly, I just wanted to add that you can use this testing approach regardless of whether you are using WebClient, Retrofit, Ktor Client, or any other HTTP client.

Video Tutorial

If you prefer video content, then check out my video:

If you find this content useful, please leave a subscription πŸ™‚

Which WebClient Testing Strategy Is The Best?

But before we head to the practice part, let’s try to answer the above question.

Quick answer- there is no such thing as the best testing strategy.

And for the longer answer, let’s take a look at the visualization of the testing pyramid first:

Image presents a diagram with testing pyramid including e2e tests, integration tests, and unit tests helping to better understand the best approach for WebClient WireMock testing.

The tests are ordered based on their scope and execution time. The closer we are to the top, the longer the tests take, but they also give us more confidence.

  • unit tests form the base, as they are quick to run and pinpoint specific code issues,
  • integration tests come next, validating the interactions between units,
  • E2E tests, with a broader scope, are placed at the top.

So long story short, although the e2e tests give us the most confidence, we cannot rely only on them, because the feedback loop would take way too long. And this would affect the development process.

So usually, we make concessions and introduce more integration and unit tests.

But which of these two should we pick for the WebClient?

In my opinion- integration tests.

Whenever we are talking about logic responsible for communicating with external API, we must make sure that:

  • we query the appropriate URL,
  • we pass the necessary headers with appropriate values,
  • request/response bodies are properly serialized/deserialized,
  • and many, many more.

With unit testing, the amount of mocking here would be so significant that I would question whether we need such a test at all.

Moreover, in Spring we can write integration tests for WebClient with WireMock at almost no effort.

Set Up GitHub API Key

With that said, let’s set up the API key we will use to query the GitHub API.

Note: technically, the endpoint we will use does not require any token. However, for the learning purposes I want to show you how to deal with passing bearer tokens, too πŸ™‚

As the first step, let’s navigate to the tokens tab in GitHub settings and click the “Generate new token (classic)”:

Image is a screenshot and presents how to generate new classic GitHub token.

On the next page, let’s specify the name, expiration, and desired scopes:

Image is a screenshot from GitHub tokens page.

If you don’t know what scopes you will need, please follow the principle of lowest privilege. At the end of the day, we can generate a new token at some point in the future when the requirements change.

When we are ready, let’s hit the generate button copy the token value, and store it securely. We will not see it again!

Query The Api With WebClient

Before we implement the WireMock tests, let’s prepare a simple project responsible for querying the external API with WebClient. So, even if you are here only for the WireMock part, I encourage you to briefly check this paragraph to be on the same page πŸ˜‰

Long story short, we will implement WebClient with coroutines that will query the “List repositories for a user” endpoint. For more details about it, check out the GitHub API documentation, please.

Generate New Project

As the first step, let’s navigate to the https://start.spring.io/ page and generate a fresh Spring Boot project:

Image presents the Spring Initializr page screenshot of a project setting that we will use to implement WebClient logic and test with WireMock.

In general, the most important thing is that we will use Kotlin and Spring Reactive Web this time. Versions and metadata are totally up to you.

Of course, let’s click the “Generate” button, download the ZIP file, and import that to our IDE. Personally, I am using IntelliJ IDEA and if you would like to learn more about its setup, then check out my video on how to install IntelliJ for Kotlin development.

Update Application Properties

As the next step, let’s navigate to the resources package and update the application.yaml file:

api:
  github:
    key: ${GITHUB_API_TOKEN:123}
    url: ${GITHUB_API_BASE_URL:http://localhost:8090}
    version: ${GITHUB_API_VERSION:2022-11-28}

Above we can see our custom properties.

We will source values for our GitHub key, URL, and version from environment variables.

Not only is this approach secure (hardcoding the key with token value would be the worst thing you could do), but also flexible (we can pass different values depending on the environment we are running our app in).

WebClient Configuration

Following, let’s configure a WebClient bean that we will use whenever we want to call the GitHub API.

Firstly, let’s add the config package and put the GitHubApiProperties data class in there:

@ConfigurationProperties("api.github")
data class GitHubApiProperties(
    val url: String,
    val key: String,
    val version: String,
)

In my opinion, the @ConfigurationProperties annotation is the cleanest way to inject values from properties. As we can see above, the prefix and field names simply reflect the structure from the previous step.

With that done, let’s add the GitHubApiConfig in the same package:

@Configuration
@EnableConfigurationProperties(GitHubApiProperties::class)
class GitHubApiConfig(
    private val gitHubApiProperties: GitHubApiProperties,
) {
    @Bean
    fun webClient(builder: WebClient.Builder): WebClient =
        builder
            .baseUrl(gitHubApiProperties.url)
            .defaultHeader("Authorization", "Bearer ${gitHubApiProperties.key}")
            .defaultHeader("X-GitHub-Api-Version", gitHubApiProperties.version)
            .defaultHeader("Accept", "application/vnd.github+json")
            .build()
}

Quite a few things are happening here, so let’s dive into each one.

Firstly, we annotate the class with @Configuration and @EnableConfigurationproperties. The second annotation is necessary to for the previous steps to work.

Then, we inject the GitHubApiProperties instance and use its values to configure a new WebClient bean. This configuration is quite straightforward. We set the base URL and a bunch of headers that will be sent with every request: the authorization token, GH API version, and the accept header (those two are specific to the GitHub API, so let’s focus on that too much).

Prepare Model Classes

With that done, let’s prepare a bunch of data classes that we will need to deserialize JSON responses from the API (+ one exception class). If you would like to add other fields, please check out the documentation link I shared with you before.

As the first step, let’s add the model package and the PageableGitHubResponse:

data class PageableGitHubResponse<T>(
    val items: List<T>,
    val hasMoreItems: Boolean,
)

The API endpoints allow us to get only a chunk of data using the page and per_page parameters. And this class can be reused whenever we’re using a paginated endpoint.

As the next step, let’s implement the GitHubRepoResponse along with GitHubOwnerResponse:

data class GitHubRepoResponse(
    val fork: Boolean,
    val name: String,
    val owner: GitHubOwnerResponse,
)

data class GitHubOwnerResponse(
    val login: String,
)

As I mentioned at the beginning of this paragraph, we need those two in order to deserialize the actual JSON response payload.

Lastly, let’s add the UpstreamApiException:

import org.springframework.http.HttpStatusCode

data class UpstreamApiException(
    val msg: String,
    val statusCode: HttpStatusCode,
) : RuntimeException(msg)

This way, we will be able to throw custom exceptions. Such an exception could be handled later, for example with @RestControllerAdvice.

Make Use Of WebClient

Finally, let’s combine all of that together.

So, let’s add the api package and implement the GitHubApi class:

@Component
class GitHubApi(
  private val webClient: WebClient,
) {

  suspend fun listRepositoriesByUsername(
    username: String,
    page: Int,
    perPage: Int,
  ): PageableGitHubResponse<GitHubRepoResponse>? =
    webClient.get()
      .uri("/users/$username/repos?page=$page&per_page=$perPage")
      .awaitExchangeOrNull(::mapToPageableResponse)
}

Don’t worry about the mapToPageableResponse, we will add it in a moment.

But before that, let’s analyze the above code.

Firstly, we annotate the class as the @Component to make it a Spring bean and inject the WebClient instance we configured earlier. We don’t have other WebClient instances, so there is no need to use @Qualifier or anything like that.

Following, we add the listRepositoriesByUsername function that lets us pass the username along with page and perPage. This way, we make this function reusable and allow the invoker to decide how to tackle the pagination.

Additionally, we mark it as a suspend function because the awaitExchangeOrNull is a suspend function. However, if you are interested in this topic, then I refer you to my other article about WebClient with coroutines.

Lastly, let’s add the missing code:

private suspend inline fun <reified T> mapToPageableResponse(clientResponse: ClientResponse): PageableGitHubResponse<T>? {
  val hasNext = checkIfMorePagesToFetch(clientResponse)

  return when (val statusCode = clientResponse.statusCode()) {
    HttpStatus.OK ->
      PageableGitHubResponse(
        items = clientResponse.awaitBody<List<T>>(),
        hasMoreItems = hasNext,
      )

    HttpStatus.NOT_FOUND -> null

    else -> throw UpstreamApiException(
      msg = "GitHub API request failed.",
      statusCode = statusCode,
    )
  }
}

fun checkIfMorePagesToFetch(clientResponse: ClientResponse) =
  clientResponse.headers()
    .header("link")
    .firstOrNull()
    ?.contains("next")
    ?: false

Long story short, the above code works as follows:

  1. Firstly, we take the response and check if there are more pages. How? Well, without going into details, GitHub API returns the link header. And if its value contains the rel="next", it means that there are more pages to fetch.
  2. Finally, we check the status code. If it is 200 OK, we simply read the JSON value. When we get 404 Not Found, we return null (we can expect this status code when we pass a non-existing username, and in my opinion, this case is not exceptional). And whenever we receive another status code, we simply throw the UpstreamApiException.

WebClient Integration Testing With WireMock

Excellent, at this point we have (almost) everything we need to test our WebClient implementation with WireMock.

We will see only a few example test cases that you will be able to easily adapt according to your needs:

  • 200 OK response with an empty list, 200 OK with items, and more pages, and 200 OK with items and no more pages to fetch
  • 404 Not Found,
  • 401 Unauthorized

When you will be implementing your own integration tests, this is the moment when you should do the homework and analyze the API you are working with. What are the possible happy paths? How should your code behave in case of any error status code? What if we add some latency? And many, many more πŸ™‚

Additional Dependencies

At this point, we defined our test cases, so we are ready to get our hands dirty.

So, let’s start by adding the necessary dependencies:

testImplementation("org.springframework.cloud:spring-cloud-contract-wiremock:4.1.1")
testImplementation("org.jetbrains.kotlinx:kotlinx-coroutines-test:1.8.0")

As we can see, the first one- the Spring Cloud Contract WireMock module- allows us to use WireMock in our app, whereas the kotlinx-coroutines-test is necessary due to our coroutines WebClient implementation.

Add Expected JSON Files

So, if we know what test cases we would like to test, we should prepare the example JSON API responses.

Personally, whenever working with WireMock I am a big fan of externalizing the expected JSON files. We put them inside the /test/resources directory and refer to them later in the tests. This way, we can improve the readability of the test classes. Of course, unless we want to prepare JSON responses dynamically.

In our case, we could simply invoke the GitHub API and persist responses for different cases. This way, we prepare the following files and put them inside the /test/resources/responses/external/github:

  • list_github_repositories_200_OK_empty_list.json
  • list_github_repositories_200_OK_page_1.json
  • list_github_repositories_401_UNAUTHORIZED.json
  • list_github_repositories_404_NOT_FOUND.json

You can find all of these files in my GH repo here.

The example for 401 Unauthorized looks as follows:

{
  "message": "Bad credentials",
  "documentation_url": "https://docs.github.com/rest"
}

Prepare Util Function

As the last thing before we implement our test class, let’s add the util function inside the util package (of course, in tests).

This function will be responsible for reading JSON files inside the test resources as String values:

import org.springframework.core.io.ClassPathResource

fun getResponseBodyAsString(path: String): String =
    ClassPathResource(path).getContentAsString(
        Charsets.UTF_8,
    )

Implement GitHubApiTest Class

Finally, let’s start implementing the GitHubApiTest class:

private const val TEST_KEY = "TEST_KEY"
private const val TEST_PORT = 8082
private const val TEST_VERSION = "2022-11-28"

@AutoConfigureWireMock(port = TEST_PORT)
@TestPropertySource(
  properties = [
    "api.github.url=http://localhost:${TEST_PORT}",
    "api.github.key=$TEST_KEY",
    "api.github.version=$TEST_VERSION",
  ],
)
@SpringBootTest(
  webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT,
)
class GitHubApiTest {
  @Autowired
  private lateinit var wireMockServer: WireMockServer

  @Autowired
  private lateinit var gitHubApi: GitHubApi

  private val page = 1
  private val perPage = 2
  private val username = UUID.randomUUID().toString()

}

Conceptually, everything starts with the @SpringBootTest and @AutoConfigureWireMock annotations.

We use the @SpringBootTest annotation in Spring Boot integration tests. This way a new ApplicationContext will be created. In our case, we set the webEnvironment with RANDOM_PORT value, so that a reactive context listening on the random port is created.

Additionally, we combine that together with @AutoConfigureWireMock and fixed test port. This way a WireMock server will be started as a part of Application Context.

Lastly, we do 3 things:

  1. We use the @TestPropertySource to pass values for properties.
  2. We inject the WireMockServer and GitHubApi instances that we will work with later. Please note that we can do that just like we would do with a standard, non-test, Spring component.
  3. We prepare some dummy data for page, perPage, and username values we will need for testing.

As a note of comment from my side- this “skeleton” is useful whenever working with WebClient and WireMock.

Test 404 Not Found API Response Case

Wonderful!

As the first test, let’s start with the 404 Not Found case. Usually, we will start with happy path cases, but for learning purposes, this one is the shortest case πŸ˜‰

@Test
fun `Given 404 NOT FOUND response When fetching repository by username Then should return null`() = runTest {
  // Given
  wireMockServer.stubFor(
    WireMock.get(WireMock.urlEqualTo("/users/$username/repos?page=$page&per_page=$perPage"))
      .withHeader("Authorization", WireMock.equalTo("Bearer $TEST_KEY"))
      .withHeader("X-GitHub-Api-Version", WireMock.equalTo(TEST_VERSION))
      .withHeader("Accept", WireMock.equalTo("application/vnd.github+json"))
      .willReturn(
        WireMock.aResponse()
          .withStatus(HttpStatus.NOT_FOUND.value())
          .withHeader(HttpHeaders.CONTENT_TYPE, "application/json; charset=utf-8")
          .withBody(
            getResponseBodyAsString("/responses/external/github/list_github_repositories_404_NOT_FOUND.json"),
          ),
      ),
  )

  // When
  val result = gitHubApi.listRepositoriesByUsername(username, page, perPage)

  // Then
  Assertions.assertNull(result)
}

We mark our test with the @Test (JUnit 5), specify the meaningful name for the test (with awesome Kotlin backticks), and wrap our test with runTest. The last one is necessary as we are working with coroutines and invoke the suspended function. Long story short, in JVM it will behave just like runBlocking (but it will skip delays).

Next comes the WireMock part. And to be even more specific- stubbing.

Stubbing is nothing else than “the ability to return canned HTTP responses for requests matching criteria” (from docs). Simply said, whenever an outgoing request that matches our criteria is made, a mocked response configured in willReturn is returned.

So, as we can see, the above logic will work only when:

  • the GET request is made,
  • URL matches,
  • Headers sent have appropriate values

And this part already tests our logic. Pretty neat, isn’t it? πŸ™‚

Lastly, we simply invoke the function and assert that null is returned. Which confirms that everything is working fine.

Test 401 Unauthorized

As we already started with error cases, let’s verify that whenever the endpoint returns 401 Unauthorized, the UpstreamApiException is thrown:

@Test
fun `Given 401 UAUTHORIZED response When fetching repository by username Then should throw UpstreamApiException`() = runTest {
  // Given
  wireMockServer.stubFor(
    WireMock.get(WireMock.urlEqualTo("/users/$username/repos?page=$page&per_page=$perPage"))
      .withHeader("Authorization", WireMock.equalTo("Bearer $TEST_KEY"))
      .withHeader("X-GitHub-Api-Version", WireMock.equalTo(TEST_VERSION))
      .withHeader("Accept", WireMock.equalTo("application/vnd.github+json"))
      .willReturn(
        WireMock.aResponse()
          .withStatus(HttpStatus.UNAUTHORIZED.value())
          .withHeader(HttpHeaders.CONTENT_TYPE, "application/json; charset=utf-8")
          .withBody(
            getResponseBodyAsString("/responses/external/github/list_github_repositories_401_UNAUTHORIZED.json"),
          ),
      ),
  )

  // When
  val exception = assertThrows<UpstreamApiException> {
    gitHubApi.listRepositoriesByUsername(username, page, perPage)
  }

  // Then
  assertNotNull(exception)

  assertEquals("GitHub API request failed.", exception.msg)
  assertEquals(HttpStatus.UNAUTHORIZED, exception.statusCode)
}

As we can see, the logic responsible for stubbing only slightly changed.

The interesting part here is the assertThrows, which not only makes sure that the exception of an appropriate type is thrown but also returns it. This way we can make additional assertions.

Verify 200 OK With Empty List

Following, let’s cover the empty list case:

@Test
fun `Given 200 OK response with empty list When fetching repository by username Then should return repository with correct properties and has next false`() =
  runTest {
    // Given
    val linkHeader = """<https://api.github.com/user/64011387/repos?page=3&per_page=2>; rel="prev","""

    wireMockServer.stubFor(
      WireMock.get(WireMock.urlEqualTo("/users/$username/repos?page=$page&per_page=$perPage"))
        .withHeader("Authorization", WireMock.equalTo("Bearer $TEST_KEY"))
        .withHeader("X-GitHub-Api-Version", WireMock.equalTo(TEST_VERSION))
        .withHeader("Accept", WireMock.equalTo("application/vnd.github+json"))
        .willReturn(
          WireMock.aResponse()
            .withStatus(HttpStatus.OK.value())
            .withHeader(HttpHeaders.CONTENT_TYPE, "application/json; charset=utf-8")
            .withHeader(HttpHeaders.LINK, linkHeader)
            .withBody(
              getResponseBodyAsString("/responses/external/github/list_github_repositories_200_OK_empty_list.json"),
            ),
        ),
    )

    // When
    val result = gitHubApi.listRepositoriesByUsername(username, page, perPage)

    // Then
    val expected =
      PageableGitHubResponse(
        items = emptyList<GitHubRepoResponse>(),
        hasMoreItems = false,
      )

    assertEquals(expected, result)
  }

This time, we assign the returned value to the result and compare that with the expected.

Check 200 OK With Has Next True

Last before least, let’s make sure that we got the expected objects in a list and that the logic responsible for the link header check returns true:

@Test
fun `Given 200 OK response with payload When fetching repository by username Then should return repository with correct properties and has next true`() =
  runTest {
    // Given
    val linkHeader = """<https://api.github.com/user/64011387/repos?page=3&per_page=2>; rel="next","""

    wireMockServer.stubFor(
      WireMock.get(WireMock.urlEqualTo("/users/$username/repos?page=$page&per_page=$perPage"))
        .withHeader("Authorization", WireMock.equalTo("Bearer $TEST_KEY"))
        .withHeader("X-GitHub-Api-Version", WireMock.equalTo(TEST_VERSION))
        .withHeader("Accept", WireMock.equalTo("application/vnd.github+json"))
        .willReturn(
          WireMock.aResponse()
            .withStatus(HttpStatus.OK.value())
            .withHeader(HttpHeaders.CONTENT_TYPE, "application/json; charset=utf-8")
            .withHeader(HttpHeaders.LINK, linkHeader)
            .withBody(
              getResponseBodyAsString("/responses/external/github/list_github_repositories_200_OK_page_1.json"),
            ),
        ),
    )

    // When
    val result = gitHubApi.listRepositoriesByUsername(username, page, perPage)

    // Then
    val expected =
      PageableGitHubResponse(
        items =
        listOf(
          GitHubRepoResponse(
            fork = false,
            name = "controlleradvice-vs-restcontrolleradvice",
            owner = GitHubOwnerResponse(login = "codersee-blog"),
          ),
          GitHubRepoResponse(
            fork = false,
            name = "freecodecamp-spring-boot-kotlin-excel",
            owner = GitHubOwnerResponse(login = "codersee-blog"),
          ),
        ),
        hasMoreItems = true,
      )

    assertEquals(expected, result)
  }

Test 200 OK With Has Next False

And finally, let’s double-check the flag logic:

@Test
fun `Given 200 OK response with payload When fetching repository by username Then should return repository with correct properties and has next false`() =
  runTest {
    // Given
    val linkHeader = """<https://api.github.com/user/64011387/repos?page=3&per_page=2>; rel="prev","""

    wireMockServer.stubFor(
      WireMock.get(WireMock.urlEqualTo("/users/$username/repos?page=$page&per_page=$perPage"))
        .withHeader("Authorization", WireMock.equalTo("Bearer $TEST_KEY"))
        .withHeader("X-GitHub-Api-Version", WireMock.equalTo(TEST_VERSION))
        .withHeader("Accept", WireMock.equalTo("application/vnd.github+json"))
        .willReturn(
          WireMock.aResponse()
            .withStatus(HttpStatus.OK.value())
            .withHeader(HttpHeaders.CONTENT_TYPE, "application/json; charset=utf-8")
            .withHeader(HttpHeaders.LINK, linkHeader)
            .withBody(
              getResponseBodyAsString("/responses/external/github/list_github_repositories_200_OK_page_1.json"),
            ),
        ),
    )

    // When
    val result = gitHubApi.listRepositoriesByUsername(username, page, perPage)

    // Then
    val expected =
      PageableGitHubResponse(
        items =
        listOf(
          GitHubRepoResponse(
            fork = false,
            name = "controlleradvice-vs-restcontrolleradvice",
            owner = GitHubOwnerResponse(login = "codersee-blog"),
          ),
          GitHubRepoResponse(
            fork = false,
            name = "freecodecamp-spring-boot-kotlin-excel",
            owner = GitHubOwnerResponse(login = "codersee-blog"),
          ),
        ),
        hasMoreItems = false,
      )

    assertEquals(expected, result)
  }

Integration Tests With WireMock And WebClient Summary

And basically, that’s all for this tutorial on how to perform integration testing for Spring WebClient that invokes external REST APIs with WireMock and JUnit 5.

I hope you enjoyed this one and I recommend you check out my other articles related to Spring Framework.

Share this:

Picture of Hi there! πŸ‘‹

Hi there! πŸ‘‹

My name is Piotr and I've created Codersee to share my knowledge about Kotlin, Spring Framework, and other related topics through practical, step-by-step guides. Always eager to chat and exchange knowledge.

Related content

One Response

  1. Hi Piotr,

    thanks a lot for this article. As always, it’s a pleasure to read. I gained many insights on how to access the GitHub-API and how to configure and integration-test SpringBoot applications.

    The Wiremock-Part, however, didn’t convince me. In my opinion wiremock’s fluent API leads to verbose testcode and a low signal/noise ratio.
    Also I’m ambivalent about the usage of json files found in `src/test/resources`. On the one hand, one can easily record actual output from github.com and paste it into a resource file. On the other hand, to validate the test’s behavior, one has to switch editors and must ignore almost all lines in `list_github_repositories_200_OK_page_1.json`.

    In the case, you are interested in alternative approaches to testing, feel free to check out [my repository](https://github.com/neblung/webclientinttest)

    Best regards
    Frank

Leave a Reply

Your email address will not be published. Required fields are marked *

Newsletter

Image presents 3 ebooks with Java, Spring and Kotlin interview questions.

Never miss any important updates from the Kotlin world and get 3 ebooks!

You may opt out any time. Terms of Use and Privacy Policy

Free tutorials and courses on Kotlin & backend

To make Codersee work, we log user data. By using our site, you agree to our Privacy Policy and Terms of Use.