How to solve Spring Webflux DataBufferLimitException

1. Introduction

In this tutorial, we will explore why we can see databufferlimit exception In Spring webflux application. Then we’ll take a look at the different ways we can address this.

2. Understanding the Problem

Let us understand the problem before jumping to the solution.

2.1. what is DataBufferLimitException?

Spring WebFlux limits the buffering of data in-memory in the codec to avoid application memory problems. By default, it is configured to 262,144 bytes, When this is not sufficient for our use case, we will end databufferlimit exception,

2.2. what one codec,

spring-web And spring-core The modules provide support for serializing and deserializing byte content from high-level objects via non-blocking I/O with reactive stream back pressure. codex Provide an alternative to Java serialization. One advantage is that, generally, objects do not need to be implemented serializable.

3. Server Side

Let’s first see how databufferlimit exception Plays out from a server perspective.

3.1. reproduce the issue

Let’s try to send a JSON payload of size 390 KB to our Spring WebFlux Server application to make an exception. we will use curl a. dispatch order Post Request to our server:

curl --location --request POST 'http://localhost:8080/1.0/process' \
  --header 'Content-Type: application/json' \
  --data-binary '@/tmp/390KB.json'

as we can see, databufferlimit exception is thrown:

org.springframework.core.io.buffer.DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144
  at org.springframework.core.io.buffer.LimitedDataBufferList.raiseLimitException(LimitedDataBufferList.java:99) ~[spring-core-5.3.23.jar:5.3.23]
  Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
  *__checkpoint ⇢ HTTP POST "/1.0/process" [ExceptionHandlingWebHandler]

3.2. solution through properties

the easiest solution would be Configure application assets spring.codec.max-in-memory-size, Let’s add the following to our application.yaml file:

spring:
    codec:
        max-in-memory-size: 500KB

With this, we should now be able to buffer payloads larger than 500 KB in our application.

3.3. code. solution through

Alternatively, we can use WebFluxConfigurer Interface for configuring the same threshold. To do this, we will add a new configuration class, WebfluxConfiguration:

@Configuration
public class WebFluxConfiguration implements WebFluxConfigurer {
    @Override
    public void configureHttpMessageCodecs(ServerCodecConfigurer configurer) {
        configurer.defaultCodecs().maxInMemorySize(500 * 1024);
    }
}

This approach will also provide us with the same result.

4. Client side

Let’s now switch gears to look at the client-side behavior.

4.1. reproduce the issue

We will try to reproduce the same behavior with webflux web client. Let’s create a handler that calls the server with a payload of 390KB:

public Mono<Users> fetch() {
    return webClient
      .post()
      .uri("/1.0/process")
      .body(BodyInserters.fromPublisher(readRequestBody(), Users.class))
      .exchangeToMono(clientResponse -> clientResponse.bodyToMono(Users.class));
}

We see again that the same exception is thrown but this time because of web client Trying to send a payload larger than allowed:

org.springframework.core.io.buffer.DataBufferLimitException: Exceeded limit on max bytes to buffer : 262144
  at org.springframework.core.io.buffer.LimitedDataBufferList.raiseLimitException(LimitedDataBufferList.java:99) ~[spring-core-5.3.23.jar:5.3.23]
  Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
    *__checkpoint ⇢ Body from POST http://localhost:8080/1.0/process [DefaultClientResponse]
    *__checkpoint ⇢ Handler com.baeldung.spring.reactive.springreactiveexceptions.handler.TriggerHandler@428eedd9 [DispatcherHandler]
    *__checkpoint ⇢ HTTP POST "/1.0/trigger" [ExceptionHandlingWebHandler]

4.2. solution through properties

again, the easiest solution is Configure application assets spring.codec.max-in-memory-size, Let’s add the following to our application.yaml file:

spring:
    codec:
        max-in-memory-size: 500KB

With this, we will now be able to send payloads larger than 500 KB from our application. It is worth noting that this configuration applies to the entire application, which means for all web clients And the server itself.

So, if we want to configure this limit only for specific web clients, this would not be an ideal solution. Additionally, there is a caveat with this approach. was about to make web client Must be auto-wired by spring as below:

@Bean("webClient")
public WebClient getSelfWebClient(WebClient.Builder builder) {
    return builder.baseUrl(host).build();
}

4.3. code. solution through

We also have a programmatic way of configuring the web client to achieve this goal. let’s make one web client With the following configuration:

@Bean("progWebClient")
    public WebClient getProgSelfWebClient() {
        return WebClient
          .builder()
          .baseUrl(host)
          .exchangeStrategies(ExchangeStrategies
	  .builder()
	  .codecs(codecs -> codecs
            .defaultCodecs()
            .maxInMemorySize(500 * 1024))
	    .build())
          .build();
}

And with that, now we will be able to successfully send payload of more than 500 KB using our web client.

5. Epilogue

In this article, we understand what databufferlimit exception and saw how to fix them on both server and client side. We saw two approaches to both, one based on property configuration and the other programmatically. We hope that this exception will no longer cause trouble for you.

As always, the full code is available on GitHub.

       

Leave a Comment