Edeki's Tech Journal

Read Request body multiple times

Edeki Nathaniel -

Let's say you were doing an integration with a third-party system who needed to send data to your system via a webhook, and as part of the webhook verification, you needed to calculate a Hash based on a shared secret and compare this to a hash which the third party system would send to you in a custom header.

The standard way to do this is to implement a Filter in Spring and register it to intercept requests to the request path, and there do the webhook verification.

To illustrate this, let's create a new Project.

Head over to Spring Initializr to generate a new Project, add the dependencies for spring-web, validation, spring security

Let's add the PaymentPayload

So far, our application startups fine

But we're also hit with a 401 when we hit the /webhook endpoint. This is because we added Spring security to the Project. Let's add a SecurityConfig and permit requests to this endpoint for now, so that we can reach the controller

We also disable CSRF so we can proceed, This is not recommended in production.

@Configuration
	@EnableWebSecurity
	static
	class SecurityConfig {
		@Bean
		SecurityFilterChain filterChain(HttpSecurity http) throws Exception {
			http.cors(Customizer.withDefaults())
					.authorizeHttpRequests(auth -> auth
							.requestMatchers("/actuator/health").permitAll()
							.requestMatchers( HttpMethod.POST, "/webhook").permitAll()
							.anyRequest().authenticated()
					)
					.csrf(AbstractHttpConfigurer::disable);
			return http.build();
		}
	}

We still anyway get a 403  😣. But then we see this log from Spring when we don't pass a request body.

When we do pass a request body, we get a warning from Spring from the same DefaultHandlerExceptionResolver for the MethodArgumentNotValidException. What's interesting is that in Spring 2.x, this would not return a 403. I also came across this stackoverflow question. But anyway, let's fix it. 


POST http://127.0.0.1:8080/webhook
Content-Type: application/json

{
  "customerName": "John Okafor"
}

Returning the right Response for Input Validation

We add a ControllerAdvice, we don't even have to handle any exceptions, but we see that we already get a 400, when the request doesn't pass validation, this is exactly what we expect. Problem solved. 

@RestControllerAdvice
static class ControllerAdvice extends ResponseEntityExceptionHandler {}

Expectations of the Filter

We extend OncePerRequestFilter from Spring and state what the filter should do. Normally, with co-pilot or one of these tools, this should be enough to generate the snippet I expect. 😹. But let's write it.

static class WebHookFilter extends OncePerRequestFilter {

		@Override
		protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException {
			//exit early if the expected request header is missing
			//exit early if the request body is missing
			//exit early if the calculated hash doesn't equal the expected hash
		}
	}

The Filter

We implement the filter like so:

static class WebHookFilter extends OncePerRequestFilter {

		private static final String secret = "fake-secret";
		private static final String xWebHookHeader = "x-webhook-hmac";

		private static final Mac mac;

		static {
			String algorithm = "HmacSHA512";
			try {
				mac = Mac.getInstance(algorithm);
				mac.init(new SecretKeySpec(secret.getBytes(StandardCharsets.UTF_8), algorithm));
			} catch (NoSuchAlgorithmException | InvalidKeyException e) {
				throw new RuntimeException(e);
			}
        }

		@Override
		protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException {
			//exit early if the expected request header is missing
			if(StringUtils.isBlank(request.getHeader(xWebHookHeader))) {
				setUnauthorizedResponse(response);
				return;
			}
			//use Spring ContentCachingRequestWrapper to cache request,
			// so it can be first read to calculate hash
			// and re-read in controller using @RequestBody
			ContentCachingRequestWrapper cachedRequest = new ContentCachingRequestWrapper(request);
			String payload = IOUtils.toString(cachedRequest.getInputStream(), StandardCharsets.UTF_8);

			//validate hash and exit early if the request body is missing
			if(!verifyRequestPayload(payload, xWebHookHeader)) {
				setForbiddenResponse(response);
			}
			//handover processing to the next filter
			doFilter(cachedRequest, response, filterChain);
		}

		private void setUnauthorizedResponse(HttpServletResponse response) throws IOException {
			response.setStatus(HttpServletResponse.SC_UNAUTHORIZED);
			response.getWriter().write("Missing Security Header");
		}

		private void setForbiddenResponse(HttpServletResponse response) throws IOException {
			response.setStatus(HttpServletResponse.SC_FORBIDDEN);
			response.getWriter().write("Invalid Security Header");
		}

		private boolean verifyRequestPayload(String payload, String headerHmac) {
			byte[] hmacBytes = mac.doFinal(payload.getBytes(StandardCharsets.UTF_8));
			String calculatedHmac = Base64.getEncoder().encodeToString(hmacBytes).trim();
			return calculatedHmac.equals(headerHmac);
		}
	}

Registering the Filter

In order for the filter to be called when we hit the route, we have to register it.

@Configuration
static class WebHookFilterConfig {
		@Bean
		FilterRegistrationBean webHookFilter() {
			var registrationBean = new FilterRegistrationBean();
			registrationBean.setFilter(new WebHookFilter());
			registrationBean.addUrlPatterns("/webhook");
			return registrationBean;
		}
	}

Probably a good time to check that our app still works as expected. Well, looks like it does. We get the "Missing Security header" message and a 401 when we don't supply the header, this comes from our Filter. So all good so far. 👍🏽

Testing

We'll do the following:

  1. Prepare the JSON request body
  2. Generate a SHA-512 hash of the request body in step 1.
  3. Base64 encode the result of step 2
  4. Pass result of step 3, the base64 encoded string in the x-webhook-hmac header of the request.

The JSON body

{
    "customerName":"John Okafor",
    "customerId": "e1221b65-6027-446e-a4ef-cf8cda75b399",
    "customerPhone": "+233898232942",
    "amount": "50.50",
    "paidAt": "2023-11-20T09:23:23Z"
}

We make a POST request to our endpoint, passing the encoded hash in the header and get a 400, with a message "failed to read request".

Spring sees an empty Request body

We get a 400 because Spring sees an empty request body at the point when we get to the Controller.

@PostMapping
void payload(@RequestBody @Valid PaymentPayload payload) {
			logger.info("payload: {}", payload);
}

The documentation for ContentCachingRequestWrapper mention that cached content is to be read using getContentAsByteArray() after we have read the request body using .getInputStream() or .getReader(). What this means is that after we have read the request body in our filter, (which caches it), If we want to access the cached content, we have to use getContentAsByteArray().

Accessing cached Request body in Controller

We change our Controller code to the following in order to access the cached request body. I find this to be verbose. We also lose the ability to validate the request body automatically. 😕 I expected more from this implementation.

@PostMapping
void payload(HttpServletRequest request) {
			if(request instanceof ContentCachingRequestWrapper cachedRequest) {
				String payload =  new String(cachedRequest.getContentAsByteArray());
				logger.info("payload: {}", payload);
   }
}

Well, there's something we can do. We can actually write our own ContentCachingWrapper

CachedBodyHttpServletRequest

We write a Custom HttpServletRequestWrapper, which will cache the request body and whenever getInputStream() or getReader() is called, the cached request body will be returned.

static class CachedBodyHttpServletRequest extends HttpServletRequestWrapper {

		byte[] cachedBody;

		CachedBodyHttpServletRequest(HttpServletRequest request) throws IOException {
			super(request);
			this.cachedBody = StreamUtils.copyToByteArray(request.getInputStream());
		}

		@Override
		public ServletInputStream getInputStream() {
			return new CachedBodyServletInputStream(this.cachedBody);
		}

		private static class CachedBodyServletInputStream extends ServletInputStream {
			private final InputStream cachedBodyServletInputStream;

			public CachedBodyServletInputStream(byte[] cachedBody) {
				this.cachedBodyServletInputStream = new ByteArrayInputStream(cachedBody);
			}

			@Override
			public boolean isFinished() {
				try {
					return cachedBodyServletInputStream.available() == 0;
				} catch (IOException e) {
					throw new RuntimeException(e);
				}
			}

			@Override
			public boolean isReady() {
				return true;
			}

			@Override
			public void setReadListener(ReadListener listener) {
				throw new UnsupportedOperationException();
			}

			@Override
			public int read() throws IOException {
				return cachedBodyServletInputStream.read();
			}
		}
	}

Our filter's doInternalFilter method becomes

@Override
protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain filterChain) throws ServletException, IOException {
			//exit early if the expected request header is missing
			String header = request.getHeader(xWebHookHeader);
			if(StringUtils.isBlank(header)) {
				setUnauthorizedResponse(response);
				return;
			}
			//use custom CachedBodyHttpServletRequest to cache request,
			// so it can be first read to calculate hash
			// and re-read in controller using @RequestBody
			CachedBodyHttpServletRequest cachedRequest = new CachedBodyHttpServletRequest(request);
			String payload = IOUtils.toString(cachedRequest.getInputStream(), StandardCharsets.UTF_8);

			//validate hash and exit early if the request body is missing
			if(!verifyRequestPayload(payload, header)) {
				setForbiddenResponse(response);
				return;
			}
			//handover processing to the next filter
			doFilter(cachedRequest, response, filterChain);
		}

We can then access the request body in our Controller using @RequestBody

@PostMapping
void payload(@RequestBody @Valid PaymentPayload payload) {
			logger.info("payload: {}", payload);
}

You can access the full code here