Spring’s RestTemplate is one of the options to make client HTTP requests to endpoints, it facilitates communication with the HTTP servers, handles the connections and transforms the XML, JSON, … request / response payloads to / from POJOs via HttpMessageConverter.
By default RestTemplate doesn’t use a connection pool to send requests to a server, it uses a SimpleClientHttpRequestFactory that wraps a standard JDK’s HttpURLConnection taking care of opening and closing the connection.
But what if an application needs to send a large number of requests to a server? Wouldn’t it be a waste of efforts to open and close a connection for each request sent to the same host? Wouldn’t it make sense to use a connection pool the same way a JDBC connection pool is used to interact with a DB or a thread pool is used to execute tasks concurrently?
In this post I’ll cover configuring RestTemplate to use a connection pool using a pooled-implementation of the ClientHttpRequestFactory interface, run a load test using JMeter, troubleshoot requests timeout and reconfigure the connection pool.
This command will create a Maven project in a folder named resttemplate-troubleshooting-svc-1 with most of the dependencies used in the accompanying source code.
Alternatively, it can also be generated using Spring Initializr tool then selecting Actuator and Web dependencies as shown below:
Spring Initializr - Generate Spring Boot App - Actuator, Web
ResttemplateTroubleshootingSvc1Application.java, the entry point to the application looks like:
It also scans the com.asimio.api.demo.rest package which includes the DemoResource class:
which exposes a single endpoint /demo whose implementation delays the response for 150ms in an attempt to simulate some computation.
4. CREATE THE DEMO SERVICE 2
Similarly to Create the Demo Service 1, this command will create a Maven project in a folder named resttemplate-troubleshooting-svc-2 and alternatively, it can also be generated using Spring Initializr tool then selecting Actuator and Web dependencies.
ResttemplateTroubleshootingSvc2Application.java, the entry point to this service looks like:
It configures a restTemplate bean with its ClientHttpRequestFactory set to HttpComponentsClientHttpRequestFactory, an implementation based on Apache HttpComponents HttpClient to replace the default implementation based on the JDK. This request factory is configured with an HttpClient with timeout-related properties (a better practice would have been to set those via a config file such as application.yml) and a pooling connection manager with a maximum number of connections set to 20.
Note:socketTimeout() (or SO_TIMEOUT) refers to the timeout for waiting for data, connectTimeout() refers to the timeout until a connection is established and connectionRequestTimeout() refers to the timeout when requesting a connection from the connection manager.
It also scans the com.asimio.api.demo.rest package which includes another DemoResource class:
exposing a single endpoint /delegate/demo whose implementation delegates the requests to Demo Service 1’s /demo using the injected RestTemplatediscussed earlier.
Tip:locahost:8800 was hard-coded to simplify this tutorial but a better approach is to either set the location of Demo Service 1 using a configuration property in application.yml or through Microservices Registration and Discovery using Spring Cloud.
5. RUNNING BOTH SERVICES
As said earlier, to keep this tutorial simple, Demo Service 2 delegates requests to Demo Service 1 via locahost:8800 so lets start Demo Service 1 on 8800:
and Demo Service 2 on 8900:
Verifying both services work as expected:
and
6. LOAD-TESTING USING JMETER
I have included a JMeter script to run a load test againt Demo Service 2, it could be found in resttemplate-troubleshooting-svc-2/src/test/resources/jmeter/loadTest.jmx of the accompanying source code section of this blog post.
Basically it will load-test 60 simultaneous thread / users with a ramp-up period of 10 minutes and the same thread will sleep for 1 second before being re-used for another request. This will change on a case by case basis but these are the settings I chose for this tutorial.
Just 11 minutes into the load issues started to be noticed, Demo Service 2 is failing to send requests to Demo Service 1 because it cannot get a connection from the pool.
JMeter shows HTTP status 500 for such requests:
JMeter Load Test RestTemplate Connection Pool - Error
How is this happening? Moving from an opening / closing connection approach to re-use connections to save time while instantiating the sockets now results in requests timing out, isn’t a pool of 20 connections enough? Attempts to increase maxTotal from 20 to a higher value won’t help, lets take a look at this command output:
Seems that even though the pool was set to a a maximum of 20 connections it’s only using 4 for such a load.
This is an real life issue we were facing when running a load test for one of the services during my time at Disney that my buddy Tim Brizard (@brizardofoz) fixed.
When defining poolingHttpClientConnectionManager bean in Create Demo Service 2’s entry point, neither defaultMaxPerRoute nor maxPerRoute were set and the implementation of PoolingHttpClientConnectionManager uses a maximum of 4 connections per host route if not specified. Lets increase that number by first editing Demo Service 2’s application.yml:
and set it during poolingHttpClientConnectionManager bean creation in ResttemplateTroubleshootingSvc2Application.java:
HttpHostsConfiguration.java
This is overriding the defaultMaxPerRoute value used by PoolingHttpClientConnectionManager to 20 (as configured in httpConnPool.defaultMaxPerRoute property) and setting the maximum connection per route for the route constructed from [scheme=http, host=localhost, port=8800] to 20.
Warning:
A word of caution, setting the defaultMaxPerRoute to a value close to maxTotal might mean a single host route might hijack the connection pool. It would be advisable to individually adjust maxPerRoute to a value based on observation from running load tests with the expected traffic or maybe to use a different RestTemplate instance.
Lets repeat the same load test:
JMeter Load Test RestTemplate Connection Pool - Success
After 30 minutes into the load, no error has been logged in the standard error or reported in JMeter.
Let’s now run the netstat command:
It can be seen more than 4 connections are now being used and the requests timeout problem has been fixed. Happy troubleshooting.
Thanks for reading and as always, feedback is very much appreciated. If you found this post helpful and would like to receive updates when content like this gets published, sign up to the newsletter.
8. SOURCE CODE
Accompanying source code for this blog post can be found at:
Orlando L Otero is a Sr Software Engineer Consultant focusing on integration, modernization, cloud adoption and migration, microservices, API design and implementation, and agile delivery.