Microservices Sidecar pattern implementation using Postgres, Spring Cloud Netflix and Docker (you are here)
Implementing Circuit Breaker using Hystrix, Dashboard using Spring Cloud Turbine Server (work in progress)
1. OVERVIEW
What’s a Sidecar? A Sidecar is a companion application of the main service, typically non-JVM, either developed in-house or a 3rd party service (eg ElasticSearch, ApacheSolr, etc.) where it’s desirable for them to take advantage of other infrastructure services such as service registration and discovery, routing, dynamic configuration, monitoring, etc..
This post covers implementing a SidecarJava application attached to a Postgres database bundled in a Docker image and a Demo client application connecting to the database after retrieving the Postgres metadata (eg host, port) from a Eureka registry.
A Eureka server instance for the Spring CloudNetflixSidecar application to register the bundled Postgres server with.
Docker, local or remote host.
3. THE SIDECAR APPLICATION
can be created like any other SpringCloud app, from your preferred IDE, http://start.spring.io or from command line:
This command will create a Maven project in a folder named Sidecar with most of the dependencies used in the accompanying source code for this post.
The Sidecar relevant files are discussed next:
pom.xml:
spring-cloud-starter-eureka includes Eureka client support for applications to register and / or discovery services metadata with / from a remote Eureka server.
spring-cloud-netflix-sidecar provides beans Spring autoconfiguration for a companion application to take care of the service registration and / or discovery of 3rd party services (JVM or not).
SidecarApplication.java:
@EnableSidecar annotation might be all needed for this Java application to behave as a companion Sidecar to a 3rd party service (JVM or not) running in the same runtime unit (eg host, VM, Docker container) whose metadata will be registered with a Eureka server.
In the case of non-JVM in-house services, all is needed from them is to provide a SpringBoot health-like endpoint returning something like:
In the case of external services such as Postgres, Elastic Search, Kafka, etc., which likely won’t provide a health check as previously described, the Sideacar app itself could be used to implement such requirement:
SidecarHealthIndicator.java:
AppConfig.java:
Assumming a Sidecar instance is going to serve as a companion to only one service, it translates to a single SidecarHealthIndicator bean needed per Sidecar application, a PostgresHealthCheck instance in this demo.
Let’s backup a little bit, how can it be verified if a Postgres DB is acceping connections? It turns out pg_isready, a Postgres command accomplishes it:
The last piece of code needed in this case, where the service desired to register with a Eureka server is outside of our control, is to expose its Health information via an endpoint the Eureka server can send requests to.
LocalStatusDelegatorController.java:
The Sidecar application exposes the endpoint /delegating-status which uses a health check to run a specific OS command to verify if the 3rd party service is usable.
Let’s look at the Sidecar configuration files:
bootstrap.yml:
The spring.application.name property value is what’s going to be used for registration and discovery purposes.
application.yml:
Notice the sidecar.hostname property (used in Eureka client configuration) is hard-coded to localhost because the Sidecar companion application is supposed to run in the same host / VM / Docker container / … as the 3rd party service intended to be discovered.
Other relevant settings are the sidecar.port property set to the port the 3rd party service listens on. The sidecar.health-uri property pointing to the /delegating-status endpoint, used by the Eureka server to get information about the availability of the service. And sidecar.postgres.enabled, which causes the application to act as a Sidecar for Postgres.
This Dockerfile bundles a Postgres DB, the Sidecar application and a shell file to start the Sidecar app in the background when a Docker container is started.
sidecar.sh:
The image can be built running:
Note:
Running a Docker container using this image causes the container to start two processes which is not a suggested practice when using Docker.
An alternative would be to start a Docker container for the Sidecar application and another container for the main application, this setup requires the Sidecar to know the host / IP of the main application as well as some code changes for the Sidecar to report the correct hostname to the Eureka registry.
Still, I decided to run the two processes in the same container in this tutorial since this would be a very close approach to take when running them on bare metal or VMs.
These Demo was run using Docker where comunication between containers is needed, so I’ll first create a Docker network for them to run on:
Logs and a request to Eureka server confirm it started successfully and no application has been registered yet.
6. STARTING THE POSTGRES DVD RENTAL DATABASE AND REGISTERING IT WITH EUREKA VIA THE SIDECAR APP
Similarly to starting the Eureka server, a container including Postgres and Sidecar apps is started specifying the same network for containers to reach each other:
See how a Postgres DB is setup first, then the Sidecar companion app successfully started and ... PostgresHealthCheck : localhost:5432 - accepting connections log indicates the Sidecar is able to connect to the Postgres server.
Sending a request to the Eureka server now results in a POSTGRES-DB_DVDRENTAL service metadata stored in the registry, with [host=172.25.0.3, port=5432].
These are the main dependencies needed for the Demo Postgres client application to locate and connect to a Postgres DB.
SidecarPostgresDemoApplication.java:
@EnableEurekaClient annotation along with configuration properties allow this application to register and / or discover metadata from a Eureka server.
AppConfig.java:
A DataSource bean is instantiated instead of relying in DataSourceAutoConfiguration because the hostname and port the Postgres server listens on is unknown until Eureka responds back with metadata.
Notice the JDBC url includes placeholders to be replaced with values coming from Eureka.
I’ll skip runnning the integration tests for now because this demo application needs a Eureka and Postgres servers to connect to while loading the Spring context. In Integration Testing using Spring Boot, Postgres and Docker I covered how to start dependant services before running each test.
8. STARTING THE POSTGRES CLIENT DEMO
Running this demo in a Docker container requires the usage of the same network the other two containers use:
Now that it has started, lets send a couple of requests to an endpoint which should successfully execute a DB query:
A relevant note before finishing showing how to implement the Sidecar pattern using Postgres, SpringCloudNetflix and Docker.
Connection-related problems arose when starting the Demo API application but the Eureka server hasn’t started yet or when the Postgres host metadata is not still available in the Eureka server while instantiating the Datasource bean.
I believe it would be a good practice for microservices to recover themselves, to self heal from a situation like this, where services startup order is ideal but not required, where an application would keep trying to connect to dependant services for a given time and / or a number of attempts. Would this be a concern of the application or a concern of some kind of platform orchestrator? Stay tuned, I might follow-up with an implementation of this approach in another post.
Thanks for reading and as always, feedback is very much appreciated. If you found this post helpful and would like to receive updates when content like this gets published, sign up to the newsletter.
9. SOURCE CODE
Accompanying source code for this blog post can be found at:
Orlando L Otero is a Sr Software Engineer Consultant focusing on integration, modernization, cloud adoption and migration, microservices, API design and implementation, and agile delivery.