Search results
Writing integration tests with Testcontainers and Cosmos DB Docker emulator for Spring Boot applications
1. OVERVIEW
As part of your organization’s modernization effort, your team is writing Spring Boot applications that store data to, and retrieve data from Azure Cosmos DBs instead of relational databases.
You might have written dynamic Cosmos DB queries using Spring Data Cosmos and ReactiveCosmosTemplate.
Your integration tests might be connecting to a dedicated Cosmos DB hosted in Azure, which increases your organization’s subscription cost.
You could instead be using a dedicated Cosmos DB emulator Docker container you would need to make sure is running, and which ports it listens on, before you seed test data and run every test.
Or you might not have integration tests at all. But you also know integration testing is one of the keys to deploy often and with confidence.
This blog post shows you how to write integration tests with Testcontainers and the Cosmos DB Docker emulator for Spring Boot applications.
Listening on available ports instead of hard-coding them. And importing the Cosmos DB self-signed emulator certificate to a temporal Java truststore your tests could use.
All these automated.
Integration tests with Testcontainers and Azure Cosmos DB Docker emulator
1.1. About Testcontainers
Briefly, Testcontainers is an open-source framework with a commercial-friendly license (MIT) that provides throwaway instances of third-party systems like databases, message brokers, etc.; running in Docker containers you can use in your integration tests.
We’ll use Testcontainers for Java library; which supports Jupiter/JUnit 5
.
2. MAVEN DEPENDENCIES
This tutorial uses Spring Boot version 2.7.14
, but it might work with Spring Boot 2.2+
, which is the first version that started using JUnit 5
by default.
I would also expect it to work with Spring Boot 3.x
.
Let’s add the relevant Maven dependencies:
pom.xml
:
<properties>
<testcontainers.version>1.19.6</testcontainers.version>
</properties>
<dependencies>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>junit-jupiter</artifactId>
<scope>test</scope>
</dependency>
...
</dependencies>
<dependencyManagement>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>testcontainers-bom</artifactId>
<version>${testcontainers.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencyManagement>
...
You still need spring-boot-starter-test
, and rest-assured
dependencies, which are not covered in this blog post.
It’s also a follow up on Seeding Cosmos DB Data to run Spring Boot 2 Integration Tests blog post, so I’ll assume you are already using spring-cloud-azure-dependencies
, spring-cloud-azure-starter-data-cosmos
.
testcontainers-bom
manages a number of Testcontainers modules/libraries.
org.testcontainers:junit-jupiter
provides support to use Testcontainers libraries with Jupiter/JUnit 5
, which is the JUnit version brought in by Spring Boot 2.7.x
’s spring-boot-starter-test
dependency.
3. CONTAINER DOCUMENTS
User.java
:
@Container(containerName = "users")
public class User {
@Id
private String id;
private String firstName;
private String lastName;
private String address;
// ...
}
users
is the name of the container space in the Cosmos database. It’ll store JSON documents corresponding to User.java POJO instances.
If you take a look at the logs after running the integration test shouldRetrieveUserWithIdEqualsTo100() method, you’ll notice this warning:
WARN ... ReactiveCosmosTemplate : The partitionKey is not id!! Consider using findById(ID id, PartitionKey partitionKey) instead to avoid the need for using a cross partition query which results in higher latency and cost than necessary. See https://aka.ms/PointReadsInSpring for more info.
That’s fine for the purposes of this blog post.
But stay tuned and sign up to the newsletter. I’ll cover modeling CosmosDB data in another blog post.
4. TEST CONFIGURATION
src/test/resources/application-integration-test.yml
:
spring:
cloud:
azure:
cosmos:
endpoint: placeholder-will-be-set-at-runtime
key: C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==
database: sampleIntegrationTestsDB
As a precaution, make sure the database name is different than the one used for development.
The key will be the same for even for different Cosmos DB emulator Docker containers.
The interesting setting here is spring.cloud.azure.cosmos.endpoint
. It’s set to a random value because the real value will come at runtime after the Cosmos DB emulator starts.
This is useful in this case because the data storage system the tests depend on will listen on a random port. This is covered in UserControllerTestContainersIntegrationTest.java.
5. CosmosDBEmulatorContainer IMPLEMENTATION
If you weren’t using Testcontainers, how would you start a Cosmos DB emulator container for local development?
You would run a command similar to:
docker run -d \
--publish 8081:8081 \
--publish 10250-10255:10250-10255 \
--memory 3g --cpus=4.0 \
--name=cosmosdb-linux-emulator \
--env AZURE_COSMOS_EMULATOR_PARTITION_COUNT=10 \
--env AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true \
--env AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE=$ipaddr \
--interactive \
--tty \
mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator
That’s what our CosmosDBEmulatorContainer implementation does.
It takes care of starting a Cosmos DB emulator Docker container and waiting until the emulator endpoint is ready.
AsimioCosmosDBEmulatorContainer.java
:
public class AsimioCosmosDBEmulatorContainer extends GenericContainer<AsimioCosmosDBEmulatorContainer> {
private static final int EMULATOR_ENDPOINT_PORT = 8081;
public AsimioCosmosDBEmulatorContainer() {
this("mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator");
}
private AsimioCosmosDBEmulatorContainer(String dockerImageName) {
super(dockerImageName);
this.addFixedExposedPort(EMULATOR_ENDPOINT_PORT, EMULATOR_ENDPOINT_PORT;
this.addFixedExposedPort(10250, 10250);
this.addFixedExposedPort(10251, 10251);
this.addFixedExposedPort(10252, 10252);
this.addFixedExposedPort(10253, 10253);
this.addFixedExposedPort(10254, 10254);
this.addFixedExposedPort(10255, 10255);
try {
this.withEnv(Map.of(
"AZURE_COSMOS_EMULATOR_ARGS", "/Port=" + EMULATOR_ENDPOINT_PORT,
"AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE", InetAddress.getLocalHost().getHostAddress(),
"AZURE_COSMOS_EMULATOR_PARTITION_COUNT", "1",
"AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE", "false")
);
} catch (UnknownHostException ex) {
throw new RuntimeException(ex);
}
WaitAllStrategy waitStrategies = new WaitAllStrategy(WaitAllStrategy.Mode.WITH_INDIVIDUAL_TIMEOUTS_ONLY)
.withStrategy(Wait.forLogMessage("(?s).*Started\\r\\n$", 1))
.withStrategy(Wait.forHttps("/_explorer/index.html")
.forPort(EMULATOR_ENDPOINT_PORT)
.allowInsecure()
.withRateLimiter(RateLimiterBuilder.newBuilder()
.withRate(6, TimeUnit.MINUTES) // 6 per minute, one request every 10 seconds
.withConstantThroughput()
.build()
)
.withStartupTimeout(Duration.ofSeconds(120)) // 2 minutes
);
this.waitingFor(waitStrategies);
}
public String getEmulatorEndpoint() {
return String.format("https://%s:%s", this.getHost(), this.getMappedPort(EMULATOR_ENDPOINT_PORT));
}
public int getEmulatorEndpointPort() {
return EMULATOR_ENDPOINT_PORT;
}
}
We are implementing our custom container using GenericContainer as the base class.
mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator
is the Docker image.
The private constructor executes:
-
addFixedExposedPort() method binds a host port to a container port.
Similarly to--publish hostPort:containerPort
as included in the shell command. -
withEnv() method passes environment variables to the container.
Similarly to--env VAR_NAME=VAR_VALUE
from the shell command.
You won’t need to keep data between multiple runs. In fact, the container name will be different between runs. That’s whyAZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE
is set tofalse
.
If you use a limited data set to run your tests, you probably need only one partition.
What’s interesting in this case is that we are also passingAZURE_COSMOS_EMULATOR_ARGS='/Port=8081'
.EMULATOR_ENDPOINT_PORT
is hard-coded to8081
.
I did that on purpose because it’s easy to refactor to bind a container port to a random, and available host port (as included in the downloadable source code your organization can purchase.
You would just need to setEMULATOR_ENDPOINT_PORT
toSocketUtils.findAvailableTcpPort()
.
Not only does this prevent to hard-code ports that might be in use, but you could also run your integration tests concurrently, starting multiple Cosmos DB emulators, possibly decreasing the build time. -
The startup process combines two Testcontainers wait strategies before returning the control back to the integration test class.
- forLogMessage() method waits for the log message
Started
to show up in the standard output when running a Cosmos DB emulator container.
But this wait strategy is not enough. The emulator endpoint might still be unavailable. - forHttps().forPort().allowInsecure() configuration sends a dozen requests per minute for up to two minutes until
/_explorer/index.html
, the Cosmos DB emulator UI home page becomes available, after that it times out.
We need this page to be available so that we can get the unsigned cert and add it to the truststore, as done below in the integration test class.
- forLogMessage() method waits for the log message
docker logs f80
...
Starting
Started 1/11 partitions
Started 2/11 partitions
Started 3/11 partitions
Started 4/11 partitions
Started 5/11 partitions
Started 6/11 partitions
Started 7/11 partitions
Started 8/11 partitions
Started 9/11 partitions
Started 10/11 partitions
Started 11/11 partitions
Started
Cosmos DB Emulator UI Home
Reuse Testcontainers initialization and configuration code with JUnit 5 Extension Callbacks in your Spring Boot Integration Tests.
6. INTEGRATION TEST CLASS
Let’s now the discuss a Spring Boot-based integration test class that uses CosmosDBEmulatorContainer to run tests against.
UserControllerTestContainersIntegrationTest.java
:
@ExtendWith(SpringExtension.class)
@SpringBootTest(classes = Application.class, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ContextConfiguration(classes = { Application.class })
@ActiveProfiles("integration-test")
@Testcontainers
public class UserControllerTestContainersIntegrationTest {
@Container
public final static AsimioCosmosDBEmulatorContainer COSMOS_DB_EMULATOR = new AsimioCosmosDBEmulatorContainer();
@LocalServerPort
private int port;
@TempDir
private static File TEMP_FOLDER;
@BeforeAll
public static void setupTruststore() throws Exception {
String pemCertPath = TEMP_FOLDER.getPath() + File.separator + "cosmos-db-emulator.pem";
KeyStoreUtils.extractPublicCert(COSMOS_DB_EMULATOR.getHost(), COSMOS_DB_EMULATOR.getEmulatorEndpointPort(), pemCertPath);
log.info("Public cert saved to {}", pemCertPath);
String trustStorePath = TEMP_FOLDER.getPath() + File.separator + "integration-tests-cosmos-emulator.truststore";
KeyStoreUtils.createKeyStore(
Lists.newArrayList(pemCertPath),
trustStorePath,
"changeit"
);
log.info("Truststore set to {}", trustStorePath);
System.setProperty("javax.net.ssl.trustStore", trustStorePath);
System.setProperty("javax.net.ssl.trustStorePassword", "changeit");
System.setProperty("javax.net.ssl.trustStoreType", "PKCS12");
}
@DynamicPropertySource
public static void cosmosDbProperties(DynamicPropertyRegistry registry) throws Exception {
log.info("Setting spring.cloud.azure.cosmos.endpoint to {}", COSMOS_DB_EMULATOR.getEmulatorEndpoint());
registry.add("spring.cloud.azure.cosmos.endpoint", COSMOS_DB_EMULATOR::getEmulatorEndpoint);
}
@Test
public void shouldCreateNewUser() throws IOException {
JsonPath jsonPath = RestAssured
.given()
.accept(ContentType.JSON)
.contentType(ContentType.JSON)
.body(FileUtils.readFileToString(
new File(UserControllerIntegrationTest.class.getClassLoader().getResource("stubs/new-user-request.json").getFile()),
"UTF-8")
)
.when()
.post("/api/users")
.then()
.statusCode(HttpStatus.OK.value())
.contentType(ContentType.JSON)
.extract()
.jsonPath();
Map<String, Object> actualUser = jsonPath.get("$");
MatcherAssert.assertThat(actualUser.get("id"), Matchers.equalTo("200"));
MatcherAssert.assertThat(actualUser.get("firstName"), Matchers.equalTo("First-200"));
}
// ...
}
Remember this project uses Jupiter/JUnit 5
.
We need to turn on the integration-test
Spring profile so that this test class uses the application-integration-test.yml properties file.
@Testcontainers annotation is a Jupiter/JUnit 5
extension that helps to start up and stop the Docker containers used by the tests.
@Container is a marker annotation specifying the Cosmos DB emulator container lifecycle to be managed by the Testcontainers Jupiter/JUnit extension implementation.
setupTruststore() method extracts the public certificate using the emulator endpoint, creates a truststore in the TEMP_FOLDER
, and sets Java truststore system properties for the tests to use.
@DynamicPropertySource-annotated cosmosDbProperties() method updates application properties at runtime.
Instead of hard-coding the emulator endpoint to https://localhost:8081
, I ran these tests using random, available ports. Thus, I needed to update spring.cloud.azure.cosmos.endpoint
before the Spring context loads.
And the test methods you would run from the IDE, or through the build process.
Integration tests with Testcontainers and Cosmos DB Docker emulator for Spring Boot applications
7. CONCLUSION
This blog post covered how to write integration tests for Spring Boot applications using Testcontainers and the Azure Cosmos DB emulator Docker image.
Writing integration tests with Jupiter/JUnit 5
and Testcontainers helps you to build and deploy Cosmos DB applications often and with confidence.
You can write your own Cosmos DB emulator Testcontainer listening on available ports, which you can then use in your Spring Boot-based integration tests.
How to seed Cosmos DB documents to run Spring Boot 2 Integration Tests.
Reuse Testcontainers initialization and configuration code with JUnit 5 Extension Callbacks in your Spring Boot Integration Tests.
Thanks for reading and as always, feedback is very much appreciated. If you found this post helpful and would like to receive updates when content like this gets published, sign up to the newsletter.
8. SOURCE CODE
Your organization can now save time and costs by purchasing a working code base, clean implementation, with support for future revisions.