1. OVERVIEW

You are deploying, or planning to deploy your Spring Boot RESTful applications to AWS.

Some of these Spring Boot applications might use Amazon DynamoDB data stores instead of relational databases.

Along with writing unit tests for REST controllers, business logic, etc., you should consider writing integration tests to verify the interactions between different parts of the system work well, including retrieving data from, and storing data to a NoSQL database like DynamoDB.

This blog post shows you how to write a custom Spring’s TestExecutionListener to seed data to a DynamoDB table.
Each Spring Boot integration test will run starting from a known DynamoDB table state, so that you won’t need to force the tests to run in a specific order.

Seed data for integration tests in Spring Boot and Amazon DynamoDB applications

2. DYNAMODB AND DOCKER

Although the code discussed here should work with a DynamoDB database hosted on AWS, this blog post uses Amazon’s dynamodb-local Docker image to run a DynamoDB instance.

Let’s first add this specific AWS profile to your local AWS credentials file:

~/.aws/credentials:

[dynamodb-localdev]
aws_access_key_id = testAccessKey
aws_secret_access_key = testSecretKey
region = localhost

Let’s now run a DynamoDB Docker container:

docker run -d -p 8000:8000 --name dynamodb amazon/dynamodb-local:latest

3. MAVEN DEPENDENCIES

pom.xml:

...
<parent>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-parent</artifactId>
  <version>3.2.11</version>
  <relativePath />
</parent>

<properties>
  <java.version>21</java.version>
  <spring-cloud-aws.version>3.2.1</spring-cloud-aws.version>
</properties>

<dependencies>
  <dependency>
    <groupId>io.awspring.cloud</groupId>
    <artifactId>spring-cloud-aws-starter-dynamodb</artifactId>
  </dependency>
...
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
  </dependency>
...
<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>io.awspring.cloud</groupId>
      <artifactId>spring-cloud-aws-dependencies</artifactId>
      <version>${spring-cloud-aws.version}</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

• This Spring Boot application uses version 3.2.11.

• AWS BOM’s spring-cloud-aws-dependencies is set to 3.2.1.
  This is the most recent version you can use with Spring Boot 3.2.x according to the compatibility table below.

• spring-cloud-aws-starter-dynamodb is one of the libraries managed by spring-cloud-aws-dependencies.

Spring Boot, Spring Cloud AWS, and AWS Java SDK compatibility table

Souce: https://github.com/awspring/spring-cloud-aws

4. DATA MODEL

This tutorial uses the Customer entity only.

The Customer access patterns the Spring Boot application implements, and this tutorial helps to test/validate are:

• Addition of a new Customer entity with attributes: id, firstName, lastName, etc..
• Retrieval of an existing Customer using its unique identifier, aided by the customer table.
• Search for Customers by first name, last name or email address, aided by the customer table’s global secondary index (GSI) named AllCustomersFirstLastEmailIndex.

The images below show a single-table and GSI design that model the access patterns above.

DynamoDB customer table

A DynamoDB customer table’s Global Secondary Index

Based on this design, the integration tests’ supporting code needs to provision a DynamoDB table and GSI similar to the create-table AWS CLI command output below.

Create AWS DynamoDB table command output

But it needs to do so in an automated fashion, and seed the table with data before running each integration test.

5. DYNAMODB ENTITY/BEAN

Customer.java:

@DynamoDbBean
// ...
public class Customer {

  public static final String GSI_ALL_CUSTOMERS_FIRST_LAST_EMAIL_INDEX_NAME = "AllCustomersFirstLastEmailIndex";
  public static final String GSI_ALL_CUSTOMERS_FIRST_LAST_EMAIL_ID_PK = "all-customers-first-last-email";

  @Getter(onMethod = @__({@DynamoDbPartitionKey, @DynamoDbAttribute("id")}))
  private String id;

  @Getter(onMethod = @__({@DynamoDbAttribute("firstName")}))
  private String firstName;

  @Getter(onMethod = @__({@DynamoDbAttribute("lastName")}))
  private String lastName;

  @Getter(onMethod = @__({@DynamoDbAttribute("emailAddress")}))
  private String emailAddress;

  @Getter(onMethod = @__({@DynamoDbAttribute("phone")}))
  private Phone phone;

  @Getter(onMethod = @__({@DynamoDbAttribute("mailingAddress")}))
  private Address mailingAddress;

  @Getter(onMethod = @__({
    @DynamoDbSecondaryPartitionKey(indexNames = GSI_ALL_CUSTOMERS_FIRST_LAST_EMAIL_INDEX_NAME),
    @DynamoDbAttribute("gsiAllCustomersFirstLastEmail_PK")})
  )
  private String gsiAllCustomersFirstLastEmailId;
}

Customer’s attributes are annotated with @Getter(onMethod = @__({....})) because this Spring Boot application uses Lombok.

If you are not using Lombok, you would place the @DynamoDbAttribute, @DynamoDbPartitionKey, @DynamoDbSortKey, @DynamoDbSecondaryPartitionKey, etc. annotations on the getter methods.

6. TESTEXECUTIONLISTENER SUPPORTING CODE

This custom Spring’s TestExecutionListener implementation hooks into the integration test life cycle to provision DynamoDB tables, GSIs, and to seed DynamoDB data before running each test, and to delete it after each test completes.

DynamoDbDataTestExecutionListener.java:

public class DynamoDbDataTestExecutionListener extends AbstractTestExecutionListener {

  @Override
  public void beforeTestMethod(TestContext testContext) throws Exception {
    super.beforeTestMethod(testContext);
    this.executeDynamoDbOperation(testContext, DynamoDb.ExecutionPhase.BEFORE_TEST_METHOD);
  }

  @Override
  public void afterTestMethod(TestContext testContext) throws Exception {
    RuntimeException resultingException = null;
    try {
      this.executeDynamoDbOperation(testContext, DynamoDb.ExecutionPhase.AFTER_TEST_METHOD);
    } catch (RuntimeException ex) {
      resultingException = ex;
      log.warn("Swallowed exception to continue with the afterTestMethod() execution chain", ex);
    }
    super.afterTestMethod(testContext);
    if (resultingException != null) {
      throw resultingException;
    }
  }

  private void executeDynamoDbOperation(TestContext testContext, DynamoDb.ExecutionPhase executionPhase) {
    boolean classLevel = false;

    Set<DynamoDb> dynamoDbAnnotations = AnnotatedElementUtils.getMergedRepeatableAnnotations(
      testContext.getTestMethod(), DynamoDb.class, DynamoDbGroup.class
    );
    if (dynamoDbAnnotations.isEmpty()) {
      dynamoDbAnnotations = AnnotatedElementUtils.getMergedRepeatableAnnotations(
        testContext.getTestClass(), DynamoDb.class, DynamoDbGroup.class
      );
      if (!dynamoDbAnnotations.isEmpty()) {
        classLevel = true;
      }
    }

    this.executeDynamoDbOperation(dynamoDbAnnotations, testContext, executionPhase, classLevel);
  }

  private void executeDynamoDbOperation(Set<DynamoDb> dynamoDbAnnotations, TestContext testContext,
      DynamoDb.ExecutionPhase executionPhase, boolean classLevel) {

    dynamoDbAnnotations.forEach(dynamoDbAnnotation ->
      this.executeDynamoDbOperation(dynamoDbAnnotation, testContext, executionPhase, classLevel)
    );
  }

  private void executeDynamoDbOperation(DynamoDb dynamoDbAnnotation, TestContext testContext,
      DynamoDb.ExecutionPhase executionPhase, boolean classLevel) {

// ...

    // Entity class
    Class<?> entityClass= dynamoDbAnnotation.entityClass();

    // Table name.
    String tableName = dynamoDbAnnotation.tableName();

    // Data file
// ...

    // DynamoDB attributes definitions
    List<AttributeDefinition> attributeDefinitions = Arrays.stream(dynamoDbAnnotation.attributeDefinitions())
      .map(annotationAttrDef -> AttributeDefinition.builder()
        .attributeName(annotationAttrDef.name())
        .attributeType(annotationAttrDef.type())
        .build()
      )
      .collect(Collectors.toUnmodifiableList());

    // DynamoDB KeySchema
// ...

    // DynamoDB GSI(s)
// ...

    SeedData seedData = SeedData.builder()
      .entityClass(entityClass)
      .tableName(tableName)
      .tableProvisionedThroughput(provisionedThroughput)
      .data(dataResource)
      .attributeDefinitions(attributeDefinitions)
      .keySchema(keySchema)
      .globalSecondaryIndexes(globalSecondaryIndexes)
      .build();
// ...

    DatastorePopulator populator = new ResourceDynamoDbPopulator(seedData, dynamoDbTemplate, dynamoDbClient);
    if (dynamoDbAnnotation.executionPhase() == DynamoDb.ExecutionPhase.BEFORE_TEST_METHOD) {
      populator.populate();
    } else {
      populator.deleteAll();
    }
}

// ...
}

• beforeTestMethod() method first follows the TestExecutionListener chain by calling super.beforeTestMethod(…).
  It then executes a common executeDynamoDbOperation(…) passing the BEFORE_TEST_METHOD execution phase.

• Multiple executeDynamoDbOperation() overloaded methods:
  1. Retrieves and merges @DynamoDb configuration annotations.
     It first tries to use the @DynamoDbGroup configuration from the current test method (line 55), and if it’s not found, it then tries the one at class level (line 9).
  2. Loops through the merged @DynamoDb annotations and calls another overloaded method passing the test execution phase and each @DynamoDb annotation.
  3. Gets metadata from the @DynamoDb configuration annotation to instantiate a SeedData object and delegates provisioning and seeding a DynamoDB table, or removing data to ResourceDynamoDbPopulator.java, depending on the current test execution phase.
• afterTestMethod() is similar to beforeTestMethod().
  It first executes the same executeDynamoDbOperation(…) passing the AFTER_TEST_METHOD execution phase, but catching any exception so that it follows the super.afterTestMethod(…)’s TestExecutionListener chain to tear down or release other resources.

7. DYNAMODB DATA POPULATOR

DynamoDbDataTestExecutionListener delegates populating and deleting data to this class.

ResourceDynamoDbPopulator.java:

public class ResourceDynamoDbPopulator implements DatastorePopulator {

  private ObjectMapper objectMapper = new ObjectMapper();

  private final SeedData seedData;
  private final DynamoDbOperations dynamoDbTemplate;
  private final DynamoDbClient dynamoDbClient;
// ...

  @Override
  public void populate() {
    log.info("Creating table '{}' and seeding it with file: {}", this.seedData.getTableName(), ObjectUtils.nullSafeToString(this.seedData.getData()));

    CreateTableRequest request = CreateTableRequest.builder()
      .tableName(this.seedData.getTableName())
      .attributeDefinitions(this.seedData.getAttributeDefinitions())
      .keySchema(this.seedData.getKeySchema())
      .provisionedThroughput(this.seedData.getTableProvisionedThroughput())
      .globalSecondaryIndexes(this.seedData.getGlobalSecondaryIndexes())
      .build();
    this.dynamoDbClient.createTable(request);

    // Seed DynamoDB table
    JavaType type = this.objectMapper.getTypeFactory().constructCollectionType(List.class, this.seedData.getEntityClass());
    List<?> values = null;
    try {
      values = this.objectMapper.readValue(this.seedData.getData().getFile(), type);
    } catch (Exception ex) {
      throw new CannotReadDataException(this.seedData.getData(), ex);
    }
    values.forEach(value -> dynamoDbTemplate.save(value));
  }

  @Override
  public void deleteAll() {
    log.info("Deleting table '{}'", this.seedData.getTableName());

    DeleteTableRequest request = DeleteTableRequest.builder()
      .tableName(this.seedData.getTableName())
      .build();
    this.dynamoDbClient.deleteTable(request);
  }
}

• populate() method provisions a DynamoDB table with attributes, key schema, GSIs, etc.
  It then reads a resource data file, converts a JSON array to a POJO list, and saves each object in the DynamoDB table.

• deleteAll() method deletes the DynamoDB table.

8. TEST DATA FILES

This is a sample JSON document to seed data in a DynamoDB table.

Integration test classes reference this file in a custom @DynamoDb annotation.

src/test/resources/customers-1.json:

[
  {
    "id": "1c1ae96c-6a8d-4f37-bf1c-5a6677da8bd4",
    "firstName": "Blah_1",
    "lastName": "Meh_1",
    "emailAddress": "invalid.1@asimiotech.com",
    "phone": {
      "number": "123-456-7890",
      "type": "MOBILE"
    },
    "mailingAddress": {
      "street": "123 Main St",
      "city": "Orlando",
      "state": "FL",
      "zipcode": "32801"
    },
    "gsiAllCustomersFirstLastEmailId": "all-customers-first-last-email"
  },
  {
    "id": "4b073d5e-0616-444e-9e2b-0f5460e210d2",
    "firstName": "Blah_2",
    "lastName": "Meh_2",
    "emailAddress": "invalid.2@asimiotech.com",
    "phone": {
      "number": "234-567-8901",
      "type": "LANDLINE"
    },
    "mailingAddress": {
      "street": "234 Main Ave",
      "city": "Kissimmee",
      "state": "FL",
      "zipcode": "34741"
    },
    "gsiAllCustomersFirstLastEmailId": "all-customers-first-last-email"
  }
]

9. TEST CONFIGURATION

src/test/resources/application-integration-test.yml:

spring:
  cloud:
    aws:
      region:
        static: localhost
      credentials:
        accessKey: testAccessKey
        secretKey: testSecretKey
      dynamodb:
        endpoint: http://localhost:8000/

Note the aws.cloud.aws.* properties match those added in the AWS credentials dynamodb-localdev profile.

10. INTEGRATION TEST CLASS

Let’s write an integration test class that uses everything laid out in this post.

CustomerControllerIntegrationTest.java:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
@ExtendWith(SpringExtension.class)  // Requires a running DynamoDB instance.
@SpringBootTest(classes = Application.class, webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@ContextConfiguration(classes = { Application.class })
@ActiveProfiles("integration-test")
@TestExecutionListeners({
  DependencyInjectionTestExecutionListener.class,
  DynamoDbDataTestExecutionListener.class
})
@DynamoDbGroup({
  @DynamoDb(tableName = "customer", executionPhase = DynamoDb.ExecutionPhase.BEFORE_TEST_METHOD,
    entityClass = Customer.class, data = "classpath:customers-1.json",
    attributeDefinitions = {
      @AttributeDefinition(name = "id", type = ScalarAttributeType.S),
      @AttributeDefinition(name = "gsiAllCustomersFirstLastEmail_PK", type = ScalarAttributeType.S)
    },
    keySchema = @KeySchema(attributeName = "id", type = KeyType.HASH),
      globalSecondaryIndexes = {
        @GlobalSecondaryIndex(
          name = "AllCustomersFirstLastEmailIndex",
          keySchemas = {
            @KeySchema(attributeName = "gsiAllCustomersFirstLastEmail_PK", type = KeyType.HASH)
          },
          projectionType = ProjectionType.ALL
        )
      }
  ),
  @DynamoDb(tableName = "customer", executionPhase = DynamoDb.ExecutionPhase.AFTER_TEST_METHOD)
})
@Slf4j
public class CustomerControllerIntegrationTest {

  @LocalServerPort
  private int port;
// ...

  @Test
  public void shouldRetrieveCustomerByPrimaryKey() {
    JsonPath jsonPath = RestAssured
      .given()
        .accept(ContentType.JSON)
      .when()
        .get("/api/customers/4b073d5e-0616-444e-9e2b-0f5460e210d2")
      .then()
        .statusCode(HttpStatus.OK.value())
        .contentType(ContentType.JSON)
        .extract().jsonPath();

    Map<String, Object> actualCustomer = jsonPath.get("$");
    MatcherAssert.assertThat(actualCustomer.get("id"), Matchers.equalTo("4b073d5e-0616-444e-9e2b-0f5460e210d2"));
    MatcherAssert.assertThat(actualCustomer.get("firstName"), Matchers.equalTo("Blah_2"));
    MatcherAssert.assertThat(actualCustomer.get("lastName"), Matchers.equalTo("Meh_2"));
    // More assertions
  }

  @DynamoDbGroup({
    @DynamoDb(tableName = "customer", executionPhase = DynamoDb.ExecutionPhase.BEFORE_TEST_METHOD,
      entityClass = Customer.class, data = "classpath:customers-2.json",
      attributeDefinitions = {
        @AttributeDefinition(name = "id", type = ScalarAttributeType.S),
        @AttributeDefinition(name = "gsiAllCustomersFirstLastEmail_PK", type = ScalarAttributeType.S)
      },
      keySchema = @KeySchema(attributeName = "id", type = KeyType.HASH),
        globalSecondaryIndexes = {
          @GlobalSecondaryIndex(
            name = "AllCustomersFirstLastEmailIndex",
            keySchemas = {
              @KeySchema(attributeName = "gsiAllCustomersFirstLastEmail_PK", type = KeyType.HASH)
            },
            projectionType = ProjectionType.ALL
          )
        }
    ),
    @DynamoDb(tableName = "customer", executionPhase = DynamoDb.ExecutionPhase.AFTER_TEST_METHOD)
  })
  @Test
  public void shouldRetrieveTwoCustomersByEmailOrLastNameFilterParameters() {
    JsonPath jsonPath = RestAssured
      .given()
        .accept(ContentType.JSON)
      .when()
        .get("/api/customers?lastName=Meh_1&email=invalid.3@asimiotech.com")
      .then()
        .statusCode(HttpStatus.OK.value())
        .contentType(ContentType.JSON)
        .extract().jsonPath();

    List<Object> actualCustomers = jsonPath.get("$");
    MatcherAssert.assertThat(actualCustomers.size(), Matchers.equalTo(2));

    List<String> actualPrimaryKeys = actualCustomers.stream()
      .map(obj -> ((Map<String, String>) obj).get("id"))
      .collect(Collectors.toUnmodifiableList());
    MatcherAssert.assertThat(actualPrimaryKeys, Matchers.containsInAnyOrder("c195120e-98f2-4f35-a9ee-318e1eaf1c8e", "1c1ae96c-6a8d-4f37-bf1c-5a6677da8bd4"));
    // More assertions
  }

  @Test
  public void shouldCreateNewCustomer() throws IOException {
    JsonPath jsonPath = RestAssured
      .given()
        .accept(ContentType.JSON)
        .contentType(ContentType.JSON)
        .body(FileUtils.readFileToString(
            new File(CustomerControllerIntegrationTest.class.getClassLoader()
              .getResource("stubs/new-customer-request.json").getFile()
            ),
            "UTF-8")
        )
      .when()
        .post("/api/customers")
    .then()
      .statusCode(HttpStatus.OK.value())
      .contentType(ContentType.JSON)
      .extract().jsonPath();

    Map<String, Object> actualCustomer = jsonPath.get("$");
    MatcherAssert.assertThat(actualCustomer.get("firstName"), Matchers.equalTo("Blah_3"));
    MatcherAssert.assertThat(actualCustomer.get("lastName"), Matchers.equalTo("Meh_3"));
    // More assertions
  }
// ...
}

This integration test class uses JUnit 5 (brought in when including spring-boot-starter-test:3.2.11.

@ActiveProfiles annotation includes the integration-test profile for this integration test class to use application-integration-test.yml test properties file.

@TestExecutionListeners annotation includes:

• DependencyInjectionTestExecutionListener because it updates:

@LocalServerPort
private int port;

for RestAssured to send requests to.

• and the custom DynamoDbDataTestExecutionListener Spring’s TestExecutionListener.

DynamoDbDataTestExecutionListener is used in conjunction with @DynamoDbGroup and @DynamoDb annotations.

@DynamoDbGroup and @DynamoDb are custom configuration annotations. Their purpose is similar to Spring Test’s @SqlGroup and @Sql.
They include metadata to aid DynamoDbDataTestExecutionListener setting up tables, seeding data, and deleting DynamoDB tables before and after each integration test.
Also similar to the custom @CosmosGroup, @Cosmos annotations used with Azure Cosmos DB.

Notice the shouldRetrieveTwoCustomersByEmailOrLastNameFilterParameters() method includes @DynamoDbGroup and @DynamoDb configuration annotations.
The DynamoDbDataTestExecutionListener supports using the @DynamoDbGroup and @DynamoDb annotations at class or method level.

Seed data for integration tests in Spring Boot and DynamoDB applications

11. CONCLUSION

This blog post covered how to write integration tests for Spring Boot and DynamoDB applications the same way you would with Spring Boot and relational databases.

You can write a custom TestExecutionListener that hooks into the integration test life cycle to provision DynamoDB tables, seed them before running each test, and delete them after each test completes.
This approach ensures that each integration test runs from a known DynamoDB database state without imposing tests execution order.

You could provision a DynamoDB database to run integration tests the same way you would when using @SqlGroup, @Sql annotations to provision a relation database.

Thanks for reading and as always, feedback is very much appreciated. If you found this post helpful and would like to receive updates when content like this gets published, sign up to the newsletter.

12. SOURCE CODE

Your organization can now save time and costs by purchasing a working code base, clean implementation, with support for future revisions.