1. OVERVIEW

Often times API endpoint implementations involve retrieving data from some sort of storage. Retrieving data, even when filtering based on a search criteria might result in hundreds, thousands or millions of records. Retrieving such amount of data could lead to performance issues, not meeting a contracted SLA, ultimately affecting the user experience.

One approach to overcome this problem is to implement pagination. You could retrieve a number of records from a data storage and add pagination links in the API response along with the page metadata back to the client application.

In a previous post, I showed readers how to include HAL hypermedia in Spring Boot RESTful applications using HATEOAS. Adding related links to REST responses help the client applications deciding what they might do next.
Some of the next actions a client application could help a customer do is to navigate through a list of resources. For instance to the first page of a result list.

This is a follow-up blog post to help you adding HAL (Hypertext Application Language) pagination hypermedia to your API responses using Spring Boot 2.1.x.RELEASE and Spring HATEOAS 0.25.x.RELEASE.

Spring Boot - HAL - HATEOAS - Pagination

2. REQUIREMENTS

  • Java 8+.
  • Maven 3.2+.
  • Familiarity with Spring Framework.

3. HATEOAS PAGINATION IMPLEMENTATION

Adding HAL hypermedia to API responses using Spring HATEOAS, paginated or not, require the spring-boot-starter-hateoas dependency.

pom.xml:

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-hateoas</artifactId>
</dependency>

The RESTful responses including HAL links will be sent in application/hal+json format by default. To override this behavior you can set the use-hal-as-default-json-media-type property to false.

application.yml:

spring:
  hateoas:
    use-hal-as-default-json-media-type: false

3.1. @EnableSpringDataWebSupport

Application.java:

@SpringBootApplication
@EnableSpringDataWebSupport
...
public class Application {
...
}

The @EnableSpringDataWebSupport annotation registers a number of beans Spring MVC controllers use. They support pagination, sorting, and resources assembler injection. The latter if Spring HATEOAS is found in the classpath.

3.2. REST CONTROLLER

Let’s analyze FilmController.retrieveFilms() endpoint implementation.

@RestController
@RequestMapping(value = "/api/films", produces = { MediaType.APPLICATION_JSON_VALUE, MediaTypes.HAL_JSON_VALUE })
...
public class FilmController {
...
  private final ResourceAssembler<Film, FilmResource> filmResourceAssembler;

  @GetMapping(path = "")
  public ResponseEntity<PagedResources<FilmResource>> retrieveFilms(
    @PageableDefault(page = DEFAULT_PAGE_NUMBER, size = DEFAULT_PAGE_SIZE) Pageable pageRequest,
    PagedResourcesAssembler<Film> pagedResourcesAssembler) {

    Page<Film> films = this.dvdRentalService.retrieveFilms(pageRequest);
    Link selfLink = new Link(ServletUriComponentsBuilder.fromCurrentRequest().build().toUriString());
    PagedResources<FilmResource> result = pagedResourcesAssembler.toResource(films, this.filmResourceAssembler, selfLink);
    return new ResponseEntity<>(result, HttpStatus.OK);
  }
...
}

Notice that Pageable pageRequest and PagedResourcesAssembler<Film> pagedResourcesAssembler arguments are injected into the method. This behavior is the result of adding @EnableSpringDataWebSupport to your application.

Method argument Description
pageRequest Allows to send requests to this endpoint as /api/films?page=0&size=30.
pagedResourcesAssembler Helps adding first, previous, self, next, last navigation links along with page metadata to the response payaload as included in the requests/responses examples section.


This endpoint implementation first retrieves Film JPA entities then assembles the PagedResources response. The most relevant statement related to pagination is:

PagedResources<FilmResource> result = pagedResourcesAssembler.toResource(films, this.filmResourceAssembler, selfLink);

How are the pagedResourcesAssembler.toResource() arguments used?

pagedResourcesAssembler.toResource() uses these arguments:

  • selfLink method argument: Used to add the page and size request parameters while keeping the rest of the query parameters.
    For instance, for an incoming request like /api/films?param1=value&param2=value2&page=3&size=30, the pagination previous link would look like /api/films?param1=value&param2=value2&page=3&size=30.
    If you don’t pass the selfLink, param1=value and param2=value2 request parameters won’t be included.

  • films method argument: A Page of the Film JPA entity to be mapped to a PagedResources of Film resources. I would suggest not to expose the internal domain directly as API responses. Any change to the internal model might break the contracted API response affecting your API consumers.

  • filmResourceAssembler method argument: an instance of ResourceAssembler.java. pagedResourcesAssembler.toResource() internally executes the assembler.toResource() method where assembler is filmResourceAssembler. The Film Resource Assembler.toResource() method implementation uses a MapStruct-based mapper to map the domain entity Film to the API resource FilmResource. pagedResourcesAssembler.toResource() method implementation also adds a PageResource with page metadata and the pagination links to the RESTful response.

3.3. RESOURCE ASSEMBLER

ResourceAssembler.java:

public class ResourceAssembler<E extends Identifiable<Integer>, R extends ResourceSupport, C>
  extends IdentifiableResourceAssemblerSupport<E, R> {

  private final ResourceMapper<E, R> resourceMapper;

  public ResourceAssembler(Class<C> controllerClass, Class<R> resourceClass, ResourceMapper<E, R> resourceMapper) {
    super(controllerClass, resourceClass);
    this.resourceMapper = resourceMapper;
  }

  @Override
  public R toResource(E entity) {
    return this.resourceMapper.map(entity);
  }
}

You need instances of this class because pagedResourcesAssembler.toResource(), found in the REST controller implementation, takes a ResourceAssembler method argument. The library internally executes the overriden method toResource().

Instances of this class gets injected a MapStruct-based mapper toResource() method uses to map from the internal domain model to the response model.

ResourceAssemblerConfig.java:

@Configuration
public class ResourceAssemblerConfig {

  @Bean
  public ResourceAssembler<Film, FilmResource, FilmController> filmResourceAssembler() {
    return new ResourceAssembler<Film, FilmResource, FilmController>(FilmController.class, FilmResource.class,
      FilmResourceMapper.INSTANCE);
  }
...
}

This configuration class instantiates the filmResourceAssembler bean. Notice how you could have used the generic Java class ResourceAssembler.java to instantiate other resource assemblers.

Maybe an actorResourceAssembler:

new ResourceAssembler<Actor, ActorResource, ActorController>(ActorController.class, ActorResource.class,
      ActorResourceMapper.INSTANCE);

3.4. JPA AND REST RESOURCE MODELS

Film.java:

public class Film implements Identifiable<Integer>, Serializable {
...
  @Override
  public Integer getId() {
    return this.filmId;
  }
...
}

FilmResource.java:

@Relation(value = "film", collectionRelation = "films")
public class FilmResource extends ResourceSupport {

  @JsonProperty("id")
  private int filmId;
  private String title;
  private String description;
...
}

The domain and response models along with the explanation for Identifiable interface and ResourceSupport base class were covered while discussing Adding HAL Links to Spring Boot applications using Spring HATEOAS.

3.5. RESOURCE MAPPER

FilmResourceMapper.java:

@Mapper
public interface FilmResourceMapper extends ResourceMapper<Film, FilmResource> {

  FilmResourceMapper INSTANCE = Mappers.getMapper(FilmResourceMapper.class);

  @AfterMapping
  default void addLinks(@MappingTarget FilmResource resource, Film entity) {
    // self link
    Link selfLink = ControllerLinkBuilder.linkTo(FilmController.class)
      .slash(entity)
      .withSelfRel();
    resource.add(selfLink);

    // actors links
    entity.getFilmActors().stream()
      .forEach(filmActor -> resource.add(
        ControllerLinkBuilder.linkTo(
          ControllerLinkBuilder.methodOn(ActorController.class)
            .retrieveActor(filmActor.getActor().getId()))
            .withRel("actors")
        ));
  }
...
}

This is a MapStruct-based mapper that maps the internal domain model Film to the API response FilmResource.

It adds a self links to the film resource and also adds a link to each actor resource that performed in the movie.

4. SAMPLE RESPONSE PAYLOADS

  • An application/json example:
curl "http://localhost:8080/api/films?page=2&size=5" | json_pp
{
  "page" : {
    "number" : 2,
    "size" : 5,
    "totalPages" : 200,
    "totalElements" : 1000
  },
  "links" : [
    {
      "rel" : "first",
      "href" : "http://localhost:8080/api/films?page=0&size=5"
    },
    {
      "href" : "http://localhost:8080/api/films?page=1&size=5",
      "rel" : "prev"
    },
    {
      "href" : "http://localhost:8080/api/films?page=2&size=5",
      "rel" : "self"
    },
    {
      "rel" : "next",
      "href" : "http://localhost:8080/api/films?page=3&size=5"
    },
    {
      "href" : "http://localhost:8080/api/films?page=199&size=5",
      "rel" : "last"
    }
  ],
  "content" : [
    {
      "id" : 11,
      "rentalRate" : 0.99,
      "length" : 126,
      "lang" : "English",
      "description" : "A Boring Epistle of a Butler And a Cat who must Fight a Pastry Chef in A MySQL Convention",
      "links" : [
        {
          "rel" : "self",
          "href" : "http://localhost:8080/api/films/11"
        },
        {
          "rel" : "actors",
          "href" : "http://localhost:8080/api/actors/40"
        },
        {
          "rel" : "actors",
          "href" : "http://localhost:8080/api/actors/174"
        },
...
}

The outer links array in the response payload includes references to the first, previous, current (via "rel" : "self"), next, and last resources pages.

The content array includes all Film resources for this page. Each resource includes a nested links array with a reference to itself and an array of references to the film actors via the "rel" : "actors" objects.

The page object provides metadata about the number of pages, current page, data set size, etc.

  • The same request but accepting application/hal+json:
curl -H "Accept:application/hal+json" "http://localhost:8080/api/films?page=2&size=5" | json_pp
{
  "page" : {
    "number" : 2,
    "totalElements" : 1000,
    "size" : 5,
    "totalPages" : 200
  },
  "_links" : {
    "first" : {
      "href" : "http://localhost:8080/api/films?page=0&size=5"
    },
    "next" : {
      "href" : "http://localhost:8080/api/films?page=3&size=5"
    },
    "self" : {
      "href" : "http://localhost:8080/api/films?page=2&size=5"
    },
    "prev" : {
      "href" : "http://localhost:8080/api/films?page=1&size=5"
    },
    "last" : {
      "href" : "http://localhost:8080/api/films?page=199&size=5"
    }
  },
  "_embedded" : {
    "films" : [
      {
        "releaseYear" : "2006",
        "id" : 11,
        "lang" : "English",
        "length" : 126,
        "rentalDuration" : 6,
        "title" : "ALAMO VIDEOTAPE",
        "_links" : {
          "self" : {
            "href" : "http://localhost:8080/api/films/11"
          },
          "actors" : [
            {
              "href" : "http://localhost:8080/api/actors/40"
            },
            {
              "href" : "http://localhost:8080/api/actors/90"
            },
...

This is a HAL-formatted response payload example.

The page object is like the one included in the previous example.

The outer _links object includes the navigational references to other pages.

And the _embedded object adds the films. Each resource includes a nested _links with references related to its owning entity.

You could replace the links’ hostname and port with those of an Edge server or load balancer or Gateway via the X-FORWARDED-*` headers.

5. CONCLUSION

One of the approaches to navigate through a data set is through pagination. Including HAL links in your REST API responses hints client applications what they could do next. Some of the next action a user might want to do is to navigate to the next page to book a room in a different hotel than those found in the first page.

Spring HATEOAS helps adding navigation links and page metadata. Such links and metadata in your API responses allow client applications to improve the user experience.

This tutorial helps you getting started with spring-boot-starter-hateoas Spring Boot starter to include pagination hypermedia in your RESTful API responses.

6. UPCOMING

Lets take a quick look at the Hibernate logs these sample requests generate:

Hibernate: 
  select
    film0_.film_id as film_id1_8_,
    film0_.description as descript2_8_,
    film0_.fulltext as fulltext3_8_,
    film0_.language_id as languag13_8_,
    film0_.last_update as last_upd4_8_,
    film0_.length as length5_8_,
    film0_.rating as rating6_8_,
    film0_.release_year as release_7_8_,
    film0_.rental_duration as rental_d8_8_,
    film0_.rental_rate as rental_r9_8_,
    film0_.replacement_cost as replace10_8_,
    film0_.special_features as special11_8_,
    film0_.title as title12_8_ 
  from
    public.film film0_ limit ? offset ?
Hibernate: 
  select
    count(film0_.film_id) as col_0_0_ 
  from
    public.film film0_

Aparently there is nothing wrong here, a SQL query to retrieve film records. Notice the limit and offset Hibernate generates for the H2 engine. It then executes a second query, a count SQL statement used to provide page metadata.

But Hibernate also generates these queries:

Hibernate: 
  select
    language0_.language_id as language1_13_0_,
    language0_.last_update as last_upd2_13_0_,
    language0_.name as name3_13_0_ 
  from
    public.language language0_ 
  where
    language0_.language_id=?
Hibernate: 
  select
    filmactors0_.film_id as film_id2_9_0_,
    filmactors0_.actor_id as actor_id1_9_0_,
    filmactors0_.actor_id as actor_id1_9_1_,
    filmactors0_.film_id as film_id2_9_1_,
    filmactors0_.last_update as last_upd3_9_1_ 
  from
    public.film_actor filmactors0_ 
  where
    filmactors0_.film_id=?
Hibernate: 
  select
    filmactors0_.film_id as film_id2_9_0_,
    filmactors0_.actor_id as actor_id1_9_0_,
    filmactors0_.actor_id as actor_id1_9_1_,
    filmactors0_.film_id as film_id2_9_1_,
    filmactors0_.last_update as last_upd3_9_1_ 
  from
    public.film_actor filmactors0_ 
  where
    filmactors0_.film_id=?
...

  • One SQL query to retrieve a language.
    Hibernate executed only one SQL statement because the five films retrieved are in English, otherwise it would have generated one SQL query for each different language.

  • Many SQL queries to retrieve the actors for each film.

This is known as the N+1 SELECT problem. Stay tuned, I’ll cover how to prevent the N+1 SELECT problem using Spring Data JPA in a follow-up post.

Thanks for reading and as always, feedback is very much appreciated. If you found this post helpful and would like to receive updates when content like this gets published, sign up to the newsletter.

7. SOURCE CODE

Accompanying source code for this blog post can be found at:

8. REFERENCES