1. OVERVIEW

Logging is an important part of application development.

It helps you to troubleshoot issues, to follow execution flows, not only inside the application, but also when spawning multiple requests across different services.

Logging also helps you to capture data and replicate production bugs in a development environment.

Often times, searching logs efficiently is a daunting task. That’s why there are plenty of Log Aggregators such as Splunk, ELK, Datadog, AWS CloudWatch, and many more, that help with capturing, standardizing, and consolidating logs to assist with log indexing, analysis, and searching.

Standard-formatted log messages like:

2023-08-01 12:43:44.421  INFO 73710 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''

are easy to read by Engineers but not so easy to parse by Log Aggregators, especially if the log format keeps changing.

This blog post helps you to configure Spring Boot applications to format log messages as JSON using Slf4j, Logback and Logstash, and having them ready to be fed to Log Aggregators.

JSON-formatted logs in Spring Boot applications using Slf4j, Logback, and Logstash

2. DEPENDENCIES

You won’t need to include spring-boot-starter-logging dependency. spring-boot-starter-web and other Spring Boot starters include it transitively.

spring-boot-starter-logging uses Logback as the default logging library through the Slf4j facade.

Let’s include the Maven dependencies you would need to format your Spring Boot applications logs as JSON.

pom.xml:

...
<properties>
  <logstash-logback-encoder.version>7.3</logstash-logback-encoder.version>
</properties>

<dependency>
  <groupId>net.logstash.logback</groupId>
  <artifactId>logstash-logback-encoder</artifactId>
  <version>${logstash-logback-encoder.version}</version>
</dependency>

3. LOGBACK CONFIGURATION

Next, let’s create src/main/resources/logback-spring.xml and configure Logback to display logs in JSON format.

logback-spring.xml:

<configuration>

<springProfile name="default">
  <include resource="org/springframework/boot/logging/logback/base.xml" />
</springProfile>

<springProfile name="json-logs">
  <appender name="jsonConsoleAppender" class="ch.qos.logback.core.ConsoleAppender">
    <encoder class="net.logstash.logback.encoder.LogstashEncoder" />
  </appender>

  <root level="INFO">
    <appender-ref ref="jsonConsoleAppender" />
  </root>
</springProfile>

</configuration>

You might not want JSON-formatted logs while you are using your IDE. They might be difficult to read.

{"@timestamp":"2023-08-01T12:46:11.512-04:00","@version":"1","message":"Starting ProtocolHandler [\"http-nio-8080\"]","logger_name":"org.apache.coyote.http11.Http11NioProtocol","thread_name":"main","level":"INFO","level_value":20000}
{"@timestamp":"2023-08-01T12:46:11.534-04:00","@version":"1","message":"Tomcat started on port(s): 8080 (http) with context path ''","logger_name":"org.springframework.boot.web.embedded.tomcat.TomcatWebServer","thread_name":"main","level":"INFO","level_value":20000}
{"@timestamp":"2023-08-01T12:46:11.55-04:00","@version":"1","message":"Started Application in 1.99 seconds (JVM running for 2.594)","logger_name":"com.asimiotech.demo.Application","thread_name":"main","level":"INFO","level_value":20000}
{"@timestamp":"2023-08-01T12:46:15.186-04:00","@version":"1","message":"Application shutdown requested.","logger_name":"org.springframework.boot.admin.SpringApplicationAdminMXBeanRegistrar$SpringApplicationAdmin","thread_name":"RMI TCP Connection(4)-127.0.0.1","level":"INFO","level_value":20000}
{"@timestamp":"2023-08-01T12:46:15.198-04:00","@version":"1","message":"Pausing ProtocolHandler [\"http-nio-8080\"]","logger_name":"org.apache.coyote.http11.Http11NioProtocol","thread_name":"RMI TCP Connection(4)-127.0.0.1","level":"INFO","level_value":20000}
{"@timestamp":"2023-08-01T12:46:15.199-04:00","@version":"1","message":"Stopping service [Tomcat]","logger_name":"org.apache.catalina.core.StandardService","thread_name":"RMI TCP Connection(4)-127.0.0.1","level":"INFO","level_value":20000}
{"@timestamp":"2023-08-01T12:46:15.207-04:00","@version":"1","message":"Stopping ProtocolHandler [\"http-nio-8080\"]","logger_name":"org.apache.coyote.http11.Http11NioProtocol","thread_name":"RMI TCP Connection(4)-127.0.0.1","level":"INFO","level_value":20000}

• default springProfile

That’s why I included the default springProfile configuration. It helps you to log messages in a traditional format in your IDE’s console; which might help you to better troubleshoot your application and increase your productivity.

It also helps you to read log messages in your CI/CD build output for those unit or integration tests that are not configured to run under any Spring profile.

If you run or debug your Spring Boot application inside your IDE specifying one or more Spring profiles, you would need to include them for the springProfile element that has the <include resource="org/springframework/boot/logging/logback/base.xml" /> configuration.

Here is how the logs look like using the above configuration when running your application in your IDE:

2023-08-01 12:43:44.374  INFO 73710 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 1 endpoint(s) beneath base path '/actuator'
2023-08-01 12:43:44.421  INFO 73710 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2023-08-01 12:43:44.436  INFO 73710 --- [           main] com.asimiotech.demo.Application          : Started Application in 1.949 seconds (JVM running for 2.657)
2023-08-01 12:43:49.225  INFO 73710 --- [on(1)-127.0.0.1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2023-08-01 12:43:49.226  INFO 73710 --- [on(1)-127.0.0.1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2023-08-01 12:43:49.227  INFO 73710 --- [on(1)-127.0.0.1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 1 ms

• json-logs springProfile

The second json-logs springProfile configuration means to apply this Logback configuration to the json-logs Spring profile.

It uses the LogstashEncoder class to format log messages as JSON.

If you start your Spring Boot application with -Dspring.profiles.active=json-logs, you should get JSON-formatted logs like:

{"@timestamp":"2023-08-01T12:46:11.512-04:00","@version":"1","message":"Starting ProtocolHandler [\"http-nio-8080\"]","logger_name":"org.apache.coyote.http11.Http11NioProtocol","thread_name":"main","level":"INFO","level_value":20000}
{"@timestamp":"2023-08-01T12:46:11.534-04:00","@version":"1","message":"Tomcat started on port(s): 8080 (http) with context path ''","logger_name":"org.springframework.boot.web.embedded.tomcat.TomcatWebServer","thread_name":"main","level":"INFO","level_value":20000}
{"@timestamp":"2023-08-01T12:46:11.55-04:00","@version":"1","message":"Started Application in 1.99 seconds (JVM running for 2.594)","logger_name":"com.asimiotech.demo.Application","thread_name":"main","level":"INFO","level_value":20000}
{"@timestamp":"2023-08-01T12:46:15.186-04:00","@version":"1","message":"Application shutdown requested.","logger_name":"org.springframework.boot.admin.SpringApplicationAdminMXBeanRegistrar$SpringApplicationAdmin","thread_name":"RMI TCP Connection(4)-127.0.0.1","level":"INFO","level_value":20000}
{"@timestamp":"2023-08-01T12:46:15.198-04:00","@version":"1","message":"Pausing ProtocolHandler [\"http-nio-8080\"]","logger_name":"org.apache.coyote.http11.Http11NioProtocol","thread_name":"RMI TCP Connection(4)-127.0.0.1","level":"INFO","level_value":20000}
{"@timestamp":"2023-08-01T12:46:15.199-04:00","@version":"1","message":"Stopping service [Tomcat]","logger_name":"org.apache.catalina.core.StandardService","thread_name":"RMI TCP Connection(4)-127.0.0.1","level":"INFO","level_value":20000}
{"@timestamp":"2023-08-01T12:46:15.207-04:00","@version":"1","message":"Stopping ProtocolHandler [\"http-nio-8080\"]","logger_name":"org.apache.coyote.http11.Http11NioProtocol","thread_name":"RMI TCP Connection(4)-127.0.0.1","level":"INFO","level_value":20000}

4. LOG LEVELS

You would now set the log levels the same way you have been doing:

application.yml:

logging:
  level:
    org.springframework: INFO
    org.springframework.transaction.interceptor: TRACE
    ROOT: INFO

5. LOGGING IN REST CONTROLLER

You could now define a LOGGER constant in your Java classes as follow:

private static final Logger LOGGER = LoggerFactory.getLogger(XyzController.class);

Or, if your Spring Boot application uses Lombok, you would add the @Slf4j annotation to a class:

// ...
@Slf4j
public class XyzController {
// ...
}

and use the LOGGER or log constant respectively to log messages, for instance:

@GetMapping(path = "/{id}")
public ResponseEntity<String> retrieveXyz(@PathVariable(name = "id") String id) {
  LOGGER.info("GET endpoint received path variable={}", id);
  return ResponseEntity.ok("Xyz resource corresponding to id: " + id);
}

6. LOGGING IN ACTION

If you start the application without any active Spring profile, and send a GET request to the previous endpoint, you will get a log in a traditional, human-readable format like:

2023-07-30 15:38:05.339  INFO 66532 --- [nio-8080-exec-2] com.asimiotech.demo.rest.XyzController   : GET endpoint received path variable=blah

If you start the application activating the json-logs Spring profile like: -Dspring.profiles.active=json-logs; sending a GET request to the previous endpoint will get you a JSON-formatted logs in like:

{"@timestamp":"2023-07-30T15:37:10.267-04:00","@version":"1","message":"GET endpoint received path variable=blah","logger_name":"com.asimiotech.demo.rest.XyzController","thread_name":"http-nio-8080-exec-2","level":"INFO","level_value":20000}

7. CONCLUSION

Logging is a vital part of application development.

JSON-formatted logs help Log Aggregators systems with capturing, standardizing, and consolidating logs; which in turn help Engineers with log analysis, and searching.

You can configure your Spring Boot applications to log messages in JSON format using Slf4j, Logback, and Logstash.

Stay tuned, I’ll cover how to feed JSON-formatted logs to Log Aggregators in a series of follow-up posts.

Thanks for reading and as always, feedback is very much appreciated. If you found this post helpful and would like to receive updates when content like this gets published, sign up to the newsletter.

8. SOURCE CODE

Accompanying source code for this blog post can be found at:

• https://bitbucket.org/asimio/springboot2-logback-json

9. REFERENCES

• https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.logging