Structured Logging in Spring Boot

Yatin BatraFebruary 20th, 2025Last Updated: February 18th, 2025

0 841 5 minutes read

Logging is an essential part of application development as it helps in debugging and monitoring. Structured logging enhances traditional logging by providing logs in a structured format (such as JSON), making it easier to analyze using log aggregators like ELK (Elasticsearch, Logstash, Kibana) or Loki. Spring Boot provides a default logging mechanism using SLF4J and Logback, which can be enhanced to support structured logging. Let us delve into understanding Spring Boot structured logging and its benefits.

1. Why Structured Logging?

Traditional logging outputs plain text logs, making it difficult to extract insights efficiently. Structured logging provides:

Enhanced Readability: Logs are formatted in JSON, making them easier to parse.
Better Searchability: Modern logging tools can query specific fields.
Improved Monitoring: Easier integration with ELK Stack, Loki, or other monitoring solutions.
Efficient Troubleshooting: Log entries contain structured metadata, allowing faster debugging.

2. Maven Dependencies

To enable structured logging, you need to add the following dependencies in your pom.xml:

<dependency>
    <groupId>net.logstash.logback</groupId>
    <artifactId>logstash-logback-encoder</artifactId>
    <version>jar__latest__version</version>
</dependency>

3. Spring Boot Default Log

Spring Boot, by default, uses Logback for logging. The default configuration outputs logs in plain text format, as shown below:

2025-02-11 12:00:00.123  INFO 12345 --- [main] com.example.demo.Application : Application started successfully.

However, this log format is not structured, making it difficult to analyze using modern logging tools.

4. Configuring Structured Output

4.1 Creating logback-spring.xml

Create a logback-spring.xml file in src/main/resources with the following content:

<configuration>
    <appender name="JSON_CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder" />
    </appender>
 
    <root level="INFO">
        <appender-ref ref="JSON_CONSOLE" />
    </root>
</configuration>

4.2 Spring Boot Application Example

Here is a Spring Boot application with structured logging:

package com.example.demo;
 
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
 
import java.util.HashMap;
import java.util.Map;
 
@SpringBootApplication
public class DemoApplication {
    private static final Logger logger = LoggerFactory.getLogger(DemoApplication.class);
 
    public static void main(String[] args) {
        SpringApplication.run(DemoApplication.class, args);
        logger.info("Spring Boot Application Started");
    }
}
 
@RestController
class LoggingController {
    private static final Logger logger = LoggerFactory.getLogger(LoggingController.class);
 
    @GetMapping("/log")
    public Map logMessage(@RequestParam String message) {
        Map response = new HashMap();
        response.put("message", message);
 
        logger.info("Received request with message: {}", message);
        return response;
    }
}

4.3 JSON Log Output

After starting the application, the logs will be structured in JSON format:

{
    "@timestamp": "2025-02-11T12:00:00.123Z",
    "@version": "1",
    "message": "Spring Boot Application Started",
    "logger_name": "com.example.demo.DemoApplication",
    "thread_name": "main",
    "level": "INFO"
}

5. Advanced Configurations

5.1 Custom Log Fields

You can add custom fields to the JSON logs using MDC (Mapped Diagnostic Context):

import org.slf4j.MDC;
 
MDC.put("userId", "12345");
logger.info("User login successful");
MDC.clear();

5.2 Configuring Logstash Destination

To send logs to a Logstash instance, modify logback-spring.xml:

<appender name="LOGSTASH" class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <destination>localhost:5000</destination>
    <encoder class="net.logstash.logback.encoder.LogstashEncoder" />
</appender>
 
<root level="INFO">
    <appender-ref ref="LOGSTASH" />
</root>

5.3 Integration with Loki

To send logs to Loki using Promtail, configure Promtail to scrape logs from the application:

scrape_configs:
  - job_name: "spring-boot"
    static_configs:
      - targets: ["localhost"]
        labels:
          job: "spring-boot-logs"
          __path__: "/var/log/app.log"

6. Elastic Common Schema (ECS) and GELF Logging

In modern applications, structured logging is essential for easy parsing and efficient searching of logs. Both Elastic Common Schema (ECS) and Graylog Extended Log Format (GELF) are popular formats for structured logging. Below are examples of each format and how they help improve log management.

6.1 ECS Log Example

The Elastic Common Schema (ECS) is a standard for structuring logs, events, and metrics to make them consistent across your environment. It allows for a unified format that can be used across different systems and platforms.

{
    "@timestamp": "2025-02-11T12:00:00.123Z",
    "@version": "1",
    "host": {
        "name": "server01",
        "ip": "192.168.1.100"
    },
    "event": {
        "module": "spring-boot",
        "dataset": "log"
    },
    "message": "Spring Boot Application Started",
    "logger_name": "com.example.demo.DemoApplication",
    "thread_name": "main",
    "level": "INFO",
    "user": {
        "id": "12345",
        "name": "john.doe"
    },
    "service": {
        "name": "demo-service",
        "environment": "production"
    }
}

To produce ECS format logs in Spring Boot application, you can specify the Logback for ECS format in src/main/resources/logback-spring.xml file.

<configuration>
    <appender name="ECS_LOGGER" class="ch.qos.logback.core.ConsoleAppender">
        <encoder>
            <pattern>
                <![CDATA[
                {
                    "@timestamp": "%date{yyyy-MM-dd'T'HH:mm:ss.SSSZ}",
                    "@version": "1",
                    "host": {
                        "name": "${HOSTNAME:-localhost}",
                        "ip": "127.0.0.1"
                    },
                    "event": {
                        "module": "spring-boot",
                        "dataset": "log"
                    },
                    "message": "%message",
                    "logger_name": "%logger",
                    "thread_name": "%thread",
                    "level": "%level",
                    "user": {
                        "id": "12345",
                        "name": "john.doe"
                    },
                    "service": {
                        "name": "demo-service",
                        "environment": "production"
                    }
                }
                ]]>
            </pattern>
        </encoder>
    </appender>
 
    <root level="INFO">
        <appender-ref ref="ECS_LOGGER"/>
    </root>
</configuration>

In this configuration, logs are printed in ECS format to the IDE console. You can replace the ConsoleAppender with FileAppender or any other appender depending on where you want to store the logs. Once completed, start the application, and the logs will be generated in ECS format.

6.1.1 Key fields

@timestamp: The timestamp when the event occurred.
@version: The version of the log format.
host: Contains information about the host where the log was generated.
event: Information about the event, such as the module and dataset.
user: User-related information.
service: The name and environment of the service generating the log.

6.2 GELF Log Example

The Graylog Extended Log Format (GELF) is designed to send structured log messages to the Graylog server. It’s designed to optimize log entries for search, filtering, and analysis.

{
    "version": "1.1",
    "host": "server01",
    "short_message": "Spring Boot Application Started",
    "full_message": "Spring Boot Application started successfully at 2025-02-11T12:00:00.123Z",
    "timestamp": 1627392000,
    "level": 6,
    "facility": "spring-boot",
    "_user_id": "12345",
    "_user_name": "john.doe",
    "_service_name": "demo-service",
    "_environment": "production"
}

To produce GELF format logs in Spring Boot application, you can specify the Logback for ECS format in src/main/resources/logback-spring.xml file.

<configuration>
    <appender name="GELF" class="org.graylog2.gelf.logback.GelfAppender">
        <host>127.0.0.1</host> <!-- Graylog server IP or hostname -->
        <port>12201</port> <!-- Graylog GELF input port -->
        <includeCallerData>false</includeCallerData>
        <facility>spring-boot</facility>
        <additionalFields>
            <user_id>12345</user_id>
            <user_name>john.doe</user_name>
            <service_name>demo-service</service_name>
            <environment>production</environment>
        </additionalFields>
    </appender>
 
    <root level="INFO">
        <appender-ref ref="GELF"/>
    </root>
</configuration>

In this configuration, once the application is started the logs are sent to a Graylog server in GELF format. Replace the host and port with your actual Graylog server details. Note that the Graylog configuration has been skipped for brevity.

6.2.1 Key fields

version: The version of the GELF format.
host: The hostname or IP address of the server where the log originated.
short_message: A short, human-readable message summarizing the log.
full_message: A longer, more detailed message about the event.
timestamp: A Unix timestamp indicating when the event occurred.
level: The log level (e.g., 6 for INFO).
facility: The application or system generating the log (e.g., “spring-boot”).
_user_id, _user_name: Custom fields to store additional metadata about the user.
_service_name, _environment: Custom fields to store service-specific information like environment (e.g., “production”).

6.3 Comparison of ECS and GELF

Both ECS and GELF have their strengths and use cases. Here’s a quick comparison:

Feature	ECS	GELF
Format	JSON with predefined schema	Flexible JSON format with customizable fields
Use Case	Ideal for centralized logging with the Elastic Stack	Used with Graylog and can also be integrated with other systems
Standardization	Highly standardized	More flexible, can be adapted to various use cases
Ease of Integration	Best suited for Elastic Stack (Elasticsearch, Logstash, Kibana)	Optimized for Graylog, but can be used with other systems too

6.4 Benefits of ECS and GELF

Using ECS and GELF provides several advantages for logging:

Consistency: Both formats provide consistent structures for logging across different systems.
Searchability: Structured logs make it easier to search, filter, and analyze logs quickly using centralized log management tools.
Integration: Both formats integrate well with centralized logging systems like Elastic Stack (for ECS) and Graylog (for GELF).
Scalability: These formats scale well in cloud environments, where multiple services may be generating logs in different formats.

6. Conclusion

Structured logging in Spring Boot enhances log readability and integration with monitoring tools. By using Logstash Logback Encoder, logs can be formatted in JSON for easy analysis. This approach is particularly useful for applications running in cloud environments where centralized logging is a necessity.

For simple applications, console-based JSON logging is sufficient.
For production environments, integrating with Logstash or Loki ensures better observability.
Using MDC allows attaching metadata like user ID, request ID, etc., to logs.

By following these practices, you can ensure that your application’s logs are structured, searchable, and useful for debugging and monitoring.

Structured Logging in Spring Boot

1. Why Structured Logging?

2. Maven Dependencies

3. Spring Boot Default Log

4. Configuring Structured Output

4.1 Creating logback-spring.xml

4.2 Spring Boot Application Example

4.3 JSON Log Output

5. Advanced Configurations

5.1 Custom Log Fields

5.2 Configuring Logstash Destination

5.3 Integration with Loki

6. Elastic Common Schema (ECS) and GELF Logging

6.1 ECS Log Example

6.1.1 Key fields

6.2 GELF Log Example

6.2.1 Key fields

6.3 Comparison of ECS and GELF

6.4 Benefits of ECS and GELF

6. Conclusion

Thank you!

Yatin Batra

Thank you!

1. Why Structured Logging?

2. Maven Dependencies

3. Spring Boot Default Log

4. Configuring Structured Output

4.1 Creating logback-spring.xml

4.2 Spring Boot Application Example

4.3 JSON Log Output

5. Advanced Configurations

5.1 Custom Log Fields

5.2 Configuring Logstash Destination

5.3 Integration with Loki

6. Elastic Common Schema (ECS) and GELF Logging

6.1 ECS Log Example

6.1.1 Key fields

6.2 GELF Log Example

6.2.1 Key fields

6.3 Comparison of ECS and GELF

6.4 Benefits of ECS and GELF

6. Conclusion

Thank you!

Related Articles

Thank you!