Intermittent Traces with Micrometer, OpenTelemetry, and Spring Boot

Question

I'm setting up distributed tracing in my Spring Boot application using Micrometer Tracing and OpenTelemetry to send data to an OpenObserve instance, but I'm facing issues where traces are being recorded only intermittently.

Here's the setup: pom.xml Dependencies:

<dependency> 
    <groupId>io.micrometer</groupId> 
    <artifactId>micrometer-registry-otlp</artifactId> 
</dependency>
<dependency> 
    <groupId>io.opentelemetry</groupId> 
    <artifactId>opentelemetry-exporter-otlp</artifactId> 
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-tracing</artifactId>
</dependency>
<dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-tracing-bridge-otel</artifactId>
</dependency>
<dependency> 
    <groupId>com.squareup.okio</groupId> 
    <artifactId>okio</artifactId> 
    <version>3.9.1</version>
</dependency>       
<dependency>
    <groupId>io.opentelemetry.instrumentation</groupId>
    <artifactId>opentelemetry-logback-appender-1.0</artifactId>
    <version>2.11.0-alpha</version>
</dependency>
<dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-exporter-otlp-logs</artifactId>
    <version>1.26.0-alpha</version>
</dependency>
<dependency>
    <groupId>io.opentelemetry</groupId>
    <artifactId>opentelemetry-sdk-logs</artifactId>
</dependency>

application.yml Configuration:

management:
  otlp:
    metrics: 
      export: 
        url: ${OPEN_OBSERVE_URL}/api/${OPEN_OBSERVE_ORG_NAME}/v1/metrics
        enabled: true 
        step: 1m
        headers: 
          enabled: true 
          Authorization: Basic ${OPEN_OBSERVE_PASSWORD}
          organization: default
          service: 
            name: default
    tracing: 
      export:
        enabled: true
        sampling:
            probability: 1.0
      endpoint: ${OPEN_OBSERVE_URL}/api/${OPEN_OBSERVE_ORG_NAME}/v1/traces
      headers: 
        active : true
        enabled: true  
        Authorization: Basic ${TECHBD_OPEN_OBSERVE_PASSWORD}
        organization: default
        stream-name: ${OPEN_OBSERVE_STREAM_NAME}-traces
        service: 
          name: default

The OPEN_OBSERVE_* environment variables are configured properly (double checked it), and I can verify them resolving in the final configurations by loging their values with the app startup. I'm using the logback configuration logback-appender-1.0 appender to properly integrate traces and spans with my existing logs using this property on the logback configuration file pattern %-X{traceId} (verified as well).

Despite these configurations, traces show up intermittently in OpenObserve.

I configured Spring Boot with Micrometer and OpenTelemetry, expecting consistent traces in OpenObserve. Instead, traces appeared intermittently.

Sabith Citrus · Accepted Answer · 2025-01-16 11:21:21Z

0

The problem was with the configuration, setting it like this resolved the problem:

management:
  tracing:
    sampling:
      probability: 1.0

By default, Spring Boot samples only 10% of requests to prevent overwhelming the trace backend. This property switches it to 100% so that every request is sent to the trace backend.

answered Jan 16 at 11:21

Sabith Citrus

113 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

Intermittent Traces with Micrometer, OpenTelemetry, and Spring Boot

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related