Setting Up OpenTelemetry for Go Applications

Posted Dec 7, 2025

By R G

12 min read

Introduction to OpenTelemetry

OpenTelemetry is an open-source observability framework that provides a unified set of APIs, libraries, and instrumentation to capture distributed traces, metrics, and logs from your applications. As a Cloud Native Computing Foundation (CNCF) project, it has become the standard for application observability.

Why OpenTelemetry for Go?

Vendor-Neutral: Export telemetry to any backend (Jaeger, Zipkin, Prometheus, etc.)
Native Go Support: Well-designed Go APIs following Go idioms
Comprehensive: Traces, metrics, and logs in one framework
Zero Dependencies: Minimal impact on your application
Production Ready: Used by major companies in production

Prerequisites

Before we begin, ensure you have:

Go 1.19 or later installed
Basic understanding of Go programming
A text editor or IDE (VS Code with Go extension recommended)
Docker (optional, for running observability backends)

Setting Up OpenTelemetry in a Go Application

Step 1: Create a New Go Project

mkdir opentelemetry-go-demo
cd opentelemetry-go-demo
go mod init github.com/yourusername/opentelemetry-go-demo

Step 2: Install OpenTelemetry Dependencies

Install the core OpenTelemetry packages:

  
# Core OpenTelemetry API and SDK
go get go.opentelemetry.io/otel
go get go.opentelemetry.io/otel/sdk

# OTLP Exporter (for sending to collectors)
go get go.opentelemetry.io/otel/exporters/otlp/otlptrace
go get go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc

# HTTP instrumentation
go get go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp

# Stdout exporter (for debugging)
go get go.opentelemetry.io/otel/exporters/stdout/stdouttrace

# Trace and resource packages
go get go.opentelemetry.io/otel/trace
go get go.opentelemetry.io/otel/sdk/resource
go get go.opentelemetry.io/otel/semconv/v1.21.0

Step 3: Initialize OpenTelemetry

Create a file telemetry/tracer.go to set up OpenTelemetry:

  
package telemetry

import (
    "context"
    "fmt"
    "time"

    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/exporters/otlp/otlptrace"
    "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc"
    "go.opentelemetry.io/otel/propagation"
    "go.opentelemetry.io/otel/sdk/resource"
    sdktrace "go.opentelemetry.io/otel/sdk/trace"
    semconv "go.opentelemetry.io/otel/semconv/v1.21.0"
)

// InitTracer initializes the OpenTelemetry tracer provider
func InitTracer(serviceName, otlpEndpoint string) (func(context.Context) error, error) {
    ctx := context.Background()

    // Create resource with service information
    res, err := resource.New(ctx,
        resource.WithAttributes(
            semconv.ServiceName(serviceName),
            semconv.ServiceVersion("1.0.0"),
        ),
    )
    if err != nil {
        return nil, fmt.Errorf("failed to create resource: %w", err)
    }

    // Create OTLP trace exporter
    traceExporter, err := otlptrace.New(ctx,
        otlptracegrpc.NewClient(
            otlptracegrpc.WithEndpoint(otlpEndpoint),
            otlptracegrpc.WithInsecure(),
        ),
    )
    if err != nil {
        return nil, fmt.Errorf("failed to create trace exporter: %w", err)
    }

    // Create tracer provider
    tracerProvider := sdktrace.NewTracerProvider(
        sdktrace.WithBatcher(traceExporter),
        sdktrace.WithResource(res),
        sdktrace.WithSampler(sdktrace.AlwaysSample()),
    )

    // Set global tracer provider
    otel.SetTracerProvider(tracerProvider)

    // Set global propagator for context propagation
    otel.SetTextMapPropagator(propagation.NewCompositeTextMapPropagator(
        propagation.TraceContext{},
        propagation.Baggage{},
    ))

    // Return cleanup function
    return tracerProvider.Shutdown, nil
}

Step 4: Create a Simple HTTP Server

Create main.go with a basic HTTP server:

  
package main

import (
    "context"
    "fmt"
    "log"
    "math/rand"
    "net/http"
    "time"

    "github.com/yourusername/opentelemetry-go-demo/telemetry"
    "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/attribute"
    "go.opentelemetry.io/otel/codes"
)

const (
    serviceName  = "go-product-service"
    otlpEndpoint = "localhost:4317"
    serverPort   = ":8080"
)

func main() {
    // Initialize OpenTelemetry
    shutdown, err := telemetry.InitTracer(serviceName, otlpEndpoint)
    if err != nil {
        log.Fatalf("Failed to initialize tracer: %v", err)
    }
    defer func() {
        if err := shutdown(context.Background()); err != nil {
            log.Printf("Error shutting down tracer provider: %v", err)
        }
    }()

    // Create HTTP handlers with instrumentation
    mux := http.NewServeMux()
    
    // Wrap handlers with OpenTelemetry instrumentation
    mux.Handle("/products", otelhttp.NewHandler(
        http.HandlerFunc(handleProducts),
        "GetProducts",
    ))
    
    mux.Handle("/products/", otelhttp.NewHandler(
        http.HandlerFunc(handleProductByID),
        "GetProductByID",
    ))
    
    mux.Handle("/orders", otelhttp.NewHandler(
        http.HandlerFunc(handleOrders),
        "CreateOrder",
    ))

    mux.Handle("/health", http.HandlerFunc(handleHealth))

    // Start server
    log.Printf("Starting server on %s", serverPort)
    if err := http.ListenAndServe(serverPort, mux); err != nil {
        log.Fatalf("Server failed: %v", err)
    }
}

func handleProducts(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()
    tracer := otel.Tracer(serviceName)
    
    // Create a custom span
    ctx, span := tracer.Start(ctx, "fetchProducts")
    defer span.End()

    // Add attributes to span
    span.SetAttributes(
        attribute.String("http.method", r.Method),
        attribute.String("http.path", r.URL.Path),
    )

    // Simulate database query
    products, err := fetchProductsFromDB(ctx)
    if err != nil {
        span.RecordError(err)
        span.SetStatus(codes.Error, err.Error())
        http.Error(w, "Failed to fetch products", http.StatusInternalServerError)
        return
    }

    span.SetAttributes(attribute.Int("product.count", len(products)))
    span.SetStatus(codes.Ok, "Products fetched successfully")

    w.Header().Set("Content-Type", "application/json")
    fmt.Fprintf(w, `{"products": %v}`, products)
}

func handleProductByID(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()
    tracer := otel.Tracer(serviceName)
    
    ctx, span := tracer.Start(ctx, "getProductByID")
    defer span.End()

    // Extract product ID from URL
    productID := r.URL.Path[len("/products/"):]
    span.SetAttributes(attribute.String("product.id", productID))

    // Simulate database query
    product, err := fetchProductByID(ctx, productID)
    if err != nil {
        span.RecordError(err)
        span.SetStatus(codes.Error, err.Error())
        http.Error(w, "Product not found", http.StatusNotFound)
        return
    }

    span.SetStatus(codes.Ok, "Product found")
    w.Header().Set("Content-Type", "application/json")
    fmt.Fprintf(w, `{"product": %v}`, product)
}

func handleOrders(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()
    tracer := otel.Tracer(serviceName)
    
    ctx, span := tracer.Start(ctx, "createOrder")
    defer span.End()

    if r.Method != http.MethodPost {
        http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
        return
    }

    // Process order through multiple steps
    orderID, err := processOrder(ctx)
    if err != nil {
        span.RecordError(err)
        span.SetStatus(codes.Error, err.Error())
        http.Error(w, "Failed to process order", http.StatusInternalServerError)
        return
    }

    span.SetAttributes(attribute.String("order.id", orderID))
    span.SetStatus(codes.Ok, "Order created successfully")

    w.Header().Set("Content-Type", "application/json")
    fmt.Fprintf(w, `{"order_id": "%s", "status": "created"}`, orderID)
}

func handleHealth(w http.ResponseWriter, r *http.Request) {
    w.WriteHeader(http.StatusOK)
    fmt.Fprint(w, "OK")
}

func fetchProductsFromDB(ctx context.Context) ([]string, error) {
    tracer := otel.Tracer(serviceName)
    ctx, span := tracer.Start(ctx, "db.query.products")
    defer span.End()

    // Add database-related attributes
    span.SetAttributes(
        attribute.String("db.system", "postgresql"),
        attribute.String("db.operation", "SELECT"),
        attribute.String("db.statement", "SELECT * FROM products"),
    )

    // Simulate database latency
    time.Sleep(time.Duration(50+rand.Intn(50)) * time.Millisecond)

    products := []string{"Laptop", "Mouse", "Keyboard", "Monitor"}
    span.SetAttributes(attribute.Int("db.rows.returned", len(products)))
    
    return products, nil
}

func fetchProductByID(ctx context.Context, productID string) (string, error) {
    tracer := otel.Tracer(serviceName)
    ctx, span := tracer.Start(ctx, "db.query.productByID")
    defer span.End()

    span.SetAttributes(
        attribute.String("db.system", "postgresql"),
        attribute.String("db.operation", "SELECT"),
        attribute.String("product.id", productID),
    )

    // Simulate database latency
    time.Sleep(time.Duration(30+rand.Intn(30)) * time.Millisecond)

    return fmt.Sprintf("Product_%s", productID), nil
}

func processOrder(ctx context.Context) (string, error) {
    tracer := otel.Tracer(serviceName)
    ctx, span := tracer.Start(ctx, "processOrder")
    defer span.End()

    orderID := fmt.Sprintf("ORDER_%d", time.Now().Unix())
    span.SetAttributes(attribute.String("order.id", orderID))

    // Step 1: Validate order
    if err := validateOrder(ctx, orderID); err != nil {
        return "", err
    }

    // Step 2: Process payment
    if err := processPayment(ctx, orderID); err != nil {
        return "", err
    }

    // Step 3: Reserve inventory
    if err := reserveInventory(ctx, orderID); err != nil {
        return "", err
    }

    span.AddEvent("Order processed successfully")
    return orderID, nil
}

func validateOrder(ctx context.Context, orderID string) error {
    tracer := otel.Tracer(serviceName)
    _, span := tracer.Start(ctx, "validateOrder")
    defer span.End()

    span.SetAttributes(attribute.String("order.id", orderID))
    time.Sleep(40 * time.Millisecond)
    
    span.AddEvent("Order validated")
    return nil
}

func processPayment(ctx context.Context, orderID string) error {
    tracer := otel.Tracer(serviceName)
    _, span := tracer.Start(ctx, "processPayment")
    defer span.End()

    span.SetAttributes(
        attribute.String("order.id", orderID),
        attribute.String("payment.method", "credit_card"),
    )
    
    time.Sleep(100 * time.Millisecond)
    span.AddEvent("Payment processed")
    
    return nil
}

func reserveInventory(ctx context.Context, orderID string) error {
    tracer := otel.Tracer(serviceName)
    _, span := tracer.Start(ctx, "reserveInventory")
    defer span.End()

    span.SetAttributes(attribute.String("order.id", orderID))
    time.Sleep(60 * time.Millisecond)
    
    span.AddEvent("Inventory reserved")
    return nil
}

Step 5: Alternative - Using Stdout Exporter (for Development)

For local development without a collector, you can use the stdout exporter:

  
package telemetry

import (
    "context"
    "fmt"

    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/exporters/stdout/stdouttrace"
    "go.opentelemetry.io/otel/propagation"
    "go.opentelemetry.io/otel/sdk/resource"
    sdktrace "go.opentelemetry.io/otel/sdk/trace"
    semconv "go.opentelemetry.io/otel/semconv/v1.21.0"
)

func InitTracerStdout(serviceName string) (func(context.Context) error, error) {
    ctx := context.Background()

    // Create stdout exporter
    exporter, err := stdouttrace.New(
        stdouttrace.WithPrettyPrint(),
    )
    if err != nil {
        return nil, fmt.Errorf("failed to create stdout exporter: %w", err)
    }

    // Create resource
    res, err := resource.New(ctx,
        resource.WithAttributes(
            semconv.ServiceName(serviceName),
        ),
    )
    if err != nil {
        return nil, fmt.Errorf("failed to create resource: %w", err)
    }

    // Create tracer provider
    tracerProvider := sdktrace.NewTracerProvider(
        sdktrace.WithBatcher(exporter),
        sdktrace.WithResource(res),
    )

    otel.SetTracerProvider(tracerProvider)
    otel.SetTextMapPropagator(propagation.TraceContext{})

    return tracerProvider.Shutdown, nil
}

Instrumenting HTTP Clients

To trace outbound HTTP requests:

  
package main

import (
    "context"
    "net/http"
    
    "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)

func makeHTTPRequest(ctx context.Context, url string) error {
    // Create HTTP client with OpenTelemetry instrumentation
    client := &http.Client{
        Transport: otelhttp.NewTransport(http.DefaultTransport),
    }

    req, err := http.NewRequestWithContext(ctx, "GET", url, nil)
    if err != nil {
        return err
    }

    resp, err := client.Do(req)
    if err != nil {
        return err
    }
    defer resp.Body.Close()

    return nil
}

Instrumenting Database Operations

For database operations, you can use instrumentation libraries or create custom spans:

  
package database

import (
    "context"
    "database/sql"

    "go.opentelemetry.io/otel"
    "go.opentelemetry.io/otel/attribute"
    "go.opentelemetry.io/otel/codes"
)

type DB struct {
    *sql.DB
    serviceName string
}

func (db *DB) QueryWithTracing(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error) {
    tracer := otel.Tracer(db.serviceName)
    ctx, span := tracer.Start(ctx, "db.query")
    defer span.End()

    span.SetAttributes(
        attribute.String("db.system", "postgresql"),
        attribute.String("db.statement", query),
    )

    rows, err := db.DB.QueryContext(ctx, query, args...)
    if err != nil {
        span.RecordError(err)
        span.SetStatus(codes.Error, err.Error())
        return nil, err
    }

    span.SetStatus(codes.Ok, "Query executed successfully")
    return rows, nil
}

Setting Up Jaeger for Visualization

Using Docker

  
docker run -d --name jaeger \
  -e COLLECTOR_OTLP_ENABLED=true \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  jaegertracing/all-in-one:latest

Using Docker Compose

Create docker-compose.yml:

  
version: '3.8'

services:
  jaeger:
    image: jaegertracing/all-in-one:latest
    container_name: jaeger
    environment:
      - COLLECTOR_OTLP_ENABLED=true
    ports:
      - "16686:16686"  # Jaeger UI
      - "4317:4317"    # OTLP gRPC
      - "4318:4318"    # OTLP HTTP

  app:
    build: .
    container_name: go-app
    environment:
      - OTEL_SERVICE_NAME=go-product-service
      - OTEL_EXPORTER_OTLP_ENDPOINT=jaeger:4317
    ports:
      - "8080:8080"
    depends_on:
      - jaeger

Dockerfile:

  
FROM golang:1.21-alpine AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN go build -o main .

FROM alpine:latest
RUN apk --no-cache add ca-certificates
WORKDIR /root/
COPY --from=builder /app/main .
EXPOSE 8080
CMD ["./main"]

Running and Testing

Step 1: Start Jaeger

docker-compose up -d jaeger

Step 2: Run Your Application

go run main.go

Step 3: Make API Calls

  
# Get all products
curl http://localhost:8080/products

# Get specific product
curl http://localhost:8080/products/123

# Create an order
curl -X POST http://localhost:8080/orders

Step 4: View Traces

Open http://localhost:16686 and explore your traces!

Best Practices

1. Use Context Properly

Always pass context through your call chain:

  
func processRequest(ctx context.Context) error {
    ctx, span := tracer.Start(ctx, "processRequest")
    defer span.End()
    
    // Pass ctx to child functions
    return childFunction(ctx)
}

2. Add Meaningful Attributes

  
span.SetAttributes(
    attribute.String("user.id", userID),
    attribute.Int("order.items.count", itemCount),
    attribute.Float64("order.total", totalAmount),
    attribute.Bool("order.is_priority", isPriority),
)

3. Record Errors Properly

  
if err != nil {
    span.RecordError(err)
    span.SetStatus(codes.Error, err.Error())
    return err
}

4. Use Span Events for Important Milestones

  
span.AddEvent("Order validated")
span.AddEvent("Payment processed", trace.WithAttributes(
    attribute.String("payment.id", paymentID),
))

5. Configure Sampling for Production

  
tracerProvider := sdktrace.NewTracerProvider(
    sdktrace.WithSampler(sdktrace.TraceIDRatioBased(0.1)), // 10% sampling
    sdktrace.WithBatcher(traceExporter),
    sdktrace.WithResource(res),
)

Advanced: Distributed Tracing Across Services

When making HTTP requests to other services:

  
import "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"

client := &http.Client{
    Transport: otelhttp.NewTransport(http.DefaultTransport),
}

The context will automatically be propagated via HTTP headers!

Common Issues and Troubleshooting

Issue 1: Traces Not Appearing

Verify OTLP endpoint is correct
Check Jaeger is running: docker ps
Ensure shutdown is called on application exit
Check for error logs

Issue 2: Context Not Propagating

Ensure you’re passing context through function calls
Verify propagators are set correctly
Check HTTP client has otelhttp instrumentation

Issue 3: High Memory Usage

Configure batch span processor correctly
Implement proper sampling
Limit span attributes

Additional Resources

Conclusion

OpenTelemetry provides excellent support for Go applications with idiomatic APIs that feel natural to Go developers. By following this guide, you now have a solid foundation for instrumenting your Go applications with distributed tracing. Start with automatic instrumentation using the contrib libraries, and add custom spans where you need deeper visibility into your application’s behavior.

Happy tracing!

go golang opentelemetry observability monitoring distributed-tracing

This post is licensed under CC BY 4.0 by the author.