Setting Up OpenTelemetry for .NET Applications

Introduction to OpenTelemetry

OpenTelemetry is an open-source observability framework that provides a single set of APIs, libraries, agents, and instrumentation to capture distributed traces and metrics from your applications. It’s a CNCF (Cloud Native Computing Foundation) project that has become the industry standard for observability.

Why OpenTelemetry?

• Vendor-Neutral: Works with multiple observability backends (Jaeger, Zipkin, Prometheus, etc.)
• Unified Standard: Single API for traces, metrics, and logs
• Auto-Instrumentation: Automatic instrumentation for popular libraries and frameworks
• Extensible: Easy to customize and extend
• Community-Driven: Strong community support and regular updates

Prerequisites

Before we begin, make sure you have:

• .NET 6.0 or later installed
• Visual Studio 2022 or Visual Studio Code
• Basic understanding of ASP.NET Core

Setting Up OpenTelemetry in a .NET Application

Step 1: Create a New .NET Web API Project

dotnet new webapi -n OpenTelemetryDemo
cd OpenTelemetryDemo

Step 2: Install Required NuGet Packages

Install the core OpenTelemetry packages:

dotnet add package OpenTelemetry
dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Instrumentation.Http
dotnet add package OpenTelemetry.Exporter.Console
dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol

For additional instrumentation (optional):

# For SQL Server instrumentation
dotnet add package OpenTelemetry.Instrumentation.SqlClient

# For Entity Framework Core instrumentation
dotnet add package OpenTelemetry.Instrumentation.EntityFrameworkCore

# For Redis instrumentation
dotnet add package OpenTelemetry.Instrumentation.StackExchangeRedis

Step 3: Configure OpenTelemetry in Program.cs

Update your Program.cs file to configure OpenTelemetry:

using OpenTelemetry.Resources;
using OpenTelemetry.Trace;
using OpenTelemetry.Metrics;

var builder = WebApplication.CreateBuilder(args);

// Add services to the container
builder.Services.AddControllers();
builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

// Configure OpenTelemetry
builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource => resource
        .AddService(
            serviceName: "OpenTelemetryDemo",
            serviceVersion: "1.0.0"))
    .WithTracing(tracing => tracing
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddConsoleExporter()
        .AddOtlpExporter(options =>
        {
            options.Endpoint = new Uri("http://localhost:4317");
        }))
    .WithMetrics(metrics => metrics
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddConsoleExporter()
        .AddOtlpExporter(options =>
        {
            options.Endpoint = new Uri("http://localhost:4317");
        }));

var app = builder.Build();

// Configure the HTTP request pipeline
if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI();
}

app.UseHttpsRedirection();
app.UseAuthorization();
app.MapControllers();

app.Run();

Step 4: Create a Sample Controller

Create a new controller to demonstrate tracing:

using Microsoft.AspNetCore.Mvc;
using System.Diagnostics;

namespace OpenTelemetryDemo.Controllers
{
    [ApiController]
    [Route("[controller]")]
    public class WeatherForecastController : ControllerBase
    {
        private static readonly string[] Summaries = new[]
        {
            "Freezing", "Bracing", "Chilly", "Cool", "Mild", "Warm", "Balmy", "Hot", "Sweltering", "Scorching"
        };

        private readonly ILogger<WeatherForecastController> _logger;
        private readonly ActivitySource _activitySource;

        public WeatherForecastController(ILogger<WeatherForecastController> logger)
        {
            _logger = logger;
            _activitySource = new ActivitySource("OpenTelemetryDemo");
        }

        [HttpGet(Name = "GetWeatherForecast")]
        public async Task<IEnumerable<WeatherForecast>> Get()
        {
            using var activity = _activitySource.StartActivity("GetWeatherForecast");
            
            activity?.SetTag("custom.tag", "weather-data");
            
            _logger.LogInformation("Fetching weather forecast");
            
            // Simulate some processing time
            await Task.Delay(100);
            
            var forecast = Enumerable.Range(1, 5).Select(index => new WeatherForecast
            {
                Date = DateOnly.FromDateTime(DateTime.Now.AddDays(index)),
                TemperatureC = Random.Shared.Next(-20, 55),
                Summary = Summaries[Random.Shared.Next(Summaries.Length)]
            })
            .ToArray();
            
            activity?.SetTag("forecast.count", forecast.Length);
            
            return forecast;
        }
    }

    public class WeatherForecast
    {
        public DateOnly Date { get; set; }
        public int TemperatureC { get; set; }
        public int TemperatureF => 32 + (int)(TemperatureC / 0.5556);
        public string? Summary { get; set; }
    }
}

Step 5: Add Custom Tracing (Advanced)

For more control over your traces, you can create custom spans:

using System.Diagnostics;

public class OrderService
{
    private static readonly ActivitySource _activitySource = new("OrderService");

    public async Task<Order> ProcessOrder(int orderId)
    {
        using var activity = _activitySource.StartActivity("ProcessOrder");
        activity?.SetTag("order.id", orderId);

        try
        {
            // Validate order
            using (var validateActivity = _activitySource.StartActivity("ValidateOrder"))
            {
                validateActivity?.SetTag("order.id", orderId);
                await ValidateOrder(orderId);
            }

            // Process payment
            using (var paymentActivity = _activitySource.StartActivity("ProcessPayment"))
            {
                paymentActivity?.SetTag("order.id", orderId);
                await ProcessPayment(orderId);
            }

            activity?.SetTag("order.status", "completed");
            return new Order { Id = orderId, Status = "Completed" };
        }
        catch (Exception ex)
        {
            activity?.SetTag("error", true);
            activity?.SetTag("error.message", ex.Message);
            throw;
        }
    }

    private Task ValidateOrder(int orderId) => Task.Delay(50);
    private Task ProcessPayment(int orderId) => Task.Delay(100);
}

public class Order
{
    public int Id { get; set; }
    public string Status { get; set; }
}

Step 6: Configure Application Settings

Add OpenTelemetry configuration to your appsettings.json:

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft.AspNetCore": "Warning"
    }
  },
  "OpenTelemetry": {
    "ServiceName": "OpenTelemetryDemo",
    "ServiceVersion": "1.0.0",
    "OtlpEndpoint": "http://localhost:4317"
  }
}

Setting Up a Collector (Optional)

To visualize your telemetry data, you need a backend. Here’s how to set up a local collector using Docker:

Using Jaeger

docker run -d --name jaeger \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  jaegertracing/all-in-one:latest

Then access Jaeger UI at: http://localhost:16686

Using Docker Compose

Create a docker-compose.yml file:

version: '3.8'

services:
  jaeger:
    image: jaegertracing/all-in-one:latest
    ports:
      - "16686:16686"
      - "4317:4317"
      - "4318:4318"
    environment:
      - COLLECTOR_OTLP_ENABLED=true

Run with:

docker-compose up -d

Running and Testing

1. Start your .NET application:

dotnet run

1. Make some API calls:

curl https://localhost:7000/WeatherForecast

1. Open Jaeger UI at http://localhost:16686 and search for traces from your service

2. You should see traces with all the instrumented HTTP requests and custom spans

Best Practices

1. Use Semantic Conventions: Follow OpenTelemetry semantic conventions for attribute naming
2. Don’t Over-Instrument: Only add custom spans where they provide value
3. Use Sampling: Configure sampling to reduce overhead in production
4. Add Meaningful Tags: Include relevant business context in your spans
5. Handle Errors: Properly tag spans with error information
6. Use Resource Attributes: Add deployment environment, version, and instance information
7. Configure Batching: Use batch processors for better performance

Configuring Sampling

For production environments, configure sampling to control data volume:

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing => tracing
        .SetSampler(new TraceIdRatioBasedSampler(0.1)) // Sample 10% of traces
        .AddAspNetCoreInstrumentation()
        .AddHttpClientInstrumentation()
        .AddOtlpExporter());

Environment-Specific Configuration

Configure different settings for different environments:

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing =>
    {
        tracing.AddAspNetCoreInstrumentation()
               .AddHttpClientInstrumentation();

        if (builder.Environment.IsDevelopment())
        {
            tracing.AddConsoleExporter();
        }
        else
        {
            tracing.AddOtlpExporter(options =>
            {
                options.Endpoint = new Uri(builder.Configuration["OpenTelemetry:OtlpEndpoint"]);
            });
        }
    });

Common Issues and Troubleshooting

Issue 1: Traces Not Appearing

• Verify the OTLP endpoint is correct and accessible
• Check that the collector is running
• Ensure the service name is configured correctly

Issue 2: High Performance Overhead

• Configure sampling to reduce the number of traces
• Use batch processors
• Reduce the number of custom spans

Issue 3: Missing HTTP Client Traces

• Make sure AddHttpClientInstrumentation() is added
• Verify that HttpClient is being used (not HttpWebRequest)

Additional Resources

• OpenTelemetry .NET Documentation
• OpenTelemetry .NET GitHub Repository
• OpenTelemetry Semantic Conventions
• CNCF OpenTelemetry Project

Conclusion

OpenTelemetry provides a powerful and standardized way to instrument your .NET applications for observability. By following this guide, you should now have a working OpenTelemetry setup that captures traces and metrics from your application. As you become more comfortable with OpenTelemetry, explore advanced features like custom metrics, context propagation, and integration with various observability backends.

Happy monitoring!