Go OpenTelemetry Instrumentation

Automatically create traces/spans on HTTP requests

OpenTelemetry can help you jumpstart your way to observability by providing automatic instrumentation for HTTP requests. You have your choice of request routers in OpenTelemetry or you can use the standard HTTP handler. You should pick the mux that’s right for your framework.

Automatic instrumentation with request routers

If you are using gin/gonic:

# Add one line to your import() stanza depending upon your request router:
middleware "go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin"

and then inject OpenTelemetry middleware

router.Use(middleware.Middleware(serviceName))

serviceName is found from the env variable. If this line is in main.go then it is already there

If you are using gorillamux:

# Add one line to your import() stanza depending upon your request router:
middleware "go.opentelemetry.io/contrib/instrumentation/github.com/gorilla/mux/otelmux"

and then inject OpenTelemetry middleware

router.Use(middleware.Middleware(serviceName))

If you are using echo:

# Add one line to your import() stanza depending upon your request router:
middleware "go.opentelemetry.io/contrib/instrumentation/github.com/labstack/echo/otelecho"

and then inject OpenTelemetry middleware

router.Use(middleware.Middleware(serviceName))

Run Command

SERVICE_NAME=<service_name> INSECURE_MODE=true OTEL_METRICS_EXPORTER=none OTEL_EXPORTER_OTLP_ENDPOINT=<IP of SigNoz backend:4317> go run main.go

<service_name> is the name of the service

If you don’t use a request router

import (
  "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)

In each place where you pass an http.Handler to a ServeMux, you’ll wrap the handler function. For instance, you’ll make the following replacements:

- mux.Handle("/path", h)
+ mux.Handle("/path", otelhttp.NewHandler(h, "description of path"))

- mux.Handle("/path", http.HandlerFunc(f))
+ mux.Handle("/path", otelhttp.NewHandler(http.HandlerFunc(f), "description of path"))

In this fashion, you can ensure that every function you wrap with othttp will automatically have its metadata collected and a corresponding trace started.

Automatically create traces/spans on gRPC server requests

Similarly, OpenTelemetry can also help you automatically instrument gRPC requests. To instrument any gRPC servers you have, add an Interceptor to the instantiation of the server.

import (
  grpcotel "go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc"
)

func main() {
  [...]

    s := grpc.NewServer(
        grpc.UnaryInterceptor(grpcotel.UnaryServerInterceptor()),
        grpc.StreamInterceptor(grpcotel.StreamServerInterceptor()),
    )

}

We have a blog Monitor gRPC calls with OpenTelemetry - explained with a Golang example, do refer to that in case you need a helping hand to work with gRPC server.

Instrumentation of a sample Golang application

We have included a sample gin/gonic application with README.md at https://github.com/SigNoz/sample-golang-app.

Feel free to use this repo to test out OpenTelemetry instrumentation and how to send telemetry data to SigNoz.

Validate installation by checking for traces

With your application running, you can now verify that you’ve installed OpenTelemetry correctly by confirming that telemetry data is being reported to your observability backend.

To do this, you need to make sure that your application is actually generating data. Applications will generally not produce traces unless they are being interacted with, and opentelemetry will often buffer data before sending it. So it may take some amount of time and interaction before your application data begins to appear in your backend.

Configuring to send data to SigNoz

    // main.go
    package main

    import (
      "context"
      "log"
      "google.golang.org/grpc/credentials"

      "go.opentelemetry.io/otel"
      "go.opentelemetry.io/otel/attribute"
      "go.opentelemetry.io/otel/exporters/otlp/otlptrace"
      "go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc"

      "go.opentelemetry.io/otel/sdk/resource"
      sdktrace "go.opentelemetry.io/otel/sdk/trace"
    )

    var (
      serviceName  = os.Getenv("SERVICE_NAME")
      collectorURL = os.Getenv("OTEL_EXPORTER_OTLP_ENDPOINT")
      insecure     = os.Getenv("INSECURE_MODE")
    )

    func initTracer() func(context.Context) error {

      secureOption := otlptracegrpc.WithTLSCredentials(credentials.NewClientTLSFromCert(nil, ""))
      if len(insecure) > 0 {
        secureOption = otlptracegrpc.WithInsecure()
      }

      exporter, err := otlptrace.New(
        context.Background(),
        otlptracegrpc.NewClient(
          secureOption,
          otlptracegrpc.WithEndpoint(collectorURL),
        ),
      )

      if err != nil {
        log.Fatal(err)
      }
      resources, err := resource.New(
        context.Background(),
        resource.WithAttributes(
          attribute.String("service.name", serviceName),
          attribute.String("library.language", "go"),
        ),
      )
      if err != nil {
        log.Printf("Could not set resources: ", err)
      }

      otel.SetTracerProvider(
        sdktrace.NewTracerProvider(
          sdktrace.WithSampler(sdktrace.AlwaysSample()),
          sdktrace.WithBatcher(exporter),
          sdktrace.WithResource(resources),
        ),
      )
      return exporter.Shutdown
    }

    func main() {

      cleanup := initTracer()
      defer cleanup(context.Background())

      // rest of initialization, including creating HTTP and gRPC servers/handlers...
    }

Library and framework support

Besides OpenTelemetry core modules, it is important to install instrumentation packages for every important library and framework which your service depends upon. Beyond the critical telemetry data these components emit, library and framework integrations are often required to ensure that the trace context is properly propagated.

OpenTelemetry automatically provides instrumentation for a large number of libraries and frameworks, right out of the box.

The full list of supported instrumentation can be found in the README.

 

Frequently Asked Questions

1. How to find what to use in IP of SigNoz if I have installed SigNoz in Kubernetes cluster?

   Based on where you have installed your application and where you have installed SigNoz, you need to find the right value for this. Please use this grid to find the value you should use for IP of SigNoz

2. I am sending data from my application to SigNoz, but I don't see any events or graphs in the SigNoz dashboard. What should I do?

   This could be because of one of the following reasons:

   1. Your application is generating telemetry data, but not able to connect with SigNoz installation

      Please use this troubleshooting guide to find if your application is able to access SigNoz installation and send data to it.

   2. Your application is not actually generating telemetry data

      Please check if the application is generating telemetry data first. You can use Console Exporter to just print your telemetry data in console first. Join our Slack Community if you need help on how to export your telemetry data in console

   3. Your SigNoz installation is not running or behind a firewall

      Please double check if the pods in SigNoz installation are running fine. docker ps or kubectl get pods -n platform are your friends for this.