OpenTelemetry
Packrat accepts data from any OpenTelemetry SDK. You install the SDK for your language, then point its exporter at Packrat with a few environment variables.
If your app already exports OpenTelemetry to another vendor, you can keep that vendor and add Packrat beside it. See If you already export OpenTelemetry.
Before you begin
-
Sign up for Packrat. It's free.
-
Copy your ingest key from the setup page. It starts with
packrat_ingest_. -
Install the Packrat CLI and sign in, so you can check what Packrat receives:
curl -fsSL https://packrat.sh/cli | sh packrat login
Set up your app
Choose your framework. Each tab has every step for that framework.
Use the
packrat-railsgem instead. It gives you a simpler setup and still uses OpenTelemetry. See the Ruby on Rails guide.
1. Install the gems
Add the SDK, the exporter and the instrumentation to your Gemfile:
# Gemfile
gem "opentelemetry-sdk", require: false
gem "opentelemetry-exporter-otlp", require: false
gem "opentelemetry-instrumentation-all", require: false
gem "opentelemetry-logs-sdk", require: false
gem "opentelemetry-exporter-otlp-logs", require: false
gem "opentelemetry-instrumentation-logger", require: false
bundle install
2. Add an initializer
Create config/initializers/opentelemetry.rb:
# config/initializers/opentelemetry.rb
require "opentelemetry/sdk"
require "opentelemetry/exporter/otlp"
require "opentelemetry/instrumentation/all"
require "opentelemetry-logs-sdk"
require "opentelemetry-exporter-otlp-logs"
require "opentelemetry-instrumentation-logger"
# Don't send data from your test suite.
return if Rails.env.test?
# Keep traces that start with a request or a job. Drop the rest, such as the database polls that
# Solid Queue and Solid Cable make about ten times a second.
class RootKindSampler
KEEP = %i[server consumer].freeze
def should_sample?(trace_id:, parent_context:, links:, name:, kind:, attributes:)
keep = KEEP.include?(kind) || name == ErrorSpans::NAME
decision = keep ? OpenTelemetry::SDK::Trace::Samplers::Decision::RECORD_AND_SAMPLE
: OpenTelemetry::SDK::Trace::Samplers::Decision::DROP
tracestate = OpenTelemetry::Trace.current_span(parent_context).context.tracestate
OpenTelemetry::SDK::Trace::Samplers::Result.new(decision:, tracestate:)
end
def description
"RootKindSampler{#{KEEP.join(',')}}"
end
end
OpenTelemetry::SDK.configure do |c|
# Skip the health check, which runs every few seconds.
c.use_all("OpenTelemetry::Instrumentation::Rack" => { untraced_endpoints: [ "/up" ] })
end
OpenTelemetry.tracer_provider.sampler =
OpenTelemetry::SDK::Trace::Samplers.parent_based(root: RootKindSampler.new)
# Send the last batch of spans and log lines when the process exits.
at_exit do
OpenTelemetry.tracer_provider.shutdown
OpenTelemetry.logger_provider.shutdown
end
# Record every error that Rails.error reports: unhandled errors in requests and jobs, errors your
# app reports itself, and errors in runner scripts.
class ErrorSpans
NAME = "Rails.error"
def report(error, handled:, severity:, context:, source: nil)
attributes = { "exception.handled" => handled, "rails.error.source" => source }.compact
span = OpenTelemetry::Trace.current_span
return span.record_exception(error, attributes:) if span.recording?
span = OpenTelemetry.tracer_provider.tracer("rails.error").start_root_span(NAME)
span.record_exception(error, attributes:)
span.finish
end
end
Rails.error.subscribe(ErrorSpans.new)
Here's what each part does:
| Part | What it does |
|---|---|
use_all |
Traces Rack, Action Pack, Active Record, Active Job, Net::HTTP and more, and sends Rails.logger lines as logs |
untraced_endpoints |
Skips /up, the health check |
RootKindSampler |
Keeps traces that start with a request or a job, and drops background polling |
at_exit |
Sends the last batch of data when the process exits. The SDK doesn't do this on its own |
ErrorSpans |
Records each error Rails.error reports. Packrat groups these into issues |
3. Point the exporter at Packrat
Set these environment variables where your app runs. Replace <your ingest key>, and set
OTEL_SERVICE_NAME to your app's name:
OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.packrat.sh
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_HEADERS=Authorization=<your ingest key>
OTEL_SERVICE_NAME=shop
OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production
4. Run your app
bin/rails server
1. Install the packages
Install the API and the package that traces Express, Fastify, Koa, http, pg and other common
libraries:
npm install @opentelemetry/api @opentelemetry/auto-instrumentations-node
2. Point the exporter at Packrat
Set these environment variables where your app runs. Replace <your ingest key>, and set
OTEL_SERVICE_NAME to your app's name:
OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.packrat.sh
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_HEADERS=Authorization=<your ingest key>
OTEL_SERVICE_NAME=shop
OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production
3. Run your app
Load the instrumentation before your app's code. You don't change your code:
NODE_OPTIONS="--require @opentelemetry/auto-instrumentations-node/register" node app.js
1. Install the packages
Install the distro and the exporter. Then run opentelemetry-bootstrap, which adds instrumentation
for the libraries your app uses, such as Django, Flask, FastAPI, requests and psycopg:
pip install opentelemetry-distro opentelemetry-exporter-otlp
opentelemetry-bootstrap -a install
2. Point the exporter at Packrat
Set these environment variables where your app runs. Replace <your ingest key>, and set
OTEL_SERVICE_NAME to your app's name:
OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.packrat.sh
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_HEADERS=Authorization=<your ingest key>
OTEL_SERVICE_NAME=shop
OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production
3. Run your app
Start your app with opentelemetry-instrument. You don't change your code:
OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true opentelemetry-instrument python app.py
Put the same command in front of gunicorn, uvicorn or flask run. The variable sends your
logging output to Packrat as logs.
1. Download the agent
The OpenTelemetry Java agent traces Spring, servlets, JDBC and common HTTP clients:
curl -L -O https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/latest/download/opentelemetry-javaagent.jar
2. Point the exporter at Packrat
Set these environment variables where your app runs. Replace <your ingest key>, and set
OTEL_SERVICE_NAME to your app's name:
OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.packrat.sh
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_HEADERS=Authorization=<your ingest key>
OTEL_SERVICE_NAME=shop
OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production
3. Run your app
Start the JVM with the agent. You don't change your code:
java -javaagent:opentelemetry-javaagent.jar -jar app.jar
1. Install the packages
Add the SDK, the exporter, and the ASP.NET Core and HttpClient instrumentation:
dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
dotnet add package OpenTelemetry.Instrumentation.Http
2. Register OpenTelemetry
Add OpenTelemetry where your app builds its services:
// Program.cs
using OpenTelemetry;
using OpenTelemetry.Metrics;
using OpenTelemetry.Trace;
builder.Services.AddOpenTelemetry()
.UseOtlpExporter()
.WithTracing(tracing => tracing
.AddAspNetCoreInstrumentation()
.AddHttpClientInstrumentation())
.WithMetrics(metrics => metrics
.AddAspNetCoreInstrumentation()
.AddHttpClientInstrumentation())
.WithLogging();
UseOtlpExporter sends traces, metrics and logs, and reads its settings from the environment
variables in the next step.
3. Point the exporter at Packrat
Set these environment variables where your app runs. Replace <your ingest key>, and set
OTEL_SERVICE_NAME to your app's name:
OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.packrat.sh
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_HEADERS=Authorization=<your ingest key>
OTEL_SERVICE_NAME=shop
OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production
4. Run your app
dotnet run
1. Install the packages
Add the SDK, the OTLP/HTTP exporter and the net/http instrumentation:
go get go.opentelemetry.io/otel \
go.opentelemetry.io/otel/sdk \
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp \
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp
2. Start the tracer
The Go SDK doesn't instrument your app automatically, so start the tracer and wrap your handler:
// main.go
package main
import (
"context"
"log"
"net/http"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp"
sdktrace "go.opentelemetry.io/otel/sdk/trace"
)
func main() {
ctx := context.Background()
// Reads the endpoint, headers and service name from the environment variables in the next step.
exporter, err := otlptracehttp.New(ctx)
if err != nil {
log.Fatal(err)
}
provider := sdktrace.NewTracerProvider(sdktrace.WithBatcher(exporter))
defer provider.Shutdown(ctx)
otel.SetTracerProvider(provider)
mux := http.NewServeMux()
// mux.HandleFunc(...)
log.Fatal(http.ListenAndServe(":8080", otelhttp.NewHandler(mux, "server")))
}
3. Point the exporter at Packrat
Set these environment variables where your app runs. Replace <your ingest key>, and set
OTEL_SERVICE_NAME to your app's name:
OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.packrat.sh
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_HEADERS=Authorization=<your ingest key>
OTEL_SERVICE_NAME=shop
OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production
4. Run your app
go run .
Any OpenTelemetry SDK works.
1. Install the SDK
Find your language in the OpenTelemetry documentation, and install its SDK and OTLP exporter. If your language offers more than one exporter, choose the HTTP one. Packrat doesn't accept gRPC.
2. Initialize the SDK
Initialize the SDK as early as possible when your app starts, as your language's documentation shows. Don't put the endpoint or the key in your code: every SDK reads them from the environment variables in the next step.
3. Point the exporter at Packrat
Set these environment variables where your app runs. Replace <your ingest key>, and set
OTEL_SERVICE_NAME to your app's name:
OTEL_EXPORTER_OTLP_ENDPOINT=https://ingest.packrat.sh
OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
OTEL_EXPORTER_OTLP_HEADERS=Authorization=<your ingest key>
OTEL_SERVICE_NAME=shop
OTEL_RESOURCE_ATTRIBUTES=deployment.environment=production
4. Run your app
Start your app as you usually do.
What the variables do
These variables work the same way in every language:
| Variable | Value | What it does |
|---|---|---|
OTEL_EXPORTER_OTLP_ENDPOINT |
https://ingest.packrat.sh |
Where the SDK sends data. It adds /v1/traces, /v1/logs and /v1/metrics itself. |
OTEL_EXPORTER_OTLP_PROTOCOL |
http/protobuf |
Packrat accepts OTLP over HTTP as protobuf. It doesn't accept gRPC or JSON. |
OTEL_EXPORTER_OTLP_HEADERS |
Authorization=<your ingest key> |
Your ingest key. Bearer <key> also works if you write the space as %20, because the value is URL-encoded. |
OTEL_SERVICE_NAME |
<your app name> |
Your app's name in Packrat. |
OTEL_RESOURCE_ATTRIBUTES |
deployment.environment=production |
Your app's environment in Packrat. |
Packrat keeps each service and environment as a separate app, such as shop/production, so you can compare environments.
Verify
Load a page in your app, then check what Packrat receives:
Run packrat status with your app's name, as <service>/<environment>:
packrat status --app shop/production
The output lists each signal (exceptions, traces, logs, metrics and replays), the sources that send
it, when each source last sent, and whether Packrat keeps it. A signal with no source hasn't arrived
yet. To see the names your apps report under, run packrat apps.
If you already export OpenTelemetry
Keep your current exporter and its environment variables. Then add a second OTLP/HTTP exporter to your OpenTelemetry setup that sends to Packrat:
- Traces:
https://ingest.packrat.sh/v1/traces - Logs:
https://ingest.packrat.sh/v1/logs - Metrics:
https://ingest.packrat.sh/v1/metrics
Send your ingest key in the Authorization header. Both vendors receive every span, and nothing
changes for the other vendor.