Logs Auto-Instrumentation Example

This page demonstrates how to use Python logs auto-instrumentation in OpenTelemetry.

Unlike Traces and Metrics, there is no equivalent Logs API. There is only an SDK. For Python, you use the Python logger library, and then the OTel SDK attaches an OTLP handler to the root logger, turning the Python logger into an OTLP logger. One way to accomplish this is documented in the logs example in OpenTelemetry Python repository.

Another way this is accomplished is through Python’s support for auto-instrumentation of logs. The example below is based on the logs example in OpenTelemetry Python repository.

There is a logs bridge API; however, it is different from the Traces and Metrics API, because it’s not used by application developers to create logs. Instead, they would use this bridge API to setup log appenders in the standard language-specific logging libraries. More information can be found here.

Start by creating the examples directory and the example Python file:

mkdir python-logs-example
cd python-logs-example
touch example.py

Paste the following contents into example.py:

import logging

from opentelemetry import trace

tracer = trace.get_tracer_provider().get_tracer(__name__)

# Trace context correlation
with tracer.start_as_current_span("foo"):
    # Do something
    current_span = trace.get_current_span()
    current_span.add_event("This is a span event")
    logging.getLogger().error("This is a log message")

Grab a copy of the OTel Collector configuration from here, and save it to python-logs-example/otel-collector-config.yaml

Prepare

Execute the following example, we recommend using a virtual environment to do so. Run the following commands to prepare for logs auto-instrumentation:

mkdir python_logs_example
virtualenv python_logs_example
source python_logs_example/bin/activate

Install

The following commands install the appropriate packages. The opentelemetry-distro package depends on a few others, like opentelemetry-sdk for custom instrumentation of your own code and opentelemetry-instrumentation which provides several commands that help automatically instrument a program.

pip install opentelemetry-distro
pip install opentelemetry-exporter-otlp

The examples that follow send instrumentation results to the console. Learn more about installing and configuring the OpenTelemetry Distro to send telemetry to other destinations, like an OpenTelemetry Collector.

Note: To use automatic instrumentation through opentelemetry-instrument, you must configure it via environment variables or the command line. The agent creates a telemetry pipeline that cannot be modified other than through these means. If you need more customization for your telemetry pipelines, then you need to forego the agent and import the OpenTelemetry SDK and instrumentation libraries into your code and configure them there. You may also extend automatic instrumentation by importing the OpenTelemetry API. For more details, see the API reference.

Execute

This section guides you through the process of executing an automatically instrumented logs.

Open up a new terminal window and start the OTel Collector:

docker run -it --rm -p 4317:4317 -p 4318:4318 \
  -v $(pwd)/otel-collector-config.yaml:/etc/otelcol-config.yml \
  --name otelcol \
  otel/opentelemetry-collector-contrib:0.76.1 \
  "--config=/etc/otelcol-config.yml"

Open up another terminal and run the Python program:

source python_logs_example/bin/activate

export OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED=true
opentelemetry-instrument \
  --traces_exporter console,otlp \
  --metrics_exporter console,otlp \
  --logs_exporter console,otlp \
  --service_name python-logs-example \
  python $(pwd)/example.py

Sample output:

...
ScopeSpans #0
ScopeSpans SchemaURL:
InstrumentationScope __main__
Span #0
    Trace ID       : 389d4ac130a390d3d99036f9cd1db75e
    Parent ID      :
    ID             : f318281c4654edc5
    Name           : foo
    Kind           : Internal
    Start time     : 2023-08-18 17:04:05.982564 +0000 UTC
    End time       : 2023-08-18 17:04:05.982667 +0000 UTC
    Status code    : Unset
    Status message :
Events:
SpanEvent #0
     -> Name: This is a span event
     -> Timestamp: 2023-08-18 17:04:05.982586 +0000 UTC

...

ScopeLogs #0
ScopeLogs SchemaURL:
InstrumentationScope opentelemetry.sdk._logs._internal
LogRecord #0
ObservedTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2023-08-18 17:04:05.982605056 +0000 UTC
SeverityText: ERROR
SeverityNumber: Error(17)
Body: Str(This is a log message)
Attributes:
     -> otelSpanID: Str(f318281c4654edc5)
     -> otelTraceID: Str(389d4ac130a390d3d99036f9cd1db75e)
     -> otelTraceSampled: Bool(true)
     -> otelServiceName: Str(python-logs-example)
Trace ID: 389d4ac130a390d3d99036f9cd1db75e
Span ID: f318281c4654edc5
...

Note that the Span Event and the Log both have the same SpanID (f318281c4654edc5). The logging SDK appends the SpanID of the current Span to any logged events to improve the ability to correlate telemetry.