opentelemetry-go/sdk/log/DESIGN.md

Logs SDK

Abstract

go.opentelemetry.io/otel/sdk/log provides Logs SDK compliant with the specification.

The main and recommended use case is to configure the SDK to use an OTLP exporter with a batch processor.¹ Therefore, the design aims to be high-performant in this scenario.

The prototype was created in #4955.

Modules structure

The SDK is published as a single go.opentelemetry.io/otel/sdk/log Go module.

The exporters are going to be published as following Go modules:

• go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc
• go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp
• go.opentelemetry.io/otel/exporters/stdout/stdoutlog

LoggerProvider

The LoggerProvider is implemented as LoggerProvider struct in provider.go.

LogRecord limits

The LogRecord limits can be configured using following options:

func WithAttributeCountLimit(limit int) LoggerProviderOption
func WithAttributeValueLengthLimit(limit int) LoggerProviderOption

The limits can be also configured using the OTEL_LOGRECORD_* environment variables as defined by the specification.

Processor

The LogRecordProcessor is defined as Processor interface in processor.go.

The user set processors for the LoggerProvider using func WithProcessor(processor Processor) LoggerProviderOption.

The user can configure custom processors and decorate built-in processors.

SimpleProcessor

The Simple processor is implemented as SimpleProcessor struct in simple.go.

BatchProcessor

The Batching processor is implemented as BatchProcessor struct in batch.go.

The Batcher can be also configured using the OTEL_BLRP_* environment variables as defined by the specification.

Exporter

The LogRecordExporter is defined as Exporter interface in exporter.go.

The slice passed to Export must not be retained by the implementation (like e.g. io.Writer) so that the caller can reuse the passed slice (e.g. using sync.Pool) to avoid heap allocations on each call.

Record

The ReadWriteLogRecord is defined as Record struct in record.go.

The Record is designed similarly to log.Record in order to reduce the number of heap allocations when processing attributes.

The SDK does not have have an additional definition of ReadableLogRecord as the specification does not say that the exporters must not be able to modify the log records. It simply requires them to be able to read the log records. Having less abstractions reduces the API surface and makes the design simpler.

Benchmarking

The benchmarks are supposed to test end-to-end scenarios and avoid I/O that could affect the stability of the results.

The benchmark results can be found in the prototype.

Rejected alternatives

Represent both LogRecordProcessor and LogRecordExporter as Expoter

Because the LogRecordProcessor and the LogRecordProcessor abstractions are so similar, there was a proposal to unify them under single Expoter interface.²

However, introducing a Processor interface makes it easier to create custom processor decorators³ and makes the design more aligned with the specifiation.

Embedd log.Record

Because Record and log.Record are very similar, there was a proposal to embedd log.Record in Record definition.

log.Record supports only adding attributes. In the SDK, we also need to be able to modify the attributes (e.g. removal) provided via API.

Moreover it is safer to have these abstraction decoupled. E.g. there can be a need for some fields that can be set via API and cannot be modified by the processors.

---

1. OpenTelemetry Logging ↩︎

2. Conversation on representing LogRecordProcessor and LogRecordExporter via a single Expoter interface ↩︎

3. Introduce Processor ↩︎