go/opentelemetry-go
1
0
Fork 0
mirror of https://github.com/open-telemetry/opentelemetry-go.git synced 2025-07-17 01:12:45 +02:00
Code Issues Releases Activity

Simplify log design docs (#5077)

Browse Source
This commit is contained in:
Robert Pająk
2024-03-16 11:06:06 +01:00
committed by GitHub
parent 3a72c5ea94
commit 6fb46a1211
2 changed files with 20 additions and 465 deletions
Download Patch File Download Diff File Expand all files Collapse all files

186
log/DESIGN.md
View File

@ -5,8 +5,8 @@
`go.opentelemetry.io/otel/log` provides `go.opentelemetry.io/otel/log` provides
[Logs Bridge API](https://opentelemetry.io/docs/specs/otel/logs/bridge-api/). [Logs Bridge API](https://opentelemetry.io/docs/specs/otel/logs/bridge-api/).
The initial version of the design and the prototype The prototype was created in
was created in [#4725](https://github.com/open-telemetry/opentelemetry-go/pull/4725). [#4725](https://github.com/open-telemetry/opentelemetry-go/pull/4725).
## Background ## Background
@ -43,14 +43,7 @@ Rejected alternative:
### LoggerProvider ### LoggerProvider
The [`LoggerProvider` abstraction](https://opentelemetry.io/docs/specs/otel/logs/bridge-api/#loggerprovider) The [`LoggerProvider` abstraction](https://opentelemetry.io/docs/specs/otel/logs/bridge-api/#loggerprovider)
is defined as an interface: is defined as `LoggerProvider` interface in [provider.go](provider.go).
```go
type LoggerProvider interface {
embedded.LoggerProvider
Logger(name string, options ...LoggerOption) Logger
}
```
The specification may add new operations to `LoggerProvider`. The specification may add new operations to `LoggerProvider`.
The interface may have methods added without a package major version bump. The interface may have methods added without a package major version bump.
@ -64,13 +57,7 @@ The `Logger` method implements the [`Get a Logger` operation](https://openteleme
The required `name` parameter is accepted as a `string` method argument. The required `name` parameter is accepted as a `string` method argument.
The following options are defined to support optional parameters: The `LoggerOption` options are defined to support optional parameters.
```go
func WithInstrumentationVersion(version string) LoggerOption
func WithInstrumentationAttributes(attr ...attribute.KeyValue) LoggerOption
func WithSchemaURL(schemaURL string) LoggerOption
```
Implementation requirements: Implementation requirements:
@ -94,14 +81,7 @@ Rejected alternative:
### Logger ### Logger
The [`Logger` abstraction](https://opentelemetry.io/docs/specs/otel/logs/bridge-api/#logger) The [`Logger` abstraction](https://opentelemetry.io/docs/specs/otel/logs/bridge-api/#logger)
is defined as an interface: is defined as `Logger` interface in [logger.go](logger.go).
```go
type Logger interface {
embedded.Logger
Emit(ctx context.Context, record Record)
}
```
The specification may add new operations to `Logger`. The specification may add new operations to `Logger`.
The interface may have methods added without a package major version bump. The interface may have methods added without a package major version bump.
@ -119,23 +99,7 @@ is accepted as a `context.Context` method argument.
Calls to `Emit` are supposed to be on the hot path. Calls to `Emit` are supposed to be on the hot path.
Therefore, in order to reduce the number of heap allocations, Therefore, in order to reduce the number of heap allocations,
the [`LogRecord` abstraction](https://opentelemetry.io/docs/specs/otel/logs/bridge-api/#emit-a-logrecord), the [`LogRecord` abstraction](https://opentelemetry.io/docs/specs/otel/logs/bridge-api/#emit-a-logrecord),
is defined as a `Record` type: is defined as `Record` struct in [record.go](record.go).
```go
type Record struct {
timestamp time.Time
observedTimestamp time.Time
severity Severity
severityText string
body Value
// The fields below are for optimizing the implementation of
// attributes.
front [5]KeyValue
nFront int // The number of attributes in front.
back []KeyValue
}
```
[`Timestamp`](https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-timestamp) [`Timestamp`](https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-timestamp)
is accessed using following methods: is accessed using following methods:
@ -161,53 +125,11 @@ func (r *Record) Severity() Severity
func (r *Record) SetSeverity(s Severity) func (r *Record) SetSeverity(s Severity)
``` ```
`Severity` type is defined and constants are based on `Severity` type is defined in [severity.go](severity.go).
The constants are are based on
[Displaying Severity recommendation](https://opentelemetry.io/docs/specs/otel/logs/data-model/#displaying-severity). [Displaying Severity recommendation](https://opentelemetry.io/docs/specs/otel/logs/data-model/#displaying-severity).
Additionally, `Severity[Level]1` constants are defined to make the API more readable and user friendly. Additionally, `Severity[Level]1` constants are defined to make the API more readable and user friendly.
```go
type Severity int
const (
SeverityTrace1 Severity = 1 // TRACE
SeverityTrace2 Severity = 2 // TRACE2
SeverityTrace3 Severity = 3 // TRACE3
SeverityTrace4 Severity = 4 // TRACE4
SeverityDebug1 Severity = 5 // DEBUG
SeverityDebug2 Severity = 6 // DEBUG2
SeverityDebug3 Severity = 7 // DEBUG3
SeverityDebug4 Severity = 8 // DEBUG4
SeverityInfo1 Severity = 9 // INFO
SeverityInfo2 Severity = 10 // INFO2
SeverityInfo3 Severity = 11 // INFO3
SeverityInfo4 Severity = 12 // INFO4
SeverityWarn1 Severity = 13 // WARN
SeverityWarn2 Severity = 14 // WARN2
SeverityWarn3 Severity = 15 // WARN3
SeverityWarn4 Severity = 16 // WARN4
SeverityError1 Severity = 17 // ERROR
SeverityError2 Severity = 18 // ERROR2
SeverityError3 Severity = 19 // ERROR3
SeverityError4 Severity = 20 // ERROR4
SeverityFatal1 Severity = 21 // FATAL
SeverityFatal2 Severity = 22 // FATAL2
SeverityFatal3 Severity = 23 // FATAL3
SeverityFatal4 Severity = 24 // FATAL4
SeverityTrace = SeverityTrace1
SeverityDebug = SeverityDebug1
SeverityInfo = SeverityInfo1
SeverityWarn = SeverityWarn1
SeverityError = SeverityError1
SeverityFatal = SeverityFatal1
)
```
[`SeverityText`](https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-severitytext) [`SeverityText`](https://opentelemetry.io/docs/specs/otel/logs/data-model/#field-severitytext)
is accessed using following methods: is accessed using following methods:
@ -247,95 +169,9 @@ while keeping the API user friendly.
It relieves the user from making his own improvements It relieves the user from making his own improvements
for reducing the number of allocations when passing attributes. for reducing the number of allocations when passing attributes.
The following definitions are implementing the abstractions The abstractions described in
described in [the specification](https://opentelemetry.io/docs/specs/otel/logs/#new-first-party-application-logs): [the specification](https://opentelemetry.io/docs/specs/otel/logs/#new-first-party-application-logs)
are defined in [keyvalue.go](keyvalue.go).
```go
type Value struct{}
type Kind int
const (
KindEmpty Kind = iota
KindBool
KindFloat64
KindInt64
KindString
KindBytes
KindSlice
KindMap
)
func (v Value) Kind() Kind
// Value factories:
func StringValue(value string) Value
func IntValue(v int) Value
func Int64Value(v int64) Value
func Float64Value(v float64) Value
func BoolValue(v bool) Value
func BytesValue(v []byte) Value
func SliceValue(vs ...Value) Value
func MapValue(kvs ...KeyValue) Value
// Value accessors:
func (v Value) AsString() string
func (v Value) AsInt64() int64
func (v Value) AsBool() bool
func (v Value) AsFloat64() float64
func (v Value) AsBytes() []byte
func (v Value) AsSlice() []Value
func (v Value) AsMap() []KeyValue
func (v Value) Empty() bool
// Value equality comparison:
func (v Value) Equal(w Value) bool
type KeyValue struct {
Key string
Value Value
}
// KeyValue factories:
func String(key, value string) KeyValue
func Int64(key string, value int64) KeyValue
func Int(key string, value int) KeyValue
func Float64(key string, v float64) KeyValue
func Bool(key string, v bool) KeyValue
func Bytes(key string, v []byte) KeyValue
func Slice(key string, args ...Value) KeyValue
func Map(key string, args ...KeyValue) KeyValue
// KeyValue equality comparison:
func (a KeyValue) Equal(b KeyValue) bool
```
`Value` is representing `any`. `Value` is representing `any`.
`KeyValue` is representing a key(string)-value(`any`) pair. `KeyValue` is representing a key(string)-value(`any`) pair.

299
sdk/log/DESIGN.md
View File

@ -25,38 +25,7 @@ The exporters are going to be published as following Go modules:
## LoggerProvider ## LoggerProvider
The [LoggerProvider](https://opentelemetry.io/docs/specs/otel/logs/sdk/#loggerprovider) The [LoggerProvider](https://opentelemetry.io/docs/specs/otel/logs/sdk/#loggerprovider)
is defined as follows: is implemented as `LoggerProvider` struct in [provider.go](provider.go).
```go
type LoggerProvider struct {
embedded.LoggerProvider
}
// NewLoggerProvider returns a new and configured LoggerProvider.
//
// By default, the returned LoggerProvider is configured with the default
// Resource and no Processors. Processors cannot be added after a LoggerProvider is
// created. This means the returned LoggerProvider, one created with no
// Processors, will perform no operations.
func NewLoggerProvider(opts ...LoggerProviderOption) *LoggerProvider
// Logger returns a new log.Logger with the provided name and configuration.
//
// This method can be called concurrently.
//
// Logger implements the log.LoggerProvider interface.
func (*LoggerProvider) Logger(name string, options ...log.LoggerOption) log.Logger
type LoggerProviderOption interface { /* ... */ }
// WithResource associates a Resource with a LoggerProvider. This Resource
// represents the entity producing telemetry and is associated with all Loggers
// the LoggerProvider will create.
//
// By default, if this Option is not used, the default Resource from the
// go.opentelemetry.io/otel/sdk/resource package will be used.
func WithResource(res *resource.Resource) LoggerProviderOption
```
## LogRecord limits ## LogRecord limits
@ -64,32 +33,7 @@ The [LogRecord limits](https://opentelemetry.io/docs/specs/otel/logs/sdk/#logrec
can be configured using following options: can be configured using following options:
```go ```go
// WithAttributeCountLimit sets the maximum allowed log record attribute count.
// Any attribute added to a log record once this limit is reached will be dropped.
//
// Setting this to zero means no attributes will be recorded.
//
// Setting this to a negative value means no limit is applied.
//
// If the OTEL_LOGRECORD_ATTRIBUTE_COUNT_LIMIT environment variable is set,
// and this option is not passed, that variable value will be used.
//
// By default, if an environment variable is not set, and this option is not
// passed, no limit 128 will be used.
func WithAttributeCountLimit(limit int) LoggerProviderOption func WithAttributeCountLimit(limit int) LoggerProviderOption
// AttributeValueLengthLimit sets the maximum allowed attribute value length.
//
// This limit only applies to string and string slice attribute values.
// Any string longer than this value will be truncated to this length.
//
// Setting this to a negative value means no limit is applied.
//
// If the OTEL_LOGRECORD_ATTRIBUTE_VALUE_LENGTH_LIMIT environment variable is set,
// and this option is not passed, that variable value will be used.
//
// By default, if an environment variable is not set, and this option is not
// passed, no limit (-1) will be used.
func WithAttributeValueLengthLimit(limit int) LoggerProviderOption func WithAttributeValueLengthLimit(limit int) LoggerProviderOption
``` ```
@ -99,140 +43,22 @@ The limits can be also configured using the `OTEL_LOGRECORD_*` environment varia
### Processor ### Processor
The [LogRecordProcessor](https://opentelemetry.io/docs/specs/otel/logs/sdk/#logrecordprocessor) The [LogRecordProcessor](https://opentelemetry.io/docs/specs/otel/logs/sdk/#logrecordprocessor)
is defined as follows: is defined as `Processor` interface in [processor.go](processor.go).
```go The user set processors for the `LoggerProvider` using
// WithProcessor associates Processor with a LoggerProvider. `func WithProcessor(processor Processor) LoggerProviderOption`.
//
// By default, if this option is not used, the LoggerProvider will perform no
// operations; no data will be exported without a processor.
//
// Each WithProcessor creates a separate pipeline. Use custom decorators
// for advanced scenarios such as enriching with attributes.
//
// Use NewBatchingProcessor to batch log records before they are exported.
// Use NewSimpleProcessor to synchronously export log records.
func WithProcessor(processor Processor) LoggerProviderOption
// Processor handles the processing of log records.
//
// Any of the Processor's methods may be called concurrently with itself
// or with other methods. It is the responsibility of the Processor to manage
// this concurrency.
type Processor interface {
// OnEmit is called when a Record is emitted.
//
// Implementation should not interrupt the record processing
// if the context is canceled.
//
// All retry logic must be contained in this function. The SDK does not
// implement any retry logic. All errors returned by this function are
// considered unrecoverable and will be reported to a configured error
// Handler.
//
// Before modifying a Record, the implementation must use Record.Clone
// to create a copy that shares no state with the original.
OnEmit(ctx context.Context, record Record) error
// Shutdown is called when the SDK shuts down. Any cleanup or release of
// resources held by the exporter should be done in this call.
//
// The deadline or cancellation of the passed context must be honored. An
// appropriate error should be returned in these situations.
//
// After Shutdown is called, calls to Export, Shutdown, or ForceFlush
// should perform no operation and return nil error.
Shutdown(ctx context.Context) error
// ForceFlush exports log records to the configured Exporter that have not yet
// been exported.
//
// The deadline or cancellation of the passed context must be honored. An
// appropriate error should be returned in these situations.
ForceFlush(ctx context.Context) error
}
```
The user can configure custom processors and decorate built-in processors. The user can configure custom processors and decorate built-in processors.
### SimpleProcessor ### SimpleProcessor
The [Simple processor](https://opentelemetry.io/docs/specs/otel/logs/sdk/#simple-processor) The [Simple processor](https://opentelemetry.io/docs/specs/otel/logs/sdk/#simple-processor)
is defined as follows: is implemented as `SimpleProcessor` struct in [simple.go](simple.go).
```go
// SimpleProcessor implements Processor.
type SimpleProcessor struct { /* ... */ }
// NewSimpleProcessor is a simple Processor adapter.
//
// Any of the exporter's methods may be called concurrently with itself
// or with other methods. It is the responsibility of the exporter to manage
// this concurrency.
func NewSimpleProcessor(exporter Exporter) *SimpleProcessor
```
### BatchingProcessor ### BatchingProcessor
The [Batching processor](https://opentelemetry.io/docs/specs/otel/logs/sdk/#batching-processor) The [Batching processor](https://opentelemetry.io/docs/specs/otel/logs/sdk/#batching-processor)
is defined as follows: is implemented as `BatchingProcessor` struct in [batch.go](batch.go).
```go
// BatchingProcessor implements Processor.
type BatchingProcessor struct { /* ... */ }
// NewBatchingProcessor decorates the provided exporter
// so that the log records are batched before exporting.
//
// All of the exporter's methods are called from a single dedicated
// background goroutine. Therefore, the expoter does not need to
// be concurrent safe.
func NewBatchingProcessor(exporter Exporter, opts ...BatchingOption) *BatchingProcessor
// BatchingOption applies a configuration to a Batcher.
type BatchingOption interface { /* ... */ }
// WithMaxQueueSize sets the maximum queue size used by the Batcher.
// After the size is reached log records are dropped.
//
// If the OTEL_BLRP_MAX_QUEUE_SIZE environment variable is set,
// and this option is not passed, that variable value will be used.
//
// By default, if an environment variable is not set, and this option is not
// passed, 2048 will be used.
// The default value is also used when the provided value is not a positive value.
func WithMaxQueueSize(max int) BatchingOption
// WithExportInterval sets the maximum duration between batched exports.
//
// If the OTEL_BSP_SCHEDULE_DELAY environment variable is set,
// and this option is not passed, that variable value will be used.
//
// By default, if an environment variable is not set, and this option is not
// passed, 1s will be used.
// The default value is also used when the provided value is not a positive value.
func WithExportInterval(d time.Duration) BatchingOption
// WithExportTimeout sets the duration after which a batched export is canceled.
//
// If the OTEL_BSP_EXPORT_TIMEOUT environment variable is set,
// and this option is not passed, that variable value will be used.
//
// By default, if an environment variable is not set, and this option is not
// passed, 30s will be used.
// The default value is also used when the provided value is not a positive value.
func WithExportTimeout(d time.Duration) BatchingOption
// WithExportMaxBatchSize sets the maximum batch size of every export.
//
// If the OTEL_BSP_MAX_EXPORT_BATCH_SIZE environment variable is set,
// and this option is not passed, that variable value will be used.
//
// By default, if an environment variable is not set, and this option is not
// passed, 512 will be used.
// The default value is also used when the provided value is not a positive value.
func WithExportMaxBatchSize(max int) BatchingOption
```
The `Batcher` can be also configured using the `OTEL_BLRP_*` environment variables as The `Batcher` can be also configured using the `OTEL_BLRP_*` environment variables as
[defined by the specification](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#batch-logrecord-processor). [defined by the specification](https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#batch-logrecord-processor).
@ -240,45 +66,7 @@ The `Batcher` can be also configured using the `OTEL_BLRP_*` environment variabl
### Exporter ### Exporter
The [LogRecordExporter](https://opentelemetry.io/docs/specs/otel/logs/sdk/#logrecordexporter) The [LogRecordExporter](https://opentelemetry.io/docs/specs/otel/logs/sdk/#logrecordexporter)
is defined as follows: is defined as `Exporter` interface in [exporter.go](exporter.go).
```go
// Exporter handles the delivery of log records to external receivers.
type Exporter interface {
// Export transmits log records to a receiver.
//
// The deadline or cancellation of the passed context must be honored. An
// appropriate error should be returned in these situations.
//
// All retry logic must be contained in this function. The SDK does not
// implement any retry logic. All errors returned by this function are
// considered unrecoverable and will be reported to a configured error
// Handler.
//
// Implementations must not retain the records slice.
//
// Before modifying a Record, the implementation must use Record.Clone
// to create a copy that shares no state with the original.
Export(ctx context.Context, records []Record) error
// Shutdown is called when the SDK shuts down. Any cleanup or release of
// resources held by the exporter should be done in this call.
//
// The deadline or cancellation of the passed context must be honored. An
// appropriate error should be returned in these situations.
//
// After Shutdown is called, calls to Export, Shutdown, or ForceFlush
// should perform no operation and return nil error.
Shutdown(ctx context.Context) error
// ForceFlush exports log records to the configured Exporter that have not yet
// been exported.
//
// The deadline or cancellation of the passed context must be honored. An
// appropriate error should be returned in these situations.
ForceFlush(ctx context.Context) error
}
```
The slice passed to `Export` must not be retained by the implementation The slice passed to `Export` must not be retained by the implementation
(like e.g. [`io.Writer`](https://pkg.go.dev/io#Writer)) (like e.g. [`io.Writer`](https://pkg.go.dev/io#Writer))
@ -289,76 +77,7 @@ to avoid heap allocations on each call.
### Record ### Record
The [ReadWriteLogRecord](https://opentelemetry.io/docs/specs/otel/logs/sdk/#readwritelogrecord) The [ReadWriteLogRecord](https://opentelemetry.io/docs/specs/otel/logs/sdk/#readwritelogrecord)
is defined as follows: is defined as `Record` struct in [record.go](record.go).
```go
type Record struct { /* ... */ }
func (r *Record) Timestamp()
func (r *Record) SetTimestamp(t time.Time)
func (r *Record) ObservedTimestamp() time.Time
func (r *Record) SetObservedTimestamp(t time.Time)
func (r *Record) Severity() log.Severity
func (r *Record) SetSeverity(level log.Severity)
func (r *Record) SeverityText() string
func (r *Record) SetSeverityText(text string)
func (r *Record) Body() log.Value
func (r *Record) SetBody(v log.Value)
func (r *Record) WalkAttributes(f func(log.KeyValue) bool)
func (r *Record) AddAttributes(attrs ...log.KeyValue)
// SetAttributes sets and overrides the attributes of the log record.
func (r *Record) SetAttributes(attrs ...log.KeyValue)
func (r *Record) TraceID() trace.TraceID
func (r *Record) SetTraceID(id trace.TraceID)
func (r *Record) SpanID() trace.SpanID
func (r *Record) SetSpanID(id trace.SpanID)
func (r *Record) TraceFlags() trace.TraceFlags
func (r *Record) SetTraceFlags(flags trace.TraceFlags)
// Resource returns the entity that collected the log.
func (r *Record) Resource() resource.Resource
// InstrumentationScope returns the scope that the Logger was created with.
func (r *Record) InstrumentationScope() instrumentation.Scope
// AttributeValueLengthLimit is the maximum allowed attribute value length.
//
// This limit only applies to string and string slice attribute values.
// Any string longer than this value should be truncated to this length.
//
// Negative value means no limit should be applied.
func (r *Record) AttributeValueLengthLimit() int
// AttributeCountLimit is the maximum allowed log record attribute count. Any
// attribute added to a log record once this limit is reached should be dropped.
//
// Zero means no attributes should be recorded.
//
// Negative value means no limit should be applied.
func (r *Record) AttributeCountLimit() int
// Clone returns a copy of the record with no shared state. The original record
// and the clone can both be modified without interfering with each other.
func (r *Record) Clone() Record
```
The `Record` is designed similarly to [`log.Record`](https://pkg.go.dev/go.opentelemetry.io/otel/log#Record) The `Record` is designed similarly to [`log.Record`](https://pkg.go.dev/go.opentelemetry.io/otel/log#Record)
in order to reduce the number of heap allocations when processing attributes. in order to reduce the number of heap allocations when processing attributes.
@ -372,7 +91,7 @@ Having less abstractions reduces the API surface and makes the design simpler.
## Benchmarking ## Benchmarking
The benchmarks are supposed to test end-to-end scenarios The benchmarks are supposed to test end-to-end scenarios
and avoid I/O that could affect the stability of the results, and avoid I/O that could affect the stability of the results.
The benchmark results can be found in [the prototype](https://github.com/open-telemetry/opentelemetry-go/pull/4955). The benchmark results can be found in [the prototype](https://github.com/open-telemetry/opentelemetry-go/pull/4955).