### Added
- Add `go.opentelemetry.io/otel/sdk/metric/exemplar.AlwaysOffFilter`,
which can be used to disable exemplar recording. (#5850)
- Add `go.opentelemetry.io/otel/sdk/metric.WithExemplarFilter`, which
can be used to configure the exemplar filter used by the metrics SDK.
(#5850)
- Add `ExemplarReservoirProviderSelector` and
`DefaultExemplarReservoirProviderSelector` to
`go.opentelemetry.io/otel/sdk/metric`, which defines the exemplar
reservoir to use based on the aggregation of the metric. (#5861)
- Add `ExemplarReservoirProviderSelector` to
`go.opentelemetry.io/otel/sdk/metric.Stream` to allow using views to
configure the exemplar reservoir to use for a metric. (#5861)
- Add `ReservoirProvider`, `HistogramReservoirProvider` and
`FixedSizeReservoirProvider` to
`go.opentelemetry.io/otel/sdk/metric/exemplar` to make it convenient to
use providers of Reservoirs. (#5861)
- The `go.opentelemetry.io/otel/semconv/v1.27.0` package.
The package contains semantic conventions from the `v1.27.0` version of
the OpenTelemetry Semantic Conventions. (#5894)
- Add `Attributes attribute.Set` field to `Scope` in
`go.opentelemetry.io/otel/sdk/instrumentation`. (#5903)
- Add `Attributes attribute.Set` field to `ScopeRecords` in
`go.opentelemetry.io/otel/log/logtest`. (#5927)
- `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc` adds
instrumentation scope attributes. (#5934)
- `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp` adds
instrumentation scope attributes. (#5934)
- `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`
adds instrumentation scope attributes. (#5935)
- `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`
adds instrumentation scope attributes. (#5935)
- `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc` adds
instrumentation scope attributes. (#5933)
- `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp` adds
instrumentation scope attributes. (#5933)
- `go.opentelemetry.io/otel/exporters/prometheus` adds instrumentation
scope attributes in `otel_scope_info` metric as labels. (#5932)
### Changed
- Support scope attributes and make them as identifying for `Tracer` in
`go.opentelemetry.io/otel` and `go.opentelemetry.io/otel/sdk/trace`.
(#5924)
- Support scope attributes and make them as identifying for `Meter` in
`go.opentelemetry.io/otel` and `go.opentelemetry.io/otel/sdk/metric`.
(#5926)
- Support scope attributes and make them as identifying for `Logger` in
`go.opentelemetry.io/otel` and `go.opentelemetry.io/otel/sdk/log`.
(#5925)
- Make schema URL and scope attributes as identifying for `Tracer` in
`go.opentelemetry.io/otel/bridge/opentracing`. (#5931)
- Clear unneeded slice elements to allow GC to collect the objects in
`go.opentelemetry.io/otel/sdk/metric` and
`go.opentelemetry.io/otel/sdk/trace`. (#5804)
### Fixed
- Global MeterProvider registration unwraps global instrument Observers,
the undocumented Unwrap() methods are now private. (#5881)
- `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`
now keeps the metadata already present in the context when `WithHeaders`
is used. (#5892)
- `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc` now
keeps the metadata already present in the context when `WithHeaders` is
used. (#5911)
- `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc` now
keeps the metadata already present in the context when `WithHeaders` is
used. (#5915)
- Fix `go.opentelemetry.io/otel/exporters/prometheus` trying to add
exemplars to Gauge metrics, which is unsupported. (#5912)
- Fix `WithEndpointURL` to always use a secure connection when an https
URL is passed in
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`.
(#5944)
- Fix `WithEndpointURL` to always use a secure connection when an https
URL is passed in
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`.
(#5944)
- Fix `WithEndpointURL` to always use a secure connection when an https
URL is passed in
`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`.
(#5944)
- Fix `WithEndpointURL` to always use a secure connection when an https
URL is passed in
`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`.
(#5944)
- Fix incorrect metrics generated from callbacks when multiple readers
are used in `go.opentelemetry.io/otel/sdk/metric`. (#5900)
### Removed
- Remove all examples under `go.opentelemetry.io/otel/example` as they
are moved to [Contrib
repository](https://github.com/open-telemetry/opentelemetry-go-contrib/tree/main/examples).
(#5930)
```go
type interInst struct {
x int
}
type inter interface {
}
var sink []inter
func BenchmarkX(b *testing.B) {
sink = make([]inter, b.N)
for i := 0; i < b.N; i++ {
sink[i] = &interInst{}
}
clear(sink)
sink = sink[:0]
runtime.GC()
var ms runtime.MemStats
runtime.ReadMemStats(&ms)
b.Log(b.N, ms.Frees) // Frees is the cumulative count of heap objects freed.
}
```
```
clear:
ioz_test.go:35: 1 589
ioz_test.go:35: 100 711
ioz_test.go:35: 10000 10729
ioz_test.go:35: 1000000 1010750 <-- 1m+ freed
ioz_test.go:35: 16076874 17087643
ioz_test.go:35: 19514749 36602412
```
```
no clear:
ioz_test.go:35: 1 585
ioz_test.go:35: 100 606
ioz_test.go:35: 10000 725
ioz_test.go:35: 1000000 10745 <-- some "overhead" objects freed, not the slice.
ioz_test.go:35: 16391445 1010765
ioz_test.go:35: 21765238 17402230
```
This is documented at https://go.dev/wiki/SliceTricks:
> NOTE If the type of the element is a pointer or a struct with pointer
fields, which need to be garbage collected, the above implementations of
Cut and Delete have a potential memory leak problem: some elements with
values are still referenced by slice a’s underlying array, just not
“visible” in the slice. Because the “deleted” value is referenced in the
underlying array, the deleted value is still “reachable” during GC, even
though the value cannot be referenced by your code. If the underlying
array is long-lived, this represents a leak.
Followed by examples of how zeroing out the slice elements solves
the problem. This PR does the same.
### Added
- Add `go.opentelemetry.io/otel/sdk/metric/exemplar` package which
includes `Exemplar`, `Filter`, `TraceBasedFilter`, `AlwaysOnFilter`,
`HistogramReservoir`, `FixedSizeReservoir`, `Reservoir`, `Value` and
`ValueType` types. These will be used for configuring the exemplar
reservoir for the metrics sdk. (#5747, #5862)
- Add `WithExportBufferSize` option to log batch processor.(#5877)
### Changed
- Enable exemplars by default in `go.opentelemetry.io/otel/sdk/metric`.
Exemplars can be disabled by setting
`OTEL_METRICS_EXEMPLAR_FILTER=always_off` (#5778)
- `Logger.Enabled` in `go.opentelemetry.io/otel/log` now accepts a newly
introduced `EnabledParameters` type instead of `Record`. (#5791)
- `FilterProcessor.Enabled` in
`go.opentelemetry.io/otel/sdk/log/internal/x` now accepts
`EnabledParameters` instead of `Record`. (#5791)
- The `Record` type in `go.opentelemetry.io/otel/log` is no longer
comparable. (#5847)
- Performance improvements for the trace SDK `SetAttributes` method in
`Span`. (#5864)
- Reduce memory allocations for the `Event` and `Link` lists in `Span`.
(#5858)
- Performance improvements for the trace SDK `AddEvent`, `AddLink`,
`RecordError` and `End` methods in `Span`. (#5874)
### Deprecated
- Deprecate all examples under `go.opentelemetry.io/otel/example` as
they are moved to [Contrib
repository](https://github.com/open-telemetry/opentelemetry-go-contrib/tree/main/examples).
(#5854)
### Fixed
- The race condition for multiple `FixedSize` exemplar reservoirs
identified in #5814 is resolved. (#5819)
- Fix log records duplication in case of heterogeneous resource
attributes by correctly mapping each log record to it's resource and
scope. (#5803)
- Fix timer channel drain to avoid hanging on Go 1.23. (#5868)
- Fix delegation for global meter providers, and panic when calling
otel.SetMeterProvider. (#5827)
- Change the `reflect.TypeOf` to use a nil pointer to not allocate on
the heap unless necessary. (#5827)
### Added
- Support `OTEL_EXPORTER_OTLP_LOGS_INSECURE` and
`OTEL_EXPORTER_OTLP_INSECURE` environments in
`go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`. (#5739)
- The `WithResource` option for `NewMeterProvider` now merges the
provided resources with the ones from environment variables. (#5773)
- The `WithResource` option for `NewLoggerProvider` now merges the
provided resources with the ones from environment variables. (#5773)
- Add UTF-8 support to `go.opentelemetry.io/otel/exporters/prometheus`.
(#5755)
### Fixed
- Fix memory leak in the global `MeterProvider` when identical
instruments are repeatedly created. (#5754)
- Fix panic instruments creation when setting meter provider. (#5758)
- Fix panic on instruments creation when setting meter provider. (#5758)
- Fix an issue where `SetMeterProvider` in `go.opentelemetry.io/otel`
might miss the delegation for instruments and registries. (#5780)
### Removed
- Drop support for [Go 1.21](https://go.dev/doc/go1.21). (#5736, #5740,
#5800)
This release is the last to support [Go 1.21]. The next release will
require at least [Go 1.22].
### Added
- Add MacOS ARM64 platform to the compatibility testing suite. (#5577)
- Add `InstrumentationScope` field to `SpanStub` in
`go.opentelemetry.io/otel/sdk/trace/tracetest`, as a replacement for the
deprecated `InstrumentationLibrary`. (#5627)
- Make the initial release of
`go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc`. This new
module contains an OTLP exporter that transmits log telemetry using
gRPC. This module is unstable and breaking changes may be introduced.
See our [versioning policy](VERSIONING.md) for more information about
these stability guarantees. (#5629)
- Add `Walk` function to `TraceState` in
`go.opentelemetry.io/otel/trace` to iterate all the key-value pairs.
(#5651)
- Bridge the trace state in
`go.opentelemetry.io/otel/bridge/opencensus`. (#5651)
- Zero value of `SimpleProcessor` in `go.opentelemetry.io/otel/sdk/log`
no longer panics. (#5665)
- The `FilterProcessor` interface type is added in
`go.opentelemetry.io/otel/sdk/log/internal/x`. This is an optional and
experimental interface that log `Processor`s can implement to instruct
the `Logger` if a `Record` will be processed or not. It replaces the
existing `Enabled` method that is removed from the `Processor` interface
itself. It does not fall within the scope of the OpenTelemetry Go
versioning and stability [policy](./VERSIONING.md) and it may be changed
in backwards incompatible ways or removed in feature releases. (#5692)
- Support [Go 1.23]. (#5720)
### Changed
- `NewMemberRaw`, `NewKeyProperty` and `NewKeyValuePropertyRaw` in
`go.opentelemetry.io/otel/baggage` allow UTF-8 string in key. (#5132)
- `Processor.OnEmit` in `go.opentelemetry.io/otel/sdk/log` now accepts a
pointer to `Record` instead of a value so that the record modifications
done in a processor are propagated to subsequent registered processors.
(#5636)
- `SimpleProcessor.Enabled` in `go.opentelemetry.io/otel/sdk/log` now
returns `false` if the exporter is `nil`. (#5665)
- Update the concurrency requirements of `Exporter` in
`go.opentelemetry.io/otel/sdk/log`. (#5666)
- `SimpleProcessor` in `go.opentelemetry.io/otel/sdk/log` synchronizes
`OnEmit` calls. (#5666)
- The `Processor` interface in `go.opentelemetry.io/otel/sdk/log` no
longer includes the `Enabled` method. See the `FilterProcessor`
interface type added in `go.opentelemetry.io/otel/sdk/log/internal/x` to
continue providing this functionality. (#5692)
- The `SimpleProcessor` type in `go.opentelemetry.io/otel/sdk/log` is no
longer comparable. (#5693)
- The `BatchProcessor` type in `go.opentelemetry.io/otel/sdk/log` is no
longer comparable. (#5693)
### Fixed
- Correct comments for the priority of the `WithEndpoint` and
`WithEndpointURL` options and their corresponding environment variables
in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`.
(#5584)
- Pass the underlying error rather than a generic retry-able failure in
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`,
`go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp` and
`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`.
(#5541)
- Correct the `Tracer`, `Meter`, and `Logger` names used in
`go.opentelemetry.io/otel/example/dice`. (#5612)
- Correct the `Tracer` names used in
`go.opentelemetry.io/otel/example/namedtracer`. (#5612)
- Correct the `Tracer` name used in
`go.opentelemetry.io/otel/example/opencensus`. (#5612)
- Correct the `Tracer` and `Meter` names used in
`go.opentelemetry.io/otel/example/otel-collector`. (#5612)
- Correct the `Tracer` names used in
`go.opentelemetry.io/otel/example/passthrough`. (#5612)
- Correct the `Meter` name used in
`go.opentelemetry.io/otel/example/prometheus`. (#5612)
- Correct the `Tracer` names used in
`go.opentelemetry.io/otel/example/zipkin`. (#5612)
- Correct comments for the priority of the `WithEndpoint` and
`WithEndpointURL` options and their corresponding environment variables
in `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc`
and `go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`.
(#5641)
- Correct comments for the priority of the `WithEndpoint` and
`WithEndpointURL` options and their corresponding environment variables
in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`.
(#5650)
- Stop percent encoding header environment variables in
`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`,
`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`,
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc` and
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`
(#5705)
- Remove invalid environment variable header keys in
`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`,
`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`,
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc` and
`go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp`
(#5705)
### Removed
- The `Enabled` method of the `SimpleProcessor` in
`go.opentelemetry.io/otel/sdk/log` is removed. (#5692)
- The `Enabled` method of the `BatchProcessor` in
`go.opentelemetry.io/otel/sdk/log` is removed. (#5692)
[Go 1.23]: https://go.dev/doc/go1.23
[Go 1.22]: https://go.dev/doc/go1.22
[Go 1.21]: https://go.dev/doc/go1.21
Closes#5425
Our current log `Processor` interface contains more functionality than
the [OTel
spec](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/sdk.md#logrecordprocessor-operations).
The additional functionality allows processors to report back to the API
if a Record should be constructed and emitted or not, which is quite
helpful[^1][^2][^3][^4][^5].
This removes the `Enabled` method from the `Processor` type. It adds
this functionality a new optional and experimental `FilterProcessor`
interface type. The logger and provider are updated to check for this
optional interface to be implemented with the configured processors and
uses them to back the `Logger.Enabled` method, preserving existing
functionality.
By making this change:
- The `Processor` interface is now compliant with the OTel spec and does
not contain any additional unspecified behavior.
- All `Processor` implementations are no longer required to implement an
`Enabled` method. The default, when they do not implement this method,
is to assume they are enabled.
### Benchmark
```terminal
goos: linux
goarch: amd64
pkg: go.opentelemetry.io/otel/sdk/log
cpu: Intel(R) Core(TM) i7-8550U CPU @ 1.80GHz
│ old.txt │ new7.txt │
│ sec/op │ sec/op vs base │
LoggerEnabled-8 133.30n ± 3% 32.36n ± 3% -75.72% (p=0.000 n=10)
│ old.txt │ new7.txt │
│ B/op │ B/op vs base │
LoggerEnabled-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
¹ all samples are equal
│ old.txt │ new7.txt │
│ allocs/op │ allocs/op vs base │
LoggerEnabled-8 0.000 ± 0% 0.000 ± 0% ~ (p=1.000 n=10) ¹
¹ all samples are equal
```
This is a significant performance improvement due to the `Record` no
longer being converted from the API version to the SDK version.
[^1]: https://pkg.go.dev/go.opentelemetry.io/contrib/processors/minsev
[^2]:
https://pkg.go.dev/go.opentelemetry.io/otel/sdk/log#BatchProcessor.Enabled
[^3]:
https://pkg.go.dev/go.opentelemetry.io/otel/sdk/log#SimpleProcessor.Enabled
[^4]:
af75717ac4/bridges/otelslog/handler.go (L206-L211)
[^5]:
d0309ddd8c/bridges/otelzap/core.go (L142-L146)
---------
Co-authored-by: Robert Pająk <pellared@hotmail.com>
Co-authored-by: Sam Xie <sam@samxie.me>
Ensure the `Record` and `Provider` continue to be non-comparable.
Restrict `BatchProcessor` and `SimpleProcessor` to be non-comparable.
There is no obvious reason an end-user will need to compare these types,
and we want to keep the possibility of changing the internals without
changing this behavior.
if comparability is required by end-users in the future we can add
`Equal(other T)` methods in the future.
From
https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/sdk.md#export:
> `Export` will never be called concurrently for the same exporter
instance.
From our SDK perspective this change will make
https://pkg.go.dev/go.opentelemetry.io/otel/exporters/stdout/stdoutlog
concurrent-safe when used with the simple processor. Before the change,
there can be multiple goroutines calling `Write` in parallel (via
`json.Encoder.Encode`).
### Remarks
[Maybe we should simply state that "whole" exporter implementation does
not need to be
concurrent-safe?](https://github.com/open-telemetry/opentelemetry-specification/pull/4173#discussion_r1699958752)
However:
1. we would need to make complex changes in `BatchExporter` to
synchronize the `Export`, `Shutdown`, `ForceFlush` calls
2. we would need to update all exporters (remove synchronization) and
simple exporter (add locks to `Shutdown`, `ForceFlush`)
3. I am 100% not sure if this would be compliant with the specification
- I think it would be complaint because we would simply give stronger
safety-measures
We should probably discuss it separately, but I wanted to highlight my
though process.
Even if we decide that simple and batch processors to synchronize all
calls then I would prefer to address it in a separate PR.
Related spec clarification PR:
-
https://github.com/open-telemetry/opentelemetry-specification/pull/4173
Add a benchmark that shows that a processor that adds an attribute may
not introduce a heap allocation (as long as the log record still has <=
5 attributes).
### Added
- The `IsEmpty` method is added to the `Instrument` type in
`go.opentelemetry.io/otel/sdk/metric`.
This method is used to check if an `Instrument` instance is a
zero-value. (#5431)
- Store and provide the emitted `context.Context` in `ScopeRecords` of
`go.opentelemetry.io/otel/sdk/log/logtest`. (#5468)
- The `go.opentelemetry.io/otel/semconv/v1.26.0` package.
The package contains semantic conventions from the `v1.26.0` version of
the OpenTelemetry Semantic Conventions. (#5476)
- The `AssertRecordEqual` method to
`go.opentelemetry.io/otel/log/logtest` to allow comparison of two log
records in tests. (#5499)
- The `WithHeaders` option to
`go.opentelemetry.io/otel/exporters/zipkin` to allow configuring custom
http headers while exporting spans. (#5530)
### Changed
- `Tracer.Start` in `go.opentelemetry.io/otel/trace/noop` no longer
allocates a span for empty span context. (#5457)
- Upgrade `go.opentelemetry.io/otel/semconv/v1.25.0` to
`go.opentelemetry.io/otel/semconv/v1.26.0` in
`go.opentelemetry.io/otel/example/otel-collector`. (#5490)
- Upgrade `go.opentelemetry.io/otel/semconv/v1.25.0` to
`go.opentelemetry.io/otel/semconv/v1.26.0` in
`go.opentelemetry.io/otel/example/zipkin`. (#5490)
- Upgrade `go.opentelemetry.io/otel/semconv/v1.25.0` to
`go.opentelemetry.io/otel/semconv/v1.26.0` in
`go.opentelemetry.io/otel/exporters/zipkin`. (#5490)
- The exporter no longer exports the deprecated "otel.library.name" or
"otel.library.version" attributes.
- Upgrade `go.opentelemetry.io/otel/semconv/v1.25.0` to
`go.opentelemetry.io/otel/semconv/v1.26.0` in
`go.opentelemetry.io/otel/sdk/resource`. (#5490)
- Upgrade `go.opentelemetry.io/otel/semconv/v1.25.0` to
`go.opentelemetry.io/otel/semconv/v1.26.0` in
`go.opentelemetry.io/otel/sdk/trace`. (#5490)
- `SimpleProcessor.OnEmit` in `go.opentelemetry.io/otel/sdk/log` no
longer allocates a slice which makes it possible to have a
zero-allocation log processing using `SimpleProcessor`. (#5493)
- Use non-generic functions in the `Start` method of
`"go.opentelemetry.io/otel/sdk/trace".Trace` to reduce memory
allocation. (#5497)
- `service.instance.id` is populated for a `Resource` created with
`"go.opentelemetry.io/otel/sdk/resource".Default` with a default value
when `OTEL_GO_X_RESOURCE` is set. (#5520)
- Improve performance of metric instruments in
`go.opentelemetry.io/otel/sdk/metric` by removing unnecessary calls to
`time.Now`. (#5545)
### Fixed
- Log a warning to the OpenTelemetry internal logger when a `Record` in
`go.opentelemetry.io/otel/sdk/log` drops an attribute due to a limit
being reached. (#5376)
- Identify the `Tracer` returned from the global `TracerProvider` in
`go.opentelemetry.io/otel/global` with its schema URL. (#5426)
- Identify the `Meter` returned from the global `MeterProvider` in
`go.opentelemetry.io/otel/global` with its schema URL. (#5426)
- Log a warning to the OpenTelemetry internal logger when a `Span` in
`go.opentelemetry.io/otel/sdk/trace` drops an attribute, event, or link
due to a limit being reached. (#5434)
- Document instrument name requirements in
`go.opentelemetry.io/otel/metric`. (#5435)
- Prevent random number generation data-race for experimental rand
exemplars in `go.opentelemetry.io/otel/sdk/metric`. (#5456)
- Fix counting number of dropped attributes of `Record` in
`go.opentelemetry.io/otel/sdk/log`. (#5464)
- Fix panic in baggage creation when a member contains `0x80` char in
key or value. (#5494)
- Correct comments for the priority of the `WithEndpoint` and
`WithEndpointURL` options and their corresponding environment variables
in `go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc`.
(#5508)
- Retry trace and span ID generation if it generated an invalid one in
`go.opentelemetry.io/otel/sdk/trace`. (#5514)
- Fix stale timestamps reported by the last-value aggregation. (#5517)
- Indicate the `Exporter` in
`go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp` must be
created by the `New` method. (#5521)
- Improved performance in all `{Bool,Int64,Float64,String}SliceValue`
functions of `go.opentelemetry.io/attributes` by reducing the number of
allocations. (#5549)
Towards https://github.com/open-telemetry/opentelemetry-go/issues/5065
Follow our own docs. From `Processor.Enabled` docs:
> Before modifying a Record, the implementation must use Record.Clone to
create a copy that shares no state with the original.
This adds a benchmark to create a logger from a logger provider.
Related: #5054.
```
BenchmarkLoggerProviderLogger-10 3145390 548.8 ns/op 330 B/op 1 allocs/op
```
This benchmark currently tests two rather different methods within the
same loop, which makes it hard to see what could be causing a
performance degradation.
Related: #5054.
```
BenchmarkSetAddAttributes/SetAttributes-10 14066331 82.80 ns/op 48 B/op 1 allocs/op
BenchmarkSetAddAttributes/AddAttributes-10 19333711 114.7 ns/op 0 B/op 0 allocs/op
```
As discussed in a previous SIG meeting, this PR adds support for setting
a default value for
[`service.instance.id`](https://github.com/open-telemetry/semantic-conventions/tree/main/docs/resource#service-experimental)
according to semantic conventions:
> Implementations, such as SDKs, are recommended to generate a random
Version 1 or Version 4 [RFC 4122](https://www.ietf.org/rfc/rfc4122.txt)
UUID, but are free to use an inherent unique ID as the source of this
value if stability is desirable. In that case, the ID SHOULD be used as
source of a UUID Version 5 and SHOULD use the following UUID as the
namespace: `4d63009a-8d0f-11ee-aad7-4c796ed8e320`.
This PR follows the recommendation and populates `service.instance.id`
with a random Version 4 UUID. The functionality is guarded by the
`OTEL_GO_X_RESOURCE` feature flag environment variable.
There are plans to declare `service.instance.id` stable and also make it
a required attribute (similar to `service.name`). Once this happens, the
functionality can be made available regardless of whether
`OTEL_GO_X_RESOURCE` is set.
Closes
https://github.com/open-telemetry/opentelemetry-go-contrib/issues/5423
---------
Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
Co-authored-by: Tyler Yahn <codingalias@gmail.com>
The reason for this improvement (apart that, in general, it is good to
have better performance) is there may be good use case to use a
`SimpleProcessor` to emit logs efficiently to standard output. It could
be one of the most efficient solutions (from application performance
perspective) and thanks to such configuration the user would not lose
any logs if the application suddenly crashes. For instance, a useful
configuration could be a simple processor with an OTLP file exporter
(https://github.com/open-telemetry/opentelemetry-go/issues/5408).
I think we might consider changing the following portion of
`SimpleProcessor` documentation (but I would prefer to do it as separate
PR):
> // This Processor is not recommended for production use. The
synchronous
// nature of this Processor make it good for testing, debugging, or
// showing examples of other features, but it can be slow and have a
high
// computation resource usage overhead. [NewBatchProcessor] is
recommended
// for production use instead.
```
goos: linux
goarch: amd64
pkg: go.opentelemetry.io/otel/sdk/log
cpu: Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
Processor/Simple-16 449.4n ± 7% 156.2n ± 5% -65.25% (p=0.000 n=10)
Processor/ModifyTimestampSimple-16 468.0n ± 6% 171.3n ± 15% -63.40% (p=0.000 n=10)
Processor/ModifyAttributesSimple-16 515.8n ± 3% 233.2n ± 8% -54.77% (p=0.000 n=10)
geomean 476.9n 184.1n -61.40%
│ old.txt │ new.txt │
│ B/op │ B/op vs base │
Processor/Simple-16 417.0 ± 0% 0.0 ± 0% -100.00% (p=0.000 n=10)
Processor/ModifyTimestampSimple-16 417.0 ± 0% 0.0 ± 0% -100.00% (p=0.000 n=10)
Processor/ModifyAttributesSimple-16 465.00 ± 0% 48.00 ± 0% -89.68% (p=0.000 n=10)
geomean 432.4 ? ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean
│ old.txt │ new.txt │
│ allocs/op │ allocs/op vs base │
Processor/Simple-16 1.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10)
Processor/ModifyTimestampSimple-16 1.000 ± 0% 0.000 ± 0% -100.00% (p=0.000 n=10)
Processor/ModifyAttributesSimple-16 2.000 ± 0% 1.000 ± 0% -50.00% (p=0.000 n=10)
geomean 1.260 ? ¹ ²
¹ summaries must be >0 to compute geomean
² ratios must be >0 to compute geomean
```
---------
Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
Applying attribute limits in `Record` uses value receiver.
But it should add count of dropped attrs.
In this PR I add using of pointer receiver.
Also it's slightly faster with pointer receiver:
```
goos: darwin
goarch: arm64
pkg: go.opentelemetry.io/otel/sdk/log
│ old.txt │ new.txt │
│ sec/op │ sec/op vs base │
SetAddAttributes-10 175.7n ± 3% 159.8n ± 4% -9.08% (p=0.000 n=10)
│ old.txt │ new.txt │
│ B/op │ B/op vs base │
SetAddAttributes-10 48.00 ± 0% 48.00 ± 0% ~ (p=1.000 n=10) ¹
¹ all samples are equal
│ old.txt │ new.txt │
│ allocs/op │ allocs/op vs base │
SetAddAttributes-10 1.000 ± 0% 1.000 ± 0% ~ (p=1.000 n=10) ¹
¹ all samples are equal
```
Towards https://github.com/open-telemetry/opentelemetry-go/issues/5219
Towards https://github.com/open-telemetry/opentelemetry-go/issues/5054
This benchmarks are supposed
- Validate the the `Processor` interface design from performance
perspective. E.g. they are used to check if a processor that is
modifying a log record is causing an additional heap allocations.
- Benchmark the processors supported by the SDK.
These are "almost-end-to-end" benchmarks (with noopExporter) so that it
checks the performance of the SDK log processing without the actual
exporting part.
```
cpu: Intel(R) Core(TM) i9-10885H CPU @ 2.40GHz
BenchmarkProcessor/Simple-16 1990946 644.6 ns/op 417 B/op 1 allocs/op
BenchmarkProcessor/Batch-16 835135 1211 ns/op 597 B/op 0 allocs/op
BenchmarkProcessor/ModifyTimestampSimple-16 1782510 644.3 ns/op 417 B/op 1 allocs/op
BenchmarkProcessor/ModifyTimestampBatch-16 945699 1222 ns/op 637 B/op 0 allocs/op
BenchmarkProcessor/ModifyAttributesSimple-16 1570214 717.6 ns/op 465 B/op 2 allocs/op
BenchmarkProcessor/ModifyAttributesBatch-16 768399 1383 ns/op 653 B/op 1 allocs/op
```
Fix#5342
Run of `go test -v -timeout 10s -short -count=10000 ./...
-run='TestBatchProcessor/ForceFlush/ErrorPartialFlush'`
**Before**:
Failed with `panic: test timed out after 10s`
**After**:
Passed
Problem was with `bufferExporter.input` chan.
In test expected:
> 1 export being performed, 1 export in buffer chan, >1 batch
But buffer chan wasn't checked. Now we ensure, that record is in buffer
chan
Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
Fix#5384
Run of `go test -count=1000000 -run="TestBatchProcessor/DroppedLogs"`
**Before**:
Failed with either `Condition never satisfied` or panic
**After**:
Passed
First, bytes.Buffer is not thread-safe, so writing log and reading
(`bytes.String()`) caused panic. Added `concurrentBuffer`
Second, fixed flaky test with 4 records:
1. Record goes to `testExporter.Export` function and blocks in this
function because of `ExportTrigger`
(19ee6d4775/sdk/log/exporter_test.go (L87))
2. Record goes to `bufferExporter.input`
(7c5e64cccc/sdk/log/exporter.go (L129))
3. Record goes to `BatchProcessor.q` queue and it could not be enqueued
to export, because `bufferExporter.input` is full
4. Record goes to `BatchProcessor.q` and because of overfill, drops
third record
---------
Co-authored-by: Sam Xie <sam@samxie.me>
Fix#5317
According to the
[specification](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/sdk.md#logrecord-limits),
there should be a message printed in the SDK's log to indicate to the
user that an attribute was discarded due to such a limit. To prevent
excessive logging, the message must be printed at most once per
`LogRecord` (i.e., not per discarded attribute).
This change centralizes the `Record` dropped field writes and calls a
global logging function. This will at most log once per any `Record`
dropping attributes, meeting the specification requirement.
This does not log once per `Record` when an attribute is dropped. To do
that we would need to maintain state within the `Record` (i.e.
`sync.Mutex` or `sync.Once`). These types cannot be copied, meaning the
`Record` would take on this "no copy" requirement. This seems too
restrictive and with the permissive specification allowing a single log
line, that is the solution added.
---------
Co-authored-by: Robert Pająk <pellared@hotmail.com>
Leftover after
https://github.com/open-telemetry/opentelemetry-go/pull/5230
We want to have the benchmarks working with 5+ attributes as this is
when allocations kick in.
Before changes:
```
BenchmarkLoggerNewRecord/5_attributes-16 4016042 309.0 ns/op 0 B/op 0 allocs/op
BenchmarkLoggerNewRecord/10_attributes-16 2150197 543.4 ns/op 0 B/op 0 allocs/op
```
After changes:
```
BenchmarkLoggerNewRecord/5_attributes-16 3779966 311.1 ns/op 0 B/op 0 allocs/op
BenchmarkLoggerNewRecord/10_attributes-16 1000000 1314 ns/op 610 B/op 4 allocs/op
```
