Fix#5317
According to the
[specification](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/logs/sdk.md#logrecord-limits),
there should be a message printed in the SDK's log to indicate to the
user that an attribute was discarded due to such a limit. To prevent
excessive logging, the message must be printed at most once per
`LogRecord` (i.e., not per discarded attribute).
This change centralizes the `Record` dropped field writes and calls a
global logging function. This will at most log once per any `Record`
dropping attributes, meeting the specification requirement.
This does not log once per `Record` when an attribute is dropped. To do
that we would need to maintain state within the `Record` (i.e.
`sync.Mutex` or `sync.Once`). These types cannot be copied, meaning the
`Record` would take on this "no copy" requirement. This seems too
restrictive and with the permissive specification allowing a single log
line, that is the solution added.
---------
Co-authored-by: Robert Pająk <pellared@hotmail.com>
Leftover after
https://github.com/open-telemetry/opentelemetry-go/pull/5230
We want to have the benchmarks working with 5+ attributes as this is
when allocations kick in.
Before changes:
```
BenchmarkLoggerNewRecord/5_attributes-16 4016042 309.0 ns/op 0 B/op 0 allocs/op
BenchmarkLoggerNewRecord/10_attributes-16 2150197 543.4 ns/op 0 B/op 0 allocs/op
```
After changes:
```
BenchmarkLoggerNewRecord/5_attributes-16 3779966 311.1 ns/op 0 B/op 0 allocs/op
BenchmarkLoggerNewRecord/10_attributes-16 1000000 1314 ns/op 610 B/op 4 allocs/op
```
Fix#5384
On single-threaded, or in general slow, systems where the log message
may not be emitted, use assert.Everything to allow multiple lookups for
the expected log message.
Co-authored-by: Sam Xie <sam@samxie.me>
This release includes the first beta release of the OpenTelemetry Logs
Bridge API and SDK for Go.
### Added
- Add example for `go.opentelemetry.io/otel/exporters/stdout/stdoutlog`.
(#5242)
- Add `RecordFactory` in `go.opentelemetry.io/otel/sdk/log/logtest` to
facilitate testing exporter and processor implementations. (#5258)
- Add `RecordFactory` in `go.opentelemetry.io/otel/log/logtest` to
facilitate testing bridge implementations. (#5263)
- The count of dropped records from the `BatchProcessor` in
`go.opentelemetry.io/otel/sdk/log` is logged. (#5276)
- Add metrics in the `otel-collector` example. (#5283)
- Add the synchronous gauge instrument to
`go.opentelemetry.io/otel/metric`. (#5304)
- An `int64` or `float64` synchronous gauge instrument can now be
created from a `Meter`.
- All implementations of the API
(`go.opentelemetry.io/otel/metric/noop`,
`go.opentelemetry.io/otel/sdk/metric`) are updated to support this
instrument.
- Add logs to `go.opentelemetry.io/otel/example/dice`. (#5349)
### Changed
- The `Shutdown` method of `Exporter` in
`go.opentelemetry.io/otel/exporters/stdout/stdouttrace` ignores the
context cancellation and always returns `nil`. (#5189)
- The `ForceFlush` and `Shutdown` methods of the exporter returned by
`New` in `go.opentelemetry.io/otel/exporters/stdout/stdoutmetric` ignore
the context cancellation and always return `nil`. (#5189)
- Apply the value length limits to `Record` attributes in
`go.opentelemetry.io/otel/sdk/log`. (#5230)
- De-duplicate map attributes added to a `Record` in
`go.opentelemetry.io/otel/sdk/log`. (#5230)
- `go.opentelemetry.io/otel/exporters/stdout/stdoutlog` won't print
timestamps when `WithoutTimestamps` option is set. (#5241)
- The `go.opentelemetry.io/otel/exporters/stdout/stdoutlog` exporter
won't print `AttributeValueLengthLimit` and `AttributeCountLimit` fields
now, instead it prints the `DroppedAttributes` field. (#5272)
- Improved performance in the `Stringer` implementation of
`go.opentelemetry.io/otel/baggage.Member` by reducing the number of
allocations. (#5286)
- Set the start time for last-value aggregates in
`go.opentelemetry.io/otel/sdk/metric`. (#5305)
- The `Span` in `go.opentelemetry.io/otel/sdk/trace` will record links
without span context if either non-empty `TraceState` or attributes are
provided. (#5315)
- Upgrade all dependencies of `go.opentelemetry.io/otel/semconv/v1.24.0`
to `go.opentelemetry.io/otel/semconv/v1.25.0`. (#5374)
### Fixed
- Comparison of unordered maps for
`go.opentelemetry.io/otel/log.KeyValue` and
`go.opentelemetry.io/otel/log.Value`. (#5306)
- Fix the empty output of `go.opentelemetry.io/otel/log.Value` in
`go.opentelemetry.io/otel/exporters/stdout/stdoutlog`. (#5311)
- Split the behavior of `Recorder` in
`go.opentelemetry.io/otel/log/logtest` so it behaves as a
`LoggerProvider` only. (#5365)
- Fix wrong package name of the error message when parsing endpoint URL
in `go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp`.
(#5371)
- Identify the `Logger` returned from the global `LoggerProvider` in
`go.opentelemetry.io/otel/log/global` with its schema URL. (#5375)
#5304 introduced the following incompatible changes:
- `InstrumentKindObservableCounter`: value changed from 4 to 5
- `InstrumentKindObservableGauge`: value changed from 6 to 7
- `InstrumentKindObservableUpDownCounter`: value changed from 5 to 6
This reverts that change, making `InstrumentKindGauge` explicitly `7`.
Additionally, this removes the use of `iota` to prevent this kind of
breaking change from being accidentally introduced in the future.
* record links with empty span context
* add global trace state
* fix test comments and changelog
---------
Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
* Refactor exemplars to not use generic argument
* Update internal/aggregate
* Update metric SDK
* Test exemplar value type
* Add TestCollectExemplars
* Fix lint
---------
Co-authored-by: Sam Xie <sam@samxie.me>
* emit slices as their json representation
* add changelog
* fix resource tests
* indicate invalid slice if we couldn't turn them into json
* move changelog entry to the unreleased section
---------
Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
build(deps): bump google.golang.org/protobuf from 1.33.0 to 1.34.0 in /exporters/otlp/otlptrace/otlptracehttp
build(deps): bump codecov/codecov-action from 4.3.0 to 4.3.1
build(deps): bump google.golang.org/protobuf from 1.33.0 to 1.34.0 in /exporters/otlp/otlptrace
build(deps): bump google.golang.org/protobuf from 1.33.0 to 1.34.0 in /exporters/otlp/otlptrace/otlptracegrpc
build(deps): bump google.golang.org/protobuf from 1.33.0 to 1.34.0 in /exporters/prometheus
build(deps): bump golang.org/x/sys from 0.19.0 to 0.20.0 in /sdk
build(deps): bump github.com/golangci/golangci-lint from 1.57.2 to 1.58.0 in /internal/tools
build(deps): bump google.golang.org/protobuf from 1.33.0 to 1.34.0 in /exporters/otlp/otlpmetric/otlpmetrichttp
build(deps): bump google.golang.org/protobuf from 1.33.0 to 1.34.0 in /exporters/otlp/otlplog/otlploghttp
build(deps): bump github.com/openzipkin/zipkin-go from 0.4.2 to 0.4.3 in /exporters/zipkin
build(deps): bump google.golang.org/protobuf from 1.33.0 to 1.34.0 in /exporters/otlp/otlpmetric/otlpmetricgrpc
* Bump versions in versions.yaml
* Prepare stable-v1 for version v1.26.0
* Prepare experimental-metrics for version v0.48.0
* Prepare experimental-logs for version v0.2.0-alpha
* Prepare experimental-schema for version v0.0.9
* Update CHANGELOG
* Unbump experimental-schema
* Update CHANGELOG.md
Co-authored-by: Robert Pająk <pellared@hotmail.com>
---------
Co-authored-by: Robert Pająk <pellared@hotmail.com>
Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
* [WIP] Implement the BatchingProcessor
* Add TestExportSync
* Add TestChunker
* Test export error default to ErrorHandler
* Fix lint
* Fix chunk smaller than size error
* Add batch tests
* Fix lint
* Update OnEmit test
Check the len of records in eventually assertion given that is what we
are going to measure.
* Revert unneeded change to BatchingProcessor doc
* Add batch type
* Refactor testing of batching config
The BatchingProcessor is not expected to ultimately contain
configuration fields for queue size or export parameters (see #5093).
This will break TestNewBatchingProcessorConfiguration which tests the
configuration by evaluating the BatchingProcessor directly.
Instead, test the batchingConfig and rename the test to
TestNewBatchingConfig to match what is being tested.
* Implement the BatchingProcessor without polling
* Add TestBatchingProcessor
* Add ConcurrentSafe test
* Expand Shutdown tests
* Test context canceled for ForceFlush
* Refactor batch to queue
* Use exportSync
* Update docs and naming
* Split buffered export to its own type
* Update comments and naming
* Fix lint
* Remove redundant triggered type
* Add interval polling
* Refactor test structure
* Add custom ring implimementation
* Add BenchmarkBatchingProcessor
* Fix merge
* Remove custom ring impl
* Remove BenchmarkBatchingProcessor
* Update dev docs
* Test nil exporter
* Update OnEmit test
Ensure the poll goroutine will completely flush the queue of batches.
* Test RetriggerFlushNonBlocking
* Update ascii diagram
* Fix flaky OnEmit
* Revert unnecessary change to test pkg name
* Use batching term in docs
* Document EnqueueExport
* Return from EnqueueExport if blocked
Do not wait for the enqueue to succeed.
* Do not drop failed flush log records
* Use cancelable ctx in concurrency test
* Fix comments
* Apply feedback
Do not spawn a goroutine for the flush operation.
* Return true from EnqueueExport when stopped
* Update sdk/log/batch.go
Co-authored-by: Robert Pająk <pellared@hotmail.com>
* Remove TODO
* Comment re-trigger in poll
---------
Co-authored-by: Robert Pająk <pellared@hotmail.com>
* Replace Record lim methods with DroppedAttributes
* Add changelog entry
* Add TestRecordDroppedAttributes
* Add TestRecordCompactAttr
* Add an indexPool
* Fix gramatical error
* Apply feedback
Reduce indentation level.
* Apply feedback
Comment compactAttr and deduplicate.
* Deduplicate all attributes when added
* Comment why head is not used
* Clarify comments
* Move TestAllocationLimits to new file
Do not run this test when the race detector is on.
* Comment follow-up task
build(deps): bump golang.org/x/tools from 0.19.0 to 0.20.0 in /internal/tools
build(deps): bump go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp from 0.49.0 to 0.50.0 in /example/dice
build(deps): bump golang.org/x/sys from 0.18.0 to 0.19.0 in /sdk
* Bump versions in versions.yaml
* Prepare stable-v1 for version v1.25.0
* Prepare experimental-metrics for version v0.47.0
* Prepare experimental-logs for version v0.1.0-alpha
* Prepare experimental-schema for version v0.0.8
* Update changelog
* Run go mod tidy
* Update CHANGELOG.md
---------
Co-authored-by: Damien Mathieu <damien.mathieu@elastic.co>
Co-authored-by: Robert Pająk <pellared@hotmail.com>
* Add chunker exporter
The batching log processor needs to be able to export payloads in
chuncks. This adds a chunker type that will forward all Shutdown and
ForceFlush calls to the embedded exporter and chunk data passed to
Export.
* Concurrent safe testExporter
* Add test for zero size
* Fix lint
* Refactor chunker into chunkExporter
* Remove ExportTrigger
* Add export sync
The batching log processor will generate records from 4 different
locations (polling, OnEmit, ForceFlush, Shutdown). In order to ensure an
Exporter is called serially, as is required by the interface, this
function will be used in the processor to centralize the interaction
with its Exporter.
Part of #5063.
See #5093 for the implementation use.
* Concurrent safe testExporter
---------
Co-authored-by: Sam Xie <sam@samxie.me>
The BatchingProcessor is not expected to ultimately contain
configuration fields for queue size or export parameters (see #5093).
This will break TestNewBatchingProcessorConfiguration which tests the
configuration by evaluating the BatchingProcessor directly.
Instead, test the batchingConfig and rename the test to
TestNewBatchingConfig to match what is being tested.
* Add the Enabled method to the Logger
* Add a changelog entry
* Rename enabled.go to min_sev.go
* Remove MinSeverityProcessor
* Document lack of interaction between OnEmit and Enabled
* Update sdk/log/processor.go
---------
Co-authored-by: Robert Pająk <pellared@hotmail.com>
build(deps): bump google.golang.org/grpc from 1.62.0 to 1.62.1 in /exporters/otlp/otlpmetric/otlpmetricgrpc
build(deps): bump google.golang.org/protobuf from 1.32.0 to 1.33.0 in /exporters/otlp/otlpmetric/otlpmetricgrpc
build(deps): bump google.golang.org/protobuf from 1.32.0 to 1.33.0 in /exporters/prometheus
build(deps): bump google.golang.org/grpc from 1.62.0 to 1.62.1 in /exporters/otlp/otlptrace/otlptracegrpc
build(deps): bump google.golang.org/protobuf from 1.32.0 to 1.33.0 in /exporters/otlp/otlptrace/otlptracegrpc
build(deps): bump google.golang.org/grpc from 1.62.0 to 1.62.1 in /bridge/opentracing/test
build(deps): bump google.golang.org/protobuf from 1.32.0 to 1.33.0 in /exporters/otlp/otlptrace/otlptracehttp
build(deps): bump google.golang.org/grpc from 1.62.0 to 1.62.1 in /exporters/otlp/otlptrace/otlptracehttp
build(deps): bump google.golang.org/protobuf from 1.32.0 to 1.33.0 in /exporters/otlp/otlptrace
build(deps): bump google.golang.org/protobuf from 1.32.0 to 1.33.0 in /exporters/otlp/otlpmetric/otlpmetrichttp
build(deps): bump google.golang.org/grpc from 1.62.0 to 1.62.1 in /exporters/otlp/otlpmetric/otlpmetrichttp
build(deps): bump google.golang.org/grpc from 1.62.0 to 1.62.1 in /example/otel-collector
build(deps): bump golang.org/x/sys from 0.17.0 to 0.18.0 in /sdk
build(deps): bump golang.org/x/tools from 0.18.0 to 0.19.0 in /internal/tools
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /sdk/metric
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /exporters/otlp/otlptrace/otlptracegrpc
build(deps): bump codecov/codecov-action from 3.1.5 to 4.1.0
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /exporters/zipkin
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /metric
build(deps): bump github.com/prometheus/client_golang from 1.18.0 to 1.19.0 in /example/prometheus
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /exporters/stdout/stdouttrace
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /exporters/otlp/otlptrace/otlptracehttp
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /exporters/otlp/otlpmetric/otlpmetricgrpc
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /exporters/otlp/otlptrace
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /trace
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /exporters/stdout/stdoutmetric
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /bridge/opentracing
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /schema
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /log
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /sdk
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /bridge/opencensus
build(deps): bump go.opentelemetry.io/build-tools/gotmpl from 0.12.0 to 0.13.0 in /internal/tools
build(deps): bump go.opentelemetry.io/build-tools/multimod from 0.12.0 to 0.13.0 in /internal/tools
build(deps): bump go.opentelemetry.io/build-tools/dbotconf from 0.12.0 to 0.13.0 in /internal/tools
build(deps): bump go.opentelemetry.io/build-tools/semconvgen from 0.12.0 to 0.13.0 in /internal/tools
build(deps): bump go.opentelemetry.io/build-tools/crosslink from 0.12.0 to 0.13.0 in /internal/tools
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /exporters/otlp/otlpmetric/otlpmetrichttp
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /bridge/opentracing/test
build(deps): bump github.com/prometheus/client_golang from 1.18.0 to 1.19.0 in /exporters/prometheus
build(deps): bump github.com/stretchr/testify from 1.8.4 to 1.9.0 in /exporters/prometheus
* Update README.md
* Remove 1.20 support from CI workflows
* Update all go mod
* Add changelog entry
* Update go mod tidy target
* Run go mod tidy
* Replace sliceEqualFunc with slices.EqualFunc
* Replace grow with slices.Grow
* Replace ensureAttributesCapacity with slices.Grow
* Replace conditional with min
* Use slices module for slice comparison in metricdatatest
* Export getLogger
Refactor the declaration of globalLogger to incorporate the logic of
init() so there is no data race between GetLogger and init being called.
* Use GetLogger in log testing
* Restore logger in batch span processor testing
* Remove unused URL in globalLogger doc
* Bump versions
* Prepare stable-v1 for version v1.24.0
* Prepare experimental-metrics for version v0.46.0
* Prepare experimental-logs for version v0.0.1-alpha
* Update changelog
* Fix changelog entry
* Return merged Resource on schema conflict
* Add changes to changelog
* Doc returned resource to have no schema URL
* Refactor Merge based on feedback
* Add the schema URLs to the returned error
* Ensure no schema URL when merge conflict on detect
* Replaced isErr with wantErr in TestNew
* Update sdk/resource/auto_test.go
Co-authored-by: Robert Pająk <pellared@hotmail.com>
* Update TestDetect based on feedback
---------
Co-authored-by: Robert Pająk <pellared@hotmail.com>
* Add the experimental exemplar feature
* Add exemplars to EXPERIMENTAL.md
* Add changelog entry
* Fix hist buckets > 1 detection
* Collect instead of Flush res about to be deleted
* Add e2e test
* Do not pre-alloc ResourceMetrics
This only has a single use.
* Fix grammatical error in comment
* Add test cases
Default and invalid OTEL_METRICS_EXEMPLAR_FILTER.
Test sampled and non-sampled context for trace_based.
* Comment nCPU
* Doc OTEL_METRICS_EXEMPLAR_FILTER
* Add the random fixed size exemplar reservoir
* Rename fixed.go to storage.go
* Update sdk/metric/internal/exemplar/rand.go
Co-authored-by: David Ashpole <dashpole@google.com>
* Remove stale ref to spec recommendation
* Add comments to clarify the reset/advance/Collect methods
* Apply comment from feedback
* Add random func to gen rand float64 on (0,1)
* Use random in TestFixedSizeSamplingCorrectness
* Add clarifying algorithm comments
Include a high-level overview of the algorithm implemented and clarify
parameter names to be consistent.
* Fix duplicate word
* Update sdk/metric/internal/exemplar/rand.go
* Comment TestFixedSizeSamplingCorrectness
* Update test delta
* Test collect less than cap
* Remove measurement.Valid method
---------
Co-authored-by: David Ashpole <dashpole@google.com>
Co-authored-by: Chester Cheung <cheung.zhy.csu@gmail.com>
This Reservoir implementaiton is used at the MeterProvider level to
pre-filter measurements offered to a wrapped Reservoir.
Co-authored-by: Robert Pająk <pellared@hotmail.com>
* Add the sdk/metric/internal/exemplar pkg
* Rename testReservoir to ReservoirTest
This is a test helper function that is not exported outside of testing
for the package. Use the exported name so the CI system doesn't complain
about it not being used. It will be used in follow-up PRs.
* Run go mod tidy
* Update versions.yaml
Move experimental-metrics modules that are stbilizing to the stable-v1
module-set.
* Prepare stable-v1 for version v1.23.0-rc.1
* Update changelog
Bump google.golang.org/grpc from 1.60.0 to 1.60.1 in /exporters/otlp/otlpmetric/otlpmetrichttp
Bump google.golang.org/protobuf from 1.31.0 to 1.32.0 in /exporters/otlp/otlpmetric/otlpmetrichttp
Bump github.com/go-logr/logr from 1.3.0 to 1.4.1
Bump github.com/go-logr/logr from 1.3.0 to 1.4.1 in /sdk/metric
Bump google.golang.org/protobuf from 1.31.0 to 1.32.0 in /exporters/prometheus
Bump github.com/go-logr/logr from 1.3.0 to 1.4.1 in /sdk
Bump google.golang.org/protobuf from 1.31.0 to 1.32.0 in /exporters/otlp/otlptrace/otlptracehttp
Bump google.golang.org/grpc from 1.60.0 to 1.60.1 in /exporters/otlp/otlptrace/otlptracehttp
Bump google.golang.org/grpc from 1.60.0 to 1.60.1 in /bridge/opentracing/test
Bump google.golang.org/grpc from 1.60.0 to 1.60.1 in /exporters/otlp/otlpmetric/otlpmetricgrpc
Bump google.golang.org/protobuf from 1.31.0 to 1.32.0 in /exporters/otlp/otlpmetric/otlpmetricgrpc
Bump google.golang.org/protobuf from 1.31.0 to 1.32.0 in /exporters/otlp/otlptrace
Bump github.com/go-logr/logr from 1.3.0 to 1.4.1 in /exporters/zipkin
Bump google.golang.org/grpc from 1.60.0 to 1.60.1 in /example/otel-collector
Bump google.golang.org/grpc from 1.60.0 to 1.60.1 in /exporters/otlp/otlptrace/otlptracegrpc
Bump google.golang.org/protobuf from 1.31.0 to 1.32.0 in /exporters/otlp/otlptrace/otlptracegrpc
