Resolve https://github.com/open-telemetry/opentelemetry-go/issues/5249
### Spec
> exemplar_reservoir: A functional type that generates an exemplar
reservoir a MeterProvider will use when storing exemplars. This
functional type needs to be a factory or callback similar to aggregation
selection functionality which allows different reservoirs to be chosen
by the aggregation.
> Users can provide an exemplar_reservoir, but it is up to their
discretion. Therefore, the stream configuration parameter needs to be
structured to accept an exemplar_reservoir, but MUST NOT obligate a user
to provide one. If the user does not provide an exemplar_reservoir
value, the MeterProvider MUST apply a [default exemplar
reservoir](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#exemplar-defaults).
Also,
> the reservoir MUST be given the Attributes associated with its
timeseries point either at construction so that additional sampling
performed by the reservoir has access to all attributes from a
measurement in the "offer" method.
### Changes
In sdk/metric/exemplar, add:
* `exemplar.ReservoirProvider`
* `exemplar.HistogramReservoirProvider`
* `exemplar.FixedSizeReservoirProvider`
In sdk/metric, add:
* `metric.ExemplarReservoirProviderSelector` (func Aggregation ->
ReservoirProvider)
* `metric.DefaultExemplarReservoirProviderSelector` (our default
implementation)
* `ExemplarReservoirProviderSelector` added to `metric.Stream`
Note: the only required public types are
`metric.ExemplarReservoirProviderSelector` and
`ExemplarReservoirProviderSelector` in `metric.Stream`. Others are for
convenience and readability.
### Alternatives considered
#### Add ExemplarReservoirProvider directly to metric.Stream, instead of
ExemplarReservoirProviderSelector
This would mean users can configure a `func() exemplar.Reservoir`
instead of a `func(Aggregation) func() exemplar.Reservoir`.
I don't think this complies with the statement: `This functional type
needs to be a factory or callback similar to aggregation selection
functionality which allows different reservoirs to be chosen by the
aggregation.`. I'm interpreting "allows different reservoirs to be
chosen by the aggregation" as meaning "allows different reservoirs to be
chosen **based on the** aggregation", rather than meaning that the
aggregation is somehow choosing the reservoir.
### Future work
There is some refactoring I plan to do after this to simplify the
interaction between the internal/aggregate and exemplar package. I've
omitted that from this PR to keep the diff smaller.
---------
Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
Co-authored-by: Robert Pająk <pellared@hotmail.com>
Part of https://github.com/open-telemetry/opentelemetry-go/issues/5249
This makes all existing types designed to implement the public Exemplar
API public by moving most of `internal/exemplar` to `exemplar`. The only
types that are not being made public are `exemplar.Drop`, and
`exemplar.FilteredReservoir`. Those types are moved to
`internal/aggregate`, and are renamed to `DropReservoir` and
`FilteredExemplarReservoir`.
The following types are made public:
* `exemplar.Exemplar`
* `exemplar.Filter`
* `exemplar.SampledFilter`
* `exemplar.AlwaysOnFilter`
* `exemplar.HistogramReservoir`
* `exemplar.FixedSizeReservoir`
* `exemplar.Reservoir`
* `exemplar.Value`
* `exemplar.ValueType`
* Refactor exemplars to not use generic argument
* Update internal/aggregate
* Update metric SDK
* Test exemplar value type
* Add TestCollectExemplars
* Fix lint
---------
Co-authored-by: Sam Xie <sam@samxie.me>
* Add the experimental exemplar feature
* Add exemplars to EXPERIMENTAL.md
* Add changelog entry
* Fix hist buckets > 1 detection
* Collect instead of Flush res about to be deleted
* Add e2e test
* Do not pre-alloc ResourceMetrics
This only has a single use.
* Fix grammatical error in comment
* Add test cases
Default and invalid OTEL_METRICS_EXEMPLAR_FILTER.
Test sampled and non-sampled context for trace_based.
* Comment nCPU
* Doc OTEL_METRICS_EXEMPLAR_FILTER
* Add agg limiting func
* Add unit test for limitAttr
* Add limiting to aggregate types
* Add internal x pkg for experimental feature-flagging
* Connect cardinality limit to metric SDK
* Replace limitAttr fn with limiter type
The Attribute method is still inlinable.
* Use x.CardinalityLimit directly
* Simplify limiter test
* Add limiter benchmark
* Document the AggregationLimit field
* Test sum limits
* Test limit for last value
* Test histogram limit
* Refactor expo hist test to use existing fixtures
The tests for the exponential histogram create their own testing
fixtures. There is nothing these new fixtures do that cannot already be
done with the existing testing fixtures used by all the other aggregate
functions. Unify the exponential histogram testing to use the existing
fixtures.
* Test the ExponentialHistogram limit
* Fix lint
* Add docs
* Rename aggregation field to aggLimit
---------
Co-authored-by: Robert Pająk <pellared@hotmail.com>
* Refactor expo hist test to use existing fixtures
The tests for the exponential histogram create their own testing
fixtures. There is nothing these new fixtures do that cannot already be
done with the existing testing fixtures used by all the other aggregate
functions. Unify the exponential histogram testing to use the existing
fixtures.
* Add alt input for cumulative test
* Use gofumpt instead of gofmt in golangci-lint conf
* Run gofumpt fixes
* Format generated templates
---------
Co-authored-by: Chester Cheung <cheung.zhy.csu@gmail.com>
* Add a benchmark target to makefile
* update CI to reflect how the benchmarks will be used
* Add auto-push to benchmarks branch.
---------
Co-authored-by: Robert Pająk <pellared@hotmail.com>
* Drop support for Go 1.19
* Add change to changelog
* Bump all modules to 1.20
* Update exponential_histogram_test.go
---------
Co-authored-by: Robert Pająk <pellared@hotmail.com>
A guard was added in #4446 to prevent non-normal float64 from being
recorded. This was added in the low-level `record` method meaning that
the higher-level `measure` method will still keep a record of the
invalid value measurement, just with a zero-value.
This fixes that issue by moving the guard to the `measure` method.
Both functions receive parameters from an expoHistogramDataPoint and are
only ever used by other methods of an expoHistogramDataPoint. Make the
functions methods of expoHistogramDataPoint so the parameter arguments
can be dropped and the functions are scoped to the type they are used
for.
Co-authored-by: Aaron Clawson <3766680+MadVikingGod@users.noreply.github.com>
* Deprecate the aggregation pkg
* Decouple the internal/aggregate from aggregation pkg
* Add Aggregation to the metric pkg
* Do not use sdk/metric/aggregation in stdoutmetric exporter
* Update all generated templates
* Update prom exporter
* Fix view example
* Add changes to changelog
* Update CHANGELOG.md
Co-authored-by: Robert Pająk <pellared@hotmail.com>
* Rename Sum to AggregationSum
* Fix comments
* Centralize validation of aggregation in pipeline
* Remove validation of agg in manual_reader selector opt
* Fix merge
---------
Co-authored-by: Robert Pająk <pellared@hotmail.com>
* Adds Exponential Histograms aggregator
* Added aggregation to the pipeline.
Adjust to new bucket
* Add no allocation if cap is available.
* Expand tests
* Fix lint
* Fix 64 bit math on 386 platform.
* Fix tests to work in go 1.19.
Fix spelling error
* fix codespell
* Add example
* Update sdk/metric/aggregation/aggregation.go
Co-authored-by: Robert Pająk <pellared@hotmail.com>
* Update sdk/metric/aggregation/aggregation.go
* Update sdk/metric/aggregation/aggregation.go
* Changelog
* Fix move
* Address feedback from the PR.
* Update expo histo to new aggregator format.
* Fix lint
* Remove Zero Threshold from config of expo histograms
* Remove DefaultExponentialHistogram()
* Refactor GetBin, and address PR Feedback
* Address PR feedback
* Fix comment in wrong location
* Fix misapplied PR feedback
* Fix codespell
---------
Co-authored-by: Robert Pająk <pellared@hotmail.com>
Co-authored-by: Chester Cheung <cheung.zhy.csu@gmail.com>
Instead of treating the returned *lastValue as an aggregator from
newLastValue, just use the type directly to construct the Measure and
ComputeAggregation functions returned from the Builder.
Accept a destination type for the underlying computeAggregation. This
allows memory reuse for collections which adds a considerable
optimization.
Simplify the integration testing of the last-value aggregate.
Update benchmarking.
* Replace filter aggregator with direct filter on measure
Part of #4220
Instead of using an aggregator to filter measured attributes, directly
filter the attributes in the constructed Measure function the Builder
creates.
Include unit and integration testing of new filtering.
* Update sdk/metric/internal/aggregate/aggregate.go
Co-authored-by: David Ashpole <dashpole@google.com>
---------
Co-authored-by: David Ashpole <dashpole@google.com>
* Allow histogram for all instruments
Any instrument that can record negative values, do not include a sum in
the produced aggregation (like the specification recommends).
Resolves#4161
* Add changes to changelog
* Fix TestBucketsSum
