1
0
mirror of https://github.com/open-telemetry/opentelemetry-go.git synced 2025-04-23 11:58:56 +02:00

16 Commits

Author SHA1 Message Date
David Ashpole
0485de287e
Move time.Now call into exemplar reservoir to improve performance (#5545)
Part of addressing
https://github.com/open-telemetry/opentelemetry-go/issues/5542.

### Motivation

This removes the `time.Now()` call from filtered-out Exemplars by only
invoking `time.Now()` after the filtering decision is made. This
improvement is especially noticeable for measurements without any
attributes.

```
goos: linux
goarch: amd64
pkg: go.opentelemetry.io/otel/sdk/metric
cpu: AMD EPYC 7B12
                                                                 │   old.txt    │                new.txt                │
                                                                 │    sec/op    │   sec/op     vs base                  │
SyncMeasure/NoView/Int64Counter/Attributes/0-24                    158.20n ± 4%   99.83n ± 1%  -36.90% (p=0.000 n=10)
SyncMeasure/NoView/Int64Counter/Attributes/1-24                     333.3n ± 4%   274.8n ± 1%  -17.55% (p=0.000 n=10)
SyncMeasure/NoView/Int64Counter/Attributes/10-24                    1.640µ ± 1%   1.600µ ± 1%   -2.41% (p=0.000 n=10)
SyncMeasure/NoView/Float64Counter/Attributes/0-24                   159.0n ± 3%   101.3n ± 0%  -36.27% (p=0.000 n=10)
SyncMeasure/NoView/Float64Counter/Attributes/1-24                   340.0n ± 2%   272.0n ± 1%  -20.00% (p=0.000 n=10)
SyncMeasure/NoView/Float64Counter/Attributes/10-24                  1.661µ ± 1%   1.597µ ± 0%   -3.85% (p=0.000 n=10)
SyncMeasure/NoView/Int64UpDownCounter/Attributes/0-24               159.8n ± 1%   103.1n ± 0%  -35.50% (p=0.000 n=10)
SyncMeasure/NoView/Int64UpDownCounter/Attributes/1-24               339.5n ± 1%   273.1n ± 0%  -19.57% (p=0.000 n=10)
SyncMeasure/NoView/Int64UpDownCounter/Attributes/10-24              1.656µ ± 0%   1.589µ ± 0%   -4.05% (p=0.000 n=10)
SyncMeasure/NoView/Float64UpDownCounter/Attributes/0-24             159.3n ± 2%   100.8n ± 0%  -36.74% (p=0.000 n=10)
SyncMeasure/NoView/Float64UpDownCounter/Attributes/1-24             337.9n ± 2%   271.8n ± 1%  -19.55% (p=0.000 n=10)
SyncMeasure/NoView/Float64UpDownCounter/Attributes/10-24            1.657µ ± 0%   1.593µ ± 1%   -3.83% (p=0.000 n=10)
SyncMeasure/NoView/Int64Histogram/Attributes/0-24                  144.65n ± 4%   89.38n ± 0%  -38.21% (p=0.000 n=10)
SyncMeasure/NoView/Int64Histogram/Attributes/1-24                   235.7n ± 2%   183.5n ± 0%  -22.15% (p=0.000 n=10)
SyncMeasure/NoView/Int64Histogram/Attributes/10-24                  900.8n ± 1%   836.8n ± 0%   -7.10% (p=0.000 n=10)
SyncMeasure/NoView/Float64Histogram/Attributes/0-24                145.60n ± 5%   93.48n ± 1%  -35.80% (p=0.000 n=10)
SyncMeasure/NoView/Float64Histogram/Attributes/1-24                 240.9n ± 1%   183.0n ± 0%  -24.06% (p=0.000 n=10)
SyncMeasure/NoView/Float64Histogram/Attributes/10-24                905.6n ± 1%   826.3n ± 0%   -8.76% (p=0.000 n=10)
SyncMeasure/DropView/Int64Counter/Attributes/0-24                   20.33n ± 0%   20.35n ± 0%        ~ (p=0.302 n=10)
SyncMeasure/DropView/Int64Counter/Attributes/1-24                   26.46n ± 0%   26.45n ± 1%        ~ (p=0.868 n=10)
SyncMeasure/DropView/Int64Counter/Attributes/10-24                  26.50n ± 0%   26.47n ± 0%        ~ (p=0.208 n=10)
SyncMeasure/DropView/Float64Counter/Attributes/0-24                 20.34n ± 1%   20.27n ± 0%   -0.34% (p=0.009 n=10)
SyncMeasure/DropView/Float64Counter/Attributes/1-24                 26.55n ± 0%   26.60n ± 1%        ~ (p=0.109 n=10)
SyncMeasure/DropView/Float64Counter/Attributes/10-24                26.59n ± 1%   26.57n ± 1%        ~ (p=0.926 n=10)
SyncMeasure/DropView/Int64UpDownCounter/Attributes/0-24             20.38n ± 1%   20.38n ± 0%        ~ (p=0.725 n=10)
SyncMeasure/DropView/Int64UpDownCounter/Attributes/1-24             26.39n ± 0%   26.44n ± 0%        ~ (p=0.238 n=10)
SyncMeasure/DropView/Int64UpDownCounter/Attributes/10-24            26.52n ± 0%   26.42n ± 0%   -0.36% (p=0.049 n=10)
SyncMeasure/DropView/Float64UpDownCounter/Attributes/0-24           20.30n ± 0%   20.25n ± 0%        ~ (p=0.196 n=10)
SyncMeasure/DropView/Float64UpDownCounter/Attributes/1-24           26.57n ± 0%   26.54n ± 1%        ~ (p=0.540 n=10)
SyncMeasure/DropView/Float64UpDownCounter/Attributes/10-24          26.57n ± 0%   26.51n ± 1%        ~ (p=0.643 n=10)
SyncMeasure/DropView/Int64Histogram/Attributes/0-24                 20.37n ± 0%   20.36n ± 1%        ~ (p=1.000 n=10)
SyncMeasure/DropView/Int64Histogram/Attributes/1-24                 26.41n ± 0%   26.50n ± 0%   +0.32% (p=0.007 n=10)
SyncMeasure/DropView/Int64Histogram/Attributes/10-24                26.44n ± 0%   26.55n ± 1%   +0.42% (p=0.012 n=10)
SyncMeasure/DropView/Float64Histogram/Attributes/0-24               20.30n ± 0%   20.45n ± 0%   +0.74% (p=0.000 n=10)
SyncMeasure/DropView/Float64Histogram/Attributes/1-24               26.52n ± 0%   26.48n ± 0%        ~ (p=0.127 n=10)
SyncMeasure/DropView/Float64Histogram/Attributes/10-24              26.55n ± 0%   26.48n ± 0%   -0.26% (p=0.002 n=10)
SyncMeasure/AttrFilterView/Int64Counter/Attributes/0-24             170.5n ± 2%   110.8n ± 0%  -35.03% (p=0.000 n=10)
SyncMeasure/AttrFilterView/Int64Counter/Attributes/1-24             402.5n ± 1%   331.5n ± 1%  -17.64% (p=0.000 n=10)
SyncMeasure/AttrFilterView/Int64Counter/Attributes/10-24            1.363µ ± 1%   1.281µ ± 1%   -6.02% (p=0.000 n=10)
SyncMeasure/AttrFilterView/Float64Counter/Attributes/0-24           170.6n ± 1%   111.5n ± 1%  -34.64% (p=0.000 n=10)
SyncMeasure/AttrFilterView/Float64Counter/Attributes/1-24           397.1n ± 1%   335.9n ± 0%  -15.41% (p=0.000 n=10)
SyncMeasure/AttrFilterView/Float64Counter/Attributes/10-24          1.371µ ± 1%   1.279µ ± 1%   -6.71% (p=0.000 n=10)
SyncMeasure/AttrFilterView/Int64UpDownCounter/Attributes/0-24       170.1n ± 1%   112.2n ± 0%  -34.09% (p=0.000 n=10)
SyncMeasure/AttrFilterView/Int64UpDownCounter/Attributes/1-24       397.5n ± 1%   330.2n ± 0%  -16.93% (p=0.000 n=10)
SyncMeasure/AttrFilterView/Int64UpDownCounter/Attributes/10-24      1.371µ ± 1%   1.289µ ± 1%   -5.95% (p=0.000 n=10)
SyncMeasure/AttrFilterView/Float64UpDownCounter/Attributes/0-24     171.4n ± 2%   112.9n ± 0%  -34.13% (p=0.000 n=10)
SyncMeasure/AttrFilterView/Float64UpDownCounter/Attributes/1-24     397.0n ± 3%   336.4n ± 0%  -15.24% (p=0.000 n=10)
SyncMeasure/AttrFilterView/Float64UpDownCounter/Attributes/10-24    1.383µ ± 1%   1.305µ ± 1%   -5.61% (p=0.000 n=10)
SyncMeasure/AttrFilterView/Int64Histogram/Attributes/0-24          157.30n ± 2%   98.58n ± 1%  -37.33% (p=0.000 n=6+10)
```

### Changes

* Introduce `exemplar.Filter`, which is a filter function based on the
context. It will not be user-facing, so we can always add other
parameters later if needed.
* Introduce `exemplar.FilteredReservoir`, which is similar to a
reservoir, except it does not receive a timestamp. It gets the current
time after the filter decision has been made. It uses generics to avoid
the call to exemplar.NewValue(), since it is internal-only.
* The `exemplar.Reservoir` is left as-is, so that it can be made public
when exemplars are stable. It still includes a timestamp argument.
* Unit tests are updated to expect a much lower number of calls to
time.Now
* `exemplar.Drop` is now an `exemplar.FilteredReservoir` instead of a
`Reservoir`, since we don't need a Reservoir to store things in if the
measurement is always dropped.

Co-authored-by: Sam Xie <sam@samxie.me>
Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
2024-07-01 09:36:11 -07:00
Tyler Yahn
2f662dbe13
Refactor exemplars to not use generic argument (#5285)
* Refactor exemplars to not use generic argument

* Update internal/aggregate

* Update metric SDK

* Test exemplar value type

* Add TestCollectExemplars

* Fix lint

---------

Co-authored-by: Sam Xie <sam@samxie.me>
2024-05-07 08:12:59 -07:00
Tyler Yahn
fe9bab54b7
Use Distinct instead of Set for map keys (#5027) 2024-03-06 11:11:16 +01:00
Robert Pająk
7dea232a46
[chore] Simplify the license header (#4987) 2024-02-29 07:05:28 +01:00
Tyler Yahn
d50a23bf4b
Clear aggregate value maps instead of deleting entries (#4981) 2024-02-27 08:22:58 +01:00
Erica Yin
27e495d6f9
Fix output exponential histogram negative buckets (#4956)
* Fix output exponential histogram buckets

* Update CHANGELOG
2024-02-22 13:51:41 -08:00
Tyler Yahn
fecb92e366
Add the experimental exemplar feature (#4871)
* Add the experimental exemplar feature

* Add exemplars to EXPERIMENTAL.md

* Add changelog entry

* Fix hist buckets > 1 detection

* Collect instead of Flush res about to be deleted

* Add e2e test

* Do not pre-alloc ResourceMetrics

This only has a single use.

* Fix grammatical error in comment

* Add test cases

Default and invalid OTEL_METRICS_EXEMPLAR_FILTER.

Test sampled and non-sampled context for trace_based.

* Comment nCPU

* Doc OTEL_METRICS_EXEMPLAR_FILTER
2024-01-31 13:15:35 -08:00
Tyler Yahn
e3bf787c21
Add cardinality limiting to the metric SDK as an experimental feature (#4457)
* Add agg limiting func

* Add unit test for limitAttr

* Add limiting to aggregate types

* Add internal x pkg for experimental feature-flagging

* Connect cardinality limit to metric SDK

* Replace limitAttr fn with limiter type

The Attribute method is still inlinable.

* Use x.CardinalityLimit directly

* Simplify limiter test

* Add limiter benchmark

* Document the AggregationLimit field

* Test sum limits

* Test limit for last value

* Test histogram limit

* Refactor expo hist test to use existing fixtures

The tests for the exponential histogram create their own testing
fixtures. There is nothing these new fixtures do that cannot already be
done with the existing testing fixtures used by all the other aggregate
functions. Unify the exponential histogram testing to use the existing
fixtures.

* Test the ExponentialHistogram limit

* Fix lint

* Add docs

* Rename aggregation field to aggLimit

---------

Co-authored-by: Robert Pająk <pellared@hotmail.com>
2023-12-19 07:53:01 -08:00
Tyler Yahn
5dff273a1e
Use gofumpt instead of gofmt (#4623)
* Use gofumpt instead of gofmt in golangci-lint conf

* Run gofumpt fixes

* Format generated templates

---------

Co-authored-by: Chester Cheung <cheung.zhy.csu@gmail.com>
2023-10-16 10:02:21 -07:00
Tyler Yahn
16ce491d38
Fix guard of measured value to not record empty (#4452)
A guard was added in #4446 to prevent non-normal float64 from being
recorded. This was added in the low-level `record` method meaning that
the higher-level `measure` method will still keep a record of the
invalid value measurement, just with a zero-value.

This fixes that issue by moving the guard to the `measure` method.
2023-08-18 06:43:09 -07:00
Tyler Yahn
9b47674bc5
Make getBin and scaleChange methods of expoHistogramDataPoint (#4451)
Both functions receive parameters from an expoHistogramDataPoint and are
only ever used by other methods of an expoHistogramDataPoint. Make the
functions methods of expoHistogramDataPoint so the parameter arguments
can be dropped and the functions are scoped to the type they are used
for.

Co-authored-by: Aaron Clawson <3766680+MadVikingGod@users.noreply.github.com>
2023-08-17 15:06:46 -07:00
Tyler Yahn
9d9b71f58d
Remove the expoHistogramValues type (#4450)
The types only use is being embedded in the expoHistogram. Just
explicitly define the fields for the expoHistogram given that is their
only use.
2023-08-17 12:31:02 -05:00
Tyler Yahn
a5ff7af3f3
Ignore +/- Inf and NaN for exponential histogram measurement (#4446)
* Add acceptance test

* Ignore +/- Inf and NaN for expo hist record
2023-08-16 07:11:13 -07:00
Tyler Yahn
3904523917
Flatten sdk/metric/aggregation into sdk/metric (#4435)
* Deprecate the aggregation pkg

* Decouple the internal/aggregate from aggregation pkg

* Add Aggregation to the metric pkg

* Do not use sdk/metric/aggregation in stdoutmetric exporter

* Update all generated templates

* Update prom exporter

* Fix view example

* Add changes to changelog

* Update CHANGELOG.md

Co-authored-by: Robert Pająk <pellared@hotmail.com>

* Rename Sum to AggregationSum

* Fix comments

* Centralize validation of aggregation in pipeline

* Remove validation of agg in manual_reader selector opt

* Fix merge

---------

Co-authored-by: Robert Pająk <pellared@hotmail.com>
2023-08-14 08:15:15 -07:00
Tyler Yahn
b44a2bbc36
Remove commented out method in exponential_histogram.go (#4429) 2023-08-10 09:23:25 +02:00
Aaron Clawson
248413d654
Add the Exponential Histogram Aggregator. (#4245)
* Adds Exponential Histograms aggregator

* Added aggregation to the pipeline.

Adjust to new bucket

* Add no allocation if cap is available.

* Expand tests

* Fix lint

* Fix 64 bit math on 386 platform.

* Fix tests to work in go 1.19.
Fix spelling error

* fix codespell

* Add example

* Update sdk/metric/aggregation/aggregation.go

Co-authored-by: Robert Pająk <pellared@hotmail.com>

* Update sdk/metric/aggregation/aggregation.go

* Update sdk/metric/aggregation/aggregation.go

* Changelog

* Fix move

* Address feedback from the PR.

* Update expo histo to new aggregator format.

* Fix lint

* Remove Zero Threshold from config of expo histograms

* Remove DefaultExponentialHistogram()

* Refactor GetBin, and address PR Feedback

* Address PR feedback

* Fix comment in wrong location

* Fix misapplied PR feedback

* Fix codespell

---------

Co-authored-by: Robert Pająk <pellared@hotmail.com>
Co-authored-by: Chester Cheung <cheung.zhy.csu@gmail.com>
2023-08-04 11:57:44 -07:00