* prometheus: add error wrapping
This PR adds error wrapping to the prometheus exporter.
The added context should make it easier to follow the error path.
This is a follow-up to #622
* feedback
* othttp: add RemoteAddrKey for HTTP client address
Include the requesting HTTP client's network address in each top-level
span emitted by othttp.NewHandler.
* Copyedit documentation for attribute keys
Instead of batching based on the Resource pointer which is not guaranteed
to uniquely identify common Resources, use the `String` method which is.
Renames the external test package to be identified as the integration
testing. This testing structure (the mock exporter) batches exported
resources transmitted from the resource. This is needed as there is no
guarantees that a batch will be exported in a single payload, therefore,
it is not an ideal place to add resource batch testing for the exporter
itself.
Add unit tests to ensure the Exporter is correctly batching spans by
resource.
* Update OTLP SpanData transform
The ParentSpanId needs to be empty for root spans according to the OTLP
[docs](6c2a86ed2f/gen/go/trace/v1/trace.pb.go (L284-L286)).
This updates the SpanData transform function to not add the ParentSpanID
if it is not a valid span ID (which includes if it is the nil span ID
used for an unset ID).
Additionally, this adds a test to prevent regression.
* Simplify test to just check parent span ID transform
Co-authored-by: Rahul Patel <rahulpa@google.com>
* Update Resource
When looking at grouping telemetry in an exporter based on the Resource
it is ideal if a map can be make with the key being represented by a
Resource. However, given the Resource is not hashable, this is not
possible.
This add a `String` method that can be used as a map key during
grouping. Additionally, this means the Resource now implements the
`Stringer` interface providing human-readable info when prited.
The internal structure of the Resource is changed. A static slice
containing all key-values in a sorted order replaces the existing map.
Additionally a set of keys is added to accommodate lookup during
`Merge`. Also, the string representation is kept in an internal field so
as to save processing for the `String` method (all fields are assumed to
be static after creation).
The `Attributes` method now returns a sorted slice of the associated
key-values.
The `Merge` method has been updated to support the changed structure of
the Resource.
New tests are added to validate the `String` method.
* Update comment
* Change loop into returned append
* Update key-value less func
Keys are unique in this package, treat them that way.
* Remove unnecessary allocation on empty attributes
* Update `Merge` method
Remove incomplete sorting of merged slices. Instead use the `sort`
package.
Add tests to catch sorting failure identified.
* Apply suggestions from code review
Co-Authored-By: ET <evantorrie@users.noreply.github.com>
* Escape Resource string representation
To ensure uniqueness of the string representation, the key-value content
needs to be escaped.
* Switch to an eager evaluation for the `String` method
* Refactor `Merge` method
Leave optimization to the future and simplify the merge.
* Add AttributeIterator
Include a method for a user of the Resource to iterate over the related
attributes without needed to copy the attributes.
* Fix ineffectual
* Fix lint
* Add licenses
* keys -> keySet for Resource
Co-authored-by: ET <evantorrie@users.noreply.github.com>
Co-authored-by: Rahul Patel <rahulpa@google.com>
This PR modifies prometheus.Collect to reflect the change introduced
by #557.
The `export{Counter,Histogram,LastValue,Summary}` methods now all return
an error instead of calling the error callback directly.
The callback is now only called on the returned error from `ForEach`.
fixes#563
Co-authored-by: Rahul Patel <rahulpa@google.com>
* Ensure spans created by httptrace client tracer reflect operation structure
* Cleanup (clientTracer).start based on PR feedback
* Ensure a span is recorded even if end() is called before start()
* Ensure start attributes for spans started by (clientTracer).end() are recorded
Co-authored-by: Rahul Patel <rahulpa@google.com>
This PR adds histogram support to the prometheus exporter.
- Adds a new aggregator selector that returns a histogram for `MeasureKind`.
The selector can be constructed using `simple.NewWithHistogramMeasure`
- Adds support for histogram aggregators in prometheus collect method.
With this PR, the default selector is changed to use histograms.
In order to support the prometheus histogram, the `aggregator.Histogram`
interface is extended with the `Sum` method.
fixes#487
Co-authored-by: Rahul Patel <rahulpa@google.com>
Banner at the top of godoc.org pages are already asking users to
redirect to pkg.go.dev as mentioned in blog post
https://blog.golang.org/pkg.go.dev-2020
Signed-off-by: Andrew Hsu <xuzuan@gmail.com>
* Remove LabelSet frmo api/metric
* SDK tests pass
* Restore benchmarks
* All tests pass
* Remove all mentions of LabelSet
* Test RecordBatch
* Batch test
* Improves benchmark (some)
* Move the benchmark to match HEAD
* Align labels for GOARCH=386
* Add alignment test
* Disable the stress test fo GOARCH=386
* Fix bug
* Move atomic fields into their own file
* Add a TODO
* Comments
* Remove metric.Labels(...)
* FTB
Co-authored-by: Liz Fong-Jones <lizf@honeycomb.io>
This PR changes the circleci build config to build the project in both go-1.13 and go-1.14 versions.
Co-authored-by: Joshua MacDonald <jmacd@users.noreply.github.com>
Co-authored-by: Liz Fong-Jones <lizf@honeycomb.io>
Update license header to standard format for source files missed prior.
Add license header to new source files.
Add Makefile check to test all `*.go` and `*.sh` files have a copyright
notice (or comment about them being auto-generated) within the first few
lines.
* Move span transforms of the OTLP exporter to internal
Breakup and move functionality of the `transform_spans.go` file into
appropriate files in the `internal/transform` sub-package. This is in
preparation of using some of the overlapping functionality to implement
Resource support in the metric side of the exporter.
Adds more specific unit tests for some of the functionality transferred.
The tests removed used the exporter as a processing engine and the
replacement tests do not do this. The test found in the `oltp_test.go`
seem to comprehensively cover this type of test.
Include Link `Name` in the exporter span link and adds a test to check
for this.
Resolves#527
* Apply suggestions from code review
Co-Authored-By: Rahul Patel <rghetia@yahoo.com>
* Fix SpanData doc
* Consolidate span comparison
Co-authored-by: Rahul Patel <rghetia@yahoo.com>
Co-authored-by: Joshua MacDonald <jmacd@users.noreply.github.com>
* Add skeleton uniqueness checker
* Fix the build w/ new code in place
* Add sync tests
* More test
* Implement global uniqueness checking
* Set the library name
* Ensure ordered global initialization
* Use proper require statement for errors
* Comment
* Apply feedback fixes
* Comment and rename from feedback
* Make pre_release.sh script work with GNU
The `-i` sed flag is for doing in-place updates of the file. On some
systems it requires a parameter, on other (GNU) the parameter is
optional, but should be "glued" to the flag (`-i.bak` for example).
That's is problematic, so just emulate the in-place updates with
copies and removals.
* Update the version string in sdk with the pre-release script
This was a missing step, which is why we still have a version string
"0.2.3" while in reality it should be "0.3.0". Hopefully it will be
something to fix in "0.3.1" or what the next release version will be.
* Ensure clean git state when doing a release
The script does `git add .` which adds everything in the tree, even
the untracked files. Make sure that there are none of those.
* Temporarily opt-out export.Labels from label encoding stuff
* Stop passing label encoding stuff to export.Labels
* Drop label encoding stuff from SDK
* Dogstatd exporter does not need to implement label exporter anymore
* more dogstatd exporter fixes
* export labels get back to encoding stuff
in a lame way, but improvements are coming in following commits
* Get encoded labels through export.Labels
* make SDK to provide its own implementation of export.Labels
* drop dead code
* add noop label exporter
* make export simple labels immutable
* Move the default label encoder to export package
* Simplify the simple export labels a bit
* Reserve some label exporter IDs
* Document and shuffle the code a bit
* Prepare for bring the iterator benchmark test back
We can install a callback to the Batcher's process function - this is
the place where we can access the labels, and thus test the label
iterator.
* Bring back the iterator benchmarks
* Simplifications and docs
* Fix copyright to be consistent with the rest
* Fix typo
* Put reserved label encoder IDs into constants
We get fewer comments about magic numbers that way.
* Fix the label encoder as label exporter thinko
* Update License header for all source files
- Add Apache 2.0 header to source files that did not have one.
- Update all existing headers dated to 2019 to be 2020
- Remove comma from License header to comply with the Apache 2.0
guidelines.
* Update Copyright notice
Use the standard Copyright notices outlined by the
[CNCF](https://github.com/cncf/foundation/blob/master/copyright-notices.md#copyright-notices)
The `checkpoint` function is executed in a single thread so we can do
the encoding lazily before passing the encoded version of labels to
the exporter. This is a cheap and quick way to avoid encoding the
labels on every collection interval.
Co-authored-by: Rahul Patel <rahulpa@google.com>
* Add support for Resources in the SDK
Add `Config` types for the push `Controller` and the `SDK`. Included
with this are helper functions to configure the `ErrorHandler` and
`Resource`.
Add a `Resource` to the Meter `Descriptor`. The choice to add the
`Resource` here (instead of say a `Record` or the `Instrument` itself)
was motivated by the definition of the `Descriptor` as the way to
uniquely describe a metric instrument.
Update the push `Controller` and default `SDK` to pass down their configured
`Resource` from instantiation to the metric instruments.
* Update New SDK constructor documentation
* Change NewDescriptor constructor to take opts
Add DescriptorConfig and DescriptorOption to configure the metric
Descriptor with the description, unit, keys, and resource.
Update all function calls to NewDescriptor to use new function
signature.
* Apply suggestions from code review
Co-Authored-By: Rahul Patel <rghetia@yahoo.com>
* Update and add copyright notices
* Update push controller creator func
Pass the configured ErrorHandler for the controller to the SDK.
* Update Resource integration with the SDK
Add back the Resource field to the Descriptor that was moved in the
last merge with master.
Add a resource.Provider interface.
Have the default SDK implement the new resource.Provider interface and
integrate the new interface into the newSync/newAsync workflows. Now, if
the SDK has a Resource defined it will be passed to all Descriptors
created for the instruments it creates.
* Remove nil check for metric SDK config
* Fix and add test for API Options
Add an `Equal` method to the Resource so it can be compared with
github.com/google/go-cmp/cmp.
Add additional test of the API Option unit tests to ensure WithResource
correctly sets a new resource.
* Move the resource.Provider interface to the API package
Move the interface to where it is used.
Fix spelling.
* Remove errant line
* Remove nil checks for the push controller config
* Fix check SDK implements Resourcer
* Apply suggestions from code review
Co-Authored-By: Rahul Patel <rghetia@yahoo.com>
Co-authored-by: Rahul Patel <rghetia@yahoo.com>
* Do not expose a slice of labels in export.Record
This is really an inconvenient implementation detail leak - we may
want to store labels in a different way. Replace it with an iterator -
it does not force us to use slice of key values as a storage in the
long run.
* Add Len to LabelIterator
It may come in handy in several situations, where we don't have access
to export.Labels object, but only to the label iterator.
* Use reflect value label iterator for the fixed labels
* add reset operation to iterator
Makes my life easier when writing a benchmark. Might also be an
alternative to cloning the iterator.
* Add benchmarks for iterators
* Add import comment
* Add clone operation to label iterator
* Move iterator tests to a separate package
* Add tests for cloning iterators
* Pass label iterator to export labels
* Use non-addressable array reflect values
By not using the value created by `reflect.New()`, but rather by
`reflect.ValueOf()`, we get a non-addressable array in the value,
which does not infer an allocation cost when getting an element from
the array.
* Drop zero iterator
This can be substituted by a reflect value iterator that goes over a
value with a zero-sized array.
* Add a simple iterator that implements label iterator
In the long run this will completely replace the LabelIterator
interface.
* Replace reflect value iterator with simple iterator
* Pass label storage to new export labels, not label iterator
* Drop label iterator interface, rename storage iterator to label iterator
* Drop clone operation from iterator
It's a leftover from interface times and now it's pointless - the
iterator is a simple struct, so cloning it is a simple copy.
* Drop Reset from label iterator
The sole existence of Reset was actually for benchmarking convenience.
Now we can just copy the iterator cheaply, so a need for Reset is no
more.
* Drop noop iterator tests
* Move back iterator tests to export package
* Eagerly get the reflect value of ordered labels
So we won't get into problems when several goroutines want to iterate
the same labels at the same time. Not sure if this would be a big
deal, since every goroutine would compute the same reflect.Value, but
concurrent write to the same memory is bad anyway. And it doesn't cost
us any extra allocations anyway.
* Replace NewSliceLabelIterator() with a method of LabelSlice
* Add some documentation
* Documentation fixes
* Create MeterImpl interface
* Checkpoint w/ sdk.go building
* Checkpoint working on global
* api/global builds (test fails)
* Test fix
* All tests pass
* Comments
* Add two tests
* Comments and uncomment tests
* Precommit part 1
* Still working on tests
* Lint
* Add a test and a TODO
* Cleanup
* Lint
* Interface()->Implementation()
* Apply some feedback
* From feedback
* (A)Synchronous -> (A)Sync
* Add a missing comment
* Apply suggestions from code review
Co-Authored-By: Krzesimir Nowak <qdlacz@gmail.com>
* Rename a variable
Co-authored-by: Krzesimir Nowak <qdlacz@gmail.com>
* Add request filtering capability to othhtp.Handler
* Add simple and useful filters for othttp plugin
* Add note that all requests are traced in the absence of any filters
* Add copyright notice to plugin/othttp/filters/filters_test.go
Co-Authored-By: Tyler Yahn <MrAlias@users.noreply.github.com>
* Add package docstring for filters package
Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
Co-authored-by: Rahul Patel <rahulpa@google.com>
