1
0
mirror of https://github.com/open-telemetry/opentelemetry-go.git synced 2024-12-14 10:13:10 +02:00
Commit Graph

15 Commits

Author SHA1 Message Date
Krzesimir Nowak
38e76efe99
Add a split protocol driver for otlp exporter (#1418)
* Add a split protocol driver

This is a wrapper around two other protocol drivers, so it makes it
possible to send traces using a different protocol than the one used
for metrics.

* Add an example and tests for multi GRPC endpoint driver

* Update changelog

* Document the split driver
2020-12-22 09:21:45 -08:00
Krzesimir Nowak
35215264dc
Split connection management away from exporter (#1369)
* Split protocol handling away from exporter

This commits adds a ProtocolDriver interface, which the exporter
will use to connect to the collector and send both metrics and traces
to it. That way, the Exporter type is free from dealing with any
connection/protocol details, as this business is taken over by the
implementations of the ProtocolDriver interface.

The gRPC code from the exporter is moved into the implementation of
ProtocolDriver. Currently it only maintains a single connection,
just as the Exporter used to do.

With the split, most of the Exporter options became actually gRPC
connection manager's options. Currently the only option that remained
to be Exporter's is about setting the export kind selector.

* Update changelog

* Increase the test coverage of GRPC driver

* Do not close a channel with multiple senders

The disconnected channel can be used for sending by multiple
goroutines (for example, by metric controller and span processor), so
this channel should not be closed at all. Dropping this line closes a
race between closing a channel and sending to it.

* Simplify new connection handler

The callbacks never return an error, so drop the return type from it.

* Access clients under a lock

The client may change as a result on reconnection in background, so
guard against a racy access.

* Simplify the GRPC driver a bit

The config type was exported earlier to have a consistent way of
configuring the driver, when also the multiple connection driver would
appear. Since we are not going to add a multiple connection driver,
pass the options directly to the driver constructor. Also shorten the
name of the constructor to `NewGRPCDriver`.

* Merge common gRPC code back into the driver

The common code was supposed to be shared between single connection
driver and multiple connection driver, but since the latter won't be
happening, it makes no sense to keep the not-so-common code in a
separate file. Also drop some abstraction too.

* Rename the file with gRPC driver implementation

* Update changelog

* Sleep for a second to trigger the timeout

Sometimes CI has it's better moments, so it's blazing fast and manages
to finish shutting the exporter down within the 1 microsecond timeout.

* Increase the timeout for shutting down the exporter

One millisecond is quite short, and I was getting failures locally or
in CI:

go test ./... + race in ./exporters/otlp
2020/12/14 18:27:54 rpc error: code = Canceled desc = context canceled
2020/12/14 18:27:54 context deadline exceeded
--- FAIL: TestNewExporter_withMultipleAttributeTypes (0.37s)
    otlp_integration_test.go:541: resource span count: got 0, want 1
FAIL
FAIL	go.opentelemetry.io/otel/exporters/otlp	5.278s

or

go test ./... + coverage in ./exporters/otlp
2020/12/14 17:41:16 rpc error: code = Canceled desc = context canceled
2020/12/14 17:41:16 exporter disconnected
--- FAIL: TestNewExporter_endToEnd (1.53s)
    --- FAIL: TestNewExporter_endToEnd/WithCompressor (0.41s)
        otlp_integration_test.go:246: span counts: got 3, want 4
2020/12/14 17:41:18 context canceled
FAIL
coverage: 35.3% of statements in ./...
FAIL	go.opentelemetry.io/otel/exporters/otlp	4.753s

* Shut down the providers in end to end test

This is to make sure that all batched spans are actually flushed
before closing the exporter.
2020-12-21 12:49:45 -08:00
Krzesimir Nowak
5a728db2e9
Another batch of cleanups in otlp exporter (#1357)
* Move connection logic into grpcConnection object

If we will need to maintain more than one connection in future, this
splitting off will come in handy.

Co-authored-by: Stefan Prisca <stefan.prisca@gmail.com>

* Make another channel a signal channel

There is another channel that serves as a one-time signal, where
channel's data type does not matter.

* Reorder and document connection members

This is to make clear that the lock is guarding only the connection
since it can be changed by multiple goroutines, and other members are
either atomic or read-only.

* Move stop signal into connection

The stop channel was rather useless on the exporter side - the primary
reason for existence of this channel is to stop a background
reconnecting goroutine. Since the goroutine lives entirely within
grpcConnection object, move the stop channel here. Also expose a
function to unify the stop channel with the context cancellation, so
exporter can use it without knowing anything about stop channels.

Also make export functions a bit more consistent.

* Do not run reconnection routine when being stopped too

It's possible that both disconnected channel and stop channel will be
triggered around the same time, so the goroutine is as likely to start
reconnecting as to return from the goroutine. Make sure we return if
the stop channel is closed.

* Nil clients on connection error

Set clients to nil on connection error, so we don't try to send the
data over a bad connection, but return a "no client" error
immediately.

* Do not call new connection handler within critical section

It's rather risky to call a callback coming from outside within a
critical section. Move it out.

* Add context parameter to connection routines

Connecting to the collector may also take its time, so it can be
useful in some cases to pass a context with a deadline. Currently we
just pass a background context, so this commit does not really change
any behavior. The follow-up commits will make a use of it, though.

* Add context parameter to NewExporter and Start

It makes it possible to limit the time spent on connecting to the
collector.

* Stop connecting on shutdown

Dialling to grpc service ignored the closing of the stop channel, but
this can be easily changed.

* Close connection after background is shut down

That way we can make sure that there won't be a window between closing
a connection and waiting for the background goroutine to return, where
the new connection could be established.

* Remove unnecessary nil check

This member is never nil, unless the Exporter is created like
&Exporter{}, which is not a thing we support anyway.

* Update changelog

Co-authored-by: Stefan Prisca <stefan.prisca@gmail.com>
2020-11-24 11:50:05 -08:00
Krzesimir Nowak
6eb68013b5
Some cleanups in otlp exporter (#1350)
* Drop WorkerCount option

This is not a good option - the user isn't likely to know how many
worker goroutines is optimal. This should be something that an
exporter should figure out itself. The second problem with the option
is that it is specific to the metric transformation from SDK export
format into protobuf. When the exporter starts supporting other
protocols (HTTP/JSON for example), this option may be of no use. So
the option should rather belong to the protocol, not to the
exporter. Currently both mean the same, but later they will be
separated, and this option breaks the separation.

* Make stop channel a typical signalling channel

Signalling channels are idiomatically defined as chan struct{}, so
let's make it so, to avoid confusion about the meaning of the bool
type.

* Close a race when grpc connection is closed multiple times

If several goroutines call Shutdown at the same time, then the
following scenario is possible:

goroutine A locks a mutex, reads a started member, unlocks the mutex
and gets preempted

goroutine B locks a mutex, reads a started member, unlocks the mutex
and gets preempted

goroutine A does not return early in the "if !started" conditional and
continues to close the connection and execute the rest of the function
(where it finally sets the started member to false), gets preempted

goroutine B also does not return early, because it got a copy of
started before goroutine A set it to false, so it tries to close the
connection again.

* Update changelog
2020-11-19 21:03:25 -05:00
Tyler Yahn
422188a85f
Correct SDK trace Exporter interface (#1078)
* Update trace export interface

Move to conforming to the specification.

* Update documentation in export trace

* Update sdk trace provider to support new trace exporter

* Update SpanProcessors

Support the Provider changes and new trace exporter.

* Update the SDK to support the changes

* Update trace Provider to not return an error

* Update sdk with new Provider return

Also fix the testExporter ExportSpans method

* Update exporters with changes

* Update examples with changes

* Update Changelog

* Move error handling to end of shutdown

* Update exporter interface

Rename to SpanExporter to match specification. Add an error return value
to the Shutdown method based on feedback. Propagate these changes.

Remove the Stop method from the OTLP exporter to avoid confusion and
redundancy.

* Add test to check OTLP Shutdown honors context

* Add Jaeger exporter test for shutdown

* Fix race in Jaeger test

* Unify shutdown behavior and testing

* Update sdk/trace/simple_span_processor.go

Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>

Co-authored-by: Anthony Mirabella <a9@aneurysm9.com>
2020-09-09 10:19:03 -07:00
Tyler Yahn
0a9e861549
Update the OTLP exporter batching of traces (#623)
Instead of batching based on the Resource pointer which is not guaranteed
to uniquely identify common Resources, use the `String` method which is.

Renames the external test package to be identified as the integration
testing. This testing structure (the mock exporter) batches exported
resources transmitted from the resource. This is needed as there is no
guarantees that a batch will be exported in a single payload, therefore,
it is not an ideal place to add resource batch testing for the exporter
itself.

Add unit tests to ensure the Exporter is correctly batching spans by
resource.
2020-04-08 16:49:53 -07:00
Joshua MacDonald
e8546e3bc5
Remove Labelset (#595)
* Remove LabelSet frmo api/metric

* SDK tests pass

* Restore benchmarks

* All tests pass

* Remove all mentions of LabelSet

* Test RecordBatch

* Batch test

* Improves benchmark (some)

* Move the benchmark to match HEAD

* Align labels for GOARCH=386

* Add alignment test

* Disable the stress test fo GOARCH=386

* Fix bug

* Move atomic fields into their own file

* Add a TODO

* Comments

* Remove metric.Labels(...)

* FTB

Co-authored-by: Liz Fong-Jones <lizf@honeycomb.io>
2020-03-27 14:06:48 -07:00
Tyler Yahn
c97b4f726c
Update project License headers and checking (#596)
Update license header to standard format for source files missed prior.

Add license header to new source files.

Add Makefile check to test all `*.go` and `*.sh` files have a copyright
notice (or comment about them being auto-generated) within the first few
lines.
2020-03-25 14:47:17 -07:00
Krzesimir Nowak
d648712cf2
Kick label encoder out of sdk (#574)
* Temporarily opt-out export.Labels from label encoding stuff

* Stop passing label encoding stuff to export.Labels

* Drop label encoding stuff from SDK

* Dogstatd exporter does not need to implement label exporter anymore

* more dogstatd exporter fixes

* export labels get back to encoding stuff

in a lame way, but improvements are coming in following commits

* Get encoded labels through export.Labels

* make SDK to provide its own implementation of export.Labels

* drop dead code

* add noop label exporter

* make export simple labels immutable

* Move the default label encoder to export package

* Simplify the simple export labels a bit

* Reserve some label exporter IDs

* Document and shuffle the code a bit

* Prepare for bring the iterator benchmark test back

We can install a callback to the Batcher's process function - this is
the place where we can access the labels, and thus test the label
iterator.

* Bring back the iterator benchmarks

* Simplifications and docs

* Fix copyright to be consistent with the rest

* Fix typo

* Put reserved label encoder IDs into constants

We get fewer comments about magic numbers that way.

* Fix the label encoder as label exporter thinko
2020-03-24 09:30:12 -07:00
Rahul Patel
6f881b4400
update to proto v0.3.0 (#588) 2020-03-24 08:45:07 -07:00
Joshua MacDonald
d8682c1999
Refactor the SDK helpers, create MeterImpl (#560)
* Create MeterImpl interface

* Checkpoint w/ sdk.go building

* Checkpoint working on global

* api/global builds (test fails)

* Test fix

* All tests pass

* Comments

* Add two tests

* Comments and uncomment tests

* Precommit part 1

* Still working on tests

* Lint

* Add a test and a TODO

* Cleanup

* Lint

* Interface()->Implementation()

* Apply some feedback

* From feedback

* (A)Synchronous -> (A)Sync

* Add a missing comment

* Apply suggestions from code review

Co-Authored-By: Krzesimir Nowak <qdlacz@gmail.com>

* Rename a variable

Co-authored-by: Krzesimir Nowak <qdlacz@gmail.com>
2020-03-19 12:02:46 -07:00
Tyler Yahn
435c39aab4
Update OTLP exporter with latest proto (#550)
[Upstream
changes](https://github.com/open-telemetry/opentelemetry-proto/pull/126)

Co-authored-by: Rahul Patel <rahulpa@google.com>
2020-03-16 16:43:54 -07:00
Rahul Patel
6ada85adba
add resource option to Provider. (#545)
- update otlp exporter to export resources.
2020-03-13 13:07:36 -07:00
Tyler Yahn
cba1664b46
Add metrics support to the OTLP exporter (#544)
* Initial metrics addition to the OTLP exporter

* Fixes

Update to incorporate merged changes.
Fix lint issues.

* Add sum float64 transform unit test

* Fix static check

* Update comments

Fix malformed License header.
Add documentation for new transform functions.
Remove errant TODO.

* Fix test failures and handle ErrEmptyDataSet

Use `assert.NoError` instead of `assert.Nil` to correctly display
checked errors.

Use the result of `assert.NoError` to guard against `nil` pointer
dereferences.

Add check to skip `Record`s that return an `ErrEmptyDataSet` error and
include test to check this error is correctly returned from the
transform package.

Co-authored-by: Rahul Patel <rahulpa@google.com>
2020-03-13 11:42:20 -07:00
Rahul Patel
5850278197
opentelemetry collector trace exporter (#497)
* opentelemetry collector exporter

- missing load test
- missing resources

* fix review comments.

* add test for each SpanKind and Attribute Type.

* rename otelcol to otlp

* move exporter/trace/otlp to exporters/otlp

* more review comments.

* add alignment test.

* pass context to uploadSpans
2020-03-06 22:01:02 -08:00