go/opentelemetry-go
go/opentelemetry-go
1
0
Fork 0
mirror of https://github.com/open-telemetry/opentelemetry-go.git synced 2024-12-14 10:13:10 +02:00
Code Issues Releases Activity

Merge pull request #711 from stefanprisca/master

Browse Source 
OpenTelemetry Collector Demo
This commit is contained in:
Tyler Yahn 2020-05-20 09:26:53 -07:00 committed by GitHub
commit 19ee7b7a97
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
9 changed files with 1773 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
.gitignore vendored
View File

@ -17,3 +17,4 @@ coverage.*
/example/namedtracer/namedtracer
/example/prometheus/prometheus
/example/zipkin/zipkin
/example/otel-collector/otel-collector

34
example/otel-collector/Makefile Normal file
View File

@ -0,0 +1,34 @@
namespace-k8s:
kubectl apply -f k8s/namespace.yaml
jaeger-operator-k8s:
# Create the jaeger operator and necessary artifacts in ns observability
kubectl create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/crds/jaegertracing.io_jaegers_crd.yaml
kubectl create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/service_account.yaml
kubectl create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/role.yaml
kubectl create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/role_binding.yaml
kubectl create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/operator.yaml
# Create the cluster role & bindings
kubectl create -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/cluster_role.yaml
kubectl create -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/cluster_role_binding.yaml
jaeger-k8s:
kubectl apply -f k8s/jaeger.yaml
otel-collector-k8s:
kubectl apply -f k8s/otel-collector.yaml
clean-k8s:
- kubectl delete -f k8s/otel-collector.yaml
- kubectl delete -f k8s/jaeger.yaml
- kubectl delete -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/cluster_role.yaml
- kubectl delete -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/cluster_role_binding.yaml
- kubectl delete -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/crds/jaegertracing.io_jaegers_crd.yaml
- kubectl delete -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/service_account.yaml
- kubectl delete -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/role.yaml
- kubectl delete -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/role_binding.yaml
- kubectl delete -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/operator.yaml

183
example/otel-collector/README.md Normal file
View File

@ -0,0 +1,183 @@
# OpenTelemetry Collector Traces Example
This example illustrates how to export traces from the OpenTelemetry-Go SDK to the OpenTelemetry Collector, and from there to a Jaeger instance.
The complete flow is:
`Demo App -> OpenTelemetry Collector -> Jaeger`
# Prerequisites
The demo is built on Kubernetes, and uses a local instance of [microk8s](https://microk8s.io/). You will need access to a cluster in order to deploy the OpenTelemetry Collector and Jaeger components from this demo.
For simplicity, the demo application is not part of the k8s cluster, and will access the OpenTelemetry Collector through a NodePort on the cluster. Note that the NodePort opened by this demo is not secured.
Ideally you'd want to either have your application running as part of the kubernetes cluster, or use a secured connection (NodePort/LoadBalancer with TLS or an ingress extension).
# Deploying Jaeger and OpenTelemetry Collector
The first step of this demo is to deploy a Jaeger instance and a Collector to your cluster. All the necessary Kubernetes deployment files are available in this demo, in the [k8s](./k8s) folder.
There are two ways to create the necessary deployments for this demo: using the [makefile](./Makefile) or manually applying the k8s files.
## Using the makefile
For using the [makefile](./Makefile), run the following commands in order:
```bash
# Create the namespace
make namespace-k8s
# Deploy Jaeger operator
make jaeger-operator-k8s
# After the operator is deployed, create the Jaeger instance
make jaeger-k8s
# Finally, deploy the OpenTelemetry Collector
make otel-collector-k8s
```
If you want to clean up after this, you can use the `make clean-k8s` to delete all the resources created above. Note that this will not remove the namespace. Because Kubernetes sometimes gets stuck when removing namespaces, please remove this namespace manually after all the resources inside have been deleted.
## Manual deployment
For manual deployments, follow the same steps as above, but instead run the `kubectl apply` yourself.
First, the namespace needs to be created:
```bash
k apply -f k8s/namespace.yaml
```
Jaeger is then deployed via the operator, and the demo follows [these steps](https://github.com/jaegertracing/jaeger-operator#getting-started) to create it:
```bash
# Create the jaeger operator and necessary artifacts in ns observability
kubectl create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/crds/jaegertracing.io_jaegers_crd.yaml
kubectl create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/service_account.yaml
kubectl create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/role.yaml
kubectl create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/role_binding.yaml
kubectl create -n observability -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/operator.yaml
# Create the cluster role & bindings
kubectl create -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/cluster_role.yaml
kubectl create -f https://raw.githubusercontent.com/jaegertracing/jaeger-operator/master/deploy/cluster_role_binding.yaml
# Create the Jaeger instance itself:
kubectl apply -f k8s/jaeger/jaeger.yaml
```
The OpenTelemetry Collector is contained in a single k8s file, which can be deployed with one command:
```bash
k8s apply -f k8s/otel-collector.yaml
```
# Configuring the OpenTelemetry Collector
Although the above steps should deploy and configure both Jaeger and the OpenTelemetry Collector, it might be worth spending some time on the [configuration](./k8s/otel-collector.yaml) of the Collector.
One important part here is that, in order to enable our application to send traces to the OpenTelemetry Collector, we need to first configure the otlp receiver:
```yml
...
otel-collector-config: |
receivers:
# Make sure to add the otlp receiver.
# This will open up the receiver on port 55680.
otlp:
endpoint: 0.0.0.0:55680
processors:
...
```
This will create the receiver on the Collector side, and open up port `55680` for receiving traces.
The rest of the configuration is quite standard, with the only mention that we need to create the Jaeger exporter:
```yml
...
exporters:
jaeger_grpc:
endpoint: "jaeger-collector.observability.svc.cluster.local:14250"
...
```
## OpenTelemetry Collector service
One more aspect in the OpenTelemetry Collector [configuration](./k8s/otel-collector.yaml) worth looking at is the NodePort service used for accessing it:
```yaml
apiVersion: v1
kind: Service
metadata:
...
spec:
ports:
- name: otlp # Default endpoint for otlp receiver.
port: 55680
protocol: TCP
targetPort: 55680
nodePort: 30080
- name: metrics # Default endpoint for metrics.
port: 8888
protocol: TCP
targetPort: 8888
selector:
component: otel-collector
type:
NodePort
```
This service will bind the `55680` port used to access the otlp receiver to port `30080` on your cluster's node. By doing so, it makes it possible for us to access the Collector by using the static address `<node-ip>:30080`. In case you are running a local cluster, this will be `localhost:30080`. Note that you can also change this to a LoadBalancer or have an ingress extension for accessing the service.
# Writing the demo
Having the OpenTelemetry Collector started with the otlp port open for traces, and connected to Jaeger, let's look at the go app that will send traces to the Collector.
First, we need to create an exporter using the [otlp](https://pkg.go.dev/go.opentelemetry.io/otel/exporters/otlp?tab=doc) package:
```go
exp, err := otlp.NewExporter(otlp.WithInsecure(),
// use the address of the NodePort service created above
// <node-ip>:30080
otlp.WithAddress("localhost:30080"),
otlp.WithGRPCDialOption(grpc.WithBlock()))
if err != nil {
log.Fatalf("Failed to create the collector exporter: %v", err)
}
defer func() {
err := exp.Stop()
if err != nil {
log.Fatalf("Failed to stop the exporter: %v", err)
}
}()
```
This will initialize the exporter and connect to the otlp receiver at the address that we set for the [NodePort](#opentelemetry-collector-service): `localhost:30080`.
Feel free to remove the blocking operation, but it might come in handy when testing the connection.
Also, make sure to close the exporter before the app exits.
The next step is to create the TraceProvider:
```go
tp, err := sdktrace.NewProvider(
sdktrace.WithConfig(sdktrace.Config{DefaultSampler: sdktrace.AlwaysSample()}),
sdktrace.WithResourceAttributes(
// the service name used to display traces in Jaeger
core.Key(conventions.AttributeServiceName).String("test-service"),
),
sdktrace.WithBatcher(exp, // add following two options to ensure flush
sdktrace.WithScheduleDelayMillis(5),
sdktrace.WithMaxExportBatchSize(2),
))
if err != nil {
log.Fatalf("error creating trace provider: %v\n", err)
}
```
It is important here to set the [AttributeServiceName](https://github.com/open-telemetry/opentelemetry-collector/blob/master/translator/conventions/opentelemetry.go#L20) from the `github.com/open-telemetry/opentelemetry-collector/translator/conventions` package on the resource level. This will be passed to the OpenTelemetry Collector, and used as ServiceName when exporting the traces to Jaeger.
After this, you can simply start sending traces:
```go
tracer := tp.Tracer("test-tracer")
ctx, span := tracer.Start(context.Background(), "CollectorExporter-Example")
defer span.End()
```
The traces should now be visible from the Jaeger UI (if you have it installed), or thorough the jaeger-query service, under the name `test-service`.
You can find the complete code for this example in the [main.go](./main.go) file.

10
example/otel-collector/go.mod Normal file
View File

@ -0,0 +1,10 @@
module go.opentelemetry.io/otel/example/otel-collector
go 1.14
require (
github.com/open-telemetry/opentelemetry-collector v0.3.0
go.opentelemetry.io/otel v0.5.0
go.opentelemetry.io/otel/exporters/otlp v0.5.0
google.golang.org/grpc v1.29.1
)

1285
example/otel-collector/go.sum Normal file
View File

File diff suppressed because it is too large Load Diff

19
example/otel-collector/k8s/jaeger.yaml Normal file
View File

@ -0,0 +1,19 @@
# Copyright The OpenTelemetry Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: jaegertracing.io/v1
kind: Jaeger
metadata:
name: jaeger
namespace: observability

18
example/otel-collector/k8s/namespace.yaml Normal file
View File

@ -0,0 +1,18 @@
# Copyright The OpenTelemetry Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: v1
kind: Namespace
metadata:
name: observability

144
example/otel-collector/k8s/otel-collector.yaml Normal file
View File

@ -0,0 +1,144 @@
# Copyright The OpenTelemetry Authors
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# Configuration based on otel-collector k8s demo:
# https://github.com/open-telemetry/opentelemetry-collector/blob/master/examples/k8s.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: otel-collector-conf
namespace: observability
labels:
app: opentelemetry
component: otel-collector-conf
data:
otel-collector-config: |
receivers:
# Make sure to add the otlp receiver.
# This will open up the receiver on port 55680
otlp:
endpoint: 0.0.0.0:55680
processors:
extensions:
health_check: {}
exporters:
jaeger_grpc:
endpoint: "jaeger-collector.observability.svc.cluster.local:14250"
service:
extensions: [health_check]
pipelines:
traces:
receivers: [otlp]
processors: []
exporters: [jaeger_grpc]
---
apiVersion: v1
kind: Service
metadata:
name: otel-collector
namespace: observability
labels:
app: opentelemetry
component: otel-collector
spec:
ports:
- name: otlp # Default endpoint for otlp receiver.
port: 55680
protocol: TCP
targetPort: 55680
nodePort: 30080
- name: metrics # Default endpoint for metrics.
port: 8888
protocol: TCP
targetPort: 8888
selector:
component: otel-collector
type:
NodePort
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: otel-collector
namespace: observability
labels:
app: opentelemetry
component: otel-collector
spec:
selector:
matchLabels:
app: opentelemetry
component: otel-collector
minReadySeconds: 5
progressDeadlineSeconds: 120
replicas: 1 #TODO - adjust this to your own requirements
template:
metadata:
annotations:
prometheus.io/path: "/metrics"
prometheus.io/port: "8888"
prometheus.io/scrape: "true"
labels:
app: opentelemetry
component: otel-collector
spec:
containers:
- command:
- "/otelcol"
- "--config=/conf/otel-collector-config.yaml"
# Memory Ballast size should be max 1/3 to 1/2 of memory.
- "--mem-ballast-size-mib=683"
env:
- name: GOGC
value: "80"
image: otel/opentelemetry-collector:0.3.0
name: otel-collector
resources:
limits:
cpu: 1
memory: 2Gi
requests:
cpu: 200m
memory: 400Mi
ports:
- containerPort: 55680 # Default endpoint for otlp receiver.
- containerPort: 8888 # Default endpoint for querying metrics.
volumeMounts:
- name: otel-collector-config-vol
mountPath: /conf
# - name: otel-collector-secrets
# mountPath: /secrets
livenessProbe:
httpGet:
path: /
port: 13133 # Health Check extension default port.
readinessProbe:
httpGet:
path: /
port: 13133 # Health Check extension default port.
volumes:
- configMap:
name: otel-collector-conf
items:
- key: otel-collector-config
path: otel-collector-config.yaml
name: otel-collector-config-vol
# - secret:
# name: otel-collector-secrets
# items:
# - key: cert.pem
# path: cert.pem
# - key: key.pem
# path: key.pem

79
example/otel-collector/main.go Normal file
View File

@ -0,0 +1,79 @@
// Copyright The OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
// Example from otlp package: https://pkg.go.dev/go.opentelemetry.io/otel/exporters/otlp?tab=doc#example-package-Insecure
package main
import (
"context"
"fmt"
"log"
"time"
"google.golang.org/grpc"
"go.opentelemetry.io/otel/api/kv"
"go.opentelemetry.io/otel/exporters/otlp"
sdktrace "go.opentelemetry.io/otel/sdk/trace"
"github.com/open-telemetry/opentelemetry-collector/translator/conventions"
)
func main() {
// If the OpenTelemetry Collector is running on a local cluster (minikube or microk8s),
// it should be accessible through the NodePort service at the `localhost:30080` address.
// Otherwise, replace `localhost` with the address of your cluster.
// If you run the app inside k8s, then you can probably connect directly to the service through dns
exp, err := otlp.NewExporter(otlp.WithInsecure(),
otlp.WithAddress("localhost:30080"),
otlp.WithGRPCDialOption(grpc.WithBlock()))
if err != nil {
log.Fatalf("Failed to create the collector exporter: %v", err)
}
defer func() {
err := exp.Stop()
if err != nil {
log.Fatalf("Failed to stop the exporter: %v", err)
}
}()
tp, err := sdktrace.NewProvider(
sdktrace.WithConfig(sdktrace.Config{DefaultSampler: sdktrace.AlwaysSample()}),
sdktrace.WithResourceAttributes(
// the service name used to display traces in Jaeger
kv.Key(conventions.AttributeServiceName).String("test-service"),
),
sdktrace.WithBatcher(exp, // add following two options to ensure flush
sdktrace.WithScheduleDelayMillis(5),
sdktrace.WithMaxExportBatchSize(2),
))
if err != nil {
log.Fatalf("error creating trace provider: %v\n", err)
}
tracer := tp.Tracer("test-tracer")
// Then use the OpenTelemetry tracing library, like we normally would.
ctx, span := tracer.Start(context.Background(), "CollectorExporter-Example")
for i := 0; i < 10; i++ {
_, iSpan := tracer.Start(ctx, fmt.Sprintf("Sample-%d", i))
<-time.After(time.Second)
iSpan.End()
}
span.End()
// Wait 1 second before ending
<-time.After(time.Second)
}