prometheus/ssl_exporter
1
0
Fork 0
mirror of https://github.com/ribbybibby/ssl_exporter.git synced 2024-11-24 08:22:17 +02:00
Code Issues Releases Activity

Use https or tcp client based on target address

Browse Source 
There are some advantages to using a http client over tcp. For instance,
using http allows you to take advatange of a http proxy, which may be necessary
in some environments.

This commit puts the http client back, alongside tcp, and decides which one to use
based on the target address.
This commit is contained in:
Rob Best 2019-08-15 09:43:34 +01:00
parent 10353fe7fb
commit cfab972f8f
3 changed files with 235 additions and 62 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

117
README.md
View File

@ -1,80 +1,116 @@
# SSL Certificate Exporter
The [blackbox_exporter](https://github.com/prometheus/blackbox_exporter) allows you to test the expiry date of a certificate as part of its HTTP(S) probe - which is great. It doesn't, however, tell you which certificate in the chain is nearing expiry or give you any other information that might be useful when sending alerts.
The [blackbox_exporter](https://github.com/prometheus/blackbox_exporter) allows you to test the expiry date of a certificate as part of its HTTP(S) probe - which is great. It doesn't, however, tell you which certificate in the chain is nearing expiry or give you any other information that might be useful when sending alerts.
For instance, there's a definite value in knowing, upon first receiving an alert, if it's a certificate you manage directly or one further up the chain. It's also not always necessarily clear from the address you're polling what kind of certificate renewal you're looking at. Is it a Let's Encrypt, in which case it should be handled by automation? Or your organisation's wildcard? Maybe the domain is managed by a third-party and you need to submit a ticket to get it renewed.
For instance, there's a definite value in knowing, upon first receiving an alert, if it's a certificate you manage directly or one further up the chain. It's also not always necessarily clear from the address you're polling what kind of certificate renewal you're looking at. Is it a Let's Encrypt, in which case it should be handled by automation? Or your organisation's wildcard? Maybe the domain is managed by a third-party and you need to submit a ticket to get it renewed.
Whatever it is, the SSL exporter gives you visibility over those dimensions at the point at which you receive an alert. It also allows you to produce more meaningful visualisations and consoles.
## Building
make
./ssl_exporter <flags>
Similarly to the blackbox_exporter, visiting [http://localhost:9219/probe?target=example.com:443](http://localhost:9219/probe?target=example.com:443) will return certificate metrics for example.com. The ```ssl_tls_connect_success``` metric indicates if the probe has been successful.
Similarly to the blackbox_exporter, visiting [http://localhost:9219/probe?target=example.com:443](http://localhost:9219/probe?target=example.com:443) will return certificate metrics for example.com. The `ssl_tls_connect_success` metric indicates if the probe has been successful.
## Docker
docker pull ribbybibby/ssl-exporter
docker run -p 9219:9219 ssl-exporter:latest <flags>
## Flags
./ssl_exporter --help
* __`--tls.insecure`:__ Skip certificate verification (default false). This is insecure but does allow you to collect metrics in the case where a certificate has expired. That being said, I feel that it's more important to catch verification failures than it is to identify an expired certificate, especially as the former includes the latter.
* __`--tls.cacert`:__ Provide the path to an alternative bundle of root CA certificates. By default the exporter will use the host's root CA set.
* __`--tls.client-auth`:__ Enable client authentication (default false). When enabled the exporter will present the certificate and key configured by `--tls.cert` and `tls.key` to the other side of the connection.
* __`--tls.cert`:__ The path to a local certificate for client authentication (default "cert.pem"). Only used when `--tls.client-auth` is toggled on.
* __`--tls.key`:__ The path to a local key for client authentication (default "key.pem"). Only used when `--tls.client-auth` is toggled on.
* __`--web.listen-address`:__ The port (default ":9219").
* __`--web.metrics-path`:__ The path metrics are exposed under (default "/metrics")
* __`--web.probe-path`:__ The path the probe endpoint is exposed under (default "/probe")
- **`--tls.insecure`:** Skip certificate verification (default false). This is insecure but does allow you to collect metrics in the case where a certificate has expired. That being said, I feel that it's more important to catch verification failures than it is to identify an expired certificate, especially as the former includes the latter.
- **`--tls.cacert`:** Provide the path to an alternative bundle of root CA certificates. By default the exporter will use the host's root CA set.
- **`--tls.client-auth`:** Enable client authentication (default false). When enabled the exporter will present the certificate and key configured by `--tls.cert` and `tls.key` to the other side of the connection.
- **`--tls.cert`:** The path to a local certificate for client authentication (default "cert.pem"). Only used when `--tls.client-auth` is toggled on.
- **`--tls.key`:** The path to a local key for client authentication (default "key.pem"). Only used when `--tls.client-auth` is toggled on.
- **`--web.listen-address`:** The port (default ":9219").
- **`--web.metrics-path`:** The path metrics are exposed under (default "/metrics")
- **`--web.probe-path`:** The path the probe endpoint is exposed under (default "/probe")
## Metrics
Metrics are exported for each certificate in the chain individually. All of the metrics are labelled with the Issuer's Common Name and the Serial ID, which is pretty much a unique identifier.
I considered having a series for each ```ssl_cert_subject_alternative_*``` value but these labels aren't actually very cardinal, considering the most frequently they'll change is probably every three months, which is longer than most metric retention times anyway. Joining them within commas as I've done allows for easy parsing and relabelling.
I considered having a series for each `ssl_cert_subject_alternative_*` value but these labels aren't actually very cardinal, considering the most frequently they'll change is probably every three months, which is longer than most metric retention times anyway. Joining them within commas as I've done allows for easy parsing and relabelling.
| Metric | Meaning | Labels |
| ------ | ------- | ------ |
| ssl_cert_not_after | The date after which the certificate expires. Expressed as a Unix Epoch Time. | issuer_cn, serial_no |
| ssl_cert_not_before | The date before which the certificate is not valid. Expressed as a Unix Epoch Time. | issuer_cn, serial_no |
| ssl_cert_subject_common_name | The common name of the certificate. Always has a value of 1 | issuer_cn, serial_no, subject_cn |
| ssl_cert_subject_alternative_dnsnames | The subject alternative names (if any). Always has a value of 1 | issuer_cn, serial_no, dnsnames |
| ssl_cert_subject_alternative_emails | The subject alternative email addresses (if any). Always has a value of 1 | issuer_cn, serial_no, emails |
| ssl_cert_subject_alternative_ips | The subject alternative IP addresses (if any). Always has a value of 1 | issuer_cn, serial_no, ips |
| ssl_cert_subject_organization_units | The subject organization names (if any). Always has a value of 1. | issuer_cn, serial_no, subject_ou |
| ssl_tls_connect_success | Was the TLS connection successful? Boolean. | |
| Metric | Meaning | Labels |
| ------------------------------------- | ----------------------------------------------------------------------------------- | -------------------------------- |
| ssl_cert_not_after | The date after which the certificate expires. Expressed as a Unix Epoch Time. | issuer_cn, serial_no |
| ssl_cert_not_before | The date before which the certificate is not valid. Expressed as a Unix Epoch Time. | issuer_cn, serial_no |
| ssl_cert_subject_common_name | The common name of the certificate. Always has a value of 1 | issuer_cn, serial_no, subject_cn |
| ssl_cert_subject_alternative_dnsnames | The subject alternative names (if any). Always has a value of 1 | issuer_cn, serial_no, dnsnames |
| ssl_cert_subject_alternative_emails | The subject alternative email addresses (if any). Always has a value of 1 | issuer_cn, serial_no, emails |
| ssl_cert_subject_alternative_ips | The subject alternative IP addresses (if any). Always has a value of 1 | issuer_cn, serial_no, ips |
| ssl_cert_subject_organization_units | The subject organization names (if any). Always has a value of 1. | issuer_cn, serial_no, subject_ou |
| ssl_client_protocol | The protocol used by the exporter to connect to the target. Boolean. | protocol |
| ssl_tls_connect_success | Was the TLS connection successful? Boolean. | |
## Prometheus
### Configuration
Just like with the blackbox_exporter, you should pass the targets to a single instance of the exporter in a scrape config with a clever bit of relabelling. This allows you to leverage service discovery and keeps configuration centralised to your Prometheus config.
```yml
scrape_configs:
- job_name: 'ssl'
- job_name: "ssl"
metrics_path: /probe
static_configs:
- targets:
- example.com:443
- prometheus.io:443
- example.com:443
- prometheus.io:443
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- source_labels: [__param_target]
target_label: instance
- target_label: __address__
replacement: 127.0.0.1:9219 # SSL exporter.
replacement: 127.0.0.1:9219 # SSL exporter.
```
### Example Queries
Certificates that expire within 7 days, with Subject Common Name and Subject Alternative Names joined on:
((ssl_cert_not_after - time() < 86400 * 7) * on (instance,issuer_cn,serial_no) group_left (dnsnames) ssl_cert_subject_alternative_dnsnames) * on (instance,issuer_cn,serial_no) group_left (subject_cn) ssl_cert_subject_common_name
### Targets
The exporter uses the provided uri to decide which client (http or tcp) to use when connecting to the target. The uri must contain
either a protocol scheme (`https://`), a port (`:443`), or both (`https://example.com:443`).
If the `https://` scheme is provided then the exporter will use a http client to connect to the target. This allows you to take
advatange of some features not available when using tcp, like host-based proxying. The exporter doesn't understand any other L7
protocols, so it will produce an error for others, like `ldaps://` or `ftps://`.
If there's only a port, then a tcp client is used to make the TLS connection. This should allow you to connect to any TLS target, regardless
of L7 protocol.
If neither are given, the exporter assumes a https connection on port `443` (the most common case).
#### Valid targets
- `https://example.com`
- `https://example.com:443`
- `example.com:443`
- `example.com:636`
- `example.com`
#### Invalid targets
- `ldaps://example.com`
- `ldaps://example.com:636`
### Example Queries
Certificates that expire within 7 days, with Subject Common Name and Subject Alternative Names joined on:
((ssl*cert_not_after - time() < 86400 * 7) \_ on (instance,issuer_cn,serial_no) group_left (dnsnames) ssl_cert_subject_alternative_dnsnames) \* on (instance,issuer_cn,serial_no) group_left (subject_cn) ssl_cert_subject_common_name
Only return wildcard certificates that are expiring:
((ssl_cert_not_after - time() < 86400 * 7) * on (instance,issuer_cn,serial_no) group_left (subject_cn) ssl_cert_subject_common_name{subject_cn=~"\\*.*"})
Number of certificates in the chain:
count(ssl_cert_subject_common_name) by (instance)
Identify instances that have failed to create a valid SSL connection:
@ -82,14 +118,29 @@ Identify instances that have failed to create a valid SSL connection:
ssl_tls_connect_success == 0
## Client authentication
The exporter optionally supports client authentication, which can be toggled on by providing the `--tls.client-auth` flag. By default, it will use the host system's root CA bundle and attempt to use `./cert.pem` and `./key.pem` as the client certificate and key, respectively. You can override these defaults with `--tls.cacert`, `--tls.cert` and `--tls.key`.
If you do enable client authentication, keep in mind that the certificate will be passed to all targets, even those that don't necessarily require client authentication. I'm not sure what the implications of that are but I think you'd probably want to avoid passing a certificate to an unrelated server.
Also, if you want to scrape targets with different client certificate requirements, you'll need to run different instances of the exporter for each. This seemed like a better approach than overloading the exporter with the ability to pass different certificates per-target.
## Proxying
The https client used by the exporter supports the use of proxy servers discovered by the environment variables `HTTP_PROXY`,
`HTTPS_PROXY` and `ALL_PROXY`.
For instance:
$ export HTTPS_PROXY=localhost:8888
$ ./ssl_exporter
In order to use the https client, targets must be provided to the exporter with the protocol in the uri (`https://<host>:<optional port>`).
## Limitations
I've only exported a subset of the information you could extract from a certificate. It would be simple to add more, for instance organisational information, if there's a need.
## Acknowledgements
The overall structure and implementation of this exporter is based on the [consul_exporter](https://github.com/prometheus/consul_exporter). The probing functionality borrows from the blackbox_exporter.

118
ssl_exporter.go
View File

@ -3,6 +3,7 @@ package main
import (
"crypto/tls"
"crypto/x509"
"errors"
"fmt"
"io/ioutil"
"net"
@ -29,6 +30,11 @@ var (
"If the TLS connection was a success",
nil, nil,
)
clientProtocol = prometheus.NewDesc(
prometheus.BuildFQName(namespace, "", "client_protocol"),
"The protocol used by the exporter to connect to the target",
[]string{"protocol"}, nil,
)
notBefore = prometheus.NewDesc(
prometheus.BuildFQName(namespace, "", "cert_not_before"),
"NotBefore expressed as a Unix Epoch Time",
@ -76,6 +82,7 @@ type Exporter struct {
// Describe metrics
func (e *Exporter) Describe(ch chan<- *prometheus.Desc) {
ch <- tlsConnectSuccess
ch <- clientProtocol
ch <- notAfter
ch <- commonName
ch <- subjectAlernativeDNSNames
@ -86,8 +93,10 @@ func (e *Exporter) Describe(ch chan<- *prometheus.Desc) {
// Collect metrics
func (e *Exporter) Collect(ch chan<- prometheus.Metric) {
var peerCertificates []*x509.Certificate
conn, err := tls.DialWithDialer(&net.Dialer{Timeout: e.timeout}, "tcp", e.target, e.tlsConfig)
// Parse the target and return the appropriate connection protocol and target address
target, proto, err := parseTarget(e.target)
if err != nil {
log.Errorln(err)
ch <- prometheus.MustNewConstMetric(
@ -96,10 +105,75 @@ func (e *Exporter) Collect(ch chan<- prometheus.Metric) {
return
}
state := conn.ConnectionState()
ch <- prometheus.MustNewConstMetric(
clientProtocol, prometheus.GaugeValue, 1, proto,
)
if len(state.PeerCertificates) < 1 {
log.Errorln("No certificates found in connection state")
if proto == "https" {
ch <- prometheus.MustNewConstMetric(
clientProtocol, prometheus.GaugeValue, 0, "tcp",
)
// Create the http client
client := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
Transport: &http.Transport{
TLSClientConfig: e.tlsConfig,
Proxy: http.ProxyFromEnvironment,
},
Timeout: e.timeout,
}
// Issue a GET request to the target
resp, err := client.Get(e.target)
if err != nil {
log.Errorln(err)
ch <- prometheus.MustNewConstMetric(
tlsConnectSuccess, prometheus.GaugeValue, 0,
)
return
}
// Check if the response from the target is encrypted
if resp.TLS == nil {
log.Errorln("The response from " + target + " is unencrypted")
ch <- prometheus.MustNewConstMetric(
tlsConnectSuccess, prometheus.GaugeValue, 0,
)
return
}
peerCertificates = resp.TLS.PeerCertificates
} else if proto == "tcp" {
ch <- prometheus.MustNewConstMetric(
clientProtocol, prometheus.GaugeValue, 0, "https",
)
conn, err := tls.DialWithDialer(&net.Dialer{Timeout: e.timeout}, "tcp", target, e.tlsConfig)
if err != nil {
log.Errorln(err)
ch <- prometheus.MustNewConstMetric(
tlsConnectSuccess, prometheus.GaugeValue, 0,
)
return
}
state := conn.ConnectionState()
peerCertificates = state.PeerCertificates
if len(peerCertificates) < 1 {
log.Errorln("No certificates found in connection state for " + target)
ch <- prometheus.MustNewConstMetric(
tlsConnectSuccess, prometheus.GaugeValue, 0,
)
return
}
} else {
log.Errorln("Unrecognised protocol: " + string(proto) + " for target: " + target)
ch <- prometheus.MustNewConstMetric(
tlsConnectSuccess, prometheus.GaugeValue, 0,
)
@ -111,7 +185,7 @@ func (e *Exporter) Collect(ch chan<- prometheus.Metric) {
)
// Remove duplicate certificates from the response
peerCertificates := uniq(state.PeerCertificates)
peerCertificates = uniq(peerCertificates)
// Loop through returned certificates and create metrics
for _, cert := range peerCertificates {
@ -173,7 +247,6 @@ func (e *Exporter) Collect(ch chan<- prometheus.Metric) {
}
func probeHandler(w http.ResponseWriter, r *http.Request, tlsConfig *tls.Config) {
target := r.URL.Query().Get("target")
// The following timeout block was taken wholly from the blackbox exporter
@ -195,13 +268,8 @@ func probeHandler(w http.ResponseWriter, r *http.Request, tlsConfig *tls.Config)
timeout := time.Duration((timeoutSeconds) * 1e9)
t, err := parseTarget(target)
if err != nil {
t = target
}
exporter := &Exporter{
target: t,
target: target,
timeout: timeout,
tlsConfig: tlsConfig,
}
@ -235,33 +303,25 @@ func contains(certs []*x509.Certificate, cert *x509.Certificate) bool {
return false
}
// parseTarget makes an attempt at converting URLs of the form scheme://host
// into host:port
func parseTarget(target string) (parsedTarget string, err error) {
func parseTarget(target string) (parsedTarget string, proto string, err error) {
if !strings.Contains(target, "://") {
target = "//" + target
}
u, err := url.Parse(target)
if err != nil {
log.Errorln(err)
return
return "", proto, err
}
if u.Port() == "" {
switch scheme := u.Scheme; scheme {
case "https":
parsedTarget = u.Host + ":443"
case "ldaps":
parsedTarget = u.Host + ":636"
default:
parsedTarget = u.Host + ":443"
if u.Scheme != "" {
if u.Scheme == "https" {
return u.String(), "https", nil
}
} else {
parsedTarget = u.Host
return "", proto, errors.New("can't handle the scheme '" + u.Scheme + "' - try providing the target in the format <host>:<port>")
} else if u.Port() == "" {
return "https://" + u.Host, "https", nil
}
return parsedTarget, nil
return u.Host, "tcp", nil
}
func init() {

62
ssl_exporter_test.go
View File

@ -192,6 +192,68 @@ func TestProbeHandlerSpaces(t *testing.T) {
}
}
// Test with a uri protocol the exporter doesn't implement a client for
func TestProbeHandlerBadScheme(t *testing.T) {
rr, err := probe("ldaps://example.com")
if err != nil {
t.Fatalf(err.Error())
}
ok := strings.Contains(rr.Body.String(), "ssl_tls_connect_success 0")
if !ok {
t.Errorf("expected `ssl_tls_connect_success 0`")
}
}
// Test that probe uses a http client when the scheme is https://
func TestProbeHandlerHTTPSClient(t *testing.T) {
rr, err := probe("https://example.com")
if err != nil {
t.Fatalf(err.Error())
}
ok := strings.Contains(rr.Body.String(), "ssl_client_protocol{protocol=\"https\"} 1")
if !ok {
t.Errorf("expected `ssl_client_protocol{protocol=\"https\"} 1`")
}
ok = strings.Contains(rr.Body.String(), "ssl_client_protocol{protocol=\"tcp\"} 0")
if !ok {
t.Errorf("expected `ssl_client_protocol{protocol=\"tcp\"} 0`")
}
}
// Test that probe uses a tcp client when the host is of the form <host>:<port>
func TestProbeHandlerTCPClient(t *testing.T) {
rr, err := probe("example.com:443")
if err != nil {
t.Fatalf(err.Error())
}
ok := strings.Contains(rr.Body.String(), "ssl_client_protocol{protocol=\"tcp\"} 1")
if !ok {
t.Errorf("expected `ssl_client_protocol{protocol=\"tcp\"} 1`")
}
ok = strings.Contains(rr.Body.String(), "ssl_client_protocol{protocol=\"https\"} 0")
if !ok {
t.Errorf("expected `ssl_client_protocol{protocol=\"https\"} 0`")
}
}
// Test that a https client is used when there is no protocol or port in the target address
func TestProbeHandlerNoProtocolNoPort(t *testing.T) {
rr, err := probe("example.com")
if err != nil {
t.Fatalf(err.Error())
}
ok := strings.Contains(rr.Body.String(), "ssl_client_protocol{protocol=\"https\"} 1")
if !ok {
t.Errorf("expected `ssl_client_protocol{protocol=\"https\"} 1`")
}
}
// Test against a HTTP server
func TestProbeHandlerHTTP(t *testing.T) {
server, err := serverHTTP()