dockerfiles/elk

ELK

• Elasticsearch
• Logstash
• Kibana
• APM Server

How it works

Up and running

$ sysctl -w vm.max_map_count=262144
$ vi /etc/sysctl.conf
  vm.max_map_count = 262144
$ chown -R 1000:1000 data
$ docker-compose up -d

Docker Config

• https://github.com/elastic/dockerfiles
• https://www.elastic.co/guide/en/elasticsearch/reference/current/docker.html
• https://www.elastic.co/guide/en/logstash/current/docker-config.html
• https://www.elastic.co/guide/en/kibana/current/docker.html

Logstash Config

• https://www.elastic.co/guide/en/logstash/current/configuration-file-structure.html
• https://www.elastic.co/guide/en/logstash/current/event-dependent-configuration.html

Delete indices older than 7 days

File: delete-indices.yml

---
actions:
  1:
    action: delete_indices
    description: >-
      Delete indices older than 7 days (based on index name), for logstash-
      prefixed indices. Ignore the error if the filter does not result in an
      actionable list of indices (ignore_empty_list) and exit cleanly.      
    options:
      ignore_empty_list: True
      disable_action: False
    filters:
    - filtertype: pattern
      kind: prefix
      value: logstash-
    - filtertype: age
      source: name
      direction: older
      timestring: '%Y.%m.%d'
      unit: days
      unit_count: 7

File: ~/.curator/curator.yml

client:
  hosts:
    - 127.0.0.1

$ pip install elasticsearch-curator
$ curator delete-indices.yml

Send container's log to ELK

input {
  gelf {
    port => 12201
  }
}

test:
  image: alpine
  command: 'sh -c "while :; do date; sleep 1; done"'
  log_driver: gelf
  log_opt:
    gelf-address: udp://x.x.x.x:12201
    tag: test

Search tag: test in kibana to show recent logging.