mirror of
https://github.com/vimagick/dockerfiles.git
synced 2024-12-14 11:23:02 +02:00
53 lines
2.1 KiB
Markdown
53 lines
2.1 KiB
Markdown
airflow
|
|
=======
|
|
|
|
|
|
|
|
## How It Works
|
|
|
|
```
|
|
+---------+
|
|
+--> | Worker1 |
|
|
redis | +---------+
|
|
postgres |
|
|
+--------+ | +---------+
|
|
| Master | <--+--> | Worker2 |
|
|
+--------+ | +---------+
|
|
webserver |
|
|
scheduler | +---------+
|
|
flower +--> | Worker3 |
|
|
+---------+
|
|
```
|
|
|
|
## Quick Start
|
|
|
|
```bash
|
|
$ docker stack deploy -c docker-stack.yaml airflow
|
|
$ docker service update --replicas-max-per-node=1 airflow_worker
|
|
$ docker service update --replicas 3 airflow_worker
|
|
|
|
$ docker stack services airflow
|
|
$ docker service ps airflow_webserver
|
|
$ docker exec -it airflow_webserver.1.xxxxxx sh
|
|
>>> airflow create_user -r Admin -u admin -e admin@borderxlab.com -f Super -l Admin -p secret
|
|
>>> airflow list_users
|
|
╒══════╤════════════╤══════════════════════╤══════════════╤═════════════╤═════════╕
|
|
│ Id │ Username │ Email │ First name │ Last name │ Roles │
|
|
╞══════╪════════════╪══════════════════════╪══════════════╪═════════════╪═════════╡
|
|
│ 1 │ admin │ admin@borderxlab.com │ Super │ Admin │ [Admin] │
|
|
╘══════╧════════════╧══════════════════════╧══════════════╧═════════════╧═════════╛
|
|
>>> exit
|
|
|
|
$ curl http://localhost:8080/
|
|
$ curl http://localhost:5555/
|
|
```
|
|
|
|
> :warning: You need to prepare nfs server with `airflow.cfg`.
|
|
|
|
```
|
|
$ python -c 'from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())'
|
|
CD2wL7G0zt1SLuO4JQpLJuHtBaBEcXWKbQyvkvf2cZ8=
|
|
```
|
|
|
|
> :warning: You should set another value to `fernet_key` in `airflow.cfg` to improve security.
|