1
0
mirror of https://github.com/vimagick/dockerfiles.git synced 2024-12-12 11:14:57 +02:00
dockerfiles/crawlee/README.md

23 lines
591 B
Markdown
Raw Normal View History

2023-03-08 09:03:19 +02:00
crawlee
=======
[Crawlee][1] is a web scraping and browser automation library Crawlee is a web
scraping and browser automation library.
```bash
2023-03-08 09:51:09 +02:00
$ docker run --rm -e PLAYWRIGHT_SKIP_BROWSER_DOWNLOAD=1 -e PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=1 -v $PWD:/tmp -w /tmp apify/actor-node:16 npx crawlee create -t cheerio-js my-crawler
2023-03-08 09:03:19 +02:00
2023-03-08 09:51:09 +02:00
$ docker-compose build my-crawler
2023-03-08 09:03:19 +02:00
2023-03-08 09:51:09 +02:00
$ docker-compose run --rm my-crawler
$ tree my-crawler/storage/
2023-03-08 09:03:19 +02:00
├── datasets
│   └── default
2023-03-08 09:51:09 +02:00
│   └── 000000001.json
2023-03-08 09:03:19 +02:00
├── key_value_stores
└── request_queues
```
[1]: https://crawlee.dev/