self-hosted/dockerfiles
1
0
Fork 0
mirror of https://github.com/vimagick/dockerfiles.git synced 2025-01-02 03:37:40 +02:00
Code Issues Releases Activity

update scrapyd

Browse Source
This commit is contained in:
kev 2021-11-30 14:46:34 +08:00
parent be138ae446
commit 1ede9d5b4b
3 changed files with 11 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
scrapyd/README.md
View File

@ -16,15 +16,18 @@ utility which allows you to deploy your project to a Scrapyd server.
[scrapyrt][5] allows you to easily add HTTP API to your existing Scrapy project. [scrapyrt][5] allows you to easily add HTTP API to your existing Scrapy project.
[pillow][6] is the Python Imaging Library to support the ImagesPipeline. [Spidermon][6] is a framework to build monitors for Scrapy spiders.
This image is based on `debian:buster`, 6 latest python packages are installed: [pillow][7] is the Python Imaging Library to support the ImagesPipeline.
This image is based on `debian:buster`, 7 latest python packages are installed:
- `scrapy`: git+https://github.com/scrapy/scrapy.git - `scrapy`: git+https://github.com/scrapy/scrapy.git
- `scrapyd`: git+https://github.com/scrapy/scrapyd.git - `scrapyd`: git+https://github.com/scrapy/scrapyd.git
- `scrapyd-client`: git+https://github.com/scrapy/scrapyd-client.git - `scrapyd-client`: git+https://github.com/scrapy/scrapyd-client.git
- `scrapy-splash`: git+https://github.com/scrapinghub/scrapy-splash.git - `scrapy-splash`: git+https://github.com/scrapinghub/scrapy-splash.git
- `scrapyrt`: git+https://github.com/scrapinghub/scrapyrt.git - `scrapyrt`: git+https://github.com/scrapinghub/scrapyrt.git
- `spidermon`: git+https://github.com/scrapinghub/spidermon.git
- `pillow`: git+https://github.com/python-pillow/Pillow.git - `pillow`: git+https://github.com/python-pillow/Pillow.git
Please use this as base image for your own project. Please use this as base image for your own project.
@ -151,4 +154,5 @@ $ curl -s 'http://localhost:9080/crawl.json?spider_name=toscrape-css&callback=pa
[3]: https://github.com/scrapy/scrapyd-client [3]: https://github.com/scrapy/scrapyd-client
[4]: https://github.com/scrapinghub/scrapy-splash [4]: https://github.com/scrapinghub/scrapy-splash
[5]: https://github.com/scrapinghub/scrapyrt [5]: https://github.com/scrapinghub/scrapyrt
[6]: https://github.com/python-pillow/Pillow [6]: https://github.com/scrapinghub/spidermon
[7]: https://github.com/python-pillow/Pillow

2
scrapyd/arm/Dockerfile
View File

@ -9,6 +9,7 @@ ENV SCRAPY_VERSION=2.5.1
ENV SCRAPYD_VERSION=1.2.1 ENV SCRAPYD_VERSION=1.2.1
ENV SCRAPYD_CLIENT_VERSION=v1.2.0 ENV SCRAPYD_CLIENT_VERSION=v1.2.0
ENV SCRAPYRT_VERSION=v0.12 ENV SCRAPYRT_VERSION=v0.12
ENV SPIDERMON_VERSION=1.15.2
ENV PILLOW_VERSION=8.4.0 ENV PILLOW_VERSION=8.4.0
RUN set -xe \ RUN set -xe \
@ -47,6 +48,7 @@ RUN set -xe \
git+https://github.com/scrapy/scrapyd-client.git@$SCRAPYD_CLIENT_VERSION \ git+https://github.com/scrapy/scrapyd-client.git@$SCRAPYD_CLIENT_VERSION \
git+https://github.com/scrapinghub/scrapy-splash.git \ git+https://github.com/scrapinghub/scrapy-splash.git \
git+https://github.com/scrapinghub/scrapyrt.git@$SCRAPYRT_VERSION \ git+https://github.com/scrapinghub/scrapyrt.git@$SCRAPYRT_VERSION \
git+https://github.com/scrapinghub/spidermon.git@$SPIDERMON_VERSION \
git+https://github.com/python-pillow/Pillow.git@$PILLOW_VERSION \ git+https://github.com/python-pillow/Pillow.git@$PILLOW_VERSION \
&& curl -sSL https://github.com/scrapy/scrapy/raw/master/extras/scrapy_bash_completion -o /etc/bash_completion.d/scrapy_bash_completion \ && curl -sSL https://github.com/scrapy/scrapy/raw/master/extras/scrapy_bash_completion -o /etc/bash_completion.d/scrapy_bash_completion \
&& echo 'source /etc/bash_completion.d/scrapy_bash_completion' >> /root/.bashrc \ && echo 'source /etc/bash_completion.d/scrapy_bash_completion' >> /root/.bashrc \

2
scrapyd/py3/Dockerfile
View File

@ -9,6 +9,7 @@ ENV SCRAPY_VERSION=2.5.1
ENV SCRAPYD_VERSION=1.2.1 ENV SCRAPYD_VERSION=1.2.1
ENV SCRAPYD_CLIENT_VERSION=v1.2.0 ENV SCRAPYD_CLIENT_VERSION=v1.2.0
ENV SCRAPYRT_VERSION=v0.12 ENV SCRAPYRT_VERSION=v0.12
ENV SPIDERMON_VERSION=1.15.2
ENV PILLOW_VERSION=8.4.0 ENV PILLOW_VERSION=8.4.0
RUN set -xe \ RUN set -xe \
@ -46,6 +47,7 @@ RUN set -xe \
git+https://github.com/scrapy/scrapyd-client.git@$SCRAPYD_CLIENT_VERSION \ git+https://github.com/scrapy/scrapyd-client.git@$SCRAPYD_CLIENT_VERSION \
git+https://github.com/scrapinghub/scrapy-splash.git \ git+https://github.com/scrapinghub/scrapy-splash.git \
git+https://github.com/scrapinghub/scrapyrt.git@$SCRAPYRT_VERSION \ git+https://github.com/scrapinghub/scrapyrt.git@$SCRAPYRT_VERSION \
git+https://github.com/scrapinghub/spidermon.git@$SPIDERMON_VERSION \
git+https://github.com/python-pillow/Pillow.git@$PILLOW_VERSION \ git+https://github.com/python-pillow/Pillow.git@$PILLOW_VERSION \
&& curl -sSL https://github.com/scrapy/scrapy/raw/master/extras/scrapy_bash_completion -o /etc/bash_completion.d/scrapy_bash_completion \ && curl -sSL https://github.com/scrapy/scrapy/raw/master/extras/scrapy_bash_completion -o /etc/bash_completion.d/scrapy_bash_completion \
&& echo 'source /etc/bash_completion.d/scrapy_bash_completion' >> /root/.bashrc \ && echo 'source /etc/bash_completion.d/scrapy_bash_completion' >> /root/.bashrc \