From 06b6ac30412ce7a7ee5280951a4305ff38032f9a Mon Sep 17 00:00:00 2001 From: kev Date: Thu, 4 Aug 2016 19:31:24 +0800 Subject: [PATCH] update scrapyd --- scrapyd/README.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/scrapyd/README.md b/scrapyd/README.md index dc8a760..47b7db8 100644 --- a/scrapyd/README.md +++ b/scrapyd/README.md @@ -33,6 +33,34 @@ $ docker run -d --restart always --name scrapyd -p 6800:6800 vimagick/scrapyd $ firefox http://localhost:6800 ``` +``` +$ mkvirtualenv webbot +$ pip install scrapy scrapyd-client + +$ scrapy startproject myproject +$ cd myproject +$ setvirtualenvproject + +$ scrapy genspider myspider mydomain.com +$ scrapy edit myspider +$ scrapy list + +$ vi scrapy.cfg +$ scrapyd-client deploy +$ curl http://localhost:6800/schedule.json -d project=myproject -d spider=myspider +``` + +File: scrapy.cfg + +``` +[settings] +default = myproject.settings + +[deploy] +url = http://localhost:6800/ +project = myproject +``` + ## Run it as interactive-shell for scrapy ```