awesome/awesome-web-scraping
1
0
Fork 0
mirror of https://github.com/lorien/awesome-web-scraping.git synced 2024-11-24 08:32:19 +02:00
Code Issues Releases Activity

Merge pull request #77 from jancurn/master

Browse Source 
Added several missing services and libraries
This commit is contained in:
Gregory Petukhov 2019-01-28 13:58:48 +03:00 committed by GitHub
commit 6cb55ddc1d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
3 changed files with 5 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
javascript.md
View File

@ -34,6 +34,7 @@ This list contains JavaScript libraries related to web scraping and data process
## Web-Scraping Frameworks
* [node-crawler](https://github.com/sylvinus/node-crawler) - Web Crawler/Spider for NodeJS + server-side jQuery
* [node-simplecrawler](https://github.com/cgiffard/node-simplecrawler) - Flexible event driven crawler for node
* [Apify SDK](https://github.com/apifytech/apify-js) - The scalable web crawling and scraping library for JavaScript. Enables development of data extraction and web automation jobs (not only) with headless Chrome and Puppeteer.
## HTML/XML Parsing
* General
@ -191,6 +192,7 @@ This list contains JavaScript libraries related to web scraping and data process
## Proxy Server
* [toxy](https://github.com/h2non/toxy) - Hackable HTTP proxy to simulate server failure scenarios and unexpected network conditions
* [proxy-chain](https://github.com/apifytech/proxy-chain) - Node.js implementation of a proxy server (think Squid) with support for SSL, authentication and upstream proxy chaining
## Data Structure
* [immutable](https://github.com/facebook/immutable-js) - Immutable persistent data collections for Javascript which increase efficiency and simplicity.

2
proxy_services.md
View File

@ -5,6 +5,8 @@ Only proxy services with referral program are allowed to be placed into Proxy Se
* [luminaty](https://luminati.io/?affiliate=ref_54bc9069600bf7e2141ba785) - The world's largest business proxy network
 for complete global anonymity. More than 10 million IPs from all around the world, 192 countries worldwide
* [proxyrack](http://www.proxyrack.com/access/aff/go/lorien) - Over 1,250,000 Unique Private IPs With Instant Setup
* [Apify Proxy](https://www.apify.com/proxy) - Provides access to data-center and residential proxies, with intelligent
IP address rotation and health monitoring.
## Proxy Services Marketpalce
* [blackhatworld](https://www.blackhatworld.com/forums/proxies-for-sale.112/) - one of the most popular places in the world for promoting proxy services

2
web_services.md
View File

@ -12,7 +12,7 @@
* [PhantomJs.Cloud](https://phantomjscloud.com/)
* [CloudScrape](http://cloudscrape.com/)
* [DiffBot](http://www.diffbot.com/)
* [Apifier](https://www.apifier.com/)
* [Apify](https://www.apify.com/) - A serverless web scraping, data extraction and web automation platform
* [Portia](http://scrapinghub.com/portia/); also on GitHub: [scrapinghub/portia](https://github.com/scrapinghub/portia)
* [Dexi](https://dexi.io)
* [Morph.io](https://morph.io) free of charge, fully [open-source](https://github.com/openaustralia/morph) service