2019-11-22 03:34:56 +02:00
# Awesome Web Scraping
2017-03-21 18:19:25 +02:00
2020-09-11 19:35:27 +02:00
The list of tools, programming libraries and web services used in web scraping and data processing.
2020-05-12 23:42:55 +02:00
Web scraping chats: [@grablab ](https://t.me/grablab ) (English) and [@grablab_ru ](https://t.me/grablab_ru ) (Russian)
2019-11-22 03:34:56 +02:00
## Programming Libraries
2015-08-12 22:27:49 +02:00
* [Python ](http://github.com/lorien/web-scraping/blob/master/python.md )
2015-08-16 15:47:36 +02:00
* [PHP ](http://github.com/lorien/web-scraping/blob/master/php.md )
2015-08-16 18:17:31 +02:00
* [Ruby ](http://github.com/lorien/web-scraping/blob/master/ruby.md )
2015-08-21 18:19:10 +02:00
* [JavaScript ](http://github.com/lorien/web-scraping/blob/master/javascript.md )
2019-11-22 03:34:56 +02:00
* [Go ](http://github.com/lorien/web-scraping/blob/master/golang.md )
## Other Things
2019-06-21 12:17:30 +02:00
2020-05-12 23:40:38 +02:00
* [Web Scraping Services ](http://github.com/lorien/web-scraping/blob/master/web_services.md )
2019-11-22 03:34:56 +02:00
* [Console tools ](http://github.com/lorien/web-scraping/blob/master/console_tools.md )
2020-05-12 23:40:38 +02:00
* [dhamaniasad / HeadlessBrowsers ](https://github.com/dhamaniasad/HeadlessBrowsers ) - a list of (almost) all headless web browsers in existence
2020-06-28 12:05:07 +02:00
* [DNS over HTTPS providers ](https://github.com/curl/curl/wiki/DNS-over-HTTPS ) - list of DNS over HTTPs providers
2019-11-23 22:17:11 +02:00
2020-05-12 23:41:45 +02:00
## Captcha Solving Services
* [2captcha.com ](https://2captcha.com/?from=3019071 ) - 2captcha.com
* [anti-gate.com ](http://getcaptchasolution.com/ijykrofoxz ) - anti-gate.com
2020-09-11 19:40:30 +02:00
## Contributing
2015-08-12 22:49:22 +02:00
2020-09-11 19:40:30 +02:00
See [contributing how-to ](https://github.com/lorien/web-scraping/blob/master/CONTRIBUTING.md ). To add new language to Programming Libraries section use [new_language_template ](http://github.com/lorien/web-scraping/blob/master/new_language_template.md ) as starting point.
## Credits
2019-11-22 03:34:56 +02:00
2019-11-23 22:15:31 +02:00
The list is based initially on some data from these sources [awesome-python ](https://github.com/vinta/awesome-python ), [awesome-php ](https://github.com/ziadoz/awesome-php ), [awesome-ruby ](https://github.com/markets/awesome-ruby ), [ruby-nlp ](https://github.com/diasks2/ruby-nlp ), [awesome-javascript ](https://github.com/sorrycc/awesome-javascript )
