mirror of https://github.com/lorien/awesome-web-scraping.git synced 2025-02-15 13:33:11 +02:00

Go to file

lorien f20550bfcd

Merge pull request #151 from Nykakin/add-chompjs

Add chompjs

2022-12-22 09:10:40 +07:00

.gitignore

add html5-parser to python

2020-10-19 21:48:59 +03:00

console_tools.md

order fix

2022-10-10 21:58:50 +03:00

CONTRIBUTING.md

Update CONTRIBUTING.md

2022-03-16 12:57:23 +03:00

golang.md

fix url to caddy proxy server repository

2022-03-15 00:41:39 +02:00

java.md

Update java.md

2018-09-06 16:09:58 +03:00

javascript.md

Replace Apify SDK with Crawlee, its successor

2022-08-17 22:48:10 +02:00

LICENSE

Initial commit

2015-08-13 02:27:49 +06:00

Makefile

New stuff

2015-08-13 02:49:22 +06:00

perl.md

Update perl.md

2018-09-06 16:01:37 +03:00

php.md

fix format

2022-10-08 18:53:12 +03:00

python.md

Move chompjs to separate section

2022-12-21 08:57:52 +01:00

README.md

Update README.md

2022-11-14 22:50:01 +07:00

ruby.md

Add arachnid2 to Ruby scrapers

2020-06-25 16:52:15 +01:00

web_services.md

Add Scraping Fish API to web services

2022-06-27 12:30:17 +02:00

README.md

Awesome Web Scraping

The list of tools, programming libraries and web services used for web scraping and data processing.

Programming Libraries

Other Things

Web Scraping Services
Console tools
dhamaniasad / HeadlessBrowsers - a list of (almost) all headless web browsers in existence
DNS over HTTPS providers - list of DNS over HTTPs providers

Captcha Solving Services

These two links point to same captcha service, it is just a different language versions

2captcha.com - English UI language
rucaptcha.com - Russian UI language

Telegram Groups about Web Scraping

@grablab - talks in English
@grablab_ru - talks in Russian

Contributing

See Contributing document.

Credits

The list is based initially on some data from these sources awesome-python, awesome-php, awesome-ruby, ruby-nlp, awesome-javascript