mirror of https://github.com/lorien/awesome-web-scraping.git synced 2025-02-19 19:00:10 +02:00

Go to file

Merge pull request #175 from bjesus/patch-3

add pipet to the list of cli tools

2024-10-27 18:39:15 +03:00

.gitignore

add html5-parser to python

2020-10-19 21:48:59 +03:00

cli.md

add pipet to the list of golang tools

2024-10-27 16:34:02 +01:00

CONTRIBUTING.md

Update CONTRIBUTING.md

2022-03-16 12:57:23 +03:00

golang.md

Update golang.md

2024-09-13 16:02:47 +05:30

java.md

Update java.md

2018-09-06 16:09:58 +03:00

javascript.md

Replace Apify SDK with Crawlee, its successor

2022-08-17 22:48:10 +02:00

LICENSE

Initial commit

2015-08-13 02:27:49 +06:00

Makefile

New stuff

2015-08-13 02:49:22 +06:00

manuals.md

Create manuals.md

2023-08-07 13:22:39 +06:00

perl.md

Update perl.md

2018-09-06 16:01:37 +03:00

php.md

Merge pull request #168 from 3xploiton3/patch-3

2024-08-23 10:07:40 +03:00

python.md

Update python.md

2024-10-15 13:00:02 +02:00

README.md

Update README with a link to cli.md

2024-10-15 15:29:22 +03:00

ruby.md

Add arachnid2 to Ruby scrapers

2020-06-25 16:52:15 +01:00

README.md

Awesome Web Scraping

Lists of packages, services and manuals related to web scraping.

Topics

Python - Python packages
PHP - PHP packages
Ruby - Ruby packages
JavaScript - JavaScript packages
Go - Go packages
Command Line Tools - tools with a command line interface
Web Scraping Manuals - list of articles and books teaching web scraping
dhamaniasad / HeadlessBrowsers - list of (almost) all headless web browsers in existence
DNS over HTTPS providers - list of DNS over HTTPs providers
Awesome Pastebins - list of pastebin sites

Captcha Solving Services

https://2captcha.com

Proxy Server Marketplaces

Telegram Discussion Groups

@grablab - talks in English
@grablab_ru - talks in Russian

How to Contribute to This List

See Contributing guide.

Credits

The list is based initially on some data from these sources awesome-python, awesome-php, awesome-ruby, ruby-nlp, awesome-javascript