mirror of
https://github.com/lorien/awesome-web-scraping.git
synced 2024-11-24 08:32:19 +02:00
List of libraries, tools and APIs for web scraping and data processing.
captcha-bypasscaptcha-recaptchacrawlercrawlingcrawling-frameworkcrawling-pythoncrawling-toolscrapingscraping-frameworkscraping-pythonscraping-toolspiderweb-scrapingwebscraping
| .gitignore | ||
| books.md | ||
| console_tools.md | ||
| CONTRIBUTING.md | ||
| golang.md | ||
| javascript.md | ||
| LICENSE | ||
| Makefile | ||
| new_language_template.md | ||
| perl.md | ||
| php.md | ||
| proxy_services.md | ||
| python.md | ||
| README.md | ||
| ruby.md | ||
| web_services.md | ||
Project status
The project is moved to http://opendir.io Your help is welcome. Just sign in to opendir.io with github account and share cool github projects you know.
Web Scraping
The list of tools, programming libraries and APIs used in web-scraping.
- Python
- PHP
- Ruby
- JavaScript
- Golang
- Feel free to add your favourite language. Use new_language_template.md as start point.
- Proxy Services
- Web Services
- Console tools
- Books
Other Awesome List Projects
- lists - List of useful, silly and awesome lists curated on GitHub
- HeadlessBrowsers - a list of (almost) all headless web browsers in existence
Contributing
Make this list better! Your contributions are always welcome! See contributing how-to
Credits
This list partially contains data from these sources:
- awesome-python by vinta / CC BY 4.0
- awesome-php by ziadoz
- awesome-ruby by markets
- ruby-nlp by diasks2
- awesome-javascript by sorrycc