awesome-web-scraping/golang.md

Golang Web Scraping

This list contains Golang libraries related to web scraping and data processing

• Golang Web Scraping
  • Network
  • Web-scraping Frameworks
  • HTML/XML Parsing
  • Text processing
  • Specific Formats Processing
  • Natural Language Processing
  • Browser automation and emulation
  • Multiprocessing
  • Queue
  • Email
  • URL and Network Address Manipulation
  • Web Content Extracting
  • Asynchronous
  • WebSocket
  • DNS Resolving
  • Computer Vision
  • Proxy Server
  • Other Golang Lists

Network

• General
  • net
  • net/http
• Asynchronous
  • goroutine

Web-Scraping Frameworks

• Full Featured Crawlers
  • Pholcus
  • go_spider
  • ants-go
• Other
  • TODO

HTML/XML Parsing

• encoding/xml

Text Processing

Libraries for parsing and manipulating plain texts.

• General
  • regexp

Specific Formats Processing

Libraries for parsing and manipulating specific text formats.

• General
  • encoding/json
• Something
  • TODO

Natural Language Processing

Libraries for working with human languages.

• TODO

Browser automation and emulation

• TODO

Multiprocessing

• TODO

Asynchronous

Libraries for asynchronous networking programming.

• TODO

Queue

• NSQ
• NATS

Email

Libraries for parsing email.

• TODO

URL and Network Address Manipulation

Libraries for parsing/modifying URLs and network addresses.

• URL
  • net/url
• Network Address
  • TODO

Web Content Extracting

Libraries for extracting web contents.

• Text and Meta Data from HTML pages
  • x/net/html

WebSocket

Libraries for working with WebSocket.

• gorilla/websocket

DNS Resolving

• net
• miekg/dns

Computer Vision

• TODO

Proxy Server

• TODO

Other Golang lists

• TODO