
Installation:
From Binary
wget https://github.com/utkusen/socialhunter/releases/download/v0.1.1/socialhunter_0.1.1_Linux_amd64.tar.gz
tar xzvf socialhunter_0.1.1_Linux_amd64.tar.gz
./socialhunter --help
From Source
- Install Go on your system
- Run:
go get -u github.com/utkusen/socialhunter
Usage
socialhunter requires 2 parameters to run:
-f
: Path of the text file that contains URLs line by line. The crawl function is path-aware. For example, if the URL is https://utkusen.com/blog
, it only crawls the pages under /blog
path
-w
: The number of workers to run (e.g -w 10
). The default value is 5. You can increase or decrease this by testing out the capability of your system.