
Installation:
From Binary
wget https://github.com/utkusen/socialhunter/releases/download/v0.1.1/socialhunter_0.1.1_Linux_amd64.tar.gz
tar xzvf socialhunter_0.1.1_Linux_amd64.tar.gz
./socialhunter --help
From Source
- Install Go on your system
- Run:
go get -u github.com/utkusen/socialhunter
Usage
socialhunter requires 2 parameters to run:
-f : Path of the text file that contains URLs line by line. The crawl function is path-aware. For example, if the URL is https://utkusen.com/blog, it only crawls the pages under /blog path
-w : The number of workers to run (e.g -w 10). The default value is 5. You can increase or decrease this by testing out the capability of your system.