BlackWidow – Web Application Spider

Gathering Information about the target domain is very important for making the penetration testing or bug bounty hunting process successful. Collecting Subdomains, Unique URLs, URLs with Parameters, Emails of the target domain, etc information is very crucial, so to get this Information we have a tool named BlackWidow.

BlackWidow tool is a python language-based tool used for web-based application spider and gathers lots of information. Inject-X fuzzer is used in this tool for scanning Dynamic URLs for common OWASP vulnerabilities. BlackWidow tool can be used in the initial steps of web-based application vulnerability assessment for the Information Gathering phase. Inject-X fuzzer has the capability to detect XSS, SWL Injection, HTML code injection, etc. vulnerabilities.

Features of BlackWidow Tool

1. It gathers subdomains, unique dynamic URLs, emails, phone numbers.
2. It uses an Inject-X fuzzer for the detection of vulnerability.
3. It can store the results in dedicated text files in sorted order.
4. It is developed in Python Language.
5. It is open-source and free to use.

Installation of BlackWidow Tool on Kali Linux OS

Step 1: Check whether Python Environment is Established or not, use the following command:

python3

Step 2: Open up your Kali Linux terminal and move to Desktop using the following command.

cd Desktop

Step 3: You are on Desktop now create a new directory called Black-Widow using the following command. In this directory, we will complete the installation of the Black-Widow tool.

mkdir Black-Widow

Step 4: Now switch to the Black-Widow directory using the following command.

cd Black-Widow

Step 5: Now you have to install the tool. You have to clone the tool from Github.

sudo git clone https://github.com/1N3/BlackWidow.git

Step 6: The tool has been downloaded successfully in the Black-Widow directory. Now list out the contents of the tool by using the below command.

ls

Step 7: You can observe that there is a new directory created of the BlackWidow tool that has been generated while we were installing the tool. Now move to that directory using the below command:

cd BlackWidow 

Step 8: Once again to discover the contents of the tool, use the below command.

ls

Step 9: Download the required packages for running the tool, use the following command.

sudo pip3 install -r requirements.txt

Step 10: Now we are done with our installation, Use the below command to view the help (gives a better understanding of tool) index of the tool.

python3 blackwidow -h

Working with BlackWidow:

Example 1: Full URL to spider

In this example, we will be performing Full URL Spider. We have provided the target Full URL using the -u tag (https://geeksforgeeks.org) 

sudo python3 blackwidow -u https://geeksforgeeks.org

In the below Screenshot, we are getting our scan results, Subdomains of geeksforgeeks.org are displayed in the below Screenshot.

In the below Screenshot, dynamic URLs found on geeksforgeeks.org are mentioned below in the screenshot, Basically, these are parameterized URLs that can be tested for XSS and many more vulnerabilities.

In the below screenshot, the files created by the tool while scanning is shown. Each file has its own results like subdomains, emails, URLs, etc.

In the below Screenshot, You can see that Emails of geeksforgeeks.org are saved in the emails.txt file, this makes the task easier.

Example 2: Domain name to spider

In this example, rather than providing a full URL (like Example 1), we are specifying only domain names to scan. -d tag is used along with the target domain name.

sudo python3 blackwidow -d geeksforgeeks.org

Domain name to spider

In the below Screenshot, we have got the results of our Domain name Spider.

Example 3: Level of depth to traverse

In this example, we will scan the domain to a certain depth. 5 levels of depth will be scanned as we have specified a depth of 5 using the -l tag.

sudo python3 blackwidow -d geeksforgeeks.org -l 5

In the below Screenshot, the results of the depth of 5 are displayed.

Example 4: Scan all dynamic URL’s found

In this example, We will scan for all the dynamic URLs detected on the geeksforgeeks.org domain. In the below Screenshot, the list of unique_dynaminc URLs list is displayed.

sudo python3 blackwidow -d geeksforgeeks.org -s /usr/share/blackwidow/geeksforgeeks.org_80/geeksforgeeks.org_80-dynamic-unique.txt

In the below Screenshot, we are starting our Scan on dynamic URLs which are provided in the form of list.

In the below screenshot, you can see that the results of the above scan are displayed.

Example 5: Port for the URL

In this example, we will perform a scan on the domain specified with the port number. We have specified Port 80, so only Port 80 URLs will be scanned.

sudo python3 blackwidow -d geeksforgeeks.org -p 80

In the below Screenshot, we have got the results of our above Example Port of the URL.

Example 6: Set verbose mode

In this example, we will be displayed in a more verbose format. -v tag is used along with the confirmation (y/n).

sudo python3 blackwidow -d geeksforgeeks.org -v y

In the below Screenshot, the results of verbose format are displayed.