One of the primary steps in attacking an internet application is enumerating hidden directories and files. Doing so can often yield valuable information that makes it easier to execute a particular attack, leaving less room for errors and wasted time. There are many tools available to try to do this, but not all of them are created equally. Gobuster, a record scanner written in Go Language, is worth searching for. In popular directories, brute-force scanners like DirBuster and DIRB work just elegantly but can often be slow and responsive to errors. Gobuster may be a Go implementation of those tools and is obtainable in a convenient command-line format. The primary benefit Gobuster has over other directory scanners is speed. As a programming language, Go is understood to be fast. It also has excellent help for concurrency, so that Gobuster can benefit from multiple threads for quicker processing. The one defeat of Gobuster, though, is the lack of recursive directory exploration. For directories, quite one level deep, another scan is going to be needed, unfortunately. Often, this is not that big of a deal, and other scanners can intensify and fill in the gaps for Gobuster in this area.
Installation Steps of Gobuster Tool in Linux OS
Step 1:
Create a working directory to keep things neat, then change into it.
~# mkdir gobuster
~# cd gobuster/
Step 2:
We need to install Gobuster Tool since it is not included on Kali Linux by default.
~/gobuster# apt-get install gobuster
Step 3:
Then, simply type gobuster into the terminal to run the tool for use.
~/gobuster# gobuster -h
Step 4:
Installing Additional Seclists for brute-forcing Directories and Files
~/gobuster# apt-get install seclists
By default, Wordlists on Kali are located in the /usr/share/wordlists directory.
How to use Gobuster Tool for Scanning?
Gobuster tools can be launched from the terminal or command-line interface. You just have to run the command using the syntax below.
gobuster [Mode][Options]
Understanding Gobuster [Mode]
After entering the “gobuster” command in a terminal, you compulsory need to provide the mode or need to specify the purpose of the tool you are running for.
Gobuster tool have many modes :
- dir – the classic directory brute-forcing mode or Enumerating URIs for directories and files.
The Dir mode in Gobuster is mainly used to find extra content in a specific target domain or its subdomain. This additional information can include hidden directories or hidden files that can contain sensitive data. In Dir Mode, we can use the option “-u” to specify the target domain or subdomain you want to dig into the hidden directories and files. Also, the “-w” option will select the wordlist which you wish to use for brute-forcing.
dir mode options
- dns – DNS subdomain brute-forcing mode or Enumerating Subdomains
The DNS mode in Gobuster Tool is mainly used to enumerate subdomains in the target domain. You can use this mode to find some hidden or unidentifiable subdomains for a given target domain. In this mode, you can use the option “-d” to specify the target domain you want to find subdomain and the “-w” option allows you to select the wordlist you wish to use for brute-forcing.
dns mode options
- vhost – virtual host brute-forcing mode or enumerating virtual hosts (not the same as DNS!)
Finally, Vhost mode in Gobuster is used to find the virtual hosts on the victim server. Virtual Hosting is done when companies host several domain names on a single server or cluster of the server. Virtual Hosting allows one server to share its data and resources with several other hostnames. Identifying hostnames on a server can disclose additional web content belonging to a company. In host mode, it checks if the subdomains exist by actually visiting the formed URL and cross checking the IP address.
vhost mode options
Mostly, you will be using the Gobuster tool for digging directories and files. In this case, dir mode will be helpful for you.
gobuster dir [options]
Understanding Gobuster [Options]
After entering the specific mode as per requirement, you have to specify the options. Gobuster tool has a long list of options; to explore them, you can simply read the help page by typing “gobuster -h”. You could use “gobuster dns -h” to explore options that are specifically related to the dns mode).
Some examples of options are :
- -o, –output string Output file to write results to (defaults to stdout)
- -q, –quiet Don’t print the banner and other noise
- -t, –threads int Number of concurrent threads (default 10)
- -v, –verbose Verbose output (errors)
gobuster -h option result
Target Specification
So, while using the tool, we need to specify the “-u” followed by a target URL, IP address, or a hostname. This option is compulsory, as there is a target specified for getting results.
Some of the examples show how to use this option.
- gobuster dir -u https://www.geeksforgeeks.org/
- gobuster dir -u https://www.webscantest.com
- gobuster dir -u 192.168.21.154
Note that these examples will not work if the mandatory option “-u” is not specified.
Wordlist Specification
Gobuster Tool enumerates hidden directories and files in the target domain by performing a brute-force attack. A brute-force attack consists of matching a list of words or a combination of words hoping that the correct term is present in the list. So, Gobuster performs a brute attack. To force an attack, we need to specify a collection of words, i.e., wordlist. So to provide this wordlist, you need to type the “-w” option, followed by the path of the wordlist where it is located. We can use a wordlist file that is already present in the system.
gobuster dir -u https://www.geeksforgeeks.org/ -w /usr/share/wordlists/big.txt
Enumerating Files
Gobuster Tool can enumerate hidden files along with the remote directories. Gobuster allows us to use the “-x” option followed by the file extensions you’d like to search for.
Consider the example below:
gobuster dir -u https://www.geeksforgeeks.com w /usr/share/wordlists/big.txt -x php,html,htm
In this command, we are specifically searching for files that have php,htm or html extensions.
Usage of Gobuster Tool with an Example
1. Obtaining Full Path for a directory or file
Option “-e” is used for completing printing URL when extracting any hidden file or hidden directories.
gobuster dir -e -u geeksforgeeks.org -w /usr/share/wordlists/dirb/common.txt –wildcard
Obtaining Full Path for a directory or file
2. Hide Status Code
Using -n Option “no status” mode prints the results’ output without presenting the status code.
gobuster dir -u geeksforgeeks.org -w /usr/share/wordlists/dirb/common.txt -n –wildcard
Hide Status Code
3. Disable Banner
Gobuster tool constantly adds the banner to define the brief introduction of applied options while launching a brute force attack. By using the -q option, we can disable the flag to hide extra data.
gobuster dir -u geeksforgeeks.org -w /usr/share/wordlists/dirb/common.txt -q –wildcard
Disable Banner
4. Set Threads Number
Using the -t option enables the number of thread parameters to be implemented while brute-forcing sub-domain names or directories.
gobuster dns -d geeksforgeeks.org -t 100 -w /usr/share/wordlists/dirb/common.txt –wildcard
Set Thread Number
5. Obtain Sub Domain IPs
Using the -i option allows the IP parameter, which should show the IPs of selected sub-domains.
gobuster dns -d geeksforgeeks.org -t 100 -w /usr/share/wordlists/dirb/common.txt -i –wildcard
DNS mode is covered in this command
Obtain Sub Domain IPs
6. Timeout
Using the –timeout option allows the timeout parameter for HTTP requests, and 5 seconds is the default time limit for the HTTP request.
gobuster dir –timeout 5s -u geeksforgeeks.org -t 100 -w /usr/share/wordlists/dirb/common.txt –wildcard
Timeout
7. Appending Forward Slash
I am using the -f option here for appending the forward-slash while making a brute-force attack on the target URL.
gobuster dir -u geeksforgeeks.org -w /usr/share/wordlists/dirb/common.txt -f –wildcard
Appending Forward Slash
8. Enumerating Directory with Specific Extension List
There are many scenarios where we need to extract the directories of a specific extension over the victim server, and then we can use the -X parameter of this scan. This parameter allows the file extension name and then explores the given extension files over the victim server or computer.
gobuster dir -u geeksforgeeks.org -w /usr/share/wordlists/dirb/common.txt -x .php –wildcard
Enumerating Directory with Specific Extension List
9. Follow Redirect
Using -r options allows redirecting the parameters, redirecting HTTP requests to another, and changing the Status code for a directory or file.
gobuster dir -u geeksforgeeks.org -w /usr/share/wordlists/dirb/common.txt -q –wildcard
gobuster dir -u geeksforgeeks.org -r -w /usr/share/wordlists/dirb/common.txt -q –wildcard
Follow Redirect
10. HTTP AUTHORIZATION (-u username: password)
HTTP Authentication/Authentication mechanisms are all based on the use of 401-status code and WWW-Authenticate response header. The most generally used HTTP authentication mechanisms are Primary. The client sends the user name and password un-encrypted base64 encoded data.
So, to avoid this kind of authentication with the help of Gobuster, we have used the command below:
gobuster dir -u http://testphp.vulnweb.com/login.php -w /usr/share/wordlists/dirb/common.txt -U test -P test –wildcard
HTTP Authorization
11. Force Processing Brute Force
It ends by obtaining the sub-domain name if it meets any Wildcard DNS, which is a non-existing domain. Therefore, it uses the –wildcard option to allow parameters to continue the attack even if there is any Wildcard Domain.
gobuster dir -u geeksforgeeks.org -w /usr/share/wordlists/dirb/common.txt –wildcard
Force Processing Brute Force
12. Hide Process of Extracting
Using the -z option covers the process of obtaining sub-domains names while making brute force attacks.
gobuster dns -d geeksforgeeks.org -t 100 -w /usr/share/wordlists/dirb/common.txt -z –wildcard
Hide Process of Extraction
13. Extracting CNAME Records
Using the –cn option enables the CNAME Records parameter of the obtained sub-domains and their CNAME records.
gobuster dns -d geeksforgeeks.org -t 100 -w /usr/share/wordlists/dirb/common.txt -c –wildcard
Extracting CNAME Records
14. Proxy URL
Using the –p option allows proxy URL to be used for all requests; by default, it works on port 1080. As you can see, on examining the victim’s network IP in the web browser, it put up an “Access forbidden error”, which means this web page is operating backwards by some proxy.
gobuster dir -p ‘https://18.172.30:3128’ -u ‘http://18.192.172.30/’ -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt –wildcard
Proxy URL
Example :
Now that everything is set up and installed, we’re ready to go and use Gobuster. Let’s run it against our victim with the default parameters.
Target for Scanning : https://testphp.vulnweb.com
Scanning for Directories and Files
kali@kali:~$ gobuster dir -u testphp.vulnweb.com -w /usr/share/wordlists/dirb/common.txt
From the above screenshot, we are enumerating for directories on https://testphp.vulnweb.com.
The wordlist used for the scanning is located at /usr/share/wordlists/dirb/common.txt
Going to the current directory which is identified while scanning
From the above screenshot, we have identified the admin panel while brute-forcing directories. After opening the web browser and typing the URL of our target, https://testphp.vulnweb.com/ and giving the identified directory /admin/, we will provide the contents available in that directory. Being a Security Researcher, you can test the functionality of that web page.
Conclusion
In this article, we learned about Gobuster, a directory brute-force scanner written in the Go programming language. First, we learned how to install the tool and some valuable wordlists not found on Kali by default. Next, we ran it against our target and explored many of the varied options it ships with. Gobuster is a fast and powerful directory scanner that should be an essential part of any hacker’s collection, and now you know how to use it.
Like Article
Suggest improvement
Share your thoughts in the comments
Please Login to comment...