TheLazy.Dev
Portfolio GitHub

Blog

cybersecurity |webcomponents |analytics |python

Getting Started with webprobe

gerrishon

Gerrishon Sirere

Posted on August 31, 2022

Getting Started with webprobe

Twitter Follow

Logo

webprobe

Nifty and sophisticated web path scanner

A sophisticated web path scanner designed for the most descerning bug bounty hunters.

This CLI brute forces directories and files in webservers.

Installation & Usage

Requirement: python 3.8 or higher

  • Install with PyPi: pip install webprobe

How to use

Some common examples on how to use webprobe.

💡 If you need to see a list of all options, just use the -h | --help argument.

To use multiple wordlists, you can separate your wordlists with commas. Example: wordlist1.txt,wordlist2.txt

Simple usage 


webprobe -u https://example.com
Enter fullscreen mode Exit fullscreen mode 

webprobe -e php,html,js -u https://example.com
Enter fullscreen mode Exit fullscreen mode 
webprobe -e php,html,js -u https://exaple.com -w /path/to/wordlist
Enter fullscreen mode Exit fullscreen mode

Pausing progress

You can pause the scanning progress with CTRL+C from here, you can save the progress (and continue later), skip the current target, or skip the current sub-directory.

Recursion

  • Brutforcing recursively can be achieved using -r or --recursive flag.

For example, if webprobe finds admin/, it will brute-force admin/* (* is where it brute forces).


webprobe -e php,html,js -u https://example.com -r
Enter fullscreen mode Exit fullscreen mode
  • You can set the max recursion depth with --recursion-depth and status codes to recurse with --recursion-status 

webprobe -e php,html,js -u https://example.com -r --recursion-depth 3 --recursion-status 200-39
Enter fullscreen mode Exit fullscreen mode

  • You can brute force recursively all found paths, not just paths end with / using --force-recursive flag.

  • You can recursively brute-force all depths of a path (a/b/c => add a/, a/b/) using --deep-recursive flag.

  • If there are sub-directories that you do not want to brute-force recursively use --exclude-subdirs flag.


webprobe -e php,html,js -u https://example.com -r --exclude-subdirs image/,media/,css/
Enter fullscreen mode Exit fullscreen mode

Threads

Thread number (-t | --threads) reflects the number of separated brute force processes. The bigger the thread number, the faster webprobe runs. By default, the number of threads is 25, but you can increase it if you want to speed up the progress.

However, the speed still depends on the response time of the server.

💡 keep the threads number within a reasonable range because it can cause DoS (Denial of Service).


webprobe -e php,htm,js,bak,zip,tgz,txt -u https://example.com -t 20
Enter fullscreen mode Exit fullscreen mode

Filters

Use -i | --include-status and -x | --exclude-status flags to select allowed and not allowed response status-codes

For more advanced filters: --exclude-sizes, --exclude-texts, --exclude-regexps, --exclude-redirects and --exclude-response


webprobe  -e php,html,js -u https://example.com --exclude-sizes 1B,243KB
Enter fullscreen mode Exit fullscreen mode 

webprobe -e php,html,js -u https://example.com --exclude-texts "403 Forbidden"
Enter fullscreen mode Exit fullscreen mode 

webprobe -e php,html,js -u https://example.com --exclude-regexps "^Error$"
Enter fullscreen mode Exit fullscreen mode 

webprobe -e php,html,js -u https://example.com --exclude-redirects "https://(.*).okta.com/*"
Enter fullscreen mode Exit fullscreen mode 

webprobe -e php,html,js -u https://example.com --exclude-response /error.html
Enter fullscreen mode Exit fullscreen mode

Scan sub-directories

  • You can scan a list of sub-directories with --subdirs flag. 

webprobe -e php,html,js -u https://example.com --subdirs /,admin/,folder/
Enter fullscreen mode Exit fullscreen mode

Proxies

  • Webprobe supports both SOCKS and HTTP proxy. You can enlist a proxy server or a list of proxy servers from a file. 

webprobe -e php,html,js -u https://example.com --proxy 127.0.0.1:8080
Enter fullscreen mode Exit fullscreen mode 

webprobe -e php,html,js -u https://example.com --proxy socks5://10.10.0.1:8080
Enter fullscreen mode Exit fullscreen mode 

webprobe -e php,html,js -u https://example.com --proxylist proxyservers.txt
Enter fullscreen mode Exit fullscreen mode

More example commands 


cat urls.txt | python3 webprobe --stdin
Enter fullscreen mode Exit fullscreen mode 

webprobe -u https://example.com --max-time 360
Enter fullscreen mode Exit fullscreen mode 

webprobe -u https://example.com --auth admin:pass --auth-type basic
Enter fullscreen mode Exit fullscreen mode 

webprobe -u https://example.com --header-list rate-limit-bypasses.txt
Enter fullscreen mode Exit fullscreen mode

Reports

Supported report formats are: simple, plain, json, xml, md, csv, html, sqlite

💡 We will be adding yaml soon

 webprobe -e php -l URLs.txt --format plain -o report.txt
Enter fullscreen mode Exit fullscreen mode 
 webprobe -e php -u https://example.com --format html -o example.json
Enter fullscreen mode Exit fullscreen mode

License📑

License: MIT

This software is licensed under the MIT License. See the License file in the top distribution directory for the full license text.

Donate🎁

In order to for us to maintain this project and grow our community of contributors.
Donate

Code of Conduct

Code of Conduct is adapted from the Contributor Covenant, version 1.2.0 available at Code of Conduct

💖 💪 🙅 🚩
gerrishon
Gerrishon Sirere

Posted on August 31, 2022

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Getting Started with webprobe
cybersecurity Getting Started with webprobe

August 31, 2022