HttpDoom – A Tool For Response-Based Inspection Of Websites Across A Large Amount Of Hosts For Quickly Gaining An Overview Of HTTP-based Attack Surface

Validate large HTTP-based attack surfaces in a very fast way. Heavily inspired by Aquatone.

Why?

When I utilize Aquatone to flyover some hosts, I have some performance issues by the screenshot feature, and the lack of extension capabilities – like validating front-end technologies with a plugin-like system -, also, my codebase is mainly C# and Rust, and make the maintenance of a tool wrote in another language can lead to a lot of issues.

With these ideas in mind, HttpDoom is born.

Installing

In order to install HttpDoom, in the current release cycle, due to not have a runtime-independent build at this time (only devel builds are available), you must have .NET5 runtime (or SDK) – AKA dotnet – installed in your host, with the .NET toolchain available in your Linux or macOS (automatic installation for Windows is not supported at this time, your PR to installation script is welcome. WSL works fine):

$ ./installer.sh

The installer script also updates (removing the current instalation) new releases of HttpDoom.

How this works?

The description (--help) of the CLI is all you need to know:

HttpDoom:
  HttpDoom is a tool for response-based inspection of websites across a large
  surface.
  amount of hosts for quickly gaining an overview of HTTP-based attack

Usage:
  HttpDoom [options]

Options:
  -d, --debug                             Print debugging information
  -f, --follow-redirect                   HTTP client follow any automatic
                                          redirects (default is false)
  -m, --max-redirects                     Max automatic redirect depth when is
                                          enable (default is 3)
  -s, --screenshot                        Take screenshots from the alive host
                                          with ChromeDriver (default is false)
  -r, --screenshot-resolution             Set screenshot resolution (default
                                             is 1366x768)
  -F, --capture-favicon                   Download the application favicon
  -h, --headers <headers>                 Set default headers to every request
                                          User-Agent)
                                          (default is just a random
  -t, --http-timeout <http-timeout>       Timeout in milliseconds for HTTP
                                          requests (default is 5000)
  -T, --threads <threads>                 Number of concurrent threads
                                          (default is 20)
  -o, --output-directory                  Path to save the output directory
  <output-directory>
  -p, --ports <ports>                     Set of ports to check (default is
                                          80, 443, 8080 and 8433)
  -P, --proxy <proxy>                     Proxy to use for HTTP requests
  -w, --w   ord-list <word-list>             List of hosts to flyover against
  (REQUIRED)
  --version                               Show version information
  -?, -h, --help                          Show help and usage information

But it is fast?

Let’s take a look on the result of a flyover agains 5000 hosts on default HttpDoom ports (80, 443, 8080 and 8433), running in the very first working release, with 2 threads (provided by a generic Amazon EC2 instance) agains the same settings on Aquatone 1.7.0:

HttpDoom:

...
[+] Flyover is done! Enumerated #31128 responses in 2.49 minute(s)
[+] Got a total of #176 alive hosts!
...

Aquatone:

...
Writing session file...Time:
 - Started at  : 2020-12-20T08:27:43Z
 - Finished at : 2020-12-20T08:34:35Z
 - Duration    : 6m52s
...

Note: The results of these tests can vary a lot based on a series of specific conditions of your host. Make the test locally and check which tool offers the best performance.

Output

By default, we create all the necessary directories, and we also randomly choose their names (you can set this up with -o, in doubt see --help).

Within the main directory, a general.json file is created containing all the results in a single file (to facilitate the search or ingestion in some visual tool), which looks like this:

[
    {
        "Domain": "google.com",
        "Addresses": [
            "2800:3f0:4001:81a::200e",
            "172.217.28.14"
        ],
        "Requested": "https://google.com/",
        "Port": 443,
        "Content": "u003CHTMLu003Eu003CHEADu003Eu003Cmeta http-equiv=u0022content-typeu0022 content=u0022text/html;charset=utf-8u0022u003Enu003CTITLEu003E301 Movedu003C/TITLEu003Eu003C/HEADu003Eu003CBODYu003Enu003CH1u003E301 Movedu003C/H1u003EnThe document has movednu003CA HREF=u0022https://www.google.com/u0022u003Ehereu003C/Au003E.rnu003C/BODYu003Eu003C/HTMLu003Ern",
        "ScreenshotPath": "C:\Users\REDACTED\AppData\Local\Temp\c14obxml.kfy\Screenshots\0086aea9-c4d4-4bbf-89d8-728e5d2ff184.png",
        "FaviconPath": "C:\Users\REDACTED\AppData\Local\Temp\c14obxml.kfy\Favicons\172d671c-636d-443b-b5b4-30ed6e10b8aa.ico",
        "Headers": [
            {
                   "Key": "Location",
                "Value": [
                    "https://www.google.com/"
                ]
            },
            {
                "Key": "Date",
                "Value": [
                    "Tue, 02 Feb 2021 15:59:46 GMT"
                ]
            },
            {
                "Key": "Cache-Control",
                "Value": [
                    "public, max-age=2592000"
                ]
            },
            {
                "Key": "Server",
                "Value": [
                    "gws"
                ]
            },
            {
                "Key": "X-XSS-Protection",
                "Value": [
                    "0"
                ]
            },
            {
                "Key": "X-Frame-Options",
                "Value": [
                    "SAMEORIGIN"
                ]   
            },
            {
                "Key": "Alt-Svc",
                "Value": [
                    "h3-29=u0022:443u0022; ma=2592000",
                    "h3-T051=u0022:443u0022; ma=2592000",
                    "h3-Q050=u0022:443u0022; ma=2592000",
                    "h3-Q046=u0022:443u0022; ma=2592000",
                    "h3-Q043=u0022:443u0022; ma=2592000",
                    "quic=u0022:443u0022; ma=2592000"
                ]
            }
        ],
        "Cookies": [],
        "StatusCode": 301
    },
    // ...
]

A directory called Individual Results is also created, indexing the results individually, categorically based on the name of the URI used for the request, as well the screenshots, if you use HttpDoom with option -s and favicons, if the site has one, and if you use HttpDoom with option -F:

.
├── Favicons
│   ├── 31be8e61-d90b-4b40-bcef-640fb31588e7.ico
│   └── 4e097b93-12f2-4f20-9582-547cc6d20312.ico
├── Individual Results
│   ├── http:google.com:80.json
│   └── https:google.com:443.json
├── Screenshots
│   ├── 1d395ce1-b329-4379-8d9e-2868ed41e67d.png
│   └── a9f90f23-4d5c-4f13-ba3e-5d8f88aa3926.png
└── general.json

Note: The pattern of Individual Results files is scheme:address:port.But : can be an invalid character depending on what operational system you use HttpDoom. For deeper ACK, check the documentation of Path.GetInvalidFileNameChars() in MSDN.

Roadmap

The project are focused to be a really useful tool.

• 0x00: Make the satuday project work;
• 0x01: Baking the CLI options very similar to Aquatone;
• 0x02: Fix issues with large (5K+) hosts wordlists;
• 0x03: Well, this is not “threads” but work like, maybe need a better polishing;
• 0x04 Screenshots because why not;
• 0x05: Create the community-driven fingerprint engine to enumerate vulnerabilities on headers and bodies of the HTTP responses;