Teler-Waf – A Go HTTP Middleware That Provides Teler IDS Functionality To Protect Against Web-Based Attacks And Improve The Security Of Go-based Web Applications

teler-waf is a comprehensive security solution for Go-based web applications. It acts as an HTTP middleware, providing an easy-to-use interface for integrating IDS functionality with teler IDS into existing Go applications. By using teler-waf, you can help protect against a variety of web-based attacks, such as cross-site scripting (XSS) and SQL injection.

The package comes with a standard net/http.Handler, making it easy to integrate into your application’s routing. When a client makes a request to a route protected by teler-waf, the request is first checked against the teler IDS to detect known malicious patterns. If no malicious patterns are detected, the request is then passed through for further processing.

In addition to providing protection against web-based attacks, teler-waf can also help improve the overall security and integrity of your application. It is highly configurable, allowing you to tailor it to fit the specific needs of your application.

See also:

• kitabisa/teler: Real-time HTTP intrusion detection.
• dwisiswant0/cox: Cox is bluemonday-wrapper to perform a deep-clean and/or sanitization of (nested-)interfaces from HTML to prevent XSS payloads.

Features

Some core features of teler-waf include:

• HTTP middleware for Go web applications.
• Integration of teler IDS functionality.
• Detection of known malicious patterns using the teler IDS.
  • Common web attacks, such as cross-site scripting (XSS) and SQL injection, etc.
  • CVEs, covers known vulnerabilities and exploits.
  • Bad IP addresses, such as those associated with known malicious actors or botnets.
  • Bad HTTP referers, such as those that are not expected based on the application’s URL structure or are known to be associated with malicious actors.
  • Bad crawlers, covers requests from known bad crawlers or scrapers, such as those that are known to cause performance issues or attempt to extract sensitive information from the application.
  • Directory bruteforce attacks, such as by trying common directory names or using dictionary attacks.
• Configuration options to whitelist specific types of requests based on their URL or headers.
• Easy integration with many frameworks.
• High configurability to fit the specific needs of your application.

Overall, teler-waf provides a comprehensive security solution for Go-based web applications, helping to protect against web-based attacks and improve the overall security and integrity of your application.

Install

To install teler-waf in your Go application, run the following command to download and install the teler-waf package:

go get github.com/kitabisa/teler-waf

Usage

Here is an example of how to use teler-waf in a Go application:

1. Import the teler-waf package in your Go code:

import "github.com/kitabisa/teler-waf"

2. Use the New function to create a new instance of the Teler type. This function takes a variety of optional parameters that can be used to configure teler-waf to suit the specific needs of your application.

waf := teler.New()

3. Use the Handler method of the Teler instance to create a net/http.Handler. This handler can then be used in your application’s HTTP routing to apply teler-waf’s security measures to specific routes.

handler := waf.Handler(http.HandlerFunc(yourHandlerFunc))

4. Use the handler in your application’s HTTP routing to apply teler-waf’s security measures to specific routes.

http.Handle("/path", handler)

That’s it! You have configured teler-waf in your Go application.

Options:

For a list of the options available to customize teler-waf, see the teler.Options struct.

Examples

Here is an example of how to customize the options and rules for teler-waf:

// main.go
package main

import (
	"net/http"

	"github.com/kitabisa/teler-waf"
	"github.com/kitabisa/teler-waf/request"
	"github.com/kitabisa/teler-waf/threat"
)

var myHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	// This is the handler function for the route that we want to protect
	// with teler-waf's security measures.
	w.Write([]byte("hello world"))
})

var rejectHandler = http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
	// This is the handler function for the route that we want to be rejected
	// if the teler-waf's security measures are triggered.
	http.Error(w, "Sorry, your request has been denied for security reasons.", http.StatusForbidden)
})

func main() {
	// Create a new instance of the Teler type using the New function
	// and configure it using the Options struct.
	telerMiddleware := teler.New(tel   er.Options{
		// Exclude specific threats from being checked by the teler-waf.
		Excludes: []threat.Threat{
			threat.BadReferrer,
			threat.BadCrawler,
		},
		// Specify whitelisted URIs (path & query parameters), headers,
		// or IP addresses that will always be allowed by the teler-waf.
		Whitelists: []string{
			`(curl|Go-http-client|okhttp)/*`,
			`^/wp-login\.php`,
			`(?i)Referer: https?:\/\/www\.facebook\.com`,
			`192\.168\.0\.1`,
		},
		// Specify custom rules for the teler-waf to follow.
		Customs: []teler.Rule{
			{
				// Give the rule a name for easy identification.
				Name:      "Log4j Attack",
				// Specify the logical operator to use when evaluating the rule's conditions.
				Condition: "or",
				// Specify the conditions that must be met for the rule to trigger.
				Rules: []teler.Condition{
					{
						// Specify the HTTP method that the rule applies to.
				   		Method: request.GET,
						// Specify the element of the request that the rule applies to
						// (e.g. URI, headers, body).
						Element: request.URI,
						// Specify the pattern to match against the element of the request.
						Pattern: `\$\{.*:\/\/.*\/?\w+?\}`,
					},
				},
			},
		},
		// Specify the file path to use for logging.
		LogFile: "/tmp/teler.log",
	})

	// Set the rejectHandler as the handler for the telerMiddleware.
	telerMiddleware.SetHandler(rejectHandler)

	// Create a new handler using the handler method of the Teler instance
	// and pass in the myHandler function for the route we want to protect.
	app := telerMiddleware.Handler(myHandler)

	// Use the app handler as the handler for the route.
	http.ListenAndServe("127.0.0.1:3000", app)
}

: When using a whitelist, any request that matches it – regardless of the type of threat it poses, it will be returned without further analysis.

To illustrate, suppose you set up a whitelist to permit requests containing a certain string. In the event that a request contains that string, but /also/ includes a payload such as an SQL injection or cross-site scripting (“XSS”) attack, the request may not be thoroughly analyzed for common web attack threats and will be swiftly returned. See issue #25.

For more examples of how to use teler-waf or integrate it with any framework, take a look at examples/ directory.

Development

By default, teler-waf caches all incoming requests for 15 minutes & clear them every 20 minutes to improve the performance. However, if you’re still customizing the settings to match the requirements of your application, you can disable caching during development by setting the development mode option to true. This will prevent incoming requests from being cached and can be helpful for debugging purposes.

// Create a new instance of the Teler type using
// the New function & enable development mode option.
telerMiddleware := teler.New(teler.Options{
	Development: true,
})

Logs

Here is an example of what the log lines would look like if teler-waf detects a threat on a request:

{"level":"warn","ts":1672261174.5995026,"msg":"bad crawler","id":"654b85325e1b2911258a","category":"BadCrawler","request":{"method":"GET","path":"/","ip_addr":"127.0.0.1:37702","headers":{"Accept":["*/*"],"User-Agent":["curl/7.81.0"]},"body":""}}
{"level":"warn","ts":1672261175.9567692,"msg":"directory bruteforce","id":"b29546945276ed6b1fba","category":"DirectoryBruteforce","request":{"method":"GET","path":"/.git","ip_addr":"127.0.0.1:37716","headers":{"Accept":["*/*"],"User-Agent":["X"]},"body":""}}
{"level":"warn","ts":1672261177.1487508,"msg":"Detects common comment types","id":"75412f2cc0ec1cf79efd","category":"CommonWebAttack","request":{"method":"GET","path":"/?id=1%27%   20or%201%3D1%23","ip_addr":"127.0.0.1:37728","headers":{"Accept":["*/*"],"User-Agent":["X"]},"body":""}}

The id is a unique identifier that is generated when a request is rejected by teler-waf. It is included in the HTTP response headers of the request (X-Teler-Req-Id), and can be used to troubleshoot issues with requests that are being made to the website.

For example, if a request to a website returns an HTTP error status code, such as a 403 Forbidden, the teler request ID can be used to identify the specific request that caused the error and help troubleshoot the issue.

Teler request IDs are used by teler-waf to track requests made to its web application and can be useful for debugging and analyzing traffic patterns on a website.

Datasets

The teler-waf package utilizes a dataset of threats to identify and analyze each incoming request for potential security threats. This dataset is updated daily, which means that you will always have the latest resource. The dataset is initially stored in the user-level cache directory (on Unix systems, it returns $XDG_CACHE_HOME/teler-waf as specified by XDG Base Directory Specification if non-empty, else $HOME/.cache/teler-waf. On Darwin, it returns $HOME/Library/Caches/teler-waf. On Windows, it returns %LocalAppData%/teler-waf. On Plan 9, it returns $home/lib/cache/teler-waf) on your first launch. Subsequent launch will utilize the cached dataset, rather than downloading it again.

: The threat datasets are obtained from the kitabisa/teler-resources repository.

However, there may be situations where you want to disable automatic updates to the threat dataset. For example, you may have a slow or limited internet connection, or you may be using a machine with restricted file access. In these cases, you can set an option called NoUpdateCheck to true, which will prevent the teler-waf from automatically updating the dataset.

// Create a new instance of the Teler type using the New
// function & disable automatic updates to the threat dataset.
telerMiddleware := teler.New(teler.Options{
	NoUpdateCheck: true,
})

Finally, there may be cases where it’s necessary to load the threat dataset into memory rather than saving it to a user-level cache directory. This can be particularly useful if you’re running the application or service on a distroless or runtime image, where file access may be limited or slow. In this scenario, you can set an option called InMemory to true, which will load the threat dataset into memory for faster access.

// Create a new instance of the Teler type using the
// New function & enable in-memory threat datasets store.
telerMiddleware := teler.New(teler.Options{
	InMemory: true,
})

: This may also consume more system resources, so it’s worth considering the trade-offs before making this decision.

Resources

• teler WAF tester! — You are free to use the following site for testing, https://waf.teler.app.

Security

If you discover a security issue, please bring it to their attention right away, we take security seriously!

Reporting a Vulnerability

If you have information about a security issue, or vulnerability in this teler-waf package, and/or you are able to successfully execute such as cross-site scripting (XSS) and pop-up an alert in our demo site (see resources), please do NOT file a public issue — instead, kindly send your report privately via the vulnerability report form or to our official channels as per our security policy.

Limitations

Here are some limitations of using teler-waf:

• Performance overhead: teler-waf may introduce some performance overhead, as the teler-waf will need to process each incoming request. If you have a high volume of traffic, this can potentially slow down the overall performance of your application significantly, especially if you enable the CVEs threat detection. See benchmark below:

$ go test -bench . -cpu=4
goos: linux
goarch: amd64
pkg: github.com/kitabisa/teler-waf
cpu: 11th Gen Intel(R) Core(TM) i9-11900H @ 2.50GHz
BenchmarkTelerDefaultOptions-4               	   42649	     24923 ns/op	    6206 B/op	      97 allocs/op
BenchmarkTelerCommonWebAttackOnly-4          	   48589	     23069 ns/op	    5560 B/op	      89 allocs/op
BenchmarkTelerCVEOnly-4                      	   48103	     23909 ns/op	    5587 B/op	      90 allocs/op
BenchmarkTelerBadIPAddressOnly-4             	   47871	     22846 ns/op	    5470 B/op	      87 allocs/op
BenchmarkTelerBadReferrerOnly-4              	   47558	     23917 ns/op	    5649 B/op	      89 allocs/op
BenchmarkTelerBadCrawlerOnly-4               	   42138	     24010 ns/op	    5694 B/op	      86 allocs/op
BenchmarkTelerDirectoryBruteforceOnly-4      	   45274	     23523 ns/op	    5657 B/op	      86 allocs/op
BenchmarkT   elerCustomRule-4                   	   48193	     22821 ns/op	    5434 B/op	      86 allocs/op
BenchmarkTelerWithoutCommonWebAttack-4       	   44524	     24822 ns/op	    6054 B/op	      94 allocs/op
BenchmarkTelerWithoutCVE-4                   	   46023	     25732 ns/op	    6018 B/op	      93 allocs/op
BenchmarkTelerWithoutBadIPAddress-4          	   39205	     25927 ns/op	    6220 B/op	      96 allocs/op
BenchmarkTelerWithoutBadReferrer-4           	   45228	     24806 ns/op	    5967 B/op	      94 allocs/op
BenchmarkTelerWithoutBadCrawler-4            	   45806	     26114 ns/op	    5980 B/op	      97 allocs/op
BenchmarkTelerWithoutDirectoryBruteforce-4   	   44432	     25636 ns/op	    6185 B/op	      97 allocs/op
PASS
ok  	github.com/kitabisa/teler-waf	25.759s

: Benchmarking results may vary and may not be consistent. Those results were obtained when there were >1.5k CVE templates and the teler-resources dataset may have increased since then, which may impact the results.

• Configuration complexity: Configuring teler-waf to suit the specific needs of your application can be complex, and may require a certain level of expertise in web security. This can make it difficult for those who are not familiar with application firewalls and IDS systems to properly set up and use teler-waf.
• Limited protection: teler-waf is not a perfect security solution, and it may not be able to protect against all possible types of attacks. As with any security system, it is important to regularly monitor and maintain teler-waf to ensure that it is providing the desired level of protection.

Known Issues

To view a list of known issues with teler-waf, please filter the issues by the “known-issue” label.

License

This program is developed and maintained by members of Kitabisa Security Team, and this is not an officially supported Kitabisa product. This program is free software: you can redistribute it and/or modify it under the terms of the Apache license. Kitabisa teler-waf and any contributions are copyright © by Dwi Siswanto 2022-2023.