The HTTP 403 Forbidden is issued by web servers when logins to web services fail or when visitors try to access files they shouldn’t. However, some visitors — well, bots for the most part — doesn’t always get the message and keep sending request after request. This can be part of a brute-force login attempt or other scripted attempts at gaining access to the system.
Although issuing the 403 response should be enough, you sometimes need to apply stronger tools to prevent bad actors from wasting your server capacity. Fail2Ban is a tool that reads your server logs and blocks repeating offenders using temporary firewall rules once unwanted behavior is identified.
This recipe assumes Fail2Ban is already installed albeit unconfigured and that the Apache web server is already running and generating logs. Either software can be found in the package repository of most Linux and *BSD distributions.
Fail2Ban requires a filter definition file and a jail configuration file. The former file instructs Fail2Ban what to look for in Apache’s log files and the second file instructs it on how to respond to matches.
To set up the filter file, create a file in /etc/fail2ban/filter.d/httpd-forbidden.conf containing the following:
[Definition]
failregex = <HOST> - - .*HTTP/[0-9]+(.[0-9]+)?" 403
[client <HOST>] AH01797: client deniedNote that the above failregex directive consists of two separate regular expressions spread over two lines. The first filter regex matches 403 responses in the Apache access log, such as those generated by WordPress and other content systems following incorrect login attempts. The second filter regex will match against resources blocked by the server configuration itself using the Apache error log format. Both of these filters assume that the default logging format is used; you must adjust the filters if you use non-standard logging formats.
Before deploying the new filter in Fail2Ban, you can test them using the fail2ban-regex utility that comes included with Fail2Ban. Run the utility program with the first parameter pointing to the log file and the second pointing to the new filter file. You may have to adjust the paths to the log files depending on your system.
fail2ban-regex /var/log/httpd/access_log /etc/fail2ban/filter.d/httpd-forbidden.conf
fail2ban-regex /var/log/httpd/error_log /etc/fail2ban/filter.d/httpd-forbidden.confIf your logs contain a few logged 403 responses, you should see the same number of matches after running the commands. Generate a few 403 responses so your logs have some example matches.
To set up the jail configuration file, create a file in /etc/fail2ban/jails.d/httpd-forbidden.conf containing the following:
[httpd-forbidden]
enabled = true
filter = httpd-forbidden
backend = polling
logpath = /var/log/httpd/error_log
/var/log/httpd/access_log
bantime = 3600
maxretry = 6
findtime = 1800
port = 80,443
banaction = iptables-multiportThe configuration options here are pretty self-explanatory. Use the filter we created previously (referred to by name and not file path), pull in the two standard Apache logs, and block any IP address with 6-requests within 30-minutes for one hour. The banaction is overwritten from the default to a multi-destination port rule targeting the two default web ports 80 and 443.
In case you accidentally lock yourself out you can login over SSH and unblock your IP by running the following command (replacing the example IP address with your):
fail2ban-client set httpd-forbidden unbanip 192.0.2.42It’s important to include any address that will return a 403 response in the /robots.txt directive file. This will prevent good bots that honor the robots directives file like Bingbot and Googlebot from inadvertently being identified as bad bots and temporarily locked out of your website.