fixed possible RCE vulnerability, unset escape variable (default tilde) stops consider "~" char after new-line as composing escape sequence

closes GHSA-m985-3f3v-cwmm for 0.9

.github/ISSUE_TEMPLATE.md

@@ -0,0 +1,49 @@
+_We will be very grateful, if your problem was described as completely as possible,
+enclosing excerpts from logs (if possible within DEBUG mode, if no errors evident
+within INFO mode), and configuration in particular of effected relevant settings
+(e.g., with ` fail2ban-client -d | grep 'affected-jail-name' ` for a particular
+jail troubleshooting).
+Thank you in advance for the details, because such issues like "It does not work" 
+alone could not help to resolve anything!
+Thanks! (remove this paragraph and other comments upon reading)_
+
+### Environment:
+
+_Fill out and check (`[x]`) the boxes which apply. If your Fail2Ban version is outdated, 
+and you can't verify that the issue persists in the recent release, better seek support 
+from the distribution you obtained Fail2Ban from_
+
+- Fail2Ban version (including any possible distribution suffixes):
+- OS, including release name/version: 
+- [ ] Fail2Ban installed via OS/distribution mechanisms
+- [ ] You have not applied any additional foreign patches to the codebase
+- [ ] Some customizations were done to the configuration (provide details below is so)
+
+### The issue:
+
+_Summary here_
+
+#### Steps to reproduce
+
+#### Expected behavior
+
+#### Observed behavior
+
+#### Any additional information
+
+### Configuration, dump and another helpful excerpts
+
+#### Any customizations done to /etc/fail2ban/ configuration
+```
+```
+
+#### Relevant parts of /var/log/fail2ban.log file:
+_preferably obtained while running fail2ban with `loglevel = 4`_
+
+```
+```
+
+#### Relevant lines from monitored log files in question:
+
+```
+```

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,9 @@
+Before submitting your PR, please review the following checklist:
+
+- [ ] **CONSIDER adding a unit test** if your PR resolves an issue
+- [ ] **LIST ISSUES** this PR resolves
+- [ ] **MAKE SURE** this PR doesn't break existing tests 
+- [ ] **KEEP PR small** so it could be easily reviewed.
+- [ ] **AVOID** making unnecessary stylistic changes in unrelated code
+- [ ] **ACCOMPANY** each new `failregex` for filter `X` with sample log lines 
+      within `fail2ban/tests/files/logs/X` file

.mailmap

@@ -0,0 +1,5 @@
+Lee Clemens <java@leeclemens.net>
+Serg G. Brester <info@sebres.de>
+Serg G. Brester <serg.brester@sebres.de>
+Serg G. Brester <sergey.brester@W7-DEHBG0189.wincor-nixdorf.com>
+Viktor Szépe <viktor@szepe.net>

.pydevproject

@@ -1,12 +0,0 @@
-<?xml version="1.0" encoding="UTF-8" standalone="no"?>
-<?eclipse-pydev version="1.0"?>
-
-<pydev_project>
-<pydev_property name="org.python.pydev.PYTHON_PROJECT_VERSION">python 2.3</pydev_property>
-<pydev_pathproperty name="org.python.pydev.PROJECT_SOURCE_PATH">
-<path>/fail2ban-0.8/client</path>
-<path>/fail2ban-0.8/server</path>
-<path>/fail2ban-0.8/testcases</path>
-<path>/fail2ban-0.8</path>
-</pydev_pathproperty>
-</pydev_project>

.travis.yml

@@ -6,13 +6,17 @@ python:
   - 2.6
   - 2.7
   - pypy
-  - 3.2
+  # disabled until coverage module fixes up compatibility issue
+  # - 3.2
   - 3.3
   - 3.4
-  - pypy3
+  # disabled since setuptools dropped support for Python 3.0 - 3.2
+  # - pypy3
+  - pypy3.3-5.2-alpha1
 before_install:
-  - if [[ $TRAVIS_PYTHON_VERSION == 2* || $TRAVIS_PYTHON_VERSION == 'pypy' ]]; then export F2B_PY_2=true && echo "Set F2B_PY_2"; fi
-  - if [[ $TRAVIS_PYTHON_VERSION == 3* || $TRAVIS_PYTHON_VERSION == 'pypy3' ]]; then export F2B_PY_3=true && echo "Set F2B_PY_3"; fi
+  - echo "running under $TRAVIS_PYTHON_VERSION"
+  - if [[ $TRAVIS_PYTHON_VERSION == 2* || $TRAVIS_PYTHON_VERSION == pypy* && $TRAVIS_PYTHON_VERSION != pypy3* ]]; then export F2B_PY_2=true && echo "Set F2B_PY_2"; fi
+  - if [[ $TRAVIS_PYTHON_VERSION == 3* || $TRAVIS_PYTHON_VERSION == pypy3* ]]; then export F2B_PY_3=true && echo "Set F2B_PY_3"; fi
   - travis_retry sudo apt-get update -qq
   # Set this so sudo executes the correct python binary
   #   Anything not using sudo will already have the correct environment
@@ -22,7 +26,7 @@ install:
   #   coverage
   - travis_retry pip install coverage
   #   coveralls
-  - travis_retry pip install coveralls
+  - travis_retry pip install coveralls codecov
   #   dnspython or dnspython3
   - if [[ "$F2B_PY_2" ]]; then travis_retry pip install dnspython; fi
   - if [[ "$F2B_PY_3" ]]; then travis_retry pip install dnspython3; fi
@@ -40,8 +44,11 @@ script:
   - if [[ "$F2B_PY_3" ]]; then coverage run bin/fail2ban-testcases; fi
   # Use $VENV_BIN (not python) or else sudo will always run the system's python (2.7)
   - sudo $VENV_BIN/pip install .
+  # Doc files should get installed on Travis under Linux
+  - test -e /usr/share/doc/fail2ban/FILTERS
 after_success:
   - coveralls
+  - codecov
 matrix:
   fast_finish: true
 # Might be worth looking into

FILTERS

@@ -227,7 +227,7 @@ Regular expressions (failregex, ignoreregex) assume that the date/time has been
 removed from the log line (this is just how fail2ban works internally ATM).
 
 If the format is like '<date...> error 1.2.3.4 is evil' then you need to match
-the < at the start so regex should be similar to '^<> <HOST> is evil$' using
+the <> at the start so regex should be similar to '^<> error <HOST> is evil$' using
 <HOST> where the IP/domain name appears in the log line.
 
 The following general rules apply to regular expressions:

MANIFEST

@@ -1,18 +1,8 @@
-CONTRIBUTING.md
-COPYING
-ChangeLog
-DEVELOP
-FILTERS
-README.Solaris
-README.md
-RELEASE
-THANKS
-TODO
-Vagrantfile
 bin/fail2ban-client
 bin/fail2ban-regex
 bin/fail2ban-server
 bin/fail2ban-testcases
+ChangeLog
 config/action.d/apf.conf
 config/action.d/badips.conf
 config/action.d/badips.py
@@ -26,25 +16,32 @@ config/action.d/firewallcmd-allports.conf
 config/action.d/firewallcmd-ipset.conf
 config/action.d/firewallcmd-multiport.conf
 config/action.d/firewallcmd-new.conf
+config/action.d/firewallcmd-rich-logging.conf
+config/action.d/firewallcmd-rich-rules.conf
 config/action.d/hostsdeny.conf
 config/action.d/ipfilter.conf
 config/action.d/ipfw.conf
 config/action.d/iptables-allports.conf
 config/action.d/iptables-common.conf
+config/action.d/iptables.conf
 config/action.d/iptables-ipset-proto4.conf
 config/action.d/iptables-ipset-proto6-allports.conf
 config/action.d/iptables-ipset-proto6.conf
-config/action.d/iptables-multiport-log.conf
 config/action.d/iptables-multiport.conf
+config/action.d/iptables-multiport-log.conf
 config/action.d/iptables-new.conf
 config/action.d/iptables-xt_recent-echo.conf
-config/action.d/iptables.conf
 config/action.d/mail-buffered.conf
-config/action.d/mail-whois-lines.conf
-config/action.d/mail-whois.conf
 config/action.d/mail.conf
+config/action.d/mail-whois-common.conf
+config/action.d/mail-whois.conf
+config/action.d/mail-whois-lines.conf
 config/action.d/mynetwatchman.conf
-config/action.d/nsupdate.conf
+config/action.d/netscaler.conf
+config/action.d/nftables-allports.conf
+config/action.d/nftables-common.conf
+config/action.d/nftables-multiport.conf
+config/action.d/npf.conf
 config/action.d/nsupdate.conf
 config/action.d/osx-afctl.conf
 config/action.d/osx-ipfw.conf
@@ -52,14 +49,15 @@ config/action.d/pf.conf
 config/action.d/route.conf
 config/action.d/sendmail-buffered.conf
 config/action.d/sendmail-common.conf
+config/action.d/sendmail.conf
 config/action.d/sendmail-geoip-lines.conf
+config/action.d/sendmail-whois.conf
 config/action.d/sendmail-whois-ipjailmatches.conf
 config/action.d/sendmail-whois-ipmatches.conf
 config/action.d/sendmail-whois-lines.conf
 config/action.d/sendmail-whois-matches.conf
-config/action.d/sendmail-whois.conf
-config/action.d/sendmail.conf
 config/action.d/shorewall.conf
+config/action.d/shorewall-ipset-proto6.conf
 config/action.d/smtp.py
 config/action.d/symbiosis-blacklist-allports.conf
 config/action.d/ufw.conf
@@ -75,6 +73,7 @@ config/filter.d/apache-modsecurity.conf
 config/filter.d/apache-nohome.conf
 config/filter.d/apache-noscript.conf
 config/filter.d/apache-overflows.conf
+config/filter.d/apache-pass.conf
 config/filter.d/apache-shellshock.conf
 config/filter.d/assp.conf
 config/filter.d/asterisk.conf
@@ -85,60 +84,65 @@ config/filter.d/courier-auth.conf
 config/filter.d/courier-smtp.conf
 config/filter.d/cyrus-imap.conf
 config/filter.d/directadmin.conf
+config/filter.d/domino-smtp.conf
 config/filter.d/dovecot.conf
 config/filter.d/dropbear.conf
+config/filter.d/drupal-auth.conf
 config/filter.d/ejabberd-auth.conf
 config/filter.d/exim-common.conf
-config/filter.d/exim-spam.conf
 config/filter.d/exim.conf
+config/filter.d/exim-spam.conf
 config/filter.d/freeswitch.conf
+config/filter.d/froxlor-auth.conf
 config/filter.d/groupoffice.conf
 config/filter.d/gssftpd.conf
 config/filter.d/guacamole.conf
+config/filter.d/haproxy-http-auth.conf
 config/filter.d/horde.conf
 config/filter.d/ignorecommands
 config/filter.d/ignorecommands/apache-fakegooglebot
 config/filter.d/kerio.conf
 config/filter.d/lighttpd-auth.conf
+config/filter.d/mongodb-auth.conf
 config/filter.d/monit.conf
+config/filter.d/murmur.conf
 config/filter.d/mysqld-auth.conf
 config/filter.d/nagios.conf
 config/filter.d/named-refused.conf
 config/filter.d/nginx-botsearch.conf
 config/filter.d/nginx-http-auth.conf
+config/filter.d/nginx-limit-req.conf
 config/filter.d/nsd.conf
+config/filter.d/openhab.conf
 config/filter.d/openwebmail.conf
 config/filter.d/oracleims.conf
 config/filter.d/pam-generic.conf
-config/filter.d/pam-generic.conf
-config/filter.d/pam-generic.conf
 config/filter.d/perdition.conf
 config/filter.d/php-url-fopen.conf
-config/filter.d/php-url-fopen.conf
-config/filter.d/php-url-fopen.conf
 config/filter.d/portsentry.conf
+config/filter.d/postfix.conf
 config/filter.d/postfix-rbl.conf
 config/filter.d/postfix-sasl.conf
-config/filter.d/postfix-sasl.conf
-config/filter.d/postfix-sasl.conf
-config/filter.d/postfix.conf
 config/filter.d/proftpd.conf
 config/filter.d/pure-ftpd.conf
 config/filter.d/qmail.conf
 config/filter.d/recidive.conf
 config/filter.d/roundcube-auth.conf
+config/filter.d/screensharingd.conf
 config/filter.d/selinux-common.conf
 config/filter.d/selinux-ssh.conf
 config/filter.d/sendmail-auth.conf
 config/filter.d/sendmail-reject.conf
 config/filter.d/sendmail-spam.conf
 config/filter.d/sieve.conf
+config/filter.d/slapd.conf
 config/filter.d/sogo-auth.conf
 config/filter.d/solid-pop3d.conf
 config/filter.d/squid.conf
 config/filter.d/squirrelmail.conf
-config/filter.d/sshd-ddos.conf
+config/filter.d/sshd-aggressive.conf
 config/filter.d/sshd.conf
+config/filter.d/sshd-ddos.conf
 config/filter.d/stunnel.conf
 config/filter.d/suhosin.conf
 config/filter.d/tine20.conf
@@ -152,13 +156,14 @@ config/paths-common.conf
 config/paths-debian.conf
 config/paths-fedora.conf
 config/paths-freebsd.conf
+config/paths-opensuse.conf
 config/paths-osx.conf
+CONTRIBUTING.md
+COPYING
+.coveragerc
+DEVELOP
 doc/run-rootless.txt
 fail2ban-2to3
-fail2ban-testcases-all
-fail2ban-testcases-all-python3
-fail2ban/__init__.py
-fail2ban/client/__init__.py
 fail2ban/client/actionreader.py
 fail2ban/client/beautifier.py
 fail2ban/client/configparserinc.py
@@ -166,13 +171,15 @@ fail2ban/client/configreader.py
 fail2ban/client/configurator.py
 fail2ban/client/csocket.py
 fail2ban/client/fail2banreader.py
+fail2ban/client/fail2banregex.py
 fail2ban/client/filterreader.py
+fail2ban/client/__init__.py
 fail2ban/client/jailreader.py
 fail2ban/client/jailsreader.py
 fail2ban/exceptions.py
 fail2ban/helpers.py
+fail2ban/__init__.py
 fail2ban/protocol.py
-fail2ban/server/__init__.py
 fail2ban/server/action.py
 fail2ban/server/actions.py
 fail2ban/server/asyncserver.py
@@ -183,11 +190,12 @@ fail2ban/server/datetemplate.py
 fail2ban/server/faildata.py
 fail2ban/server/failmanager.py
 fail2ban/server/failregex.py
-fail2ban/server/filter.py
 fail2ban/server/filtergamin.py
 fail2ban/server/filterpoll.py
+fail2ban/server/filter.py
 fail2ban/server/filterpyinotify.py
 fail2ban/server/filtersystemd.py
+fail2ban/server/__init__.py
 fail2ban/server/iso8601.py
 fail2ban/server/jail.py
 fail2ban/server/jails.py
@@ -197,7 +205,9 @@ fail2ban/server/server.py
 fail2ban/server/strptime.py
 fail2ban/server/ticket.py
 fail2ban/server/transmitter.py
-fail2ban/tests/__init__.py
+fail2ban/setup.py
+fail2ban-testcases-all
+fail2ban-testcases-all-python3
 fail2ban/tests/action_d/__init__.py
 fail2ban/tests/action_d/test_badips.py
 fail2ban/tests/action_d/test_smtp.py
@@ -207,9 +217,11 @@ fail2ban/tests/banmanagertestcase.py
 fail2ban/tests/clientreadertestcase.py
 fail2ban/tests/config/action.d/brokenaction.conf
 fail2ban/tests/config/fail2ban.conf
+fail2ban/tests/config/filter.d/common.conf
 fail2ban/tests/config/filter.d/simple.conf
 fail2ban/tests/config/filter.d/test.conf
 fail2ban/tests/config/filter.d/test.local
+fail2ban/tests/config/filter.d/zzz-generic-example.conf
 fail2ban/tests/config/jail.conf
 fail2ban/tests/config/paths-common.conf
 fail2ban/tests/config/paths-debian.conf
@@ -218,33 +230,34 @@ fail2ban/tests/config/paths-osx.conf
 fail2ban/tests/databasetestcase.py
 fail2ban/tests/datedetectortestcase.py
 fail2ban/tests/dummyjail.py
+fail2ban/tests/fail2banregextestcase.py
 fail2ban/tests/failmanagertestcase.py
-fail2ban/tests/files/action.d/action.py
 fail2ban/tests/files/action.d/action_checkainfo.py
 fail2ban/tests/files/action.d/action_errors.py
 fail2ban/tests/files/action.d/action_modifyainfo.py
 fail2ban/tests/files/action.d/action_noAction.py
 fail2ban/tests/files/action.d/action_nomethod.py
-fail2ban/tests/files/config/apache-auth/README
+fail2ban/tests/files/action.d/action.py
+fail2ban/tests/files/config/apache-auth/basic/authz_owner/cant_get_me.html
 fail2ban/tests/files/config/apache-auth/basic/authz_owner/.htaccess
 fail2ban/tests/files/config/apache-auth/basic/authz_owner/.htpasswd
-fail2ban/tests/files/config/apache-auth/basic/authz_owner/cant_get_me.html
 fail2ban/tests/files/config/apache-auth/basic/file/.htaccess
 fail2ban/tests/files/config/apache-auth/basic/file/.htpasswd
-fail2ban/tests/files/config/apache-auth/digest.py
-fail2ban/tests/files/config/apache-auth/digest/.htaccess
-fail2ban/tests/files/config/apache-auth/digest/.htpasswd
 fail2ban/tests/files/config/apache-auth/digest_anon/.htaccess
 fail2ban/tests/files/config/apache-auth/digest_anon/.htpasswd
+fail2ban/tests/files/config/apache-auth/digest/.htaccess
+fail2ban/tests/files/config/apache-auth/digest/.htpasswd
+fail2ban/tests/files/config/apache-auth/digest.py
 fail2ban/tests/files/config/apache-auth/digest_time/.htaccess
 fail2ban/tests/files/config/apache-auth/digest_time/.htpasswd
 fail2ban/tests/files/config/apache-auth/digest_wrongrelm/.htaccess
 fail2ban/tests/files/config/apache-auth/digest_wrongrelm/.htpasswd
 fail2ban/tests/files/config/apache-auth/noentry/.htaccess
+fail2ban/tests/files/config/apache-auth/README
 fail2ban/tests/files/database_v1.db
 fail2ban/tests/files/filter.d/substition.conf
-fail2ban/tests/files/filter.d/testcase-common.conf
 fail2ban/tests/files/filter.d/testcase01.conf
+fail2ban/tests/files/filter.d/testcase-common.conf
 fail2ban/tests/files/ignorecommand.py
 fail2ban/tests/files/logs/3proxy
 fail2ban/tests/files/logs/apache-auth
@@ -256,6 +269,7 @@ fail2ban/tests/files/logs/apache-modsecurity
 fail2ban/tests/files/logs/apache-nohome
 fail2ban/tests/files/logs/apache-noscript
 fail2ban/tests/files/logs/apache-overflows
+fail2ban/tests/files/logs/apache-pass
 fail2ban/tests/files/logs/apache-shellshock
 fail2ban/tests/files/logs/assp
 fail2ban/tests/files/logs/asterisk
@@ -267,25 +281,33 @@ fail2ban/tests/files/logs/courier-auth
 fail2ban/tests/files/logs/courier-smtp
 fail2ban/tests/files/logs/cyrus-imap
 fail2ban/tests/files/logs/directadmin
+fail2ban/tests/files/logs/domino-smtp
 fail2ban/tests/files/logs/dovecot
 fail2ban/tests/files/logs/dropbear
+fail2ban/tests/files/logs/drupal-auth
 fail2ban/tests/files/logs/ejabberd-auth
 fail2ban/tests/files/logs/exim
 fail2ban/tests/files/logs/exim-spam
 fail2ban/tests/files/logs/freeswitch
+fail2ban/tests/files/logs/froxlor-auth
 fail2ban/tests/files/logs/groupoffice
 fail2ban/tests/files/logs/gssftpd
 fail2ban/tests/files/logs/guacamole
+fail2ban/tests/files/logs/haproxy-http-auth
 fail2ban/tests/files/logs/horde
 fail2ban/tests/files/logs/kerio
 fail2ban/tests/files/logs/lighttpd-auth
+fail2ban/tests/files/logs/mongodb-auth
 fail2ban/tests/files/logs/monit
+fail2ban/tests/files/logs/murmur
 fail2ban/tests/files/logs/mysqld-auth
 fail2ban/tests/files/logs/nagios
 fail2ban/tests/files/logs/named-refused
 fail2ban/tests/files/logs/nginx-botsearch
 fail2ban/tests/files/logs/nginx-http-auth
+fail2ban/tests/files/logs/nginx-limit-req
 fail2ban/tests/files/logs/nsd
+fail2ban/tests/files/logs/openhab
 fail2ban/tests/files/logs/openwebmail
 fail2ban/tests/files/logs/oracleims
 fail2ban/tests/files/logs/pam-generic
@@ -300,16 +322,19 @@ fail2ban/tests/files/logs/pure-ftpd
 fail2ban/tests/files/logs/qmail
 fail2ban/tests/files/logs/recidive
 fail2ban/tests/files/logs/roundcube-auth
+fail2ban/tests/files/logs/screensharingd
 fail2ban/tests/files/logs/selinux-ssh
 fail2ban/tests/files/logs/sendmail-auth
 fail2ban/tests/files/logs/sendmail-reject
 fail2ban/tests/files/logs/sendmail-spam
 fail2ban/tests/files/logs/sieve
+fail2ban/tests/files/logs/slapd
 fail2ban/tests/files/logs/sogo-auth
 fail2ban/tests/files/logs/solid-pop3d
 fail2ban/tests/files/logs/squid
 fail2ban/tests/files/logs/squirrelmail
 fail2ban/tests/files/logs/sshd
+fail2ban/tests/files/logs/sshd-aggressive
 fail2ban/tests/files/logs/sshd-ddos
 fail2ban/tests/files/logs/stunnel
 fail2ban/tests/files/logs/suhosin
@@ -319,14 +344,17 @@ fail2ban/tests/files/logs/vsftpd
 fail2ban/tests/files/logs/webmin-auth
 fail2ban/tests/files/logs/wuftpd
 fail2ban/tests/files/logs/xinetd-fail
-fail2ban/tests/files/testcase-journal.log
-fail2ban/tests/files/testcase-multiline.log
-fail2ban/tests/files/testcase-usedns.log
+fail2ban/tests/files/logs/zzz-generic-example
 fail2ban/tests/files/testcase01.log
 fail2ban/tests/files/testcase02.log
 fail2ban/tests/files/testcase03.log
 fail2ban/tests/files/testcase04.log
+fail2ban/tests/files/testcase-journal.log
+fail2ban/tests/files/testcase-multiline.log
+fail2ban/tests/files/testcase-usedns.log
+fail2ban/tests/files/testcase-wrong-char.log
 fail2ban/tests/filtertestcase.py
+fail2ban/tests/__init__.py
 fail2ban/tests/misctestcase.py
 fail2ban/tests/samplestestcase.py
 fail2ban/tests/servertestcase.py
@@ -334,36 +362,48 @@ fail2ban/tests/sockettestcase.py
 fail2ban/tests/utils.py
 fail2ban/version.py
 files/bash-completion
-files/cacti/README
 files/cacti/cacti_host_template_fail2ban.xml
 files/cacti/fail2ban_stats.sh
+files/cacti/README
 files/debian-initd
 files/fail2ban-logrotate
-files/fail2ban-tmpfiles.conf
 files/fail2ban.service
+files/fail2ban-tmpfiles.conf
 files/fail2ban.upstart
 files/gen_badbots
 files/gentoo-confd
 files/gentoo-initd
 files/ipmasq-ZZZzzz_fail2ban.rul
 files/logwatch/fail2ban
+files/logwatch/fail2ban-0.8.log
+files/logwatch/fail2ban-0.9.log
 files/macosx-initd
 files/monit/fail2ban
-files/nagios/README
 files/nagios/check_fail2ban
+files/nagios/README
 files/redhat-initd
 files/solaris-fail2ban.xml
 files/solaris-svc-fail2ban
 files/suse-initd
+FILTERS
 kill-server
+man/fail2ban.1
 man/fail2ban-client.1
 man/fail2ban-client.h2m
 man/fail2ban-regex.1
 man/fail2ban-regex.h2m
 man/fail2ban-server.1
 man/fail2ban-server.h2m
-man/fail2ban.1
+man/fail2ban-testcases.1
+man/fail2ban-testcases.h2m
 man/generate-man
 man/jail.conf.5
+.pylintrc
+README.md
+README.Solaris
+RELEASE
 setup.cfg
 setup.py
+THANKS
+TODO
+Vagrantfile

README.md

@@ -2,47 +2,51 @@
                         / _|__ _(_) |_  ) |__  __ _ _ _  
                        |  _/ _` | | |/ /| '_ \/ _` | ' \ 
                        |_| \__,_|_|_/___|_.__/\__,_|_||_|
-                       v0.9.3                  2015/08/01
+                       v0.9.7.dev0             2017/??/??
 
 ## Fail2Ban: ban hosts that cause multiple authentication errors
 
-Fail2Ban scans log files like /var/log/pwdfail and bans IP that makes too many
-password failures. It updates firewall rules to reject the IP address. These
-rules can be defined by the user. Fail2Ban can read multiple log files such as
-sshd or Apache web server ones.
+Fail2Ban scans log files like `/var/log/auth.log` and bans IP addresses conducting
+too many failed login attempts. It does this by updating system firewall rules
+to reject new connections from those IP addresses, for a configurable amount
+of time. Fail2Ban comes out-of-the-box ready to read many standard log files,
+such as those for sshd and Apache, and is easily configured to read any log
+file of your choosing, for any error you wish.
 
-Fail2Ban is able to reduce the rate of incorrect authentications attempts
-however it cannot eliminate the risk that weak authentication presents.
-Configure services to use only two factor or public/private authentication
+Though Fail2Ban is able to reduce the rate of incorrect authentication
+attempts, it cannot eliminate the risk presented by weak authentication.
+Set up services to use only two factor, or public/private authentication
 mechanisms if you really want to protect services.
 
-This README is a quick introduction to Fail2ban. More documentation, FAQ, HOWTOs
-are available in fail2ban(1) manpage and on the website http://www.fail2ban.org
+This README is a quick introduction to Fail2Ban. More documentation, FAQ, and HOWTOs
+to be found on fail2ban(1) manpage and the website: https://www.fail2ban.org
 
 Installation:
 -------------
 
-**It is possible that Fail2ban is already packaged for your distribution.  In
-this case, you should use it instead.**
+**It is possible that Fail2Ban is already packaged for your distribution.  In
+this case, you should use that instead.**
 
 Required:
-- [Python2 >= 2.6 or Python >= 3.2](http://www.python.org) or [PyPy](http://pypy.org)
+- [Python2 >= 2.6 or Python >= 3.2](https://www.python.org) or [PyPy](https://pypy.org)
 
 Optional:
-- [pyinotify >= 0.8.3](https://github.com/seb-m/pyinotify)
-  - Linux >= 2.6.13
+- [pyinotify >= 0.8.3](https://github.com/seb-m/pyinotify), may require:
+  * Linux >= 2.6.13
 - [gamin >= 0.0.21](http://www.gnome.org/~veillard/gamin)
-- [systemd >= 204](http://www.freedesktop.org/wiki/Software/systemd)
+- [systemd >= 204](http://www.freedesktop.org/wiki/Software/systemd) and python bindings:
+  * [python-systemd package](https://www.freedesktop.org/software/systemd/python-systemd/index.html)
 - [dnspython](http://www.dnspython.org/)
 
-To install, just do:
 
-    tar xvfj fail2ban-0.9.3.tar.bz2
-    cd fail2ban-0.9.3
+To install:
+
+    tar xvfj fail2ban-0.9.7.tar.bz2
+    cd fail2ban-0.9.7
     python setup.py install
 
 This will install Fail2Ban into the python library directory. The executable
-scripts are placed into /usr/bin, and configuration under /etc/fail2ban.
+scripts are placed into `/usr/bin`, and configuration in `/etc/fail2ban`.
 
 Fail2Ban should be correctly installed now. Just type:
 
@@ -51,11 +55,20 @@ Fail2Ban should be correctly installed now. Just type:
 to see if everything is alright. You should always use fail2ban-client and
 never call fail2ban-server directly.
 
+Please note that the system init/service script is not automatically installed.
+To enable fail2ban as an automatic service, simply copy the script for your
+distro from the `files` directory to `/etc/init.d`. Example (on a Debian-based
+system):
+
+    cp files/debian-initd /etc/init.d/fail2ban
+    update-rc.d fail2ban defaults
+    service fail2ban start
+
 Configuration:
 --------------
 
-You can configure Fail2Ban using the files in /etc/fail2ban. It is possible to
-configure the server using commands sent to it by fail2ban-client. The
+You can configure Fail2Ban using the files in `/etc/fail2ban`. It is possible to
+configure the server using commands sent to it by `fail2ban-client`. The
 available commands are described in the fail2ban-client(1) manpage.  Also see
 fail2ban(1) and jail.conf(5)  manpages for further references.
 
@@ -66,6 +79,8 @@ Code status:
 
 * [![Coverage Status](https://coveralls.io/repos/fail2ban/fail2ban/badge.png?branch=master)](https://coveralls.io/r/fail2ban/fail2ban)
 
+* [![codecov.io](https://codecov.io/github/fail2ban/fail2ban/coverage.svg?branch=master)](https://codecov.io/github/fail2ban/fail2ban?branch=master)
+
 Contact:
 --------
 
@@ -73,7 +88,7 @@ Contact:
 See [CONTRIBUTING.md](https://github.com/fail2ban/fail2ban/blob/master/CONTRIBUTING.md)
 
 ### You just appreciate this program:
-send kudos to the original author ([Cyril Jaquier](mailto: Cyril Jaquier <cyril.jaquier@fail2ban.org>))
+Send kudos to the original author ([Cyril Jaquier](mailto: Cyril Jaquier <cyril.jaquier@fail2ban.org>))
 or *better* to the [mailing list](https://lists.sourceforge.net/lists/listinfo/fail2ban-users)
 since Fail2Ban is "community-driven" for years now.
 

RELEASE

@@ -51,6 +51,11 @@ Preparation
 
      find -type f | grep -v -e '\.git' -e '/doc/' -e '\.travis' -e MANIFEST | sed -e 's,^\./,,g' | while read f; do grep -ne "^$f\$" MANIFEST >/dev/null || echo "$f" ; done
 
+  or an alternative for comparison with previous release
+
+     git diff 0.9.7 | grep -B2 'index 0000000..' | grep -B1 'new file mode' | sed -n -e '/^diff /s,.* b/,,gp' >> MANIFEST
+     sort MANIFEST | uniq | sponge MANIFEST
+
 * Run::
 
   python setup.py sdist
@@ -61,24 +66,24 @@ Preparation
 
   * Which indicates that testcases/files/logs/mysqld.log has been moved or is a directory::
 
-      tar -C /tmp -jxf dist/fail2ban-0.9.3.tar.bz2
+      tar -C /tmp -jxf dist/fail2ban-0.9.7.tar.bz2
 
-* clean up current direcory::
+* clean up current directory::
 
-    diff -rul --exclude \*.pyc . /tmp/fail2ban-0.9.3/
+    diff -rul --exclude \*.pyc . /tmp/fail2ban-0.9.7/
 
   * Only differences should be files that you don't want distributed.
 
 
 * Ensure the tests work from the tarball::
 
-    cd /tmp/fail2ban-0.9.3/ && bin/fail2ban-testcases
+    cd /tmp/fail2ban-0.9.7/ && bin/fail2ban-testcases
 
 * Add/finalize the corresponding entry in the ChangeLog
 
   * To generate a list of committers use e.g.::
 
-      git shortlog -sn 0.9.3.. | sed -e 's,^[ 0-9\t]*,,g' | tr '\n' '\|' | sed -e 's:|:, :g'
+      git shortlog -sn 0.9.7.. | sed -e 's,^[ 0-9\t]*,,g' | tr '\n' '\|' | sed -e 's:|:, :g'
 
   * Ensure the top of the ChangeLog has the right version and current date.
   * Ensure the top entry of the ChangeLog has the right version and current date.
@@ -101,7 +106,7 @@ Preparation
 * Tag the release by using a signed (and annotated) tag.  Cut/paste
   release ChangeLog entry as tag annotation::
 
-    git tag -s 0.9.3
+    git tag -s 0.9.8
 
 Pre Release
 ===========
@@ -116,7 +121,7 @@ Pre Release
 
     * http://packages.qa.debian.org/f/fail2ban.html
 
-  * FreeBSD: Christoph Theis theis@gmx.at>, Nick Hilliard <nick@foobar.org>
+  * FreeBSD: Christoph Theis theis@gmx.at>
 
     * http://svnweb.freebsd.org/ports/head/security/py-fail2ban/Makefile?view=markup
     * http://www.freebsd.org/cgi/query-pr-summary.cgi?text=fail2ban
@@ -185,14 +190,14 @@ Post Release
 
 Add the following to the top of the ChangeLog::
 
-  ver. 0.9.4 (2014/XX/XXX) - wanna-be-released
+  ver. 0.9.9 (2016/XX/XXX) - wanna-be-released
   -----------
   
-  - Fixes:
+  ### Fixes
   
-  - New Features:
+  ### New Features
   
-  - Enhancements:
+  ### Enhancements
 
 Alter the git shortlog command in the previous section to refer to the just
 released version.

THANKS

@@ -12,6 +12,7 @@ Adrien Clerc
 ache
 ag4ve (Shawn)
 Alasdair D. Campbell
+Alexandre Perrin (kAworu)
 Amir Caspi
 Amy
 Andrew St. Jean
@@ -34,12 +35,14 @@ Daniel B. Cid
 Daniel B.
 Daniel Black
 David Nutter
+David Reagan (jerrac)
 Derek Atkins
 Donald Yandt
 Eric Gerbier
 Enrico Labedzki
 Eugene Hopkinson (SlowRiot)
 ftoppi
+Florian Robert (1technophile)
 François Boulogne
 Frantisek Sumsal
 Frédéric
@@ -65,12 +68,14 @@ Joël Bertrand
 JP Espinosa
 jserrachinha
 Justin Shore
+Kevin Locke
 Kévin Drapel
 kjohnsonecl
 kojiro
 Lars Kneschke
 Lee Clemens
 leftyfb (Mike Rushton)
+M. Maraun
 Manuel Arostegui Ramirez
 Marcel Dopita
 Mark Edgington
@@ -87,6 +92,7 @@ Mika (mkl)
 Nick Munger
 onorua
 Orion Poplawski
+Pablo Rodriguez Fernandez
 Paul Marrapese
 Paul Traina
 Noel Butler
@@ -111,7 +117,9 @@ Steven Hiscocks
 TESTOVIK
 Thomas Mayer
 Tom Pike
+Tom Hendrikx
 Tomas Pihl
+Thomas Skierlo (phaleas)
 Tony Lawrence
 Tomasz Ciolek
 Tyler

bin/fail2ban-client

@@ -176,7 +176,7 @@ class Fail2banClient:
 					if showRet:
 						self.__logSocketError()
 					return False
-				except Exception, e:
+				except Exception as e:
 					if showRet:
 						logSys.error(e)
 					return False
@@ -376,8 +376,10 @@ class Fail2banClient:
 			logSys.setLevel(logging.WARNING)
 		elif verbose == 2:
 			logSys.setLevel(logging.INFO)
-		else:
+		elif verbose == 3:
 			logSys.setLevel(logging.DEBUG)
+		else:
+			logSys.setLevel(logging.HEAVYDEBUG)
 		# Add the default logging handler to dump to stderr
 		logout = logging.StreamHandler(sys.stderr)
 		# set a format which is simpler for console use
@@ -427,7 +429,7 @@ class Fail2banClient:
 						elif not cmd == "":
 							try:
 								self.__processCommand(shlex.split(cmd))
-							except Exception, e:
+							except Exception as e:
 								logSys.error(e)
 			except (EOFError, KeyboardInterrupt):
 				print
@@ -449,7 +451,7 @@ class Fail2banClient:
 			ret = self.__configurator.getOptions(jail)
 			self.__configurator.convertToProtocol()
 			self.__stream = self.__configurator.getConfigStream()
-		except Exception, e:
+		except Exception as e:
 			logSys.error("Failed during configuration: %s" % e)
 			ret = False
 		return ret

bin/fail2ban-regex

@@ -29,563 +29,6 @@ __author__ = "Fail2Ban Developers"
 __copyright__ = "Copyright (c) 2004-2008 Cyril Jaquier, 2012-2014 Yaroslav Halchenko"
 __license__ = "GPL"
 
-import getopt
-import locale
-import logging
-import os
-import shlex
-import sys
-import time
-import time
-import urllib
-from optparse import OptionParser, Option
+from fail2ban.client.fail2banregex import exec_command_line
 
-from ConfigParser import NoOptionError, NoSectionError, MissingSectionHeaderError
-
-try:
-	from systemd import journal
-	from fail2ban.server.filtersystemd import FilterSystemd
-except ImportError:
-	journal = None
-
-from fail2ban.version import version
-from fail2ban.client.filterreader import FilterReader
-from fail2ban.server.filter import Filter
-from fail2ban.server.failregex import RegexException
-
-from fail2ban.helpers import FormatterWithTraceBack, getLogger
-# Gets the instance of the logger.
-logSys = getLogger("fail2ban")
-
-def debuggexURL(sample, regex):
-	q = urllib.urlencode({ 're': regex.replace('<HOST>', '(?&.ipv4)'),
-							'str': sample,
-							'flavor': 'python' })
-	return 'http://www.debuggex.com/?' + q
-
-def shortstr(s, l=53):
-	"""Return shortened string
-	"""
-	if len(s) > l:
-		return s[:l-3] + '...'
-	return s
-
-def pprint_list(l, header=None):
-	if not len(l):
-		return
-	if header:
-		s = "|- %s\n" % header
-	else:
-		s = ''
-	print s + "|  " + "\n|  ".join(l) + '\n`-'
-
-def file_lines_gen(hdlr):
-	for line in hdlr:
-		try:
-			line = line.decode(fail2banRegex.encoding, 'strict')
-		except UnicodeDecodeError:
-			if sys.version_info >= (3,): # Python 3 must be decoded
-				line = line.decode(fail2banRegex.encoding, 'ignore')
-		yield line
-
-def journal_lines_gen(myjournal):
-	while True:
-		try:
-			entry = myjournal.get_next()
-		except OSError:
-			continue
-		if not entry:
-			break
-		yield FilterSystemd.formatJournalEntry(entry)
-
-def get_opt_parser():
-	# use module docstring for help output
-	p = OptionParser(
-				usage="%s [OPTIONS] <LOG> <REGEX> [IGNOREREGEX]\n" % sys.argv[0] + __doc__
-				+ """
-LOG:
-    string                  a string representing a log line
-    filename                path to a log file (/var/log/auth.log)
-    "systemd-journal"       search systemd journal (systemd-python required)
-
-REGEX:
-    string                  a string representing a 'failregex'
-    filename                path to a filter file (filter.d/sshd.conf)
-
-IGNOREREGEX:
-    string                  a string representing an 'ignoreregex'
-    filename                path to a filter file (filter.d/sshd.conf)
-
-Copyright (c) 2004-2008 Cyril Jaquier, 2008- Fail2Ban Contributors
-Copyright of modifications held by their respective authors.
-Licensed under the GNU General Public License v2 (GPL).
-
-Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>.
-Many contributions by Yaroslav O. Halchenko and Steven Hiscocks.
-
-Report bugs to https://github.com/fail2ban/fail2ban/issues
-""",
-				version="%prog " + version)
-
-	p.add_options([
-		Option("-d", "--datepattern",
-			   help="set custom pattern used to match date/times"),
-		Option("-e", "--encoding",
-			   help="File encoding. Default: system locale"),
-		Option("-L", "--maxlines", type=int, default=0,
-			   help="maxlines for multi-line regex"),
-		Option("-m", "--journalmatch",
-			   help="journalctl style matches overriding filter file. "
-			   "\"systemd-journal\" only"),
-		Option('-l', "--log-level", type="choice",
-			   dest="log_level",
-			   choices=('heavydebug', 'debug', 'info', 'notice', 'warning', 'error', 'critical'),
-			   default=None,
-			   help="Log level for the Fail2Ban logger to use"),
-		Option("-v", "--verbose", action='store_true',
-			   help="Be verbose in output"),
-		Option("-D", "--debuggex", action='store_true',
-			   help="Produce debuggex.com urls for debugging there"),
-		Option("--print-no-missed", action='store_true',
-			   help="Do not print any missed lines"),
-		Option("--print-no-ignored", action='store_true',
-			   help="Do not print any ignored lines"),
-		Option("--print-all-matched", action='store_true',
-			   help="Print all matched lines"),
-		Option("--print-all-missed", action='store_true',
-			   help="Print all missed lines, no matter how many"),
-		Option("--print-all-ignored", action='store_true',
-			   help="Print all ignored lines, no matter how many"),
-		Option("-t", "--log-traceback", action='store_true',
-			   help="Enrich log-messages with compressed tracebacks"),
-		Option("--full-traceback", action='store_true',
-			   help="Either to make the tracebacks full, not compressed (as by default)"),
-		])
-
-	return p
-
-
-class RegexStat(object):
-
-	def __init__(self, failregex):
-		self._stats = 0
-		self._failregex = failregex
-		self._ipList = list()
-
-	def __str__(self):
-		return "%s(%r) %d failed: %s" \
-		  % (self.__class__, self._failregex, self._stats, self._ipList)
-
-	def inc(self):
-		self._stats += 1
-
-	def getStats(self):
-		return self._stats
-
-	def getFailRegex(self):
-		return self._failregex
-
-	def appendIP(self, value):
-		self._ipList.append(value)
-
-	def getIPList(self):
-		return self._ipList
-
-
-class LineStats(object):
-	"""Just a convenience container for stats
-	"""
-	def __init__(self):
-		self.tested = self.matched = 0
-		self.matched_lines = []
-		self.missed = 0
-		self.missed_lines = []
-		self.missed_lines_timeextracted = []
-		self.ignored = 0
-		self.ignored_lines = []
-		self.ignored_lines_timeextracted = []
-
-	def __str__(self):
-		return "%(tested)d lines, %(ignored)d ignored, %(matched)d matched, %(missed)d missed" % self
-
-	# just for convenient str
-	def __getitem__(self, key):
-		return getattr(self, key)
-
-
-class Fail2banRegex(object):
-
-	def __init__(self, opts):
-		self._verbose = opts.verbose
-		self._debuggex = opts.debuggex
-		self._maxlines = 20
-		self._print_no_missed = opts.print_no_missed
-		self._print_no_ignored = opts.print_no_ignored
-		self._print_all_matched = opts.print_all_matched
-		self._print_all_missed = opts.print_all_missed
-		self._print_all_ignored = opts.print_all_ignored
-		self._maxlines_set = False		  # so we allow to override maxlines in cmdline
-		self._datepattern_set = False
-		self._journalmatch = None
-
-		self.share_config=dict()
-		self._filter = Filter(None)
-		self._ignoreregex = list()
-		self._failregex = list()
-		self._time_elapsed = None
-		self._line_stats = LineStats()
-
-		if opts.maxlines:
-			self.setMaxLines(opts.maxlines)
-		if opts.journalmatch is not None:
-			self.setJournalMatch(opts.journalmatch.split())
-		if opts.datepattern:
-			self.setDatePattern(opts.datepattern)
-		if opts.encoding:
-			self.encoding = opts.encoding
-		else:
-			self.encoding = locale.getpreferredencoding()
-
-
-
-	def setDatePattern(self, pattern):
-		if not self._datepattern_set:
-			self._filter.setDatePattern(pattern)
-			self._datepattern_set = True
-			if pattern is not None:
-				print "Use      datepattern : %s" % (
-					self._filter.getDatePattern()[1], )
-
-	def setMaxLines(self, v):
-		if not self._maxlines_set:
-			self._filter.setMaxLines(int(v))
-			self._maxlines_set = True
-			print "Use         maxlines : %d" % self._filter.getMaxLines()
-
-	def setJournalMatch(self, v):
-		if self._journalmatch is None:
-			self._journalmatch = v
-
-	def readRegex(self, value, regextype):
-		assert(regextype in ('fail', 'ignore'))
-		regex = regextype + 'regex'
-		if os.path.isfile(value) or os.path.isfile(value + '.conf'):
-			if os.path.basename(os.path.dirname(value)) == 'filter.d':
-				## within filter.d folder - use standard loading algorithm to load filter completely (with .local etc.):
-				basedir = os.path.dirname(os.path.dirname(value))
-				value = os.path.splitext(os.path.basename(value))[0]
-				print "Use %11s filter file : %s, basedir: %s" % (regex, value, basedir)
-				reader = FilterReader(value, 'fail2ban-regex-jail', {}, share_config=self.share_config, basedir=basedir)
-				if not reader.read():
-					print "ERROR: failed to load filter %s" % value
-					return False
-			else:
-				## foreign file - readexplicit this file and includes if possible:
-				print "Use %11s file : %s" % (regex, value)
-				reader = FilterReader(value, 'fail2ban-regex-jail', {}, share_config=self.share_config)
-				reader.setBaseDir(None)
-				if not reader.readexplicit():
-					print "ERROR: failed to read %s" % value
-					return False
-			reader.getOptions(None)
-			readercommands = reader.convert()
-			regex_values = [
-				RegexStat(m[3])
-				for m in filter(
-					lambda x: x[0] == 'set' and x[2] == "add%sregex" % regextype,
-					readercommands)]
-			# Read out and set possible value of maxlines
-			for command in readercommands:
-				if command[2] == "maxlines":
-					maxlines = int(command[3])
-					try:
-						self.setMaxLines(maxlines)
-					except ValueError:
-						print "ERROR: Invalid value for maxlines (%(maxlines)r) " \
-							  "read from %(value)s" % locals()
-						return False
-				elif command[2] == 'addjournalmatch':
-					journalmatch = command[3:]
-					self.setJournalMatch(journalmatch)
-				elif command[2] == 'datepattern':
-					datepattern = command[3]
-					self.setDatePattern(datepattern)
-		else:
-			print "Use %11s line : %s" % (regex, shortstr(value))
-			regex_values = [RegexStat(value)]
-
-		setattr(self, "_" + regex, regex_values)
-		for regex in regex_values:
-			getattr(
-				self._filter,
-				'add%sRegex' % regextype.title())(regex.getFailRegex())
-		return True
-
-	def testIgnoreRegex(self, line):
-		found = False
-		try:
-			ret = self._filter.ignoreLine([(line, "", "")])
-			if ret is not None:
-				found = True
-				regex = self._ignoreregex[ret].inc()
-		except RegexException, e:
-			print e
-			return False
-		return found
-
-	def testRegex(self, line, date=None):
-		orgLineBuffer = self._filter._Filter__lineBuffer
-		fullBuffer = len(orgLineBuffer) >= self._filter.getMaxLines()
-		try:
-			line, ret = self._filter.processLine(line, date, checkAllRegex=True)
-			for match in ret:
-				# Append True/False flag depending if line was matched by
-				# more than one regex
-				match.append(len(ret)>1)
-				regex = self._failregex[match[0]]
-				regex.inc()
-				regex.appendIP(match)
-		except RegexException, e:
-			print e
-			return False
-		except IndexError:
-			print "Sorry, but no <HOST> found in regex"
-			return False
-		for bufLine in orgLineBuffer[int(fullBuffer):]:
-			if bufLine not in self._filter._Filter__lineBuffer:
-				try:
-					self._line_stats.missed_lines.pop(
-						self._line_stats.missed_lines.index("".join(bufLine)))
-					self._line_stats.missed_lines_timeextracted.pop(
-						self._line_stats.missed_lines_timeextracted.index(
-							"".join(bufLine[::2])))
-				except ValueError:
-					pass
-				else:
-					self._line_stats.matched += 1
-					self._line_stats.missed -= 1
-		return line, ret
-
-	def process(self, test_lines):
-		t0 = time.time()
-		for line_no, line in enumerate(test_lines):
-			if isinstance(line, tuple):
-				line_datetimestripped, ret = fail2banRegex.testRegex(
-					line[0], line[1])
-				line = "".join(line[0])
-			else:
-				line = line.rstrip('\r\n')
-				if line.startswith('#') or not line:
-					# skip comment and empty lines
-					continue
-				line_datetimestripped, ret = fail2banRegex.testRegex(line)
-			is_ignored = fail2banRegex.testIgnoreRegex(line_datetimestripped)
-
-			if is_ignored:
-				self._line_stats.ignored += 1
-				if not self._print_no_ignored and (self._print_all_ignored or self._line_stats.ignored <= self._maxlines + 1):
-					self._line_stats.ignored_lines.append(line)
-					self._line_stats.ignored_lines_timeextracted.append(line_datetimestripped)
-
-			if len(ret) > 0:
-				assert(not is_ignored)
-				self._line_stats.matched += 1
-				if self._print_all_matched:
-					self._line_stats.matched_lines.append(line)
-			else:
-				if not is_ignored:
-					self._line_stats.missed += 1
-					if not self._print_no_missed and (self._print_all_missed or self._line_stats.missed <= self._maxlines + 1):
-						self._line_stats.missed_lines.append(line)
-						self._line_stats.missed_lines_timeextracted.append(line_datetimestripped)
-			self._line_stats.tested += 1
-
-			if line_no % 10 == 0 and self._filter.dateDetector is not None:
-				self._filter.dateDetector.sortTemplate()
-		self._time_elapsed = time.time() - t0
-
-
-
-	def printLines(self, ltype):
-		lstats = self._line_stats
-		assert(self._line_stats.missed == lstats.tested - (lstats.matched + lstats.ignored))
-		lines = lstats[ltype]
-		l = lstats[ltype + '_lines']
-		if lines:
-			header = "%s line(s):" % (ltype.capitalize(),)
-			if self._debuggex:
-				if ltype == 'missed' or ltype == 'matched':
-					regexlist = self._failregex
-				else:
-					regexlist = self._ignoreregex
-				l = lstats[ltype + '_lines_timeextracted']
-				if lines < self._maxlines or getattr(self, '_print_all_' + ltype):
-					ans = [[]]
-					for arg in [l, regexlist]:
-						ans = [ x + [y] for x in ans for y in arg ]
-					b = map(lambda a: a[0] +  ' | ' + a[1].getFailRegex() + ' |  ' + debuggexURL(a[0], a[1].getFailRegex()), ans)
-					pprint_list([x.rstrip() for x in b], header)
-				else:
-					print "%s too many to print.  Use --print-all-%s " \
-						  "to print all %d lines" % (header, ltype, lines)
-			elif lines < self._maxlines or getattr(self, '_print_all_' + ltype):
-				pprint_list([x.rstrip() for x in l], header)
-			else:
-				print "%s too many to print.  Use --print-all-%s " \
-					  "to print all %d lines" % (header, ltype, lines)
-
-	def printStats(self):
-		print
-		print "Results"
-		print "======="
-
-		def print_failregexes(title, failregexes):
-			# Print title
-			total, out = 0, []
-			for cnt, failregex in enumerate(failregexes):
-				match = failregex.getStats()
-				total += match
-				if (match or self._verbose):
-					out.append("%2d) [%d] %s" % (cnt+1, match, failregex.getFailRegex()))
-
-				if self._verbose and len(failregex.getIPList()):
-					for ip in failregex.getIPList():
-						timeTuple = time.localtime(ip[2])
-						timeString = time.strftime("%a %b %d %H:%M:%S %Y", timeTuple)
-						out.append(
-							"    %s  %s%s" % (
-								ip[1],
-								timeString,
-								ip[-1] and " (multiple regex matched)" or ""))
-
-			print "\n%s: %d total" % (title, total)
-			pprint_list(out, " #) [# of hits] regular expression")
-			return total
-
-		# Print title
-		total = print_failregexes("Failregex", self._failregex)
-		_ = print_failregexes("Ignoreregex", self._ignoreregex)
-
-
-		if self._filter.dateDetector is not None:
-			print "\nDate template hits:"
-			out = []
-			for template in self._filter.dateDetector.templates:
-				if self._verbose or template.hits:
-					out.append("[%d] %s" % (
-						template.hits, template.name))
-			pprint_list(out, "[# of hits] date format")
-
-		print "\nLines: %s" % self._line_stats,
-		if self._time_elapsed is not None:
-			print "[processed in %.2f sec]" % self._time_elapsed,
-		print
-
-		if self._print_all_matched:
-			self.printLines('matched')
-		if not self._print_no_ignored:
-			self.printLines('ignored')
-		if not self._print_no_missed:
-			self.printLines('missed')
-
-		return True
-
-
-if __name__ == "__main__":
-
-	parser = get_opt_parser()
-	(opts, args) = parser.parse_args()
-	if opts.print_no_missed and opts.print_all_missed:
-		sys.stderr.write("ERROR: --print-no-missed and --print-all-missed are mutually exclusive.\n\n")
-		parser.print_help()
-		sys.exit(-1)
-	if opts.print_no_ignored and opts.print_all_ignored:
-		sys.stderr.write("ERROR: --print-no-ignored and --print-all-ignored are mutually exclusive.\n\n")
-		parser.print_help()
-		sys.exit(-1)
-
-	print
-	print "Running tests"
-	print "============="
-	print
-
-	fail2banRegex = Fail2banRegex(opts)
-
-	# We need 2 or 3 parameters
-	if not len(args) in (2, 3):
-		sys.stderr.write("ERROR: provide both <LOG> and <REGEX>.\n\n")
-		parser.print_help()
-		sys.exit(-1)
-
-	# TODO: taken from -testcases -- move common functionality somewhere
-	if opts.log_level is not None: # pragma: no cover
-		# so we had explicit settings
-		logSys.setLevel(getattr(logging, opts.log_level.upper()))
-	else: # pragma: no cover
-		# suppress the logging but it would leave unittests' progress dots
-		# ticking, unless like with '-l critical' which would be silent
-		# unless error occurs
-		logSys.setLevel(getattr(logging, 'CRITICAL'))
-
-	# Add the default logging handler
-	stdout = logging.StreamHandler(sys.stdout)
-
-	fmt = 'D: %(message)s'
-
-	if opts.log_traceback:
-		Formatter = FormatterWithTraceBack
-		fmt = (opts.full_traceback and ' %(tb)s' or ' %(tbc)s') + fmt
-	else:
-		Formatter = logging.Formatter
-
-	# Custom log format for the verbose tests runs
-	if opts.verbose: # pragma: no cover
-		stdout.setFormatter(Formatter(' %(asctime)-15s %(thread)s' + fmt))
-	else: # pragma: no cover
-		# just prefix with the space
-		stdout.setFormatter(Formatter(fmt))
-	logSys.addHandler(stdout)
-
-	cmd_log, cmd_regex = args[:2]
-
-	fail2banRegex.readRegex(cmd_regex, 'fail') or sys.exit(-1)
-
-	if len(args) == 3:
-		fail2banRegex.readRegex(args[2], 'ignore') or sys.exit(-1)
-
-	if os.path.isfile(cmd_log):
-		try:
-			hdlr = open(cmd_log, 'rb')
-			print "Use         log file : %s" % cmd_log
-			print "Use         encoding : %s" % fail2banRegex.encoding
-			test_lines = file_lines_gen(hdlr)
-		except IOError, e:
-			print e
-			sys.exit(-1)
-	elif cmd_log == "systemd-journal":
-		if not journal:
-			print "Error: systemd library not found. Exiting..."
-			sys.exit(-1)
-		myjournal = journal.Reader(converters={'__CURSOR': lambda x: x})
-		journalmatch = fail2banRegex._journalmatch
-		fail2banRegex.setDatePattern(None)
-		if journalmatch:
-			try:
-				for element in journalmatch:
-					if element == "+":
-						myjournal.add_disjunction()
-					else:
-						myjournal.add_match(element)
-			except ValueError:
-				print "Error: Invalid journalmatch: %s" % shortstr(" ".join(journalmatch))
-				sys.exit(-1)
-		print "Use    journal match : %s" % " ".join(journalmatch)
-		test_lines = journal_lines_gen(myjournal)
-	else:
-		print "Use      single line : %s" % shortstr(cmd_log)
-		test_lines = [ cmd_log ]
-	print
-
-	fail2banRegex.process(test_lines)
-
-	fail2banRegex.printStats() or sys.exit(-1)
+exec_command_line()

bin/fail2ban-server

@@ -127,7 +127,7 @@ class Fail2banServer:
 								self.__conf["pidfile"],
 								self.__conf["force"])
 			return True
-		except Exception, e:
+		except Exception as e:
 			logSys.exception(e)
 			self.__server.quit()
 			return False

bin/fail2ban-testcases

@@ -39,10 +39,18 @@ from fail2ban.version import version
 
 from fail2ban.tests.utils import gatherTests
 from fail2ban.helpers import FormatterWithTraceBack, getLogger
+from fail2ban.setup import updatePyExec
 from fail2ban.server.mytime import MyTime
 
 from optparse import OptionParser, Option
 
+# Update fail2ban-python env to current python version (where f2b-modules located/installed)
+bindir = os.path.dirname(
+	# __file__ seems to be overwritten sometimes on some python versions (e.g. bug of 2.6 by running under cProfile, etc.):
+	sys.argv[0] if os.path.basename(sys.argv[0]) == 'fail2ban-testcases' else __file__
+)
+updatePyExec(bindir)
+
 def get_opt_parser():
 	# use module docstring for help output
 	p = OptionParser(

config/action.d/badips.conf

@@ -1,6 +1,6 @@
 # Fail2ban reporting to badips.com
 #
-# Note: This reports and IP only and does not actually ban traffic. Use 
+# Note: This reports an IP only and does not actually ban traffic. Use
 # another action in the same jail if you want bans to occur.
 #
 # Set the category to the appropriate value before use.
@@ -10,7 +10,7 @@
 
 [Definition]
 
-actionban = curl --fail  --user-agent "fail2ban v0.8.12" http://www.badips.com/add/<category>/<ip>
+actionban = curl --fail  --user-agent "<agent>" http://www.badips.com/add/<category>/<ip>
 
 [Init]
 

config/action.d/badips.py

@@ -21,7 +21,6 @@ import sys
 if sys.version_info < (2, 7):
 	raise ImportError("badips.py action requires Python >= 2.7")
 import json
-from functools import partial
 import threading
 import logging
 if sys.version_info >= (3, ):
@@ -33,7 +32,6 @@ else:
 	from urllib import urlencode
 
 from fail2ban.server.actions import ActionBase
-from fail2ban.version import version as f2bVersion
 
 
 class BadIPsAction(ActionBase):
@@ -72,6 +70,9 @@ class BadIPsAction(ActionBase):
 	updateperiod : int, optional
 		Time in seconds between updating bad IPs blacklist.
 		Default 900 (15 minutes)
+	agent : str, optional
+		User agent transmitted to server.
+		Default `Fail2Ban/ver.`
 
 	Raises
 	------
@@ -79,14 +80,18 @@ class BadIPsAction(ActionBase):
 		If invalid `category`, `score`, `banaction` or `updateperiod`.
 	"""
 
+	TIMEOUT = 10
 	_badips = "http://www.badips.com"
-	_Request = partial(
-		Request, headers={'User-Agent': "Fail2Ban %s" % f2bVersion})
+	def _Request(self, url, **argv):
+		return Request(url, headers={'User-Agent': self.agent}, **argv)
 
 	def __init__(self, jail, name, category, score=3, age="24h", key=None,
-		banaction=None, bancategory=None, bankey=None, updateperiod=900):
+		banaction=None, bancategory=None, bankey=None, updateperiod=900, agent="Fail2Ban", 
+		timeout=TIMEOUT):
 		super(BadIPsAction, self).__init__(jail, name)
 
+		self.timeout = timeout
+		self.agent = agent
 		self.category = category
 		self.score = score
 		self.age = age
@@ -117,7 +122,7 @@ class BadIPsAction(ActionBase):
 		"""
 		try:
 			response = urlopen(
-				self._Request("/".join([self._badips, "get", "categories"])))
+				self._Request("/".join([self._badips, "get", "categories"])), timeout=self.timeout)
 		except HTTPError as response:
 			messages = json.loads(response.read().decode('utf-8'))
 			self._logSys.error(
@@ -171,7 +176,7 @@ class BadIPsAction(ActionBase):
 				urlencode({'age': age})])
 			if key:
 				url = "&".join([url, urlencode({'key': key})])
-			response = urlopen(self._Request(url))
+			response = urlopen(self._Request(url), timeout=self.timeout)
 		except HTTPError as response:
 			messages = json.loads(response.read().decode('utf-8'))
 			self._logSys.error(
@@ -356,7 +361,7 @@ class BadIPsAction(ActionBase):
 			url = "/".join([self._badips, "add", self.category, aInfo['ip']])
 			if self.key:
 				url = "?".join([url, urlencode({'key': self.key})])
-			response = urlopen(self._Request(url))
+			response = urlopen(self._Request(url), timeout=self.timeout)
 		except HTTPError as response:
 			messages = json.loads(response.read().decode('utf-8'))
 			self._logSys.error(

config/action.d/blocklist_de.conf

@@ -54,7 +54,7 @@ actioncheck =
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = curl --fail --data-urlencode 'server=<email>' --data 'apikey=<apikey>' --data 'service=<service>' --data 'ip=<ip>' --data-urlencode 'logs=<matches>' --data 'format=text' --user-agent "fail2ban v0.8.12" "https://www.blocklist.de/en/httpreports.html"
+actionban = curl --fail --data-urlencode 'server=<email>' --data 'apikey=<apikey>' --data 'service=<service>' --data 'ip=<ip>' --data-urlencode 'logs=<matches>' --data 'format=text' --user-agent "<agent>" "https://www.blocklist.de/en/httpreports.html"
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -67,7 +67,7 @@ actionunban =
 [Init]
 
 # Option:  email
-# Notes    server email address, as per blocklise.de account
+# Notes    server email address, as per blocklist.de account
 # Values:  STRING  Default: None
 #
 #email =

config/action.d/bsd-ipfw.conf

@@ -14,7 +14,7 @@
 # Notes.:  command executed once at the start of Fail2Ban.
 # Values:  CMD
 #
-actionstart = ipfw show | fgrep -q 'table(<table>)' || ( ipfw show | awk 'BEGIN { b = 1 } { if ($1 <= b) { b = $1 + 1 } else { e = b } } END { if (e) exit e <br> else exit b }'; num=$?; ipfw -q add $num <blocktype> <block> from table\(<table>\) to me <port>; echo $num > "<startstatefile>" )
+actionstart = ipfw show | fgrep -c -m 1 -s 'table(<table>)' > /dev/null 2>&1 || ( ipfw show | awk 'BEGIN { b = <lowest_rule_num> } { if ($1 < b) {} else if ($1 == b) { b = $1 + 1 } else { e = b } } END { if (e) exit e <br> else exit b }'; num=$?; ipfw -q add $num <blocktype> <block> from table\(<table>\) to me <port>; echo $num > "<startstatefile>" )
 
 
 # Option:  actionstop
@@ -81,3 +81,11 @@ block = ip
 # Values:  STRING
 #
 blocktype = unreach port
+
+# Option:  lowest_rule_num
+# Notes:   When fail2ban starts with action and there is no rule for the given table yet
+#          then fail2ban will start looking for an empty slot starting with this rule number.
+# Values:  NUM
+lowest_rule_num = 111
+
+

config/action.d/cloudflare.conf

@@ -9,6 +9,8 @@
 # Referenced from http://www.normyee.net/blog/2012/02/02/adding-cloudflare-support-to-fail2ban by NORM YEE
 #
 # To get your CloudFlare API Key: https://www.cloudflare.com/a/account/my-account
+#
+# CloudFlare API error codes: https://www.cloudflare.com/docs/host-api.html#s4.2
 
 [Definition]
 
@@ -38,7 +40,12 @@ actioncheck =
 #          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
-actionban = curl -s -o /dev/null https://www.cloudflare.com/api_json.html -d 'a=ban' -d 'tkn=<cftoken>' -d 'email=<cfuser>' -d 'key=<ip>'
+# API v1
+#actionban = curl -s -o /dev/null https://www.cloudflare.com/api_json.html -d 'a=ban' -d 'tkn=<cftoken>' -d 'email=<cfuser>' -d 'key=<ip>'
+# API v4
+actionban = curl -s -o /dev/null -X POST -H 'X-Auth-Email: <cfuser>' -H 'X-Auth-Key: <cftoken>' \
+            -H 'Content-Type: application/json' -d '{ "mode": "block", "configuration": { "target": "ip", "value": "<ip>" } }' \
+            https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -48,7 +55,12 @@ actionban = curl -s -o /dev/null https://www.cloudflare.com/api_json.html -d 'a=
 #          <time>  unix timestamp of the ban time
 # Values:  CMD
 #
-actionunban = curl -s -o /dev/null https://www.cloudflare.com/api_json.html -d 'a=nul' -d 'tkn=<cftoken>' -d 'email=<cfuser>' -d 'key=<ip>'
+# API v1
+#actionunban = curl -s -o /dev/null https://www.cloudflare.com/api_json.html -d 'a=nul' -d 'tkn=<cftoken>' -d 'email=<cfuser>' -d 'key=<ip>'
+# API v4
+actionunban = curl -s -o /dev/null -X DELETE -H 'X-Auth-Email: <cfuser>' -H 'X-Auth-Key: <cftoken>' \
+              https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules/$(curl -s -X GET -H 'X-Auth-Email: <cfuser>' -H 'X-Auth-Key: <cftoken>' \
+              'https://api.cloudflare.com/client/v4/user/firewall/access_rules/rules?mode=block&configuration_target=ip&configuration_value=<ip>&page=1&per_page=1' | cut -d'"' -f6)
 
 [Init]
 

config/action.d/complain.conf

@@ -79,7 +79,7 @@ logpath = /dev/null
 # Notes.:  Your system mail command. Is passed 2 args: subject and recipient
 # Values:  CMD
 #
-mailcmd = mail -s
+mailcmd = mail -E 'set escape' -s
 
 # Option:  mailargs
 # Notes.:  Additional arguments to mail command. e.g. for standard Unix mail:

config/action.d/dshield.conf

@@ -176,7 +176,7 @@ tcpflags =
 # Notes.:  Your system mail command. Is passed 2 args: subject and recipient
 # Values:  CMD
 #
-mailcmd = mail -s
+mailcmd = mail -E 'set escape' -s
 
 # Option:  mailargs
 # Notes.:  Additional arguments to mail command. e.g. for standard Unix mail:

config/action.d/firewallcmd-allports.conf

@@ -6,7 +6,7 @@
 
 [INCLUDES]
 
-before = iptables-blocktype.conf
+before = iptables-common.conf
 
 [Definition]
 

config/action.d/firewallcmd-ipset.conf

@@ -19,9 +19,9 @@ before = iptables-common.conf
 [Definition]
 
 actionstart = ipset create fail2ban-<name> hash:ip timeout <bantime>
-              firewall-cmd --direct --add-rule ipv4 filter <chain> 0 -p <protocol> -m multiport --dports <port> -m set --match-set fail2ban-<name> src -j <blocktype>
+              firewall-cmd --direct --add-rule ipv4 filter <chain> 0 <actiontype> -m set --match-set fail2ban-<name> src -j <blocktype>
 
-actionstop = firewall-cmd --direct --remove-rule ipv4 filter <chain> 0 -p <protocol> -m multiport --dports <port> -m set --match-set fail2ban-<name> src -j <blocktype>
+actionstop = firewall-cmd --direct --remove-rule ipv4 filter <chain> 0 <actiontype> -m set --match-set fail2ban-<name> src -j <blocktype>
              ipset flush fail2ban-<name>
              ipset destroy fail2ban-<name>
 
@@ -44,6 +44,23 @@ chain = INPUT_direct
 
 bantime = 600
 
+# Option: actiontype
+# Notes.: defines additions to the blocking rule
+# Values: leave empty to block all attempts from the host
+# Default: Value of the multiport
+actiontype = <multiport>
+
+# Option: allports
+# Notes.: default addition to block all ports
+# Usage.: use in jail config:  banaction = firewallcmd-ipset[actiontype=<allports>]
+#         for all protocols:   banaction = firewallcmd-ipset[actiontype=""]
+allports = -p <protocol>
+
+# Option: multiport
+# Notes.: addition to block access only to specific ports
+# Usage.: use in jail config:  banaction = firewallcmd-ipset[actiontype=<multiport>]
+multiport = -p <protocol> -m multiport --dports <port>
+
 
 # DEV NOTES:
 #

config/action.d/firewallcmd-multiport.conf

@@ -5,15 +5,15 @@
 
 [INCLUDES]
 
-before = iptables-blocktype.conf
+before = iptables-common.conf
 
 [Definition]
 
 actionstart = firewall-cmd --direct --add-chain ipv4 filter f2b-<name>
               firewall-cmd --direct --add-rule ipv4 filter f2b-<name> 1000 -j RETURN
-              firewall-cmd --direct --add-rule ipv4 filter <chain> 0 -m state --state NEW -p <protocol> -m multiport --dports <port> -j f2b-<name>
+              firewall-cmd --direct --add-rule ipv4 filter <chain> 0 -m conntrack --ctstate NEW -p <protocol> -m multiport --dports <port> -j f2b-<name>
 
-actionstop = firewall-cmd --direct --remove-rule ipv4 filter <chain> 0 -m state --state NEW -p <protocol> -m multiport --dports <port> -j f2b-<name>
+actionstop = firewall-cmd --direct --remove-rule ipv4 filter <chain> 0 -m conntrack --ctstate NEW -p <protocol> -m multiport --dports <port> -j f2b-<name>
              firewall-cmd --direct --remove-rules ipv4 filter f2b-<name>
              firewall-cmd --direct --remove-chain ipv4 filter f2b-<name>
 

config/action.d/firewallcmd-rich-logging.conf

@@ -0,0 +1,65 @@
+# Fail2Ban configuration file
+#
+# Author: Donald Yandt 
+# 
+# Because of the rich rule commands requires firewalld-0.3.1+
+# This action uses firewalld rich-rules which gives you a cleaner iptables since it stores rules according to zones and not
+# by chain. So for an example all deny rules will be listed under <zone>_deny and all log rules under <zone>_log. 
+#
+# Also this action logs banned access attempts so you can filter that and increase ban time for offenders.
+#
+# If you use the --permanent rule you get a xml file in /etc/firewalld/zones/<zone>.xml that can be shared and parsed easliy
+#
+# Example commands to view rules:
+# firewall-cmd [--zone=<zone>] --list-rich-rules
+# firewall-cmd [--zone=<zone>] --list-all
+# firewall-cmd [--zone=zone] --query-rich-rule='rule'
+
+[Definition]
+
+actionstart = 
+
+actionstop = 
+
+actioncheck = 
+
+# you can also use zones and/or service names. 
+#
+# zone example: 
+# 	firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' port port='<port>' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <blocktype>"
+# service name example:
+#	firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' service name='<service>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <blocktype>"
+# Because rich rules can only handle single or a range of ports we must split ports and execute the command for each port. Ports can be single and ranges separated by a comma or space for an example: http, https, 22-60, 18 smtp 
+
+actionban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --add-rich-rule="rule family='ipv4' source address='<ip>' port port='$p' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <blocktype>"; done
+	   
+actionunban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --remove-rich-rule="rule family='ipv4' source address='<ip>' port port='$p' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <blocktype>"; done
+
+[Init]
+
+name = default
+
+# log levels are "emerg", "alert", "crit", "error", "warning", "notice", "info" or "debug"
+level = info
+
+# log rate per minute
+rate = 1
+
+zone = public
+
+# use command firewall-cmd --get-services to see a list of services available
+#
+# Examples:
+#
+# amanda-client amanda-k5-client bacula bacula-client dhcp dhcpv6 dhcpv6-client dns freeipa-ldap freeipa-ldaps 
+# freeipa-replication ftp high-availability http https imaps ipp ipp-client ipsec iscsi-target kadmin kerberos 
+# kpasswd ldap ldaps libvirt libvirt-tls mdns mosh mountd ms-wbt mysql nfs ntp openvpn pmcd pmproxy pmwebapi pmwebapis pop3s 
+# postgresql privoxy proxy-dhcp puppetmaster radius rpc-bind rsyncd samba samba-client sane smtp squid ssh synergy 
+# telnet tftp tftp-client tinc tor-socks transmission-client vdsm vnc-server wbem-https xmpp-bosh xmpp-client xmpp-local xmpp-server
+
+service = ssh
+
+# reject types: 'icmp-net-unreachable', 'icmp-host-unreachable', 'icmp-port-unreachable', 'icmp-proto-unreachable', 
+# 'icmp-net-prohibited', 'icmp-host-prohibited', 'icmp-admin-prohibited' or 'tcp-reset'
+
+blocktype = reject type='icmp-port-unreachable'

config/action.d/firewallcmd-rich-rules.conf

@@ -0,0 +1,57 @@
+# Fail2Ban configuration file
+#
+# Author: Donald Yandt 
+# 
+# Because of the rich rule commands requires firewalld-0.3.1+
+# This action uses firewalld rich-rules which gives you a cleaner iptables since it stores rules according to zones and not
+# by chain. So for an example all deny rules will be listed under <zone>_deny. 
+#
+# If you use the --permanent rule you get a xml file in /etc/firewalld/zones/<zone>.xml that can be shared and parsed easliy
+#
+# Example commands to view rules:
+# firewall-cmd [--zone=<zone>] --list-rich-rules
+# firewall-cmd [--zone=<zone>] --list-all
+# firewall-cmd [--zone=zone] --query-rich-rule='rule'
+
+[Definition]
+
+actionstart = 
+
+actionstop = 
+
+actioncheck = 
+
+#you can also use zones and/or service names. 
+#
+# zone example: 
+# 	firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' port port='<port>' protocol='<protocol>' <blocktype>"
+# service name example:
+#	firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' service name='<service>' <blocktype>"
+# Because rich rules can only handle single or a range of ports we must split ports and execute the command for each port. Ports can be single and ranges separated by a comma or space for an example: http, https, 22-60, 18 smtp 
+
+actionban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --add-rich-rule="rule family='ipv4' source address='<ip>' port port='$p' protocol='<protocol>' <blocktype>"; done
+	   
+actionunban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --remove-rich-rule="rule family='ipv4' source address='<ip>' port port='$p' protocol='<protocol>' <blocktype>"; done
+
+[Init]
+
+name = default
+
+zone = public
+
+# use command firewall-cmd --get-services to see a list of services available
+#
+# Examples:
+#
+# amanda-client amanda-k5-client bacula bacula-client dhcp dhcpv6 dhcpv6-client dns freeipa-ldap freeipa-ldaps 
+# freeipa-replication ftp high-availability http https imaps ipp ipp-client ipsec iscsi-target kadmin kerberos 
+# kpasswd ldap ldaps libvirt libvirt-tls mdns mosh mountd ms-wbt mysql nfs ntp openvpn pmcd pmproxy pmwebapi pmwebapis pop3s 
+# postgresql privoxy proxy-dhcp puppetmaster radius rpc-bind rsyncd samba samba-client sane smtp squid ssh synergy 
+# telnet tftp tftp-client tinc tor-socks transmission-client vdsm vnc-server wbem-https xmpp-bosh xmpp-client xmpp-local xmpp-server
+
+service = ssh
+
+# reject types: 'icmp-net-unreachable', 'icmp-host-unreachable', 'icmp-port-unreachable', 'icmp-proto-unreachable', 
+# 'icmp-net-prohibited', 'icmp-host-prohibited', 'icmp-admin-prohibited' or 'tcp-reset'
+
+blocktype = reject type='icmp-port-unreachable'

config/action.d/hostsdeny.conf

@@ -31,8 +31,7 @@ actioncheck =
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionban = IP=<ip> &&
-            printf %%b "<daemon_list>: $IP\n" >> <file>
+actionban = IP=<ip> && printf %%b "<daemon_list>: $IP\n" >> <file>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -40,7 +39,7 @@ actionban = IP=<ip> &&
 # Tags:    See jail.conf(5) man page
 # Values:  CMD
 #
-actionunban = echo "/^<daemon_list>: <ip>$/<br>d<br>w<br>q" | ed <file>
+actionunban = IP=$(echo <ip> | sed 's/\./\\./g') && sed -i "/^<daemon_list>: $IP$/d" <file>
 
 [Init]
 

config/action.d/mail-buffered.conf

@@ -14,7 +14,7 @@ actionstart = printf %%b "Hi,\n
               The jail <name> has been started successfully.\n
               Output will be buffered until <lines> lines are available.\n
               Regards,\n
-              Fail2Ban"|mail -s "[Fail2Ban] <name>: started on `uname -n`" <dest>
+              Fail2Ban"|mail -E 'set escape' -s "[Fail2Ban] <name>: started on `uname -n`" <dest>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
@@ -25,13 +25,13 @@ actionstop = if [ -f <tmpfile> ]; then
                  These hosts have been banned by Fail2Ban.\n
                  `cat <tmpfile>`
                  Regards,\n
-                 Fail2Ban"|mail -s "[Fail2Ban] <name>: Summary from `uname -n`" <dest>
+                 Fail2Ban"|mail -E 'set escape' -s "[Fail2Ban] <name>: Summary from `uname -n`" <dest>
                  rm <tmpfile>
              fi
              printf %%b "Hi,\n
              The jail <name> has been stopped.\n
              Regards,\n
-             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
+             Fail2Ban"|mail -E 'set escape' -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
@@ -52,7 +52,7 @@ actionban = printf %%b "`date`: <ip> (<failures> failures)\n" >> <tmpfile>
                 These hosts have been banned by Fail2Ban.\n
                 `cat <tmpfile>`
                 \nRegards,\n
-                Fail2Ban"|mail -s "[Fail2Ban] <name>: Summary" <dest>
+                Fail2Ban"|mail -E 'set escape' -s "[Fail2Ban] <name>: Summary" <dest>
                 rm <tmpfile>
             fi
 

config/action.d/mail-whois-lines.conf

@@ -17,7 +17,7 @@ before = mail-whois-common.conf
 actionstart = printf %%b "Hi,\n
               The jail <name> has been started successfully.\n
               Regards,\n
-              Fail2Ban"|mail -s "[Fail2Ban] <name>: started on `uname -n`" <dest>
+              Fail2Ban" | <mailcmd> "[Fail2Ban] <name>: started on `uname -n`" <dest>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
@@ -26,7 +26,7 @@ actionstart = printf %%b "Hi,\n
 actionstop = printf %%b "Hi,\n
              The jail <name> has been stopped.\n
              Regards,\n
-             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
+             Fail2Ban" | <mailcmd> "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
@@ -48,7 +48,7 @@ actionban = printf %%b "Hi,\n
             Lines containing IP:<ip> in <logpath>\n
             `grep -E <grepopts> '(^|[^0-9])<ip>([^0-9]|$)' <logpath>`\n\n
             Regards,\n
-            Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip> from  `uname -n`" <dest>
+            Fail2Ban" | <mailcmd> "[Fail2Ban] <name>: banned <ip> from  `uname -n`" <dest>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the
@@ -60,6 +60,12 @@ actionunban =
 
 [Init]
 
+# Option:  mailcmd
+# Notes.:  Your system mail command. Is passed 2 args: subject and recipient
+# Values:  CMD
+#
+mailcmd = mail -E 'set escape' -s
+
 # Default name of the chain
 #
 name = default

config/action.d/mail-whois.conf

@@ -17,7 +17,7 @@ before = mail-whois-common.conf
 actionstart = printf %%b "Hi,\n
               The jail <name> has been started successfully.\n
               Regards,\n
-              Fail2Ban"|mail -s "[Fail2Ban] <name>: started on `uname -n`" <dest>
+              Fail2Ban"|mail -E 'set escape' -s "[Fail2Ban] <name>: started on `uname -n`" <dest>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
@@ -26,7 +26,7 @@ actionstart = printf %%b "Hi,\n
 actionstop = printf %%b "Hi,\n
              The jail <name> has been stopped.\n
              Regards,\n
-             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
+             Fail2Ban"|mail -E 'set escape' -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
@@ -46,7 +46,7 @@ actionban = printf %%b "Hi,\n
             Here is more information about <ip> :\n
             `%(_whois_command)s`\n
             Regards,\n
-            Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip> from `uname -n`" <dest>
+            Fail2Ban"|mail -E 'set escape' -s "[Fail2Ban] <name>: banned <ip> from `uname -n`" <dest>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the

config/action.d/mail.conf

@@ -13,7 +13,7 @@
 actionstart = printf %%b "Hi,\n
               The jail <name> has been started successfully.\n
               Regards,\n
-              Fail2Ban"|mail -s "[Fail2Ban] <name>: started  on `uname -n`" <dest>
+              Fail2Ban"|mail -E 'set escape' -s "[Fail2Ban] <name>: started  on `uname -n`" <dest>
 
 # Option:  actionstop
 # Notes.:  command executed once at the end of Fail2Ban
@@ -22,7 +22,7 @@ actionstart = printf %%b "Hi,\n
 actionstop = printf %%b "Hi,\n
              The jail <name> has been stopped.\n
              Regards,\n
-             Fail2Ban"|mail -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
+             Fail2Ban"|mail -E 'set escape' -s "[Fail2Ban] <name>: stopped on `uname -n`" <dest>
 
 # Option:  actioncheck
 # Notes.:  command executed once before each actionban command
@@ -40,7 +40,7 @@ actionban = printf %%b "Hi,\n
             The IP <ip> has just been banned by Fail2Ban after
             <failures> attempts against <name>.\n
             Regards,\n
-            Fail2Ban"|mail -s "[Fail2Ban] <name>: banned <ip> from `uname -n`" <dest>
+            Fail2Ban"|mail -E 'set escape' -s "[Fail2Ban] <name>: banned <ip> from `uname -n`" <dest>
 
 # Option:  actionunban
 # Notes.:  command executed when unbanning an IP. Take care that the

config/action.d/mynetwatchman.conf

@@ -111,13 +111,17 @@ myip = `ip -4 addr show dev eth0 | grep inet | head -n 1 | sed -r 's/.*inet ([0-
 #
 protocol = tcp
 
+# Option:  agent
+# Default: Fail2ban
+agent = Fail2ban
+
 # Option:  getcmd
 # Notes.:  A command to fetch a URL. Should output page to STDOUT
 # Values:  CMD  Default: wget
 #
-getcmd = wget --no-verbose --tries=3 --waitretry=10 --connect-timeout=10 --read-timeout=60 --retry-connrefused --output-document=- --user-agent=Fail2Ban
+getcmd = wget --no-verbose --tries=3 --waitretry=10 --connect-timeout=10 --read-timeout=60 --retry-connrefused --output-document=- --user-agent=<agent>
 # Alternative value:
-# getcmd = curl --silent --show-error --retry 3 --connect-timeout 10 --max-time 60 --user-agent Fail2Ban
+# getcmd = curl --silent --show-error --retry 3 --connect-timeout 10 --max-time 60 --user-agent <agent>
 
 # Option:  srcport
 # Notes.:  The source port of the attack. You're unlikely to have this info, so

config/action.d/netscaler.conf

@@ -0,0 +1,33 @@
+# Fail2ban Citrix Netscaler Action
+# by Juliano Jeziorny
+# juliano@jeziorny.eu
+#
+# The script will add offender IPs to a dataset on netscaler, the dataset can then be used to block the IPs at a cs/vserver or global level
+# This dataset is then used to block IPs using responder policies on the netscaler.
+# 
+# The script assumes using HTTPS with unsecure certificate to access the netscaler, 
+# if you have a valid certificate installed remove the -k from the curl lines, or if you want http change it accordingly (and remove the -k)
+# 
+# This action depends on curl
+#
+# You need to populate the 3 options inside Init
+#
+# ns_host: IP or hostname of netslcaer appliance
+# ns_auth: username:password, suggest base64 encoded for a little added security (echo -n "username:password" | base64)
+# ns_dataset:  Name of the netscaler dataset holding the IPs to be blocked.
+# 
+# For further details on how to use it please check http://blog.ckzone.eu/2017/01/fail2ban-action-for-citrix-netscaler.html
+
+[Init]
+ns_host = 
+ns_auth = 
+ns_dataset = 
+
+[Definition]
+actionstart = curl -kH 'Authorization: Basic <ns_auth>' https://<ns_host>/nitro/v1/config
+
+actioncheck = 
+
+actionban = curl -k -H 'Authorization: Basic <ns_auth>' -X PUT -d '{"policydataset_value_binding":{"name":"<ns_dataset>","value":"<ip>"}}' https://<ns_host>/nitro/v1/config/
+
+actionunban = curl -H 'Authorization: Basic <ns_auth>' -X DELETE -k "https://<ns_host>/nitro/v1/config/policydataset_value_binding/<ns_dataset>?args=value:<ip>"

config/action.d/nftables-allports.conf

@@ -0,0 +1,22 @@
+# Fail2Ban configuration file
+#
+# Author: Cyril Jaquier
+# Modified: Yaroslav O. Halchenko <debian@onerussian.com>
+# 			made active on all ports from original iptables.conf
+# Modified: Alexander Belykh <albel727@ngs.ru>
+#                       adapted for nftables
+#
+
+[INCLUDES]
+
+before = nftables-common.conf
+
+[Definition]
+
+# Option:  nftables_mode
+# Notes.:  additional expressions for nftables filter rule
+# Values:  nftables expressions
+#
+nftables_mode = ip protocol <protocol>
+
+[Init]

config/action.d/nftables-common.conf

@@ -0,0 +1,119 @@
+# Fail2Ban configuration file
+#
+# Author: Daniel Black
+# Author: Cyril Jaquier
+# Modified: Yaroslav O. Halchenko <debian@onerussian.com>
+# 			made active on all ports from original iptables.conf
+# Modified: Alexander Belykh <albel727@ngs.ru>
+#                       adapted for nftables
+#
+# This is a included configuration file and includes the definitions for the nftables
+# used in all nftables based actions by default.
+#
+# The user can override the defaults in nftables-common.local
+
+[INCLUDES]
+
+after = nftables-common.local
+
+[Definition]
+
+# Option:  nftables_mode
+# Notes.:  additional expressions for nftables filter rule
+# Values:  nftables expressions
+#
+nftables_mode = <protocol> dport \{ <port> \}
+
+# Option:  actionstart
+# Notes.:  command executed once at the start of Fail2Ban.
+# Values:  CMD
+#
+actionstart = <nftables> add set <nftables_family> <nftables_table> f2b-<name> \{ type <nftables_type>\; \}
+              <nftables> insert rule <nftables_family> <nftables_table> <chain> %(nftables_mode)s ip saddr @f2b-<name> <blocktype>
+
+_nft_list = <nftables> --handle --numeric list chain <nftables_family> <nftables_table> <chain>
+_nft_get_handle_id = grep -m1 'ip saddr @f2b-<name> <blocktype> # handle' | grep -oe ' handle [0-9]*'
+
+# Option:  actionstop
+# Notes.:  command executed once at the end of Fail2Ban
+# Values:  CMD
+#
+actionstop = HANDLE_ID=$(%(_nft_list)s | %(_nft_get_handle_id)s)
+             <nftables> delete rule <nftables_family> <nftables_table> <chain> $HANDLE_ID
+             <nftables> delete set <nftables_family> <nftables_table> f2b-<name>
+
+# Option:  actioncheck
+# Notes.:  command executed once before each actionban command
+# Values:  CMD
+#
+actioncheck = <nftables> list chain <nftables_family> <nftables_table> <chain> | grep -q '@f2b-<name>[ \t]'
+
+# Option:  actionban
+# Notes.:  command executed when banning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    See jail.conf(5) man page
+# Values:  CMD
+#
+actionban = <nftables> add element <nftables_family> <nftables_table> f2b-<name> \{ <ip> \}
+
+# Option:  actionunban
+# Notes.:  command executed when unbanning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    See jail.conf(5) man page
+# Values:  CMD
+#
+actionunban = <nftables> delete element <nftables_family> <nftables_table> f2b-<name> \{ <ip> \}
+
+[Init]
+
+# Option:  nftables_type
+# Notes.:  address type to work with
+# Values:  [ipv4_addr | ipv6_addr]  Default: ipv4_addr
+#
+nftables_type = ipv4_addr
+
+# Option:  nftables_family
+# Notes.:  address family to work in
+# Values:  [ip | ip6 | inet]  Default: inet
+#
+nftables_family = inet
+
+# Option:  nftables_table
+# Notes.:  table in the address family to work in
+# Values:  STRING  Default: filter
+#
+nftables_table = filter
+
+# Option:  chain
+# Notes    specifies the nftables chain to which the Fail2Ban rules should be
+#          added
+# Values:  STRING  Default: input
+chain = input
+
+# Default name of the filtering set
+#
+name = default
+
+# Option:  port
+# Notes.:  specifies port to monitor
+# Values:  [ NUM | STRING ]  Default:
+#
+port = ssh
+
+# Option:  protocol
+# Notes.:  internally used by config reader for interpolations.
+# Values:  [ tcp | udp ] Default: tcp
+#
+protocol = tcp
+
+# Option:  blocktype
+# Note:    This is what the action does with rules. This can be any jump target
+#          as per the nftables man page (section 8). Common values are drop
+#          reject, reject with icmp type host-unreachable
+# Values:  STRING
+blocktype = reject
+
+# Option:  nftables
+# Notes.:  Actual command to be executed, including common to all calls options
+# Values:  STRING
+nftables = nft

config/action.d/nftables-multiport.conf

@@ -0,0 +1,22 @@
+# Fail2Ban configuration file
+#
+# Author: Cyril Jaquier
+# Modified: Yaroslav O. Halchenko <debian@onerussian.com>
+# 			made active on all ports from original iptables.conf
+# Modified: Alexander Belykh <albel727@ngs.ru>
+#                       adapted for nftables
+#
+
+[INCLUDES]
+
+before = nftables-common.conf
+
+[Definition]
+
+# Option:  nftables_mode
+# Notes.:  additional expressions for nftables filter rule
+# Values:  nftables expressions
+#
+nftables_mode = <protocol> dport \{ <port> \}
+
+[Init]

config/action.d/npf.conf

@@ -0,0 +1,61 @@
+# Fail2Ban configuration file
+#
+# NetBSD npf ban/unban
+#
+# Author: Nils Ratusznik <nils@NetBSD.org>
+# Based on pf.conf action file
+#
+
+[Definition]
+
+# Option:  actionstart
+# Notes.:  command executed once at the start of Fail2Ban.
+# Values:  CMD
+#
+# we don't enable NPF automatically, as it will be enabled elsewhere
+actionstart = 
+
+
+# Option:  actionstop
+# Notes.:  command executed once at the end of Fail2Ban
+# Values:  CMD
+#
+# we don't disable NPF automatically either
+actionstop = 
+
+
+# Option:  actioncheck
+# Notes.:  command executed once before each actionban command
+# Values:  CMD
+#
+actioncheck = 
+
+
+# Option:  actionban
+# Notes.:  command executed when banning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    <ip>  IP address
+#          <failures>  number of failures
+#          <time>  unix timestamp of the ban time
+# Values:  CMD
+#
+actionban = /sbin/npfctl table <tablename> add <ip>
+
+
+# Option:  actionunban
+# Notes.:  command executed when unbanning an IP. Take care that the
+#          command is executed with Fail2Ban user rights.
+# Tags:    <ip>  IP address
+#          <failures>  number of failures
+#          <time>  unix timestamp of the ban time
+# Values:  CMD
+#
+# note -r option used to remove matching rule
+actionunban = /sbin/npfctl table <tablename> rem <ip>
+
+[Init]
+# Option:  tablename
+# Notes.:  The pf table name.
+# Values:  [ STRING ]
+#
+tablename = fail2ban

config/action.d/route.conf

@@ -17,6 +17,9 @@
 [Definition]
 actionban   = ip route add <blocktype> <ip>
 actionunban = ip route del <blocktype> <ip>
+actioncheck =
+actionstart =
+actionstop =
 
 [Init]
 

config/action.d/shorewall.conf

@@ -9,7 +9,7 @@
 # connections. So if the attempter goes on trying using the same connection
 # he could even log in. In order to get the same behavior of the iptable
 # action (so that the ban is immediate) the /etc/shorewall/shorewall.conf
-# file should me modified with "BLACKLISTNEWONLY=No". Note that as of
+# file should be modified with "BLACKLISTNEWONLY=No". Note that as of
 # Shorewall 4.5.13 BLACKLISTNEWONLY is deprecated; however the equivalent
 # of BLACKLISTNEWONLY=No can now be achieved by setting BLACKLIST="ALL".
 # 

config/action.d/xarf-login-attack.conf

@@ -48,9 +48,9 @@ actionban = oifs=${IFS}; IFS=.;SEP_IP=( <ip> ); set -- ${SEP_IP}; ADDRESSES=$(di
             PORT=<port>
             DATE=`LC_ALL=C date --date=@<time> +"%%a, %%d %%h %%Y %%T %%z"`
             if [ ! -z "$ADDRESSES" ]; then
-                (printf -- %%b "<header>\n<message>\n<report>\n";
+                (printf -- %%b "<header>\n<message>\n<report>\n\n";
                  date '+Note: Local timezone is %%z (%%Z)';
-                 printf -- %%b "<ipmatches>\n\n<footer>") | <mailcmd> <mailargs> ${ADDRESSES//,/\" \"}
+                 printf -- %%b "\n<ipmatches>\n\n<footer>") | <mailcmd> <mailargs> ${ADDRESSES//,/\" \"}
             fi
 
 actionunban =

config/filter.d/apache-badbots.conf

@@ -7,8 +7,8 @@
 
 [Definition]
 
-badbotscustom = EmailCollector|WebEMailExtrac|TrackBack/1\.02|sogou music spider
-badbots = Atomic_Email_Hunter/4\.0|atSpider/1\.0|autoemailspider|bwh3_user_agent|China Local Browse 2\.6|ContactBot/0\.2|ContentSmartz|DataCha0s/2\.0|DBrowse 1\.4b|DBrowse 1\.4d|Demo Bot DOT 16b|Demo Bot Z 16b|DSurf15a 01|DSurf15a 71|DSurf15a 81|DSurf15a VA|EBrowse 1\.4b|Educate Search VxB|EmailSiphon|EmailSpider|EmailWolf 1\.00|ESurf15a 15|ExtractorPro|Franklin Locator 1\.8|FSurf15a 01|Full Web Bot 0416B|Full Web Bot 0516B|Full Web Bot 2816B|Guestbook Auto Submitter|Industry Program 1\.0\.x|ISC Systems iRc Search 2\.1|IUPUI Research Bot v 1\.9a|LARBIN-EXPERIMENTAL \(efp@gmx\.net\)|LetsCrawl\.com/1\.0 +http\://letscrawl\.com/|Lincoln State Web Browser|LMQueueBot/0\.2|LWP\:\:Simple/5\.803|Mac Finder 1\.0\.xx|MFC Foundation Class Library 4\.0|Microsoft URL Control - 6\.00\.8xxx|Missauga Locate 1\.0\.0|Missigua Locator 1\.9|Missouri College Browse|Mizzu Labs 2\.2|Mo College 1\.9|MVAClient|Mozilla/2\.0 \(compatible; NEWT ActiveX; Win32\)|Mozilla/3\.0 \(compatible; Indy Library\)|Mozilla/3\.0 \(compatible; scan4mail \(advanced version\) http\://www\.peterspages\.net/?scan4mail\)|Mozilla/4\.0 \(compatible; Advanced Email Extractor v2\.xx\)|Mozilla/4\.0 \(compatible; Iplexx Spider/1\.0 http\://www\.iplexx\.at\)|Mozilla/4\.0 \(compatible; MSIE 5\.0; Windows NT; DigExt; DTS Agent|Mozilla/4\.0 efp@gmx\.net|Mozilla/5\.0 \(Version\: xxxx Type\:xx\)|NameOfAgent \(CMS Spider\)|NASA Search 1\.0|Nsauditor/1\.x|PBrowse 1\.4b|PEval 1\.4b|Poirot|Port Huron Labs|Production Bot 0116B|Production Bot 2016B|Production Bot DOT 3016B|Program Shareware 1\.0\.2|PSurf15a 11|PSurf15a 51|PSurf15a VA|psycheclone|RSurf15a 41|RSurf15a 51|RSurf15a 81|searchbot admin@google\.com|ShablastBot 1\.0|snap\.com beta crawler v0|Snapbot/1\.0|Snapbot/1\.0 \(Snap Shots, +http\://www\.snap\.com\)|sogou develop spider|Sogou Orion spider/3\.0\(+http\://www\.sogou\.com/docs/help/webmasters\.htm#07\)|sogou spider|Sogou web spider/3\.0\(+http\://www\.sogou\.com/docs/help/webmasters\.htm#07\)|sohu agent|SSurf15a 11 |TSurf15a 11|Under the Rainbow 2\.2|User-Agent\: Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1\)|VadixBot|WebVulnCrawl\.unknown/1\.0 libwww-perl/5\.803|Wells Search II|WEP Search 00
+badbotscustom = EmailCollector|WebEMailExtrac|TrackBack/1\.02|sogou music spider|(?:Mozilla/\d+\.\d+ )?Jorgee
+badbots = Atomic_Email_Hunter/4\.0|atSpider/1\.0|autoemailspider|bwh3_user_agent|China Local Browse 2\.6|ContactBot/0\.2|ContentSmartz|DataCha0s/2\.0|DBrowse 1\.4b|DBrowse 1\.4d|Demo Bot DOT 16b|Demo Bot Z 16b|DSurf15a 01|DSurf15a 71|DSurf15a 81|DSurf15a VA|EBrowse 1\.4b|Educate Search VxB|EmailSiphon|EmailSpider|EmailWolf 1\.00|ESurf15a 15|ExtractorPro|Franklin Locator 1\.8|FSurf15a 01|Full Web Bot 0416B|Full Web Bot 0516B|Full Web Bot 2816B|Guestbook Auto Submitter|Industry Program 1\.0\.x|ISC Systems iRc Search 2\.1|IUPUI Research Bot v 1\.9a|LARBIN-EXPERIMENTAL \(efp@gmx\.net\)|LetsCrawl\.com/1\.0 \+http\://letscrawl\.com/|Lincoln State Web Browser|LMQueueBot/0\.2|LWP\:\:Simple/5\.803|Mac Finder 1\.0\.xx|MFC Foundation Class Library 4\.0|Microsoft URL Control - 6\.00\.8xxx|Missauga Locate 1\.0\.0|Missigua Locator 1\.9|Missouri College Browse|Mizzu Labs 2\.2|Mo College 1\.9|MVAClient|Mozilla/2\.0 \(compatible; NEWT ActiveX; Win32\)|Mozilla/3\.0 \(compatible; Indy Library\)|Mozilla/3\.0 \(compatible; scan4mail \(advanced version\) http\://www\.peterspages\.net/?scan4mail\)|Mozilla/4\.0 \(compatible; Advanced Email Extractor v2\.xx\)|Mozilla/4\.0 \(compatible; Iplexx Spider/1\.0 http\://www\.iplexx\.at\)|Mozilla/4\.0 \(compatible; MSIE 5\.0; Windows NT; DigExt; DTS Agent|Mozilla/4\.0 efp@gmx\.net|Mozilla/5\.0 \(Version\: xxxx Type\:xx\)|NameOfAgent \(CMS Spider\)|NASA Search 1\.0|Nsauditor/1\.x|PBrowse 1\.4b|PEval 1\.4b|Poirot|Port Huron Labs|Production Bot 0116B|Production Bot 2016B|Production Bot DOT 3016B|Program Shareware 1\.0\.2|PSurf15a 11|PSurf15a 51|PSurf15a VA|psycheclone|RSurf15a 41|RSurf15a 51|RSurf15a 81|searchbot admin@google\.com|ShablastBot 1\.0|snap\.com beta crawler v0|Snapbot/1\.0|Snapbot/1\.0 \(Snap Shots, \+http\://www\.snap\.com\)|sogou develop spider|Sogou Orion spider/3\.0\(\+http\://www\.sogou\.com/docs/help/webmasters\.htm#07\)|sogou spider|Sogou web spider/3\.0\(\+http\://www\.sogou\.com/docs/help/webmasters\.htm#07\)|sohu agent|SSurf15a 11 |TSurf15a 11|Under the Rainbow 2\.2|User-Agent\: Mozilla/4\.0 \(compatible; MSIE 6\.0; Windows NT 5\.1\)|VadixBot|WebVulnCrawl\.unknown/1\.0 libwww-perl/5\.803|Wells Search II|WEP Search 00
 
 failregex = ^<HOST> -.*"(GET|POST|HEAD).*HTTP.*"(?:%(badbots)s|%(badbotscustom)s)"$
 

config/filter.d/apache-botsearch.conf

@@ -3,7 +3,7 @@
 # This filter is aimed at blocking specific URLs that don't exist. This
 # could be a set of URLs places in a Disallow: directive in robots.txt or
 # just some web services that don't exist caused bots are searching for
-# exploitable content. This filter is designed to have a low false postitive
+# exploitable content. This filter is designed to have a low false positive
 # rate due.
 #
 # An alternative to this is the apache-noscript filter which blocks all
@@ -37,4 +37,4 @@ webroot = /var/www/
 
 # DEV Notes:
 #
-# Author: Daniel Black
+# Author: Daniel Black

config/filter.d/apache-modsecurity.conf

@@ -10,9 +10,10 @@ before = apache-common.conf
 [Definition]
 
 
-failregex = ^%(_apache_error_client)s ModSecurity:  (\[.*?\] )*Access denied with code [45]\d\d.*$
+failregex = ^%(_apache_error_client)s ModSecurity:\s+(?:\[(?:\w+ \"[^\"]*\"|[^\]]*)\]\s*)*Access denied with code [45]\d\d
 
 ignoreregex = 
 
 # https://github.com/SpiderLabs/ModSecurity/wiki/ModSecurity-2-Data-Formats
 # Author: Daniel Black
+#         Sergey G. Brester aka sebres (review, optimization)

config/filter.d/apache-overflows.conf

@@ -8,11 +8,15 @@ before = apache-common.conf
 
 [Definition]
 
-failregex = ^%(_apache_error_client)s ((AH0013[456]: )?Invalid (method|URI) in request .*( - possible attempt to establish SSL connection on non-SSL port)?|(AH00565: )?request failed: URI too long \(longer than \d+\)|request failed: erroneous characters after protocol string: .*|AH00566: request failed: invalid characters in URI)(, referer: \S+)?$
+failregex = ^%(_apache_error_client)s (?:(?:AH0013[456]: )?Invalid (method|URI) in request\b|(?:AH00565: )?request failed: URI too long \(longer than \d+\)|request failed: erroneous characters after protocol string:|(?:AH00566: )?request failed: invalid characters in URI\b)
 
 ignoreregex =
 
 # DEV Notes:
+#
+# [sebres] Because this apache-log could contain very long URLs (and/or referrer), 
+#          the parsing of it anchored way may be very vulnerable (at least as regards 
+#          the system resources, see gh-1790). Thus rewritten without end-anchor ($).
 # 
 # fgrep -r 'URI too long' httpd-2.*
 #   httpd-2.2.25/server/protocol.c:                          "request failed: URI too long (longer than %d)", r->server->limit_req_line);

config/filter.d/apache-shellshock.conf

@@ -23,4 +23,4 @@ ignoreregex =
 # [Thu Sep 25 09:27:18.813902 2014] [cgi:error] [pid 16860] [client 89.207.132.76:59635] AH01215: /bin/bash: warning: HTTP_TEST: ignoring function definition attempt
 # [Thu Sep 25 09:29:56.141832 2014] [cgi:error] [pid 16864] [client 162.247.73.206:41273] AH01215: /bin/bash: error importing function definition for `HTTP_TEST'
 #
-# Author: Eugene Hopkinson (riot@riot.so)
+# Author: Eugene Hopkinson (e.hopkinson@gmail.com)

config/filter.d/assp.conf

@@ -1,24 +1,43 @@
-# Fail2Ban filter for Anti-Spam SMTP Proxy Server also known as ASSP
-# 
-#    Honmepage:   http://www.magicvillage.de/~Fritz_Borgstedt/assp/0003D91C-8000001C/
-#    ProjektSite: http://sourceforge.net/projects/assp/?source=directory
+# Fail2Ban filter for Anti-Spam SMTP Proxy Server (ASSP)
+#    Filter works in theory for both ASSP V1 and V2. Recommended ASSP is V2.5.1 or later.
+#    Support for ASSP V1 ended in 2014 so if you are still running ASSP V1 an immediate upgrade is recommended.
+#
+#    Homepage:    http://sourceforge.net/projects/assp/
+#    ProjectSite: http://sourceforge.net/projects/assp/?source=directory
 #
 #
 
 [Definition] 
+# Note: First three failregex matches below are for ASSP V1 with the remaining being designed for V2. Deleting the V1 regex is recommended but I left it in for compatibility reasons.
 
 __assp_actions = (?:dropping|refusing)
 
 failregex = ^(:? \[SSL-out\])? <HOST> max sender authentication errors \(\d{,3}\) exceeded -- %(__assp_actions)s connection - after reply: \d{3} \d{1}\.\d{1}.\d{1} Error: authentication failed: \w+;$
 			^(?: \[SSL-out\])? <HOST> SSL negotiation with client failed: SSL accept attempt failed with unknown error.*:unknown protocol;$
 			^ Blocking <HOST> - too much AUTH errors \(\d{,3}\);$
+			^\s*(?:[\w\-]+\s+)*(?:\[\S+\]\s+)*<HOST> (?:\<\S+@\S+\.\S+\> )*(?:to: \S+@\S+\.\S+ )*relay attempt blocked for(?: \(parsing\))?: \S+$
+			^\s*(?:[\w\-]+\s+)*(?:\[\S+\]\s+)*<HOST> \[SMTP Error\] 535 5\.7\.8 Error: authentication failed:\s+(?:\S+|Connection lost to authentication server|Invalid authentication mechanism|Invalid base64 data in continued response)?$
 
 ignoreregex = 
 
 # DEV Notes:
+# V1 Examples matches:
+#   Apr-27-13 02:33:09 Blocking 217.194.197.97 - too much AUTH errors (41);
+#   Dec-29-12 17:10:31 [SSL-out] 200.247.87.82 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol;
+#   Dec-30-12 04:01:47 [SSL-out] 81.82.232.66 max sender authentication errors (5) exceeded 
 #
-# Examples: Apr-27-13 02:33:09 Blocking 217.194.197.97 - too much AUTH errors (41);
-#           Dec-29-12 17:10:31 [SSL-out] 200.247.87.82 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol;
-#           Dec-30-12 04:01:47 [SSL-out] 81.82.232.66 max sender authentication errors (5) exceeded 
+# V2 Examples matches:
+#   Jul-29-16 16:49:52 m1-25391-06124 [Worker_1] [TLS-out] [RelayAttempt] 0.0.0.0 <user@example.com> to: user@example.org relay attempt blocked for: someone@example.org
+#   Jul-30-16 16:59:42 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6
+#   Jul-30-16 00:15:36 m1-52131-09651 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: UGFzc3dvcmQ6
+#   Jul-31-16 06:45:59 [Worker_1] [TLS-in] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed:
+#   Jan-05-16 08:38:49 m1-01129-09140 [Worker_1] [TLS-in] [TLS-out] [RelayAttempt] 0.0.0.0 <user@example.com> relay attempt blocked for (parsing): <user2@example>
+#   Jun-12-16 16:43:37 m1-64217-12013 [Worker_1] [TLS-in] [TLS-out] [RelayAttempt] 0.0.0.0 <user@example.com> to: user2@example.com relay attempt blocked for (parsing): <a.notheruser69@example.c>
+#   Jan-22-16 22:25:51 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: Invalid authentication mechanism
+#   Mar-19-16 13:42:20 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: Invalid base64 data in continued response
+#   Jul-18-16 16:54:21 [Worker_2] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: Connection lost to authentication server
+#   Jul-18-16 17:14:23 m1-76453-02949 [Worker_1] [TLS-out] 0.0.0.0 [SMTP Error] 535 5.7.8 Error: authentication failed: Connection lost to authentication server
+
 #
 # Author: Enrico Labedzki (enrico.labedzki@deiwos.de)
+# V2 Filters: Robert Hardy (rhardy@webcon.ca)

config/filter.d/asterisk.conf

@@ -11,22 +11,24 @@ before = common.conf
 
 _daemon = asterisk
 
-__pid_re = (?:\[\d+\])
+__pid_re = (?:\s*\[\d+\])
 
 iso8601 = \d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d+[+-]\d{4}
 
 # All Asterisk log messages begin like this:
-log_prefix= (?:NOTICE|SECURITY)%(__pid_re)s:?(?:\[C-[\da-f]*\])? \S+:\d*( in \w+:)?
+log_prefix= (?:NOTICE|SECURITY|WARNING)%(__pid_re)s:?(?:\[C-[\da-f]*\])? [^:]+:\d*(?:(?: in)? \w+:)?
 
-failregex = ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?' - (Wrong password|Username/auth name mismatch|No matching peer found|Not a local domain|Device does not match ACL|Peer is not supposed to register|ACL error \(permit/deny\)|Not a local domain)$
-            ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Call from '[^']*' \(<HOST>:\d+\) to extension '\d+' rejected because extension not found in context 'default'\.$
-            ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Host <HOST> failed to authenticate as '[^']*'$
-            ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s No registration for peer '[^']*' \(from <HOST>\)$
-            ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Host <HOST> failed MD5 authentication for '[^']*' \([^)]+\)$
-            ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Failed to authenticate (user|device) [^@]+@<HOST>\S*$
-            ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s hacking attempt detected '<HOST>'$
-            ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s SecurityEvent="(FailedACL|InvalidAccountID|ChallengeResponseFailed|InvalidPassword)",EventTV="([\d-]+|%(iso8601)s)",Severity="[\w]+",Service="[\w]+",EventVersion="\d+",AccountID="(\d*|<unknown>)",SessionID=".+",LocalAddress="IPV[46]/(UDP|TCP|WS)/[\da-fA-F:.]+/\d+",RemoteAddress="IPV[46]/(UDP|TCP|WS)/<HOST>/\d+"(,Challenge="[\w/]+")?(,ReceivedChallenge="\w+")?(,Response="\w+",ExpectedResponse="\w*")?(,ReceivedHash="[\da-f]+")?(,ACLName="\w+")?$
-            ^(%(__prefix_line)s|\[\]\s*WARNING%(__pid_re)s:?(?:\[C-[\da-f]*\])? )Ext\. s: "Rejecting unknown SIP connection from <HOST>"$
+failregex = ^%(__prefix_line)s%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?' - (?:Wrong password|Username/auth name mismatch|No matching peer found|Not a local domain|Device does not match ACL|Peer is not supposed to register|ACL error \(permit/deny\)|Not a local domain)$
+            ^%(__prefix_line)s%(log_prefix)s Call from '[^']*' \(<HOST>:\d+\) to extension '[^']*' rejected because extension not found in context
+            ^%(__prefix_line)s%(log_prefix)s (?:Host )?<HOST> (?:failed (?:to authenticate\b|MD5 authentication\b)|tried to authenticate with nonexistent user\b)
+            ^%(__prefix_line)s%(log_prefix)s No registration for peer '[^']*' \(from <HOST>\)$
+            ^%(__prefix_line)s%(log_prefix)s hacking attempt detected '<HOST>'$
+            ^%(__prefix_line)s%(log_prefix)s SecurityEvent="(?:FailedACL|InvalidAccountID|ChallengeResponseFailed|InvalidPassword)"(?:(?:,(?!RemoteAddress=)\w+="[^"]*")*|.*?),RemoteAddress="IPV[46]/(UDP|TCP|WS)/<HOST>/\d+"(?:,(?!RemoteAddress=)\w+="[^"]*")*$
+            ^%(__prefix_line)s%(log_prefix)s "Rejecting unknown SIP connection from <HOST>(?::\d+)?"$
+            ^%(__prefix_line)s%(log_prefix)s Request (?:'[^']*' )?from '(?:[^']*|.*?)' failed for '<HOST>(?::\d+)?'\s\(callid: [^\)]*\) - (?:No matching endpoint found|Not match Endpoint(?: Contact)? ACL|(?:Failed|Error) to authenticate)\s*$
+
+# FreePBX (todo: make optional in v.0.10):
+#            ^(%(__prefix_line)s|\[\]\s*WARNING%(__pid_re)s:?(?:\[C-[\da-f]*\])? )[^:]+: Friendly Scanner from <HOST>$
 
 ignoreregex =
 

config/filter.d/botsearch-common.conf

@@ -11,9 +11,9 @@ webmail = roundcube|(ext)?mail|horde|(v-?)?webmail
 
 phpmyadmin = (typo3/|xampp/|admin/|)(pma|(php)?[Mm]y[Aa]dmin)
 
-wordpress = wp-(login|signup)\.php
+wordpress = wp-(login|signup|admin)\.php
 
 # DEV Notes:
 # Taken from apache-botsearch filter
 # 
-# Author: Frantisek Sumsal
+# Author: Frantisek Sumsal

config/filter.d/common.conf

@@ -26,7 +26,7 @@ __daemon_re = [\[\(]?%(_daemon)s(?:\(\S+\))?[\]\)]?:?
 
 # extra daemon info
 # EXAMPLE: [ID 800047 auth.info]
-__daemon_extra_re = (?:\[ID \d+ \S+\])
+__daemon_extra_re = \[ID \d+ \S+\]
 
 # Combinations of daemon name and PID
 # EXAMPLES: sshd[31607], pop(pam_unix)[4920]
@@ -44,14 +44,18 @@ __md5hex = (?:[\da-f]{2}:){15}[\da-f]{2}
 
 # bsdverbose is where syslogd is started with -v or -vv and results in <4.3> or
 # <auth.info> appearing before the host as per testcases/files/logs/bsd/*.
-__bsd_syslog_verbose = (<[^.]+\.[^.]+>)
+__bsd_syslog_verbose = <[^.]+\.[^.]+>
+
+__vserver = @vserver_\S+
+
+__date_ambit = (?:\[\])
 
 # Common line prefixes (beginnings) which could be used in filters
 #
 #      [bsdverbose]? [hostname] [vserver tag] daemon_id spaces
 #
 # This can be optional (for instance if we match named native log files)
-__prefix_line = \s*%(__bsd_syslog_verbose)s?\s*(?:%(__hostname)s )?(?:%(__kernel_prefix)s )?(?:@vserver_\S+ )?%(__daemon_combs_re)s?\s%(__daemon_extra_re)s?\s*
+__prefix_line = %(__date_ambit)s?\s*(?:%(__bsd_syslog_verbose)s\s+)?(?:%(__hostname)s\s+)?(?:%(__kernel_prefix)s\s+)?(?:%(__vserver)s\s+)?(?:%(__daemon_combs_re)s\s+)?(?:%(__daemon_extra_re)s\s+)?
 
 # PAM authentication mechanism check for failures, e.g.: pam_unix, pam_sss,
 # pam_ldap

config/filter.d/courier-smtp.conf

@@ -13,7 +13,7 @@ before = common.conf
 _daemon = courieresmtpd
 
 failregex = ^%(__prefix_line)serror,relay=<HOST>,.*: 550 User (<.*> )?unknown\.?$
-            ^%(__prefix_line)serror,relay=<HOST>,msg="535 Authentication failed\.",cmd:( AUTH \S+)?( [0-9a-zA-Z\+/=]+)?$
+            ^%(__prefix_line)serror,relay=<HOST>,msg="535 Authentication failed\.",cmd:( AUTH \S+)?( [0-9a-zA-Z\+/=]+)?(?: \S+)$
 
 ignoreregex = 
 

config/filter.d/cyrus-imap.conf

@@ -13,7 +13,7 @@ before = common.conf
 
 _daemon = (?:cyrus/)?(?:imap(d|s)?|pop3(d|s)?)
 
-failregex = ^%(__prefix_line)sbadlogin: \S+ ?\[<HOST>\] \S+ .*?\[?SASL\(-13\): (authentication failure|user not found): .*\]?$
+failregex = ^%(__prefix_line)sbadlogin: [^\[]*\[<HOST>\] \S+ .*?\[?SASL\(-13\): (authentication failure|user not found): .*\]?$
 
 ignoreregex = 
 

config/filter.d/domino-smtp.conf

@@ -0,0 +1,47 @@
+# Fail2Ban configuration file for IBM Domino SMTP Server TASK to detect failed login attempts
+#
+# Author: Christian Brandlehner
+#
+# $Revision: 003 $
+#
+# Configuration:
+# Set the following Domino Server parameters in notes.ini:
+#       console_log_enabled=1
+#       log_sessions=2
+# You also have to use a date and time format supported by fail2ban. Recommended notes.ini configuration is:
+#       DateOrder=DMY
+#       DateSeparator=-
+#       ClockType=24_Hour
+#       TimeSeparator=:
+#
+# Depending on your locale you might have to tweak the date and time format so fail2ban can read the log
+
+#[INCLUDES]
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+#before = common.conf
+
+[Definition]
+# Option:  failregex
+# Notes.:  regex to match the password failure messages in the logfile. The
+#          host must be matched by a group named "host". The tag "<HOST>" can
+#          be used for standard IP/hostname matching and is only an alias for
+#          (?:::f{4,6}:)?(?P<host>\S+)
+# Values:  TEXT
+#
+# Sample log entries (used different time formats and an extra sample with process info in front of date)
+# 01-23-2009 19:54:51   SMTP Server: Authentication failed for user postmaster ; connecting host 1.2.3.4
+# [28325:00010-3735542592] 22-06-2014 09:56:12   smtp: postmaster [1.2.3.4] authentication failure using internet password
+# 08-09-2014 06:14:27   smtp: postmaster [1.2.3.4] authentication failure using internet password
+# 08-09-2014 06:14:27   SMTP Server: Authentication failed for user postmaster ; connecting host 1.2.3.4
+
+__prefix = (?:\[[^\]]+\])?\s+
+failregex = ^%(__prefix)sSMTP Server: Authentication failed for user .*? \; connecting host <HOST>$
+            ^%(__prefix)ssmtp: (?:[^\[]+ )*\[<HOST>\] authentication failure using internet password\s*$
+# Option:  ignoreregex
+# Notes.:  regex to ignore. If this regex matches, the line is ignored.
+# Values:  TEXT
+#
+
+ignoreregex =
+

config/filter.d/dovecot.conf

@@ -7,12 +7,15 @@ before = common.conf
 
 [Definition]
 
-_daemon = (auth|dovecot(-auth)?|auth-worker)
+_auth_worker = (?:dovecot: )?auth(?:-worker)?
+_daemon = (?:dovecot(?:-auth)?|auth)
 
-failregex = ^%(__prefix_line)s(%(__pam_auth)s(\(dovecot:auth\))?:)?\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=dovecot ruser=\S* rhost=<HOST>(\s+user=\S*)?\s*$
-            ^%(__prefix_line)s(pop3|imap)-login: (Info: )?(Aborted login|Disconnected)(: Inactivity)? \(((auth failed, \d+ attempts)( in \d+ secs)?|tried to use (disabled|disallowed) \S+ auth)\):( user=<\S*>,)?( method=\S+,)? rip=<HOST>(, lip=(\d{1,3}\.){3}\d{1,3})?(, TLS( handshaking(: SSL_accept\(\) failed: error:[\dA-F]+:SSL routines:[TLS\d]+_GET_CLIENT_HELLO:unknown protocol)?)?(: Disconnected)?)?(, session=<\S+>)?\s*$
-            ^%(__prefix_line)s(Info|dovecot: auth\(default\)|auth-worker\(\d+\)): pam\(\S+,<HOST>\): pam_authenticate\(\) failed: (User not known to the underlying authentication module: \d+ Time\(s\)|Authentication failure \(password mismatch\?\))\s*$
-            ^%(__prefix_line)s(auth|auth-worker\(\d+\)): (pam|passwd-file)\(\S+,<HOST>\): unknown user\s*$
+__prefregex = ^%(__prefix_line)s(?:%(_auth_worker)s(?:\([^\)]+\))?: )?(?:%(__pam_auth)s(?:\(dovecot:auth\))?: |(?:pop3|imap)-login: )?(?:Info: )?
+
+failregex = ^%(__prefregex)sauthentication failure; logname=\S* uid=\S* euid=\S* tty=dovecot ruser=\S* rhost=<HOST>(?:\s+user=\S*)?\s*$
+            ^%(__prefregex)s(?:Aborted login|Disconnected)(?::(?: [^ \(]+)+)? \((?:auth failed, \d+ attempts(?: in \d+ secs)?|tried to use (?:disabled|disallowed) \S+ auth)\):(?: user=<[^>]*>,)?(?: method=\S+,)? rip=<HOST>(?:[^>]*(?:, session=<\S+>)?)\s*$
+            ^%(__prefregex)spam\(\S+,<HOST>(?:,\S*)?\): pam_authenticate\(\) failed: (?:User not known to the underlying authentication module: \d+ Time\(s\)|Authentication failure \(password mismatch\?\)|Permission denied)\s*$
+            ^%(__prefregex)s[a-z\-]{3,15}\(\S*,<HOST>(?:,\S*)?\): (?:unknown user|invalid credentials)\s*$
 
 ignoreregex = 
 
@@ -22,9 +25,11 @@ journalmatch = _SYSTEMD_UNIT=dovecot.service
 
 # DEV Notes:
 # * the first regex is essentially a copy of pam-generic.conf
-# * Probably doesn't do dovecot sql/ldap backends properly
+# * Probably doesn't do dovecot sql/ldap backends properly (resolved in edit 21/03/2016)
 # * Removed the 'no auth attempts' log lines from the matches because produces
 #    lots of false positives on misconfigured MTAs making regexp unusable
 #
 # Author: Martin Waschbuesch
 #         Daniel Black (rewrote with begin and end anchors)
+#         Martin O'Neal (added LDAP authentication failure regex)
+#         Sergey G. Brester aka sebres (reviewed, optimized, IPv6-compatibility)

config/filter.d/exim-common.conf

@@ -9,8 +9,10 @@ after = exim-common.local
 
 [Definition]
 
-host_info = H=([\w.-]+ )?(\(\S+\) )?\[<HOST>\](:\d+)? (I=\[\S+\]:\d+ )?(U=\S+ )?(P=e?smtp )?
-pid = ( \[\d+\])?
+host_info_pre = (?:H=([\w.-]+ )?(?:\(\S+\) )?)?
+host_info_suf = (?::\d+)?(?: I=\[\S+\](:\d+)?)?(?: U=\S+)?(?: P=e?smtp)?(?: F=(?:<>|[^@]+@\S+))?\s
+host_info = %(host_info_pre)s\[<HOST>\]%(host_info_suf)s
+pid = (?: \[\d+\])?
 
 # DEV Notes:
 # From exim source code: ./src/receive.c:add_host_info_for_log

config/filter.d/exim.conf

@@ -14,10 +14,13 @@ before = exim-common.conf
 [Definition]
 
 failregex = ^%(pid)s %(host_info)ssender verify fail for <\S+>: (?:Unknown user|Unrouteable address|all relevant MX records point to non-existent hosts)\s*$
-             ^%(pid)s \w+ authenticator failed for (\S+ )?\(\S+\) \[<HOST>\](:\d+)?( I=\[\S+\](:\d+)?)?: 535 Incorrect authentication data( \(set_id=.*\)|: \d+ Time\(s\))?\s*$
-             ^%(pid)s %(host_info)sF=(<>|[^@]+@\S+) rejected RCPT [^@]+@\S+: (relay not permitted|Sender verify failed|Unknown user)\s*$
-             ^%(pid)s SMTP protocol synchronization error \([^)]*\): rejected (connection from|"\S+") %(host_info)s(next )?input=".*"\s*$
-             ^%(pid)s SMTP call from \S+ \[<HOST>\](:\d+)? (I=\[\S+\](:\d+)? )?dropped: too many nonmail commands \(last was "\S+"\)\s*$
+            ^%(pid)s \w+ authenticator failed for (?:[^\[\( ]* )?(?:\(\S*\) )?\[<HOST>\](?::\d+)?(?: I=\[\S+\](:\d+)?)?: 535 Incorrect authentication data( \(set_id=.*\)|: \d+ Time\(s\))?\s*$
+            ^%(pid)s %(host_info)srejected RCPT [^@]+@\S+: (?:relay not permitted|Sender verify failed|Unknown user|Unrouteable address)\s*$
+            ^%(pid)s SMTP protocol synchronization error \([^)]*\): rejected (?:connection from|"\S+") %(host_info)s(?:next )?input=".*"\s*$
+            ^%(pid)s SMTP call from \S+ %(host_info)sdropped: too many nonmail commands \(last was "\S+"\)\s*$
+            ^%(pid)s SMTP protocol error in "[^"]+(?:"+[^"]*(?="))*?" %(host_info)sAUTH command used when not advertised\s*$
+            ^%(pid)s no MAIL in SMTP connection from (?:[^\[\( ]* )?(?:\(\S*\) )?%(host_info)sD=\d\S*s(?: C=\S*)?\s*$
+            ^%(pid)s (?:[\w\-]+ )?SMTP connection from (?:[^\[\( ]* )?(?:\(\S*\) )?%(host_info)sclosed by DROP in ACL\s*$
 
 ignoreregex = 
 
@@ -30,3 +33,4 @@ ignoreregex =
 #
 # Author: Cyril Jaquier
 #         Daniel Black (rewrote with strong regexs)
+#         Martin O'Neal (added additional regexs to detect authentication failures, protocol errors, and drops)

config/filter.d/freeswitch.conf

@@ -16,7 +16,7 @@ failregex = ^\.\d+ \[WARNING\] sofia_reg\.c:\d+ SIP auth (failure|challenge) \((
 ignoreregex =
 
 # Author: Rupa SChomaker, soapee01, Daniel Black
-# http://wiki.freeswitch.org/wiki/Fail2ban
+# https://freeswitch.org/confluence/display/FREESWITCH/Fail2Ban
 # Thanks to Jim on mailing list of samples and guidance
 #
 # No need to match the following. Its a duplicate of the SIP auth regex.

config/filter.d/haproxy-http-auth.conf

@@ -0,0 +1,37 @@
+# Fail2Ban filter configuration file to match failed login attempts to
+# HAProxy HTTP Authentication protected servers.
+#
+# PLEASE NOTE - When a user first hits the HTTP Auth a 401 is returned by the server
+# which prompts their browser to ask for login details.
+# This initial 401 is logged by HAProxy.
+# In other words, even successful logins will have at least 1 fail regex match.
+# Please keep this in mind when setting findtime and maxretry for jails.
+#
+# Author: Jordan Moeser
+#
+
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
+
+[Definition]
+
+_daemon = haproxy
+
+# Option:  failregex
+# Notes.:  regex to match the password failures messages in the logfile. The
+#          host must be matched by a group named "host". The tag "<HOST>" can
+#          be used for standard IP/hostname matching and is only an alias for
+#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
+# Values:  TEXT
+#
+failregex = ^%(__prefix_line)s<HOST>.*<NOSRV> -1/-1/-1/-1/\+*\d* 401
+
+# Option:  ignoreregex
+# Notes.:  regex to ignore. If this regex matches, the line is ignored.
+# Values:  TEXT
+#
+ignoreregex =

config/filter.d/ignorecommands/apache-fakegooglebot

@@ -1,4 +1,4 @@
-#!/usr/bin/python
+#!/usr/bin/env fail2ban-python
 # Inspired by https://isc.sans.edu/forums/diary/When+Google+isnt+Google/15968/
 #
 # Written in Python to reuse built-in Python batteries and not depend on
@@ -26,7 +26,10 @@ def is_googlebot(ip):
     from fail2ban.server.filter import DNSUtils
 
     host = DNSUtils.ipToName(ip)
-    sys.exit(0 if (host and re.match('crawl-.*\.googlebot\.com', host)) else 1)
+    if not host or not re.match('.*\.google(bot)?\.com$', host):
+       sys.exit(1)
+    host_ips = DNSUtils.dnsToIp(host)
+    sys.exit(0 if ip in host_ips else 1)
 
 if __name__ == '__main__':
     is_googlebot(process_args(sys.argv))

config/filter.d/kerio.conf

@@ -3,9 +3,14 @@
 [Definition]
 
 failregex = ^ SMTP Spam attack detected from <HOST>,
-            ^ IP address <HOST> found in DNS blacklist \S+, mail from \S+ to \S+$
+            ^ IP address <HOST> found in DNS blacklist
             ^ Relay attempt from IP address <HOST>
             ^ Attempt to deliver to unknown recipient \S+, from \S+, IP address <HOST>$
+            ^ Failed SMTP login from <HOST>
+            ^ SMTP: User \S+ doesn't exist. Attempt from IP address <HOST>
+            ^ Client with IP address <HOST> has no reverse DNS entry, connection rejected before SMTP greeting$
+            ^ Administration login into Web Administration from <HOST> failed: IP address not allowed$
+            ^ Message from IP address <HOST>, sender \S+ rejected: sender domain does not exist$
 
 ignoreregex =
 
@@ -16,5 +21,6 @@ datepattern = ^\[%%d/%%b/%%Y %%H:%%M:%%S\]
 # DEV NOTES:
 # 
 # Author: A.P. Lawrence
+# Updated by: M. Bischoff <https://github.com/herrbischoff>
 #
 # Based off: http://aplawrence.com/Kerio/fail2ban.html

config/filter.d/lighttpd-auth.conf

@@ -3,7 +3,7 @@
 
 [Definition]
 
-failregex = ^: \(http_auth\.c\.\d+\) (password doesn\'t match .* username: .*|digest: auth failed for .*: wrong password|get_password failed), IP: <HOST>\s*$
+failregex = ^: \((?:http|mod)_auth\.c\.\d+\) (?:password doesn\'t match .* username: .*|digest: auth failed for .*: wrong password|get_password failed), IP: <HOST>\s*$
 
 ignoreregex = 
 

config/filter.d/mongodb-auth.conf

@@ -0,0 +1,49 @@
+# Fail2Ban filter for unsuccesfull MongoDB authentication attempts
+#
+# Logfile /var/log/mongodb/mongodb.log
+#
+# add setting in /etc/mongodb.conf
+# logpath=/var/log/mongodb/mongodb.log
+#
+# and use of the authentication
+# auth = true
+#
+
+[Definition]
+#failregex = ^\s+\[initandlisten\] connection accepted from <HOST>:\d+ \#(?P<__connid>\d+) \(1 connection now open\)<SKIPLINES>\s+\[conn(?P=__connid)\] Failed to authenticate\s+
+failregex = ^\s+\[conn(?P<__connid>\d+)\] Failed to authenticate [^\n]+<SKIPLINES>\s+\[conn(?P=__connid)\] end connection <HOST>
+
+ignoreregex =
+
+
+[Init]
+maxlines = 10
+
+# DEV Notes:
+#
+# Regarding the multiline regex:
+#
+# There can be a nunber of non-related lines between the first and second part
+# of this regex maxlines of 10 is quite generious.
+#
+# Note the capture __connid, includes the connection ID, used in second part of regex.
+#
+# The first regex is commented out (but will match also), because it is better to use
+# the host from "end connection" line (uncommented above):
+#  -  it has the same prefix, searching begins directly with failure message
+#     (so faster, because ignores success connections at all)
+#  -  it is not so vulnerable in case of possible race condition
+#
+# Log example:
+# 2016-10-20T09:54:27.108+0200 [initandlisten] connection accepted from 127.0.0.1:53276 #1 (1 connection now open)
+# 2016-10-20T09:54:27.109+0200 [conn1]  authenticate db: test { authenticate: 1, nonce: "xxx", user: "root", key: "xxx" }
+# 2016-10-20T09:54:27.110+0200 [conn1] Failed to authenticate root@test with mechanism MONGODB-CR: AuthenticationFailed UserNotFound Could not find user root@test
+# 2016-11-09T09:54:27.894+0100 [conn1] end connection 127.0.0.1:53276 (0 connections now open)
+# 2016-11-09T11:55:58.890+0100 [initandlisten] connection accepted from 127.0.0.1:54266 #1510 (1 connection now open)
+# 2016-11-09T11:55:58.892+0100 [conn1510]  authenticate db: admin { authenticate: 1, nonce: "xxx", user: "root", key: "xxx" }
+# 2016-11-09T11:55:58.892+0100 [conn1510] Failed to authenticate root@admin with mechanism MONGODB-CR: AuthenticationFailed key mismatch
+# 2016-11-09T11:55:58.894+0100 [conn1510] end connection 127.0.0.1:54266 (0 connections now open)
+#
+# Authors: Alexander Finkhäuser
+#          Sergey G. Brester (sebres)
+

config/filter.d/monit.conf

@@ -2,9 +2,20 @@
 #
 #
 
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
 [Definition]
 
-failregex = ^\[[A-Z]+\s+\]\s*error\s*:\s*Warning:\s+Client '<HOST>' supplied unknown user '\w+' accessing monit httpd$
-            ^\[[A-Z]+\s+\]\s*error\s*:\s*Warning:\s+Client '<HOST>' supplied wrong password for user '\w+' accessing monit httpd$
+_daemon = monit
 
+# Regexp for previous (accessing monit httpd) and new (access denied) versions
+failregex = ^\[[A-Z]+\s+\]\s*error\s*:\s*Warning:\s+Client '<HOST>' supplied (?:unknown user '[^']+'|wrong password for user '[^']*') accessing monit httpd$
+            ^%(__prefix_line)s\w+: access denied -- client <HOST>: (?:unknown user '[^']+'|wrong password for user '[^']*'|empty password)$
+
+# Ignore login with empty user (first connect, no user specified)
+# ignoreregex = %(__prefix_line)s\w+: access denied -- client <HOST>: (?:unknown user '')
 ignoreregex =

config/filter.d/murmur.conf

@@ -0,0 +1,28 @@
+# Fail2Ban filter for murmur/mumble-server
+#
+
+[INCLUDES]
+
+before = common.conf
+
+
+[Definition]
+
+_daemon = murmurd
+
+# N.B. If you allow users to have usernames that include the '>' character you
+#      should change this to match the regex assigned to the 'username'
+#      variable in your server config file (murmur.ini / mumble-server.ini).
+_usernameregex = [^>]+
+
+_prefix = <W>[\n\s]*(\.\d{3})?\s+\d+ => <\d+:%(_usernameregex)s\(-1\)> Rejected connection from <HOST>:\d+:
+
+failregex = ^%(_prefix)s Invalid server password$
+            ^%(_prefix)s Wrong certificate or password for existing user$
+
+ignoreregex =
+
+
+# DEV Notes:
+#
+# Author: Ross Brown

config/filter.d/mysqld-auth.conf

@@ -1,4 +1,4 @@
-# Fail2Ban filter for unsuccesfull MySQL authentication attempts
+# Fail2Ban filter for unsuccesful MySQL authentication attempts
 #
 #
 # To log wrong MySQL access attempts add to /etc/my.cnf in [mysqld]:
@@ -17,7 +17,7 @@ before = common.conf
 
 _daemon = mysqld
 
-failregex = ^%(__prefix_line)s(\d{6} \s?\d{1,2}:\d{2}:\d{2} )?\[Warning\] Access denied for user '\w+'@'<HOST>' (to database '[^']*'|\(using password: (YES|NO)\))*\s*$
+failregex = ^%(__prefix_line)s(?:\d+ |\d{6} \s?\d{1,2}:\d{2}:\d{2} )?\[\w+\] Access denied for user '[^']+'@'<HOST>' (to database '[^']*'|\(using password: (YES|NO)\))*\s*$
 
 ignoreregex = 
 

config/filter.d/nginx-http-auth.conf

@@ -4,7 +4,7 @@
 [Definition]
 
 
-failregex = ^ \[error\] \d+#\d+: \*\d+ user "\S+":? (password mismatch|was not found in ".*"), client: <HOST>, server: \S*, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"(, referrer: "\S+")?\s*$
+failregex = ^ \[error\] \d+#\d+: \*\d+ user "(?:[^"]+|.*?)":? (?:password mismatch|was not found in "[^\"]*"), client: <HOST>, server: \S*, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"(?:, referrer: "\S+")?\s*$
 
 ignoreregex = 
 

config/filter.d/nginx-limit-req.conf

@@ -0,0 +1,45 @@
+# Fail2ban filter configuration for nginx :: limit_req
+# used to ban hosts, that were failed through nginx by limit request processing rate 
+#
+# Author: Serg G. Brester (sebres)
+#
+# To use 'nginx-limit-req' filter you should have `ngx_http_limit_req_module`
+# and define `limit_req` and `limit_req_zone` as described in nginx documentation
+# http://nginx.org/en/docs/http/ngx_http_limit_req_module.html
+#
+# Example:
+#
+#   http {
+#     ...
+#     limit_req_zone $binary_remote_addr zone=lr_zone:10m rate=1r/s;
+#     ...
+#     # http, server, or location:
+#     location ... {
+#       limit_req zone=lr_zone burst=1 nodelay;
+#       ...
+#     }
+#     ...
+#   }
+#   ...
+#
+
+[Definition]
+
+# Specify following expression to define exact zones, if you want to ban IPs limited 
+# from specified zones only.
+# Example:
+#
+#   ngx_limit_req_zones = lr_zone|lr_zone2
+#
+ngx_limit_req_zones = [^"]+
+
+# Use following full expression if you should range limit request to specified 
+# servers, requests, referrers etc. only :
+#
+# failregex = ^\s*\[[a-z]+\] \d+#\d+: \*\d+ limiting requests, excess: [\d\.]+ by zone "(?:%(ngx_limit_req_zones)s)", client: <HOST>, server: \S*, request: "\S+ \S+ HTTP/\d+\.\d+", host: "\S+"(, referrer: "\S+")?\s*$
+
+# Shortly, much faster and stable version of regexp:
+failregex = ^\s*\[[a-z]+\] \d+#\d+: \*\d+ limiting requests, excess: [\d\.]+ by zone "(?:%(ngx_limit_req_zones)s)", client: <HOST>,
+
+ignoreregex = 
+

config/filter.d/nsd.conf

@@ -22,7 +22,7 @@ _daemon = nsd
 #          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
 # Values:  TEXT
 
-failregex =  ^\[\]%(__prefix_line)sinfo: ratelimit block .* query <HOST> TYPE255$
-              ^\[\]%(__prefix_line)sinfo: .* <HOST> refused, no acl matches\.$
+failregex =  ^%(__prefix_line)sinfo: ratelimit block .* query <HOST> TYPE255$
+             ^%(__prefix_line)sinfo: .* <HOST> refused, no acl matches\.$
 
 ignoreregex =

config/filter.d/openhab.conf

@@ -0,0 +1,16 @@
+# Openhab brute force auth filter: /etc/fail2ban/filter.d/openhab.conf:
+#
+# Block IPs trying to auth openhab by web or rest api
+#
+# Matches e.g.
+# 12.34.33.22 -  -  [26/sept./2015:18:04:43 +0200] "GET /openhab.app HTTP/1.1" 401 1382 
+# 175.18.15.10 -  -  [02/sept./2015:00:11:31 +0200] "GET /rest/bindings HTTP/1.1" 401 1384
+
+[Definition] 
+failregex = ^<HOST>\s+-\s+-\s+\[\]\s+"[A-Z]+ .*" 401 \d+\s*$
+
+[Init]
+datepattern = %%d/%%b[^/]*/%%Y:%%H:%%M:%%S %%z
+
+
+

config/filter.d/phpmyadmin-syslog.conf

@@ -0,0 +1,18 @@
+# Fail2Ban fitler for the phpMyAdmin-syslog
+#
+
+[INCLUDES]
+
+before = common.conf
+
+[Definition]
+
+_daemon = phpMyAdmin
+
+failregex = ^%(__prefix_line)suser denied: (?:\S+|.*?) \(mysql-denied\) from <HOST>\s*$
+
+ignoreregex =
+
+
+# Author: Pavel Mihadyuk
+# Regex fixes: Serg G. Brester

config/filter.d/postfix-rbl.conf

@@ -10,9 +10,10 @@ before = common.conf
 
 [Definition]
 
-_daemon = postfix/smtpd
+_daemon = postfix(-\w+)?/smtpd
+_port = (?::\d+)?
 
-failregex = ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 454 4\.7\.1 Service unavailable; Client host \[\S+\] blocked using .* from=<\S*> to=<\S+> proto=ESMTP helo=<\S*>$
+failregex = ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]%(_port)s: [45]54 [45]\.7\.1 Service unavailable; Client host \[\S+\] blocked\b
 
 ignoreregex =
 

config/filter.d/postfix-sasl.conf

@@ -7,9 +7,10 @@ before = common.conf
 
 [Definition]
 
-_daemon = postfix/(submission/)?smtp(d|s)
+_daemon = postfix(-\w+)?/(?:submission/|smtps/)?smtp[ds]
+_port = (?::\d+)?
 
-failregex = ^%(__prefix_line)swarning: [-._\w]+\[<HOST>\]: SASL ((?i)LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed(: [ A-Za-z0-9+/:]*={0,2})?\s*$
+failregex = ^%(__prefix_line)swarning: [-._\w]+\[<HOST>\]%(_port)s: SASL ((?i)LOGIN|PLAIN|(?:CRAM|DIGEST)-MD5) authentication failed(:[ A-Za-z0-9+/:]*={0,2})?\s*$
 
 ignoreregex = authentication failed: Connection lost to authentication server$
 

config/filter.d/postfix.conf

@@ -10,13 +10,16 @@ before = common.conf
 
 [Definition]
 
-_daemon = postfix/(submission/)?smtp(d|s)
+_daemon = postfix(-\w+)?/(?:submission/|smtps/)?smtp[ds]
+_port = (?::\d+)?
 
-failregex = ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 554 5\.7\.1 .*$
-            ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 450 4\.7\.1 Client host rejected: cannot find your hostname, (\[\S*\]); from=<\S*> to=<\S+> proto=ESMTP helo=<\S*>$
-            ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]: 450 4\.7\.1 : Helo command rejected: Host not found; from=<> to=<> proto=ESMTP helo= *$
-            ^%(__prefix_line)sNOQUEUE: reject: VRFY from \S+\[<HOST>\]: 550 5\.1\.1 .*$
-            ^%(__prefix_line)simproper command pipelining after \S+ from [^[]*\[<HOST>\]:?$
+failregex = ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]%(_port)s: 554 5\.7\.1 .*$
+            ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]%(_port)s: 450 4\.7\.1 Client host rejected: cannot find your hostname, (\[\S*\]); from=<\S*> to=<\S+> proto=ESMTP helo=<\S*>$
+            ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]%(_port)s: 450 4\.7\.1 : Helo command rejected: Host not found; from=<> to=<> proto=ESMTP helo= *$
+            ^%(__prefix_line)sNOQUEUE: reject: EHLO from \S+\[<HOST>\]%(_port)s: 504 5\.5\.2 <\S+>: Helo command rejected: need fully-qualified hostname;
+            ^%(__prefix_line)sNOQUEUE: reject: VRFY from \S+\[<HOST>\]%(_port)s: 550 5\.1\.1 .*$
+            ^%(__prefix_line)sNOQUEUE: reject: RCPT from \S+\[<HOST>\]%(_port)s: 450 4\.1\.8 <\S*>: Sender address rejected: Domain not found; from=<\S*> to=<\S+> proto=ESMTP helo=<\S*>$
+            ^%(__prefix_line)simproper command pipelining after \S+ from [^[]*\[<HOST>\]%(_port)s:?$
 
 ignoreregex = 
 

config/filter.d/proftpd.conf

@@ -18,10 +18,13 @@ __suffix_failed_login = (User not authorized for login|No such user found|Incorr
 
 failregex = ^%(__prefix_line)s%(__hostname)s \(\S+\[<HOST>\]\)[: -]+ USER .*: no such user found from \S+ \[\S+\] to \S+:\S+ *$
             ^%(__prefix_line)s%(__hostname)s \(\S+\[<HOST>\]\)[: -]+ USER .* \(Login failed\): %(__suffix_failed_login)s\s*$
-            ^%(__prefix_line)s%(__hostname)s \(\S+\[<HOST>\]\)[: -]+ SECURITY VIOLATION: .* login attempted\. *$
+            ^%(__prefix_line)s%(__hostname)s \(\S+\[<HOST>\]\)[: -]+ SECURITY VIOLATION: .* login attempted\.? *$
             ^%(__prefix_line)s%(__hostname)s \(\S+\[<HOST>\]\)[: -]+ Maximum login attempts \(\d+\) exceeded *$
 
 ignoreregex = 
 
+[Init]
+journalmatch = _SYSTEMD_UNIT=proftpd.service
+
 # Author: Yaroslav Halchenko
 #         Daniel Black - hardening of regex

config/filter.d/pure-ftpd.conf

@@ -21,6 +21,10 @@ failregex = ^%(__prefix_line)s\(.+?@<HOST>\) \[WARNING\] %(__errmsg)s\s*$
 
 ignoreregex = 
 
+[Init]
+
+journalmatch = _SYSTEMD_UNIT=pure-ftpd.service + _COMM=pure-ftpd
+
 # Author: Cyril Jaquier
 # Modified: Yaroslav Halchenko for pure-ftpd
 # Documentation thanks to Blake on http://www.fail2ban.org/wiki/index.php?title=Fail2ban:Community_Portal

config/filter.d/screensharingd.conf

@@ -0,0 +1,31 @@
+# Fail2Ban configuration file
+#
+# Author: Simon Brown
+#
+# Filter for Mac OS X Screen Sharing service
+
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
+
+[Definition]
+
+_daemon = screensharingd
+
+# Option:  failregex
+# Notes.:  regex to match the password failures messages in the logfile. The
+#          host must be matched by a group named "host". The tag "<HOST>" can
+#          be used for standard IP/hostname matching and is only an alias for
+#          (?:::f{4,6}:)?(?P<host>[\w\-.^_]+)
+# Values:  TEXT
+#
+failregex = ^%(__prefix_line)sAuthentication: FAILED :: User Name: .+ :: Viewer Address: <HOST> :: Type: DH$
+
+# Option:  ignoreregex
+# Notes.:  regex to ignore. If this regex matches, the line is ignored.
+# Values:  TEXT
+#
+ignoreregex =

config/filter.d/sendmail-auth.conf

@@ -7,12 +7,16 @@ before = common.conf
 
 [Definition]
 
-_daemon = (?:sm-(mta|acceptingconnections))
+_daemon = (?:sendmail|sm-(?:mta|acceptingconnections))
 
 failregex = ^%(__prefix_line)s\w{14}: (\S+ )?\[<HOST>\]( \(may be forged\))?: possible SMTP attack: command=AUTH, count=\d+$
 
 ignoreregex =
 
+[Init]
+
+journalmatch = _SYSTEMD_UNIT=sendmail.service
+
 # DEV Notes:
 #
 # Author: Daniel Black

config/filter.d/sendmail-reject.conf

@@ -23,9 +23,9 @@ _daemon = (?:(sm-(mta|acceptingconnections)|sendmail))
 
 failregex = ^%(__prefix_line)s\w{14}: ruleset=check_rcpt, arg1=(?P<email><\S+@\S+>), relay=(\S+ )?\[<HOST>\]( \(may be forged\))?, reject=(550 5\.7\.1 (?P=email)\.\.\. Relaying denied\. (IP name possibly forged \[(\d+\.){3}\d+\]|Proper authentication required\.|IP name lookup failed \[(\d+\.){3}\d+\])|553 5\.1\.8 (?P=email)\.\.\. Domain of sender address \S+ does not exist|550 5\.[71]\.1 (?P=email)\.\.\. (Rejected: .*|User unknown))$
             ^%(__prefix_line)sruleset=check_relay, arg1=(?P<dom>\S+), arg2=<HOST>, relay=((?P=dom) )?\[(\d+\.){3}\d+\]( \(may be forged\))?, reject=421 4\.3\.2 (Connection rate limit exceeded\.|Too many open connections\.)$
-            ^%(__prefix_line)s\w{14}: rejecting commands from  (\S+ )?\[<HOST>\] due to pre-greeting traffic after \d+ seconds$
+            ^%(__prefix_line)s\w{14}: rejecting commands from (\S* )?\[<HOST>\] due to pre-greeting traffic after \d+ seconds$
             ^%(__prefix_line)s\w{14}: (\S+ )?\[<HOST>\]: ((?i)expn|vrfy) \S+ \[rejected\]$
-            ^(?P<__prefix>%(__prefix_line)s\w+: )<[^@]+@[^>]+>\.\.\. No such user here<SKIPLINES>(?P=__prefix)from=<[^@]+@[^>]+>, size=\d+, class=\d+, nrcpts=\d+, bodytype=\w+, proto=E?SMTP, daemon=MTA, relay=\S+ \[<HOST>\]$
+            ^(?P<__prefix>%(__prefix_line)s\w+: )<[^@]+@[^>]+>\.\.\. No such user here$<SKIPLINES>^(?P=__prefix)from=<[^@]+@[^>]+>, size=\d+, class=\d+, nrcpts=\d+, bodytype=\w+, proto=E?SMTP, daemon=MTA, relay=\S+ \[<HOST>\]$
 
 
 ignoreregex =
@@ -33,6 +33,8 @@ ignoreregex =
 
 [Init]
 
+journalmatch = _SYSTEMD_UNIT=sendmail.service
+
 # "maxlines" is number of log lines to buffer for multi-line regex searches
 maxlines = 10
 

config/filter.d/slapd.conf

@@ -0,0 +1,25 @@
+# slapd (Stand-alone LDAP Daemon) openldap daemon filter
+#
+# Detecting invalid credentials: error code 49
+# http://www.openldap.org/doc/admin24/appendix-ldap-result-codes.html#invalidCredentials (49)
+
+[INCLUDES]
+
+# Read common prefixes. If any customizations available -- read them from
+# common.local
+before = common.conf
+
+[Definition]
+
+_daemon = slapd
+
+failregex = ^(?P<__prefix>%(__prefix_line)s)conn=(?P<_conn_>\d+) fd=\d+ ACCEPT from IP=<HOST>:\d{1,5} \(IP=\S+\)\s*<SKIPLINES>(?P=__prefix)conn=(?P=_conn_) op=\d+ RESULT(?:\s(?!err)\S+=\S*)* err=49 text=[\w\s]*$
+
+ignoreregex =
+
+[Init]
+
+# "maxlines" is number of log lines to buffer for multi-line regex searches
+maxlines = 20
+
+# Author: Andrii Melnyk

config/filter.d/sogo-auth.conf

@@ -4,7 +4,7 @@
 
 [Definition]
 
-failregex = ^ sogod \[\d+\]: SOGoRootPage Login from '<HOST>' for user '.*' might not have worked( - password policy: \d*  grace: -?\d*  expire: -?\d*  bound: -?\d*)?\s*$
+failregex = ^ sogod \[\d+\]: SOGoRootPage Login from '<HOST>(?:,[^']*)?' for user '[^']*' might not have worked( - password policy: \d*  grace: -?\d*  expire: -?\d*  bound: -?\d*)?\s*$
 
 ignoreregex = 
 

config/filter.d/sshd-aggressive.conf

@@ -0,0 +1,11 @@
+# Fail2Ban aggressive ssh filter for at attempted exploit
+#
+# Includes failregex of both sshd and sshd-ddos filters
+#
+[INCLUDES]
+
+before = sshd.conf
+
+[Definition]
+
+mode = %(aggressive)s

config/filter.d/sshd-ddos.conf

@@ -10,20 +10,8 @@
 
 [INCLUDES]
 
-# Read common prefixes. If any customizations available -- read them from
-# common.local
-before = common.conf
+before = sshd.conf
 
 [Definition]
 
-_daemon = sshd
-
-failregex = ^%(__prefix_line)sDid not receive identification string from <HOST>\s*$
-
-ignoreregex = 
-
-[Init]
-
-journalmatch = _SYSTEMD_UNIT=sshd.service + _COMM=sshd
-
-# Author: Yaroslav Halchenko
+mode = %(ddos)s

config/filter.d/sshd.conf

@@ -14,26 +14,54 @@
 # common.local
 before = common.conf
 
-[Definition]
+[DEFAULT]
 
 _daemon = sshd
 
-failregex = ^%(__prefix_line)s(?:error: PAM: )?[aA]uthentication (?:failure|error) for .* from <HOST>( via \S+)?\s*$
-            ^%(__prefix_line)s(?:error: PAM: )?User not known to the underlying authentication module for .* from <HOST>\s*$
-            ^%(__prefix_line)sFailed \S+ for .*? from <HOST>(?: port \d*)?(?: ssh\d*)?(: (ruser .*|(\S+ ID \S+ \(serial \d+\) CA )?\S+ %(__md5hex)s(, client user ".*", client host ".*")?))?\s*$
-            ^%(__prefix_line)sROOT LOGIN REFUSED.* FROM <HOST>\s*$
-            ^%(__prefix_line)s[iI](?:llegal|nvalid) user .* from <HOST>\s*$
-            ^%(__prefix_line)sUser .+ from <HOST> not allowed because not listed in AllowUsers\s*$
-            ^%(__prefix_line)sUser .+ from <HOST> not allowed because listed in DenyUsers\s*$
-            ^%(__prefix_line)sUser .+ from <HOST> not allowed because not in any group\s*$
-            ^%(__prefix_line)srefused connect from \S+ \(<HOST>\)\s*$
-            ^%(__prefix_line)sReceived disconnect from <HOST>: 3: \S+: Auth fail$
-            ^%(__prefix_line)sUser .+ from <HOST> not allowed because a group is listed in DenyGroups\s*$
-            ^%(__prefix_line)sUser .+ from <HOST> not allowed because none of user's groups are listed in AllowGroups\s*$
-            ^(?P<__prefix>%(__prefix_line)s)User .+ not allowed because account is locked<SKIPLINES>(?P=__prefix)(?:error: )?Received disconnect from <HOST>: 11: .+ \[preauth\]$
-            ^(?P<__prefix>%(__prefix_line)s)Disconnecting: Too many authentication failures for .+? \[preauth\]<SKIPLINES>(?P=__prefix)(?:error: )?Connection closed by <HOST> \[preauth\]$
-            ^(?P<__prefix>%(__prefix_line)s)Connection from <HOST> port \d+(?: on \S+ port \d+)?<SKIPLINES>(?P=__prefix)Disconnecting: Too many authentication failures for .+? \[preauth\]$
-            ^%(__prefix_line)spam_unix\(sshd:auth\):\s+authentication failure;\s*logname=\S*\s*uid=\d*\s*euid=\d*\s*tty=\S*\s*ruser=\S*\s*rhost=<HOST>\s.*$
+# optional prefix (logged from several ssh versions) like "error: ", "error: PAM: " or "fatal: "
+__pref = (?:(?:error|fatal): (?:PAM: )?)?
+# optional suffix (logged from several ssh versions) like " [preauth]"
+__suff = (?: \[preauth\])?\s*
+__on_port_opt = (?: port \d+)?(?: on \S+(?: port \d+)?)?
+
+# single line prefix:
+__prefix_line_sl = %(__prefix_line)s%(__pref)s
+# multi line prefixes (for first and second lines):
+__prefix_line_ml1 = (?P<__prefix>%(__prefix_line)s)%(__pref)s
+__prefix_line_ml2 = %(__suff)s$<SKIPLINES>^(?P=__prefix)%(__pref)s
+
+mode = %(normal)s
+
+normal = ^%(__prefix_line_sl)s[aA]uthentication (?:failure|error|failed) for .* from <HOST>( via \S+)?\s*%(__suff)s$
+         ^%(__prefix_line_sl)sUser not known to the underlying authentication module for .* from <HOST>\s*%(__suff)s$
+         ^%(__prefix_line_sl)sFailed \S+ for (?P<cond_inv>invalid user )?(?P<user>(?P<cond_user>\S+)|(?(cond_inv)(?:(?! from ).)*?|[^:]+)) from <HOST>%(__on_port_opt)s(?: ssh\d*)?(?(cond_user): |(?:(?:(?! from ).)*)$)
+         ^%(__prefix_line_sl)sROOT LOGIN REFUSED.* FROM <HOST>\s*%(__suff)s$
+         ^%(__prefix_line_sl)s[iI](?:llegal|nvalid) user .*? from <HOST>%(__on_port_opt)s\s*$
+         ^%(__prefix_line_sl)sUser .+ from <HOST> not allowed because not listed in AllowUsers\s*%(__suff)s$
+         ^%(__prefix_line_sl)sUser .+ from <HOST> not allowed because listed in DenyUsers\s*%(__suff)s$
+         ^%(__prefix_line_sl)sUser .+ from <HOST> not allowed because not in any group\s*%(__suff)s$
+         ^%(__prefix_line_sl)srefused connect from \S+ \(<HOST>\)\s*%(__suff)s$
+         ^%(__prefix_line_sl)sReceived disconnect from <HOST>%(__on_port_opt)s:\s*3: .*: Auth fail%(__suff)s$
+         ^%(__prefix_line_sl)sUser .+ from <HOST> not allowed because a group is listed in DenyGroups\s*%(__suff)s$
+         ^%(__prefix_line_sl)sUser .+ from <HOST> not allowed because none of user's groups are listed in AllowGroups\s*%(__suff)s$
+         ^%(__prefix_line_sl)spam_unix\(sshd:auth\):\s+authentication failure;\s*logname=\S*\s*uid=\d*\s*euid=\d*\s*tty=\S*\s*ruser=\S*\s*rhost=<HOST>\s.*%(__suff)s$
+         ^%(__prefix_line_sl)s(error: )?maximum authentication attempts exceeded for .* from <HOST>%(__on_port_opt)s(?: ssh\d*)? \[preauth\]$
+         ^%(__prefix_line_ml1)sUser .+ not allowed because account is locked%(__prefix_line_ml2)sReceived disconnect from <HOST>: 11: .+%(__suff)s$
+         ^%(__prefix_line_ml1)sDisconnecting: Too many authentication failures for .+?%(__prefix_line_ml2)sConnection closed by <HOST>%(__suff)s$
+         ^%(__prefix_line_ml1)sConnection from <HOST>%(__on_port_opt)s%(__prefix_line_ml2)sDisconnecting: Too many authentication failures for .+%(__suff)s$
+
+ddos =   ^%(__prefix_line_sl)sDid not receive identification string from <HOST>%(__suff)s$
+         ^%(__prefix_line_sl)sReceived disconnect from <HOST>%(__on_port_opt)s:\s*14: No supported authentication methods available%(__suff)s$
+         ^%(__prefix_line_sl)sUnable to negotiate with <HOST>%(__on_port_opt)s: no matching (?:cipher|key exchange method) found.
+         ^%(__prefix_line_ml1)sConnection from <HOST>%(__on_port_opt)s%(__prefix_line_ml2)sUnable to negotiate a (?:cipher|key exchange method)%(__suff)s$
+         ^%(__prefix_line_ml1)sSSH: Server;Ltype: (?:Authname|Version|Kex);Remote: <HOST>-\d+;[A-Z]\w+:.*%(__prefix_line_ml2)sRead from socket failed: Connection reset by peer%(__suff)s$
+
+aggressive = %(normal)s
+             %(ddos)s
+
+[Definition]
+
+failregex = %(mode)s
 
 ignoreregex = 
 

config/filter.d/suhosin.conf

@@ -17,7 +17,7 @@ _daemon = (?:lighttpd|suhosin)
 
 _lighttpd_prefix = (?:\(mod_fastcgi\.c\.\d+\) FastCGI-stderr:\s)
 
-failregex = ^%(__prefix_line)s%(_lighttpd_prefix)s?ALERT - .* \(attacker '<HOST>', file '.*'(?:, line \d+)?\)$
+failregex = ^%(__prefix_line)s%(_lighttpd_prefix)s?ALERT - .*? \(attacker '<HOST>', file '[^']*'(?:, line \d+)?\)$
 
 ignoreregex = 
 

config/filter.d/vsftpd.conf

@@ -14,7 +14,7 @@ __pam_re=\(?%(__pam_auth)s(?:\(\S+\))?\)?:?
 _daemon =  vsftpd
 
 failregex = ^%(__prefix_line)s%(__pam_re)s\s+authentication failure; logname=\S* uid=\S* euid=\S* tty=(ftp)? ruser=\S* rhost=<HOST>(?:\s+user=.*)?\s*$
-            ^ \[pid \d+\] \[.+\] FAIL LOGIN: Client "<HOST>"\s*$
+            ^ \[pid \d+\] \[[^\]]+\] FAIL LOGIN: Client "<HOST>"(?:\s*$|,)
 
 ignoreregex = 
 

config/filter.d/zoneminder.conf

@@ -0,0 +1,21 @@
+# Fail2Ban filter for Zoneminder login failures
+
+[INCLUDES]
+before = apache-common.conf
+
+[Definition]
+
+# pattern: [Wed Apr 27 23:12:07.736196 2016] [:error] [pid 2460] [client 10.1.1.1:47296] WAR [Login denied for user "test"], referer: https://zoneminderurl/index.php
+#
+#
+# Option:  failregex
+# Notes.:  regex to match the password failure messages in the logfile.
+
+failregex = ^%(_apache_error_client)s WAR \[Login denied for user "[^"]*"\]
+
+ignoreregex =
+
+# Notes:
+#	Tested on Zoneminder 1.29.0
+#
+# Author: John Marzella

config/jail.conf

@@ -46,7 +46,7 @@ before = paths-debian.conf
 
 # "ignoreip" can be an IP address, a CIDR mask or a DNS host. Fail2ban will not
 # ban a host which matches an address in this list. Several addresses can be
-# defined using space separator.
+# defined using space (and/or comma) separator.
 ignoreip = 127.0.0.1/8
 
 # External command that will take an tagged arguments to ignore, e.g. <ip>,
@@ -80,7 +80,7 @@ maxretry = 5
 # auto:      will try to use the following backends, in order:
 #              pyinotify, gamin, polling.
 #
-# Note: if systemd backend is choses as the default but you enable a jail
+# Note: if systemd backend is chosen as the default but you enable a jail
 #       for which logs are present only in its own log files, specify some other
 #       backend for that jail (e.g. polling) and provide empty value for
 #       journalmatch. See https://github.com/fail2ban/fail2ban/issues/959#issuecomment-74901200
@@ -94,6 +94,7 @@ backend = auto
 #        but it will be logged as a warning.
 # no:    if a hostname is encountered, will not be used for banning,
 #        but it will be logged as info.
+# raw:   use raw value (no hostname), allow use it for no-host filters/actions (example user)
 usedns = warn
 
 # "logencoding" specifies the encoding of the log files handled by the jail
@@ -146,6 +147,9 @@ chain = INPUT
 # Usually should be overridden in a particular jail
 port = 0:65535
 
+# Format of user-agent https://tools.ietf.org/html/rfc7231#section-5.5.3
+fail2ban_agent = Fail2Ban/%(fail2ban_version)s
+
 #
 # Action shortcuts. To be used to define action parameter
 
@@ -154,18 +158,19 @@ port = 0:65535
 # action_* variables. Can be overridden globally or per
 # section within jail.local file
 banaction = iptables-multiport
+banaction_allports = iptables-allports
 
 # The simplest action to take: ban only
 action_ = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
 
 # ban & send an e-mail with whois report to the destemail.
 action_mw = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
-            %(mta)s-whois[name=%(__name__)s, dest="%(destemail)s", protocol="%(protocol)s", chain="%(chain)s"]
+            %(mta)s-whois[name=%(__name__)s, sender="%(sender)s", dest="%(destemail)s", protocol="%(protocol)s", chain="%(chain)s"]
 
 # ban & send an e-mail with whois report and relevant log lines
 # to the destemail.
 action_mwl = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(port)s", protocol="%(protocol)s", chain="%(chain)s"]
-             %(mta)s-whois-lines[name=%(__name__)s, dest="%(destemail)s", logpath=%(logpath)s, chain="%(chain)s"]
+             %(mta)s-whois-lines[name=%(__name__)s, sender="%(sender)s", dest="%(destemail)s", logpath=%(logpath)s, chain="%(chain)s"]
 
 # See the IMPORTANT note in action.d/xarf-login-attack for when to use this action
 #
@@ -177,16 +182,17 @@ action_xarf = %(banaction)s[name=%(__name__)s, bantime="%(bantime)s", port="%(po
 # ban IP on CloudFlare & send an e-mail with whois report and relevant log lines
 # to the destemail.
 action_cf_mwl = cloudflare[cfuser="%(cfemail)s", cftoken="%(cfapikey)s"]
-                %(mta)s-whois-lines[name=%(__name__)s, dest="%(destemail)s", logpath=%(logpath)s, chain="%(chain)s"]
+                %(mta)s-whois-lines[name=%(__name__)s, sender="%(sender)s", dest="%(destemail)s", logpath=%(logpath)s, chain="%(chain)s"]
 
 # Report block via blocklist.de fail2ban reporting service API
 # 
-# See the IMPORTANT note in action.d/blocklist_de.conf for when to
-# use this action. Create a file jail.d/blocklist_de.local containing
-# [Init]
-# blocklist_de_apikey = {api key from registration]
+# See the IMPORTANT note in action.d/blocklist_de.conf for when to use this action.
+# Specify expected parameters in file action.d/blocklist_de.local or if the interpolation
+# `action_blocklist_de` used for the action, set value of `blocklist_de_apikey`
+# in your `jail.local` globally (section [DEFAULT]) or per specific jail section (resp. in 
+# corresponding jail.d/my-jail.local file).
 #
-action_blocklist_de  = blocklist_de[email="%(sender)s", service=%(filter)s, apikey="%(blocklist_de_apikey)s"]
+action_blocklist_de  = blocklist_de[email="%(sender)s", service=%(filter)s, apikey="%(blocklist_de_apikey)s", agent="%(fail2ban_agent)s"]
 
 # Report ban via badips.com, and use as blacklist
 #
@@ -196,7 +202,11 @@ action_blocklist_de  = blocklist_de[email="%(sender)s", service=%(filter)s, apik
 # NOTE: This action relies on banaction being present on start and therefore
 # should be last action defined for a jail.
 #
-action_badips = badips.py[category="%(name)s", banaction="%(banaction)s"]
+action_badips = badips.py[category="%(__name__)s", banaction="%(banaction)s", agent="%(fail2ban_agent)s"]
+#
+# Report ban via badips.com (uses action.d/badips.conf for reporting only)
+#
+action_badips_report = badips[category="%(__name__)s", agent="%(fail2ban_agent)s"]
 
 # Choose default action.  To change, just override value of 'action' with the
 # interpolation to the chosen action shortcut (e.g.  action_mw, action_mwl, etc) in jail.local
@@ -214,8 +224,11 @@ action = %(action_)s
 
 [sshd]
 
+# To use more aggressive sshd filter (inclusive sshd-ddos failregex):
+#filter = sshd-aggressive
 port    = ssh
 logpath = %(sshd_log)s
+backend = %(sshd_backend)s
 
 
 [sshd-ddos]
@@ -224,19 +237,20 @@ logpath = %(sshd_log)s
 # in the body.
 port    = ssh
 logpath = %(sshd_log)s
+backend = %(sshd_backend)s
 
 
 [dropbear]
 
 port     = ssh
 logpath  = %(dropbear_log)s
+backend  = %(dropbear_backend)s
 
 
 [selinux-ssh]
 
 port     = ssh
 logpath  = %(auditd_log)s
-maxretry = 5
 
 
 #
@@ -262,7 +276,6 @@ maxretry = 1
 
 port     = http,https
 logpath  = %(apache_error_log)s
-maxretry = 6
 
 
 [apache-overflows]
@@ -300,23 +313,41 @@ port     = http,https
 logpath  = %(apache_error_log)s
 maxretry = 2
 
+
 [apache-shellshock]
 
 port    = http,https
 logpath = %(apache_error_log)s
 maxretry = 1
 
+
+[openhab-auth]
+
+filter = openhab
+action = iptables-allports[name=NoAuthFailures]
+logpath = /opt/openhab/logs/request.log
+
+
 [nginx-http-auth]
 
 port    = http,https
 logpath = %(nginx_error_log)s
 
+# To use 'nginx-limit-req' jail you should have `ngx_http_limit_req_module` 
+# and define `limit_req` and `limit_req_zone` as described in nginx documentation
+# http://nginx.org/en/docs/http/ngx_http_limit_req_module.html
+# or for example see in 'config/filter.d/nginx-limit-req.conf'
+[nginx-limit-req]
+port    = http,https
+logpath = %(nginx_error_log)s
+
 [nginx-botsearch]
 
 port     = http,https
 logpath  = %(nginx_error_log)s
 maxretry = 2
 
+
 # Ban attackers that try to use PHP's URL-fopen() functionality
 # through GET/POST variables. - Experimental, with more than a year
 # of usage in production environments.
@@ -348,7 +379,7 @@ logpath = %(lighttpd_error_log)s
 [roundcube-auth]
 
 port     = http,https
-logpath  = logpath = %(roundcube_errors_log)s
+logpath  = %(roundcube_errors_log)s
 
 
 [openwebmail]
@@ -381,7 +412,6 @@ logpath  = /var/log/sogo/sogo.log
 
 logpath  = /var/log/tine20/tine20.log
 port     = http,https
-maxretry = 5
 
 
 #
@@ -393,6 +423,7 @@ maxretry = 5
 
 port     = http,https
 logpath  = %(syslog_daemon)s
+backend  = %(syslog_backend)s
 
 [guacamole]
 
@@ -401,7 +432,6 @@ logpath  = /var/log/tomcat*/catalina.out
 
 [monit]
 #Ban clients brute-forcing the monit gui login
-filter   = monit
 port = 2812
 logpath  = /var/log/monit
 
@@ -410,12 +440,14 @@ logpath  = /var/log/monit
 
 port    = 10000
 logpath = %(syslog_authpriv)s
+backend = %(syslog_backend)s
 
 
 [froxlor-auth]
 
 port    = http,https
 logpath  = %(syslog_authpriv)s
+backend  = %(syslog_backend)s
 
 
 #
@@ -444,27 +476,28 @@ logpath = /var/log/3proxy.log
 
 port     = ftp,ftp-data,ftps,ftps-data
 logpath  = %(proftpd_log)s
+backend  = %(proftpd_backend)s
 
 
 [pure-ftpd]
 
 port     = ftp,ftp-data,ftps,ftps-data
 logpath  = %(pureftpd_log)s
-maxretry = 6
+backend  = %(pureftpd_backend)s
 
 
 [gssftpd]
 
 port     = ftp,ftp-data,ftps,ftps-data
 logpath  = %(syslog_daemon)s
-maxretry = 6
+backend  = %(syslog_backend)s
 
 
 [wuftpd]
 
 port     = ftp,ftp-data,ftps,ftps-data
 logpath  = %(wuftpd_log)s
-maxretry = 6
+backend  = %(wuftpd_backend)s
 
 
 [vsftpd]
@@ -491,18 +524,21 @@ logpath  = /root/path/to/assp/logs/maillog.txt
 
 port     = smtp,465,submission
 logpath  = %(syslog_mail)s
+backend  = %(syslog_backend)s
 
 
 [postfix]
 
 port     = smtp,465,submission
 logpath  = %(postfix_log)s
+backend  = %(postfix_backend)s
 
 
 [postfix-rbl]
 
 port     = smtp,465,submission
-logpath  = %(syslog_mail)s
+logpath  = %(postfix_log)s
+backend  = %(postfix_backend)s
 maxretry = 1
 
 
@@ -510,12 +546,14 @@ maxretry = 1
 
 port    = submission,465,smtp
 logpath = %(syslog_mail)s
+backend = %(syslog_backend)s
 
 
 [sendmail-reject]
 
 port     = smtp,465,submission
 logpath  = %(syslog_mail)s
+backend  = %(syslog_backend)s
 
 
 [qmail-rbl]
@@ -531,12 +569,14 @@ logpath = /service/qmail/log/main/current
 
 port    = pop3,pop3s,imap,imaps,submission,465,sieve
 logpath = %(dovecot_log)s
+backend = %(dovecot_backend)s
 
 
 [sieve]
 
 port   = smtp,465,submission
 logpath = %(dovecot_log)s
+backend = %(dovecot_backend)s
 
 
 [solid-pop3d]
@@ -572,6 +612,7 @@ logpath = /opt/kerio/mailserver/store/logs/security.log
 
 port     = smtp,465,submission,imap3,imaps,pop3,pop3s
 logpath  = %(syslog_mail)s
+backend  = %(syslog_backend)s
 
 
 [postfix-sasl]
@@ -581,12 +622,14 @@ port     = smtp,465,submission,imap3,imaps,pop3,pop3s
 # running postfix since it would provide the same log lines at the
 # "warn" level but overall at the smaller filesize.
 logpath  = %(postfix_log)s
+backend  = %(postfix_backend)s
 
 
 [perdition]
 
 port   = imap3,imaps,pop3,pop3s
 logpath = %(syslog_mail)s
+backend = %(syslog_backend)s
 
 
 [squirrelmail]
@@ -599,12 +642,14 @@ logpath = /var/lib/squirrelmail/prefs/squirrelmail_access_log
 
 port   = imap3,imaps
 logpath = %(syslog_mail)s
+backend = %(syslog_backend)s
 
 
 [uwimap-auth]
 
 port   = imap3,imaps
 logpath = %(syslog_mail)s
+backend = %(syslog_backend)s
 
 
 #
@@ -686,7 +731,14 @@ maxretry = 10
 
 port     = 3306
 logpath  = %(mysql_log)s
-maxretry = 5
+backend  = %(mysql_backend)s
+
+
+# Log wrong MongoDB auth (for details see filter 'filter.d/mongodb-auth.conf')
+[mongodb-auth]
+# change port when running with "--shardsvr" or "--configsvr" runtime operation
+port     = 27017
+logpath  = /var/log/mongodb/mongodb.log
 
 
 # Jail for more extended banning of persistent abusers
@@ -699,10 +751,9 @@ maxretry = 5
 [recidive]
 
 logpath  = /var/log/fail2ban.log
-banaction = iptables-allports
+banaction = %(banaction_allports)s
 bantime  = 604800  ; 1 week
 findtime = 86400   ; 1 day
-maxretry = 5
 
 
 # Generic filter for PAM. Has to be used with action which bans all
@@ -710,14 +761,16 @@ maxretry = 5
 
 [pam-generic]
 # pam-generic filter can be customized to monitor specific subset of 'tty's
-banaction = iptables-allports
+banaction = %(banaction_allports)s
 logpath  = %(syslog_authpriv)s
+backend  = %(syslog_backend)s
 
 
 [xinetd-fail]
 
 banaction = iptables-multiport-log
 logpath   = %(syslog_daemon)s
+backend   = %(syslog_backend)s
 maxretry  = 2
 
 
@@ -736,7 +789,6 @@ logpath = /var/log/ejabberd/ejabberd.log
 [counter-strike]
 
 logpath = /opt/cstrike/logs/L[0-9]*.log
-# Firewall: http://www.cstrike-planet.com/faq/6
 tcpport = 27030,27031,27032,27033,27034,27035,27036,27037,27038,27039
 udpport = 1200,27000,27001,27002,27003,27004,27005,27006,27007,27008,27009,27010,27011,27012,27013,27014,27015
 action  = %(banaction)s[name=%(__name__)s-tcp, port="%(tcpport)s", protocol="tcp", chain="%(chain)s", actname=%(banaction)s-tcp]
@@ -746,33 +798,30 @@ action  = %(banaction)s[name=%(__name__)s-tcp, port="%(tcpport)s", protocol="tcp
 # nobody except your own Nagios server should ever probe nrpe
 [nagios]
 
-enabled  = false
 logpath  = %(syslog_daemon)s     ; nrpe.cfg may define a different log_facility
+backend  = %(syslog_backend)s
 maxretry = 1
 
 
 [oracleims]
 # see "oracleims" filter file for configuration requirement for Oracle IMS v6 and above
-enabled = false
 logpath = /opt/sun/comms/messaging64/log/mail.log_current
-maxretry = 6
-banaction = iptables-allports
+banaction = %(banaction_allports)s
 
 [directadmin]
-enabled = false
 logpath = /var/log/directadmin/login.log
 port = 2222
 
 [portsentry]
-enabled  = false
 logpath  = /var/lib/portsentry/portsentry.history
 maxretry = 1
 
 [pass2allow-ftp]
 # this pass2allow example allows FTP traffic after successful HTTP authentication
 port         = ftp,ftp-data,ftps,ftps-data
-# knocking_url variable must be overridden to some secret value in filter.d/apache-pass.local
-filter       = apache-pass
+# knocking_url variable must be overridden to some secret value in jail.local
+knocking_url = /knocking/
+filter       = apache-pass[knocking_url="%(knocking_url)s"]
 # access log of the website with HTTP auth
 logpath      = %(apache_access_log)s
 blocktype    = RETURN
@@ -780,3 +829,48 @@ returntype   = DROP
 bantime      = 3600
 maxretry     = 1
 findtime     = 1
+
+
+[murmur]
+# AKA mumble-server
+port     = 64738
+action   = %(banaction)s[name=%(__name__)s-tcp, port="%(port)s", protocol=tcp, chain="%(chain)s", actname=%(banaction)s-tcp]
+           %(banaction)s[name=%(__name__)s-udp, port="%(port)s", protocol=udp, chain="%(chain)s", actname=%(banaction)s-udp]
+logpath  = /var/log/mumble-server/mumble-server.log
+
+
+[screensharingd]
+# For Mac OS Screen Sharing Service (VNC)
+logpath  = /var/log/system.log
+logencoding = utf-8
+
+[haproxy-http-auth]
+# HAProxy by default doesn't log to file you'll need to set it up to forward
+# logs to a syslog server which would then write them to disk.
+# See "haproxy-http-auth" filter for a brief cautionary note when setting
+# maxretry and findtime.
+logpath  = /var/log/haproxy.log
+
+[slapd]
+port    = ldap,ldaps
+filter  = slapd
+logpath = /var/log/slapd.log
+
+[domino-smtp]
+port    = smtp,ssmtp
+filter  = domino-smtp
+logpath = /home/domino01/data/IBM_TECHNICAL_SUPPORT/console.log
+
+[phpmyadmin-syslog]
+port    = http,https
+filter  = phpmyadmin-syslog
+logpath = %(syslog_authpriv)s
+backend = %(syslog_backend)s
+
+
+[zoneminder]
+# Zoneminder HTTP/HTTPS web interface auth
+# Logs auth failures to apache2 error log
+port    = http,https
+logpath = %(apache_error_log)s
+

config/paths-common.conf

@@ -7,9 +7,13 @@ after  = paths-overrides.local
 
 [DEFAULT]
 
+default_backend = auto
+
 sshd_log = %(syslog_authpriv)s
+sshd_backend = %(default_backend)s
 
 dropbear_log = %(syslog_authpriv)s
+dropbear_backend = %(default_backend)s
 
 # There is no sensible generic defaults for syslog log targets, thus
 # leaving them empty here so that no errors while parsing/interpolating configs
@@ -18,15 +22,17 @@ syslog_ftp =
 syslog_local0 =
 syslog_mail_warn =
 syslog_user =
+# Set the default syslog backend target to default_backend
+syslog_backend = %(default_backend)s
 
 # from /etc/audit/auditd.conf
 auditd_log = /var/log/audit/audit.log
 
 exim_main_log = /var/log/exim/mainlog
 
-nginx_error_log = /var/log/nginx/error.log
+nginx_error_log = /var/log/nginx/*error.log
 
-nginx_access_log = /var/log/nginx/access.log
+nginx_access_log = /var/log/nginx/*access.log
 
 
 lighttpd_error_log = /var/log/lighttpd/error.log
@@ -34,18 +40,22 @@ lighttpd_error_log = /var/log/lighttpd/error.log
 # http://www.hardened-php.net/suhosin/configuration.html#suhosin.log.syslog.facility
 # syslog_user is the default. Lighttpd also hooks errors into its log.
 
-suhosin_log = %(syslog_user)s %(lighttpd_error_log)s
+suhosin_log = %(syslog_user)s
+              %(lighttpd_error_log)s
 
 # defaults to ftp or local2 if ftp doesn't exist
 proftpd_log = %(syslog_ftp)s
+proftpd_backend = %(default_backend)s
 
 # http://svnweb.freebsd.org/ports/head/ftp/proftpd/files/patch-src_proftpd.8.in?view=markup
 # defaults to ftp but can be overwritten.
 pureftpd_log = %(syslog_ftp)s
+pureftpd_backend = %(default_backend)s
 
 # ftp, daemon and then local7 are tried at configure time however it is overwriteable at configure time
 #
 wuftpd_log = %(syslog_ftp)s
+wuftpd_backend = %(default_backend)s
 
 # syslog_enable defaults to no. so it defaults to vsftpd_log_file setting of /var/log/vsftpd.log
 # No distro seems to set it to syslog by default
@@ -54,13 +64,16 @@ vsftpd_log = /var/log/vsftpd.log
 
 # Technically syslog_facility in main.cf can overwrite but no-one sane does this.
 postfix_log = %(syslog_mail_warn)s
+postfix_backend = %(default_backend)s
 
 dovecot_log = %(syslog_mail_warn)s
+dovecot_backend = %(default_backend)s
 
 # Seems to be set at compile time only to LOG_LOCAL0 (src/const.h) at Notice level
 solidpop3d_log = %(syslog_local0)s
 
 mysql_log = %(syslog_daemon)s
+mysql_backend = %(default_backend)s
 
 roundcube_errors_log = /var/log/roundcube/errors
 

config/paths-debian.conf

@@ -32,6 +32,8 @@ apache_access_log = /var/log/apache2/*access.log
 
 exim_main_log = /var/log/exim4/mainlog
 
+dovecot_log = %(syslog_mail)s
+
 # was in debian squeezy but not in wheezy
 # /etc/proftpd/proftpd.conf (SystemLog)
 proftpd_log = /var/log/proftpd/proftpd.log

config/paths-fedora.conf

@@ -34,6 +34,18 @@ apache_access_log = /var/log/httpd/*access_log
 
 exim_main_log = /var/log/exim/main.log
 
-mysql_log = /var/lib/mysql/mysqld.log
+mysql_log = /var/log/mariadb/mariadb.log
+            /var/log/mysqld.log
 
 roundcube_errors_log = /var/log/roundcubemail/errors
+
+# These services will log to the journal via syslog, so use the journal by
+# default.
+syslog_backend = systemd
+sshd_backend = systemd
+dropbear_backend = systemd
+proftpd_backend = systemd
+pureftpd_backend = systemd
+wuftpd_backend = systemd
+postfix_backend = systemd
+dovecot_backend = systemd

config/paths-freebsd.conf

@@ -34,13 +34,13 @@ auditd_log = /dev/null
 # http://svnweb.freebsd.org/ports/head/www/apache24/files/patch-config.layout
 # http://svnweb.freebsd.org/ports/head/www/apache22/files/patch-config.layout
 
-apache_error_log = /usr/local/www/logs/*error[_.]log
+apache_error_log = /var/log/httpd-error.log
 
-apache_access_log = /usr/local/www/logs/*access[_.]log
+apache_access_log = /var/log/httpd-access.log
 
 # http://svnweb.freebsd.org/ports/head/www/nginx/Makefile?view=markup
 
-nginx_error_log = /var/log/nginx-error.log
+nginx_error_log = /var/log/nginx/error.log
 
-nginx_access_log = /var/log/nginx-access.log
+nginx_access_log = /var/log/nginx/access.log
 

config/paths-opensuse.conf

@@ -0,0 +1,50 @@
+# openSUSE log-file locations
+
+[INCLUDES]
+
+before = paths-common.conf
+
+after  = paths-overrides.local
+
+
+[DEFAULT]
+
+syslog_local0  = /var/log/messages
+
+syslog_mail = /var/log/mail
+
+syslog_mail_warn = %(syslog_mail)s
+
+syslog_authpriv = %(syslog_local0)s
+
+syslog_user =  %(syslog_local0)s
+
+syslog_ftp  = %(syslog_local0)s
+
+syslog_daemon  = %(syslog_local0)s
+
+apache_error_log = /var/log/apache2/*error_log
+
+apache_access_log = /var/log/apache2/*access_log
+
+pureftpd_log = %(syslog_local0)s
+
+exim_main_log = /var/log/exim/main.log
+
+mysql_log = /var/log/mysql/mysqld.log
+
+roundcube_errors_log = /srv/www/roundcubemail/logs/errors
+
+solidpop3d_log = %(syslog_mail)s
+
+# These services will log to the journal via syslog, so use the journal by
+# default.
+syslog_backend = systemd
+sshd_backend = systemd
+dropbear_backend = systemd
+proftpd_backend = systemd
+pureftpd_backend = systemd
+wuftpd_backend = systemd
+postfix_backend = systemd
+dovecot_backend = systemd
+mysql_backend = systemd

fail2ban/__init__.py

@@ -34,7 +34,9 @@ Below derived from:
 	https://mail.python.org/pipermail/tutor/2007-August/056243.html
 """
 
+logging.MSG = logging.INFO - 2
 logging.NOTICE = logging.INFO + 5
+logging.addLevelName(logging.MSG, 'MSG')
 logging.addLevelName(logging.NOTICE, 'NOTICE')
 
 

fail2ban/client/configparserinc.py

@@ -168,7 +168,7 @@ after = 1.conf
 			parser, i = self._getSharedSCPWI(resource)
 			if not i:
 				return []
-		except UnicodeDecodeError, e:
+		except UnicodeDecodeError as e:
 			logSys.error("Error decoding config file '%s': %s" % (resource, e))
 			return []
 		

fail2ban/client/configreader.py

@@ -221,7 +221,7 @@ class ConfigReaderUnshared(SafeConfigParserWithIncludes):
 				if not pOptions is None and option[1] in pOptions:
 					continue
 				values[option[1]] = v
-			except NoSectionError, e:
+			except NoSectionError as e:
 				# No "Definition" section or wrong basedir
 				logSys.error(e)
 				values[option[1]] = option[2]
@@ -285,8 +285,10 @@ class DefinitionInitConfigReader(ConfigReader):
 		
 		if self.has_section("Init"):
 			for opt in self.options("Init"):
+				v = self.get("Init", opt)
+				self._initOpts['known/'+opt] = v
 				if not opt in self._initOpts:
-					self._initOpts[opt] = self.get("Init", opt)
+					self._initOpts[opt] = v
 	
 	def convert(self):
 		raise NotImplementedError

fail2ban/client/fail2banregex.py

@@ -0,0 +1,599 @@
+#!/usr/bin/python
+# emacs: -*- mode: python; py-indent-offset: 4; indent-tabs-mode: t -*-
+# vi: set ft=python sts=4 ts=4 sw=4 noet :
+#
+# This file is part of Fail2Ban.
+#
+# Fail2Ban is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# Fail2Ban is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with Fail2Ban; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA.
+"""
+Fail2Ban  reads log file that contains password failure report
+and bans the corresponding IP addresses using firewall rules.
+
+This tools can test regular expressions for "fail2ban".
+
+"""
+
+__author__ = "Fail2Ban Developers"
+__copyright__ = "Copyright (c) 2004-2008 Cyril Jaquier, 2012-2014 Yaroslav Halchenko"
+__license__ = "GPL"
+
+import getopt
+import locale
+import logging
+import os
+import shlex
+import sys
+import time
+import time
+import urllib
+from optparse import OptionParser, Option
+
+from ConfigParser import NoOptionError, NoSectionError, MissingSectionHeaderError
+
+try:
+	from ..server.filtersystemd import FilterSystemd
+except ImportError:
+	FilterSystemd = None
+
+from ..version import version
+from .jailreader import JailReader
+from .filterreader import FilterReader
+from ..server.filter import Filter, FileContainer
+from ..server.failregex import RegexException
+
+from ..helpers import FormatterWithTraceBack, getLogger
+# Gets the instance of the logger.
+logSys = getLogger("fail2ban")
+
+def debuggexURL(sample, regex):
+	q = urllib.urlencode({ 're': regex.replace('<HOST>', '(?&.ipv4)'),
+							'str': sample,
+							'flavor': 'python' })
+	return 'https://www.debuggex.com/?' + q
+
+def output(args):
+	print(args)
+
+def shortstr(s, l=53):
+	"""Return shortened string
+	"""
+	if len(s) > l:
+		return s[:l-3] + '...'
+	return s
+
+def pprint_list(l, header=None):
+	if not len(l):
+		return
+	if header:
+		s = "|- %s\n" % header
+	else:
+		s = ''
+	output( s + "|  " + "\n|  ".join(l) + '\n`-' )
+
+def journal_lines_gen(flt, myjournal): # pragma: no cover
+	while True:
+		try:
+			entry = myjournal.get_next()
+		except OSError:
+			continue
+		if not entry:
+			break
+		yield flt.formatJournalEntry(entry)
+
+def get_opt_parser():
+	# use module docstring for help output
+	p = OptionParser(
+				usage="%s [OPTIONS] <LOG> <REGEX> [IGNOREREGEX]\n" % sys.argv[0] + __doc__
+				+ """
+LOG:
+    string                  a string representing a log line
+    filename                path to a log file (/var/log/auth.log)
+    "systemd-journal"       search systemd journal (systemd-python required)
+
+REGEX:
+    string                  a string representing a 'failregex'
+    filename                path to a filter file (filter.d/sshd.conf)
+
+IGNOREREGEX:
+    string                  a string representing an 'ignoreregex'
+    filename                path to a filter file (filter.d/sshd.conf)
+
+Copyright (c) 2004-2008 Cyril Jaquier, 2008- Fail2Ban Contributors
+Copyright of modifications held by their respective authors.
+Licensed under the GNU General Public License v2 (GPL).
+
+Written by Cyril Jaquier <cyril.jaquier@fail2ban.org>.
+Many contributions by Yaroslav O. Halchenko and Steven Hiscocks.
+
+Report bugs to https://github.com/fail2ban/fail2ban/issues
+""",
+				version="%prog " + version)
+
+	p.add_options([
+		Option("-d", "--datepattern",
+			   help="set custom pattern used to match date/times"),
+		Option("-e", "--encoding",
+			   help="File encoding. Default: system locale"),
+		Option("-r", "--raw", action='store_true',
+			   help="Raw hosts, don't resolve dns"),
+		Option("-L", "--maxlines", type=int, default=0,
+			   help="maxlines for multi-line regex"),
+		Option("-m", "--journalmatch",
+			   help="journalctl style matches overriding filter file. "
+			   "\"systemd-journal\" only"),
+		Option('-l', "--log-level", type="choice",
+			   dest="log_level",
+			   choices=('heavydebug', 'debug', 'info', 'notice', 'warning', 'error', 'critical'),
+			   default=None,
+			   help="Log level for the Fail2Ban logger to use"),
+		Option("-v", "--verbose", action='store_true',
+			   help="Be verbose in output"),
+		Option("-D", "--debuggex", action='store_true',
+			   help="Produce debuggex.com urls for debugging there"),
+		Option("--print-no-missed", action='store_true',
+			   help="Do not print any missed lines"),
+		Option("--print-no-ignored", action='store_true',
+			   help="Do not print any ignored lines"),
+		Option("--print-all-matched", action='store_true',
+			   help="Print all matched lines"),
+		Option("--print-all-missed", action='store_true',
+			   help="Print all missed lines, no matter how many"),
+		Option("--print-all-ignored", action='store_true',
+			   help="Print all ignored lines, no matter how many"),
+		Option("-t", "--log-traceback", action='store_true',
+			   help="Enrich log-messages with compressed tracebacks"),
+		Option("--full-traceback", action='store_true',
+			   help="Either to make the tracebacks full, not compressed (as by default)"),
+		])
+
+	return p
+
+
+class RegexStat(object):
+
+	def __init__(self, failregex):
+		self._stats = 0
+		self._failregex = failregex
+		self._ipList = list()
+
+	def __str__(self):
+		return "%s(%r) %d failed: %s" \
+		  % (self.__class__, self._failregex, self._stats, self._ipList)
+
+	def inc(self):
+		self._stats += 1
+
+	def getStats(self):
+		return self._stats
+
+	def getFailRegex(self):
+		return self._failregex
+
+	def appendIP(self, value):
+		self._ipList.append(value)
+
+	def getIPList(self):
+		return self._ipList
+
+
+class LineStats(object):
+	"""Just a convenience container for stats
+	"""
+	def __init__(self):
+		self.tested = self.matched = 0
+		self.matched_lines = []
+		self.missed = 0
+		self.missed_lines = []
+		self.missed_lines_timeextracted = []
+		self.ignored = 0
+		self.ignored_lines = []
+		self.ignored_lines_timeextracted = []
+
+	def __str__(self):
+		return "%(tested)d lines, %(ignored)d ignored, %(matched)d matched, %(missed)d missed" % self
+
+	# just for convenient str
+	def __getitem__(self, key):
+		return getattr(self, key) if hasattr(self, key) else ''
+
+
+class Fail2banRegex(object):
+
+	def __init__(self, opts):
+		self._verbose = opts.verbose
+		self._debuggex = opts.debuggex
+		self._maxlines = 20
+		self._print_no_missed = opts.print_no_missed
+		self._print_no_ignored = opts.print_no_ignored
+		self._print_all_matched = opts.print_all_matched
+		self._print_all_missed = opts.print_all_missed
+		self._print_all_ignored = opts.print_all_ignored
+		self._maxlines_set = False		  # so we allow to override maxlines in cmdline
+		self._datepattern_set = False
+		self._journalmatch = None
+
+		self.share_config=dict()
+		self._filter = Filter(None)
+		self._ignoreregex = list()
+		self._failregex = list()
+		self._time_elapsed = None
+		self._line_stats = LineStats()
+
+		if opts.maxlines:
+			self.setMaxLines(opts.maxlines)
+		if opts.journalmatch is not None:
+			self.setJournalMatch(opts.journalmatch.split())
+		if opts.datepattern:
+			self.setDatePattern(opts.datepattern)
+		if opts.encoding:
+			self.encoding = opts.encoding
+		else:
+			self.encoding = locale.getpreferredencoding()
+		self.raw = True if opts.raw else False
+
+	def decode_line(self, line):
+		return FileContainer.decode_line('<LOG>', self.encoding, line)
+
+	def encode_line(self, line):
+		return line.encode(self.encoding, 'ignore')
+
+	def setDatePattern(self, pattern):
+		if not self._datepattern_set:
+			self._filter.setDatePattern(pattern)
+			self._datepattern_set = True
+			if pattern is not None:
+				output( "Use      datepattern : %s" % (
+					self._filter.getDatePattern()[1], ) )
+
+	def setMaxLines(self, v):
+		if not self._maxlines_set:
+			self._filter.setMaxLines(int(v))
+			self._maxlines_set = True
+			output( "Use         maxlines : %d" % self._filter.getMaxLines() )
+
+	def setJournalMatch(self, v):
+		if self._journalmatch is None:
+			self._journalmatch = v
+
+	def readRegex(self, value, regextype):
+		assert(regextype in ('fail', 'ignore'))
+		regex = regextype + 'regex'
+		if os.path.isfile(value) or os.path.isfile(value + '.conf'):
+			if os.path.basename(os.path.dirname(value)) == 'filter.d':
+				## within filter.d folder - use standard loading algorithm to load filter completely (with .local etc.):
+				basedir = os.path.dirname(os.path.dirname(value))
+				value = os.path.splitext(os.path.basename(value))[0]
+				output( "Use %11s filter file : %s, basedir: %s" % (regex, value, basedir) )
+				reader = FilterReader(value, 'fail2ban-regex-jail', {}, share_config=self.share_config, basedir=basedir)
+				if not reader.read():
+					output( "ERROR: failed to load filter %s" % value )
+					return False
+			else:
+				## foreign file - readexplicit this file and includes if possible:
+				output( "Use %11s file : %s" % (regex, value) )
+				reader = FilterReader(value, 'fail2ban-regex-jail', {}, share_config=self.share_config)
+				reader.setBaseDir(None)
+				if not reader.readexplicit():
+					output( "ERROR: failed to read %s" % value )
+					return False
+			reader.getOptions(None)
+			readercommands = reader.convert()
+			regex_values = [
+				RegexStat(m[3])
+				for m in filter(
+					lambda x: x[0] == 'set' and x[2] == "add%sregex" % regextype,
+					readercommands)]
+			# Read out and set possible value of maxlines
+			for command in readercommands:
+				if command[2] == "maxlines":
+					maxlines = int(command[3])
+					try:
+						self.setMaxLines(maxlines)
+					except ValueError:
+						output( "ERROR: Invalid value for maxlines (%(maxlines)r) " \
+							  "read from %(value)s" % locals() )
+						return False
+				elif command[2] == 'addjournalmatch':
+					journalmatch = command[3:]
+					self.setJournalMatch(journalmatch)
+				elif command[2] == 'datepattern':
+					datepattern = command[3]
+					self.setDatePattern(datepattern)
+		else:
+			output( "Use %11s line : %s" % (regex, shortstr(value)) )
+			regex_values = [RegexStat(value)]
+
+		setattr(self, "_" + regex, regex_values)
+		for regex in regex_values:
+			getattr(
+				self._filter,
+				'add%sRegex' % regextype.title())(regex.getFailRegex())
+		return True
+
+	def testIgnoreRegex(self, line):
+		found = False
+		try:
+			ret = self._filter.ignoreLine([(line, "", "")])
+			if ret is not None:
+				found = True
+				regex = self._ignoreregex[ret].inc()
+		except RegexException as e:
+			output( e )
+			return False
+		return found
+
+	def testRegex(self, line, date=None):
+		orgLineBuffer = self._filter._Filter__lineBuffer
+		fullBuffer = len(orgLineBuffer) >= self._filter.getMaxLines()
+		try:
+			line, ret = self._filter.processLine(line, date, checkAllRegex=True, returnRawHost=self.raw)
+			for match in ret:
+				# Append True/False flag depending if line was matched by
+				# more than one regex
+				match.append(len(ret)>1)
+				regex = self._failregex[match[0]]
+				regex.inc()
+				regex.appendIP(match)
+		except RegexException as e:
+			output( e )
+			return False
+		except IndexError:
+			output( "Sorry, but no <HOST> found in regex" )
+			return False
+		for bufLine in orgLineBuffer[int(fullBuffer):]:
+			if bufLine not in self._filter._Filter__lineBuffer:
+				try:
+					self._line_stats.missed_lines.pop(
+						self._line_stats.missed_lines.index("".join(bufLine)))
+					self._line_stats.missed_lines_timeextracted.pop(
+						self._line_stats.missed_lines_timeextracted.index(
+							"".join(bufLine[::2])))
+				except ValueError:
+					pass
+				else:
+					self._line_stats.matched += 1
+					self._line_stats.missed -= 1
+		return line, ret
+
+	def process(self, test_lines):
+		t0 = time.time()
+		for line_no, line in enumerate(test_lines):
+			if isinstance(line, tuple):
+				line_datetimestripped, ret = self.testRegex(
+					line[0], line[1])
+				line = "".join(line[0])
+			else:
+				line = line.rstrip('\r\n')
+				if line.startswith('#') or not line:
+					# skip comment and empty lines
+					continue
+				line_datetimestripped, ret = self.testRegex(line)
+			is_ignored = self.testIgnoreRegex(line_datetimestripped)
+
+			if is_ignored:
+				self._line_stats.ignored += 1
+				if not self._print_no_ignored and (self._print_all_ignored or self._line_stats.ignored <= self._maxlines + 1):
+					self._line_stats.ignored_lines.append(line)
+					self._line_stats.ignored_lines_timeextracted.append(line_datetimestripped)
+
+			if len(ret) > 0:
+				assert(not is_ignored)
+				self._line_stats.matched += 1
+				if self._print_all_matched:
+					self._line_stats.matched_lines.append(line)
+			else:
+				if not is_ignored:
+					self._line_stats.missed += 1
+					if not self._print_no_missed and (self._print_all_missed or self._line_stats.missed <= self._maxlines + 1):
+						self._line_stats.missed_lines.append(line)
+						self._line_stats.missed_lines_timeextracted.append(line_datetimestripped)
+			self._line_stats.tested += 1
+
+			if line_no % 10 == 0 and self._filter.dateDetector is not None:
+				self._filter.dateDetector.sortTemplate()
+		self._time_elapsed = time.time() - t0
+
+	def printLines(self, ltype):
+		lstats = self._line_stats
+		assert(self._line_stats.missed == lstats.tested - (lstats.matched + lstats.ignored))
+		lines = lstats[ltype]
+		l = lstats[ltype + '_lines']
+		if lines:
+			header = "%s line(s):" % (ltype.capitalize(),)
+			if self._debuggex:
+				if ltype == 'missed' or ltype == 'matched':
+					regexlist = self._failregex
+				else:
+					regexlist = self._ignoreregex
+				l = lstats[ltype + '_lines_timeextracted']
+				if lines < self._maxlines or getattr(self, '_print_all_' + ltype):
+					ans = [[]]
+					for arg in [l, regexlist]:
+						ans = [ x + [y] for x in ans for y in arg ]
+					b = map(lambda a: a[0] +  ' | ' + a[1].getFailRegex() + ' |  ' + 
+						debuggexURL(self.encode_line(a[0]), a[1].getFailRegex()), ans)
+					pprint_list([x.rstrip() for x in b], header)
+				else:
+					output( "%s too many to print.  Use --print-all-%s " \
+						  "to print all %d lines" % (header, ltype, lines) )
+			elif lines < self._maxlines or getattr(self, '_print_all_' + ltype):
+				pprint_list([x.rstrip() for x in l], header)
+			else:
+				output( "%s too many to print.  Use --print-all-%s " \
+					  "to print all %d lines" % (header, ltype, lines) )
+
+	def printStats(self):
+		output( "" )
+		output( "Results" )
+		output( "=======" )
+
+		def print_failregexes(title, failregexes):
+			# Print title
+			total, out = 0, []
+			for cnt, failregex in enumerate(failregexes):
+				match = failregex.getStats()
+				total += match
+				if (match or self._verbose):
+					out.append("%2d) [%d] %s" % (cnt+1, match, failregex.getFailRegex()))
+
+				if self._verbose and len(failregex.getIPList()):
+					for ip in failregex.getIPList():
+						timeTuple = time.localtime(ip[2])
+						timeString = time.strftime("%a %b %d %H:%M:%S %Y", timeTuple)
+						out.append(
+							"    %s  %s%s" % (
+								ip[1],
+								timeString,
+								ip[-1] and " (multiple regex matched)" or ""))
+
+			output( "\n%s: %d total" % (title, total) )
+			pprint_list(out, " #) [# of hits] regular expression")
+			return total
+
+		# Print title
+		total = print_failregexes("Failregex", self._failregex)
+		_ = print_failregexes("Ignoreregex", self._ignoreregex)
+
+
+		if self._filter.dateDetector is not None:
+			output( "\nDate template hits:" )
+			out = []
+			for template in self._filter.dateDetector.templates:
+				if self._verbose or template.hits:
+					out.append("[%d] %s" % (
+						template.hits, template.name))
+			pprint_list(out, "[# of hits] date format")
+
+		output( "\nLines: %s" % self._line_stats, )
+		if self._time_elapsed is not None:
+			output( "[processed in %.2f sec]" % self._time_elapsed, )
+		output( "" )
+
+		if self._print_all_matched:
+			self.printLines('matched')
+		if not self._print_no_ignored:
+			self.printLines('ignored')
+		if not self._print_no_missed:
+			self.printLines('missed')
+
+		return True
+
+	def file_lines_gen(self, hdlr):
+		for line in hdlr:
+			yield self.decode_line(line)
+
+	def start(self, opts, args):
+
+		cmd_log, cmd_regex = args[:2]
+
+		if not self.readRegex(cmd_regex, 'fail'):
+			return False
+
+		if len(args) == 3 and not self.readRegex(args[2], 'ignore'):
+			return False
+
+		if os.path.isfile(cmd_log):
+			try:
+				hdlr = open(cmd_log, 'rb')
+				output( "Use         log file : %s" % cmd_log )
+				output( "Use         encoding : %s" % self.encoding )
+				test_lines = self.file_lines_gen(hdlr)
+			except IOError as e:
+				output( e )
+				return False
+		elif cmd_log.startswith("systemd-journal"): # pragma: no cover
+			if not FilterSystemd:
+				output( "Error: systemd library not found. Exiting..." )
+				return False
+			output( "Use         systemd journal" )
+			output( "Use         encoding : %s" % self.encoding )
+			backend, beArgs = JailReader.extractOptions(cmd_log)
+			flt = FilterSystemd(None, **beArgs)
+			flt.setLogEncoding(self.encoding)
+			myjournal = flt.getJournalReader()
+			journalmatch = self._journalmatch
+			self.setDatePattern(None)
+			if journalmatch:
+				flt.addJournalMatch(journalmatch)
+			output( "Use    journal match : %s" % " ".join(journalmatch) )
+			test_lines = journal_lines_gen(flt, myjournal)
+		else:
+			output( "Use      single line : %s" % shortstr(cmd_log) )
+			test_lines = [ cmd_log ]
+		output( "" )
+
+		self.process(test_lines)
+
+		if not self.printStats():
+			return False
+
+		return True
+
+
+def exec_command_line():
+	parser = get_opt_parser()
+	(opts, args) = parser.parse_args()
+	if opts.print_no_missed and opts.print_all_missed:
+		sys.stderr.write("ERROR: --print-no-missed and --print-all-missed are mutually exclusive.\n\n")
+		parser.print_help()
+		sys.exit(-1)
+	if opts.print_no_ignored and opts.print_all_ignored:
+		sys.stderr.write("ERROR: --print-no-ignored and --print-all-ignored are mutually exclusive.\n\n")
+		parser.print_help()
+		sys.exit(-1)
+
+	# We need 2 or 3 parameters
+	if not len(args) in (2, 3):
+		sys.stderr.write("ERROR: provide both <LOG> and <REGEX>.\n\n")
+		parser.print_help()
+		return False
+
+	output( "" )
+	output( "Running tests" )
+	output( "=============" )
+	output( "" )
+
+	# TODO: taken from -testcases -- move common functionality somewhere
+	if opts.log_level is not None:
+		# so we had explicit settings
+		logSys.setLevel(getattr(logging, opts.log_level.upper()))
+	else:
+		# suppress the logging but it would leave unittests' progress dots
+		# ticking, unless like with '-l critical' which would be silent
+		# unless error occurs
+		logSys.setLevel(getattr(logging, 'CRITICAL'))
+
+	# Add the default logging handler
+	stdout = logging.StreamHandler(sys.stdout)
+
+	fmt = 'D: %(message)s'
+
+	if opts.log_traceback:
+		Formatter = FormatterWithTraceBack
+		fmt = (opts.full_traceback and ' %(tb)s' or ' %(tbc)s') + fmt
+	else:
+		Formatter = logging.Formatter
+
+	# Custom log format for the verbose tests runs
+	if opts.verbose:
+		stdout.setFormatter(Formatter(' %(asctime)-15s %(thread)s' + fmt))
+	else:
+		# just prefix with the space
+		stdout.setFormatter(Formatter(fmt))
+	logSys.addHandler(stdout)
+
+	fail2banRegex = Fail2banRegex(opts)
+	if not fail2banRegex.start(opts, args):
+		sys.exit(-1)

fail2ban/client/jailreader.py

@@ -27,12 +27,13 @@ __license__ = "GPL"
 import glob
 import json
 import os.path
-import re
 
 from .configreader import ConfigReaderUnshared, ConfigReader
 from .filterreader import FilterReader
 from .actionreader import ActionReader
+from ..version import version
 from ..helpers import getLogger
+from ..helpers import extractOptions, splitwords
 
 # Gets the instance of the logger.
 logSys = getLogger(__name__)
@@ -40,10 +41,6 @@ logSys = getLogger(__name__)
 
 class JailReader(ConfigReader):
 	
-	optionCRE = re.compile("^((?:\w|-|_|\.)+)(?:\[(.*)\])?$")
-	optionExtractRE = re.compile(
-		r'([\w\-_\.]+)=(?:"([^"]*)"|\'([^\']*)\'|([^,]*))(?:,|$)')
-	
 	def __init__(self, name, force_enable=False, **kwargs):
 		ConfigReader.__init__(self, **kwargs)
 		self.__name = name
@@ -107,6 +104,10 @@ class JailReader(ConfigReader):
 				["string", "filter", ""],
 				["string", "action", ""]]
 
+		# Before interpolation (substitution) add static options always available as default:
+		defsec = self._cfg.get_defaults()
+		defsec["fail2ban_version"] = version
+
 		# Read first options only needed for merge defaults ('known/...' from filter):
 		self.__opts = ConfigReader.getOptions(self, self.__name, opts1st)
 		if not self.__opts:
@@ -115,7 +116,7 @@ class JailReader(ConfigReader):
 		if self.isEnabled():
 			# Read filter
 			if self.__opts["filter"]:
-				filterName, filterOpt = JailReader.extractOptions(
+				filterName, filterOpt = extractOptions(
 					self.__opts["filter"])
 				self.__filter = FilterReader(
 					filterName, self.__name, filterOpt, share_config=self.share_config, basedir=self.getBaseDir())
@@ -144,7 +145,7 @@ class JailReader(ConfigReader):
 				try:
 					if not act:			  # skip empty actions
 						continue
-					actName, actOpt = JailReader.extractOptions(act)
+					actName, actOpt = extractOptions(act)
 					if actName.endswith(".py"):
 						self.__actions.append([
 							"set",
@@ -165,7 +166,7 @@ class JailReader(ConfigReader):
 							self.__actions.append(action)
 						else:
 							raise AttributeError("Unable to read action")
-				except Exception, e:
+				except Exception as e:
 					logSys.error("Error in action definition " + act)
 					logSys.debug("Caught exception: %s" % (e,))
 					return False
@@ -186,7 +187,7 @@ class JailReader(ConfigReader):
 		stream = []
 		for opt in self.__opts:
 			if opt == "logpath" and	\
-					self.__opts.get('backend', None) != "systemd":
+					not self.__opts.get('backend', None).startswith("systemd"):
 				found_files = 0
 				for path in self.__opts[opt].split("\n"):
 					path = path.rsplit(" ", 1)
@@ -208,10 +209,8 @@ class JailReader(ConfigReader):
 			elif opt == "maxretry":
 				stream.append(["set", self.__name, "maxretry", self.__opts[opt]])
 			elif opt == "ignoreip":
-				for ip in self.__opts[opt].split():
-					# Do not send a command if the rule is empty.
-					if ip != '':
-						stream.append(["set", self.__name, "addignoreip", ip])
+				for ip in splitwords(self.__opts[opt]):
+					stream.append(["set", self.__name, "addignoreip", ip])
 			elif opt == "findtime":
 				stream.append(["set", self.__name, "findtime", self.__opts[opt]])
 			elif opt == "bantime":
@@ -240,18 +239,4 @@ class JailReader(ConfigReader):
 		stream.insert(0, ["add", self.__name, backend])
 		return stream
 	
-	@staticmethod
-	def extractOptions(option):
-		match = JailReader.optionCRE.match(option)
-		if not match:
-			# TODO proper error handling
-			return None, None
-		option_name, optstr = match.groups()
-		option_opts = dict()
-		if optstr:
-			for optmatch in JailReader.optionExtractRE.finditer(optstr):
-				opt = optmatch.group(1)
-				value = [
-					val for val in optmatch.group(2,3,4) if val is not None][0]
-				option_opts[opt.strip()] = value.strip()
-		return option_name, option_opts
+JailReader.extractOptions = staticmethod(extractOptions)

fail2ban/helpers.py

@@ -127,3 +127,42 @@ def excepthook(exctype, value, traceback):
 	getLogger("fail2ban").critical(
 		"Unhandled exception in Fail2Ban:", exc_info=True)
 	return sys.__excepthook__(exctype, value, traceback)
+
+def splitwords(s):
+	"""Helper to split words on any comma, space, or a new line
+
+	Returns empty list if input is empty (or None) and filters
+	out empty entries
+	"""
+	if not s:
+		return []
+	return filter(bool, map(str.strip, re.split('[ ,\n]+', s)))
+
+
+#
+# Following function used for parse options from parameter (e.g. `name[p1=0, p2="..."][p3='...']`).
+#
+
+# regex, to extract list of options:
+OPTION_CRE = re.compile(r"^([^\[]+)(?:\[(.*)\])?\s*$", re.DOTALL)
+# regex, to iterate over single option in option list, syntax:
+# `action = act[p1="...", p2='...', p3=...]`, where the p3=... not contains `,` or ']'
+# since v0.10 separator extended with `]\s*[` for support of multiple option groups, syntax 
+# `action = act[p1=...][p2=...]`
+OPTION_EXTRACT_CRE = re.compile(
+	r'([\w\-_\.]+)=(?:"([^"]*)"|\'([^\']*)\'|([^,\]]*))(?:,|\]\s*\[|$)', re.DOTALL)
+
+def extractOptions(option):
+	match = OPTION_CRE.match(option)
+	if not match:
+		# TODO proper error handling
+		return None, None
+	option_name, optstr = match.groups()
+	option_opts = dict()
+	if optstr:
+		for optmatch in OPTION_EXTRACT_CRE.finditer(optstr):
+			opt = optmatch.group(1)
+			value = [
+				val for val in optmatch.group(2,3,4) if val is not None][0]
+			option_opts[opt.strip()] = value.strip()
+	return option_name, option_opts

fail2ban/protocol.py

@@ -89,7 +89,7 @@ protocol = [
 ["set <JAIL> unbanip <IP>", "manually Unban <IP> in <JAIL>"], 
 ["set <JAIL> maxretry <RETRY>", "sets the number of failures <RETRY> before banning the host for <JAIL>"], 
 ["set <JAIL> maxlines <LINES>", "sets the number of <LINES> to buffer for regex search for <JAIL>"], 
-["set <JAIL> addaction <ACT>[ <PYTHONFILE> <JSONKWARGS>]", "adds a new action named <NAME> for <JAIL>. Optionally for a Python based action, a <PYTHONFILE> and <JSONKWARGS> can be specified, else will be a Command Action"], 
+["set <JAIL> addaction <ACT>[ <PYTHONFILE> <JSONKWARGS>]", "adds a new action named <ACT> for <JAIL>. Optionally for a Python based action, a <PYTHONFILE> and <JSONKWARGS> can be specified, else will be a Command Action"], 
 ["set <JAIL> delaction <ACT>", "removes the action <ACT> from <JAIL>"], 
 ["", "COMMAND ACTION CONFIGURATION", ""],
 ["set <JAIL> action <ACT> actionstart <CMD>", "sets the start command <CMD> of the action <ACT> for <JAIL>"], 

fail2ban/server/action.py

@@ -55,6 +55,9 @@ _RETCODE_HINTS = {
 signame = dict((num, name)
 	for name, num in signal.__dict__.iteritems() if name.startswith("SIG"))
 
+# max tag replacement count:
+MAX_TAG_REPLACE_COUNT = 10
+
 
 class CallingMap(MutableMapping):
 	"""A Mapping type which returns the result of callable values.
@@ -390,22 +393,22 @@ class CommandAction(ActionBase):
 		"""
 		t = re.compile(r'<([^ <>]+)>')
 		# repeat substitution while embedded-recursive (repFlag is True)
+		done = cls._escapedTags.copy()
 		while True:
 			repFlag = False
 			# substitute each value:
 			for tag in tags.iterkeys():
-				if tag in cls._escapedTags:
-					# Escaped so won't match
-					continue
+				# ignore escaped or already done:
+				if tag in done: continue
 				value = str(tags[tag])
 				# search and replace all tags within value, that can be interpolated using other tags:
 				m = t.search(value)
-				done = []
+				refCounts = {}
 				#logSys.log(5, 'TAG: %s, value: %s' % (tag, value))
 				while m:
 					found_tag = m.group(1)
 					#logSys.log(5, 'found: %s' % found_tag)
-					if found_tag == tag or found_tag in done:
+					if found_tag == tag or refCounts.get(found_tag, 1) > MAX_TAG_REPLACE_COUNT:
 						# recursive definitions are bad
 						#logSys.log(5, 'recursion fail tag: %s value: %s' % (tag, value) )
 						return False
@@ -417,7 +420,9 @@ class CommandAction(ActionBase):
 						continue
 					value = value.replace('<%s>' % found_tag , tags[found_tag])
 					#logSys.log(5, 'value now: %s' % value)
-					done.append(found_tag)
+					# increment reference count:
+					refCounts[found_tag] = refCounts.get(found_tag, 0) + 1
+					# the next match for replace:
 					m = t.search(value, m.start())
 				#logSys.log(5, 'TAG: %s, newvalue: %s' % (tag, value))
 				# was substituted?
@@ -426,6 +431,9 @@ class CommandAction(ActionBase):
 					if t.search(value):
 						repFlag = True
 					tags[tag] = value
+				# no more sub tags (and no possible composite), add this tag to done set (just to be faster):
+				if '<' not in value: done.add(tag)
+			# stop interpolation, if no replacements anymore:
 			if not repFlag:
 				break
 		return tags
@@ -560,32 +568,33 @@ class CommandAction(ActionBase):
 			return True
 
 		_cmd_lock.acquire()
-		try: # Try wrapped within another try needed for python version < 2.5
+		try:
+			retcode = None  # to guarantee being defined upon early except
 			stdout = tempfile.TemporaryFile(suffix=".stdout", prefix="fai2ban_")
 			stderr = tempfile.TemporaryFile(suffix=".stderr", prefix="fai2ban_")
-			try:
-				popen = subprocess.Popen(
-					realCmd, stdout=stdout, stderr=stderr, shell=True,
-					preexec_fn=os.setsid  # so that killpg does not kill our process
-				)
-				stime = time.time()
+
+			popen = subprocess.Popen(
+				realCmd, stdout=stdout, stderr=stderr, shell=True,
+				preexec_fn=os.setsid  # so that killpg does not kill our process
+			)
+			stime = time.time()
+			retcode = popen.poll()
+			while time.time() - stime <= timeout and retcode is None:
+				time.sleep(0.1)
 				retcode = popen.poll()
-				while time.time() - stime <= timeout and retcode is None:
+			if retcode is None:
+				logSys.error("%s -- timed out after %i seconds." %
+				    (realCmd, timeout))
+				pgid = os.getpgid(popen.pid)
+				os.killpg(pgid, signal.SIGTERM)  # Terminate the process
+				time.sleep(0.1)
+				retcode = popen.poll()
+				if retcode is None:  # Still going...
+					os.killpg(pgid, signal.SIGKILL)  # Kill the process
 					time.sleep(0.1)
 					retcode = popen.poll()
-				if retcode is None:
-					logSys.error("%s -- timed out after %i seconds." %
-						(realCmd, timeout))
-					pgid = os.getpgid(popen.pid)
-					os.killpg(pgid, signal.SIGTERM)  # Terminate the process
-					time.sleep(0.1)
-					retcode = popen.poll()
-					if retcode is None: # Still going...
-						os.killpg(pgid, signal.SIGKILL)  # Kill the process
-						time.sleep(0.1)
-						retcode = popen.poll()
-			except OSError, e:
-				logSys.error("%s -- failed with %s" % (realCmd, e))
+		except OSError as e:
+			logSys.error("%s -- failed with %s" % (realCmd, e))
 		finally:
 			_cmd_lock.release()
 
@@ -603,15 +612,16 @@ class CommandAction(ActionBase):
 			return True
 		elif retcode is None:
 			logSys.error("%s -- unable to kill PID %i" % (realCmd, popen.pid))
-		elif retcode < 0:
-			logSys.error("%s -- killed with %s" %
-				(realCmd, signame.get(-retcode, "signal %i" % -retcode)))
+		elif retcode < 0 or retcode > 128:
+			# dash would return negative while bash 128 + n
+			sigcode = -retcode if retcode < 0 else retcode - 128
+			logSys.error("%s -- killed with %s (return code: %s)" %
+				(realCmd, signame.get(sigcode, "signal %i" % sigcode), retcode))
 		else:
 			msg = _RETCODE_HINTS.get(retcode, None)
 			logSys.error("%s -- returned %i" % (realCmd, retcode))
 			if msg:
 				logSys.info("HINT on %i: %s"
 							% (retcode, msg % locals()))
-			return False
-		raise RuntimeError("Command execution failed: %s" % realCmd)
+		return False