- improved usage output (don't put a long help if an error occurs);
- new option `--no-check-all` to avoid check of all regex's (first matched only);
- new option `-o`, `--out` to set token provided in output (disables check-all and outputs only expected data);
- test cases optimized and extended
provides new replacement tags for failregex to match subnets in form of IP-addresses with CIDR mask (gh-2559):
- `<CIDR>` - helper regex to match CIDR (simple integer form of net-mask);
- `<SUBNET>` - regex to match sub-net adresses (in form of IP/CIDR, also single IP is matched, so part /CIDR is optional);
this is avoided if some sets were added manually or can be avoided via overwriting of parameter `_nft_shutdown_table`, for example:
banaction = nftables[_nft_shutdown_table=''][...]
- ignore errors from mod_evasive in `normal` mode (mode-controlled now) (gh-2548);
- extended with option `mode` - `normal` (default) and `aggressive`
close gh-2548
`banaction = nftables[type=multiport]` with `protocol="tcp,udp,sctp"` in jail replace 3 separate actions.
more robust if deleting multiple references to set (rules in chain)
- nftables-common is removed
- nftables-allports is obsolete, replaced by nftables[type=allports]
- nftables-multiport is obsolete, replaced by nftables[type=multiport]
* be sure usedns is before all regex(s) in stream (this option is also allowed in the config of filter now)
* logpath after all log-related data (backend, date-pattern, etc)
normalize stream generation of filter-related parameters across FilterReader and JailReader (uses stream generator of filter now);
test cases extended (testOverrideFilterOptInJail) to cover this possibility.
Log format changed since ver. 9.11.0
Ref. ftp://ftp.isc.org/isc/bind9/9.11.0/RELEASE-NOTES-bind-9.11.0.html
"The logging format used for querylog has been altered. It now includes an additional field indicating the address in memory of the client object processing the query."
- common.conf: rewritten using section-based handling round about option logtype;
- option `logtype` extended with `rfc5424` to cover RFC 5424 log-format (see #2309);
now unban will happen not later than 10 tickets get banned regardless there are still active bans available (precedence of ban is 10 now);
closes gh-2410
also included within modes `normal`, `more` (`extra` and `aggressive`), since postfix
parameter `smtpd_hard_error_limit` is default 20 (additionally consider `maxretry`)
- specify default options (`logtype`) in default sections of filter-config (this allows to overwrite such options in Definition/Init sections within filter.local or includes also without setting that in the jail);
- fail2ban-regex: output real filter-options (after combine/interpolate) if verbose (`-vv`) or debug (`-l debug`).
sshd-journal: new test covering sshd journal logging format (matches short prefix-line simulating output of formatJournalEntry);
samplestestcase-factory extended with new option `fileOptions` to set common filter/test options for whole test-file
common.conf: differentiate "__prefix_line" for file/journal logtype's (speedup and fix parsing of systemd-journal);
samplestestcase.py: extends testSampleRegexsFactory to allow coverage of journal logtype;
closes gh-2383: asterisk can log timestamp if logs into systemd-journal (regex extended with optional part matching this)
fail2ban-regex: catch errors/exceptions by set of parameter, more verbose output if needed (`-v` or log-level `debug` would produce output of call-stack additionally).
extended with new option `--with-time` to provide more pretty and informative result (separated by new-line, including time strings: time of ban + ban-time = end of ban):
192.0.2.1 2019-01-06 22:24:48 + 300 = 2019-01-06 22:29:48
192.0.2.2 2019-01-06 22:24:48 + 600 = 2019-01-06 22:34:48
also it is possible now to provide separator-character as extra-parameter after `get <jail> banip ?sep-char?` (default is space).
removed unneeded test-cases (test code-base minimization) and unexpected manually changed files.
additionally OrderedDict is used now for cache (if available, so >= 2.7) - avoids (slow) search of expired items in full cache and always prefers older objects to remove (like FIFO).
Tests files are not always needed especially when installing on low
resource systems like an embedded one.
This patch adds the --without-tests option to skip building and
installing of tests files.
Signed-off-by: Angelo Compagnucci <angelo@amarulasolutions.com>
filter.d/sshd.conf: extended with new rules:
- Disconnecting ...: Change of username or service not allowed
- Disconnected from ... [preauth] (extra/aggressive mode only)
- recognizes failures logged using another format (something like session-id, IP enclosed in square brackets);
- failregex extended to catch connections rejected for policy reasons (gh-2228);
- extended with new default date-pattern `^(?:%%Y-)?%%m-%%d[ T]%%H:%%M:%%S(?:\.%%f)?` to cover
`YYYY-mm-dd HH:MM::SS.ms` as well as `mm-dd HH:MM::SS.ms` (so year is optional);
- more optional arguments in log-line (so accept [WARN] as well as [WARNING] and optional [SOFIA] hereafter);
jailreader.py: additionally relocate the option `logpath` after all log-related data (backend, date-pattern, etc) that may be needed by the first usage (gh-2173).
Thanks to Matt Stancliff (mattsta)
When new log paths are configured, their start offset is immediately determined
by a filter searching for (now - findTime).
But, since findTime is configured *after* the log is loaded and
searched, logs are only searched back by the default 10 minute findTime,
regardless of user configuration of jail settings.
So, findTime must be configured before logpath or else the default findtime
is used, which ignores any findtime time defined by the user.
This fixes new reads on startup for actual log files. The systemd filter
always performed as expected due to being setup after the jail's
findtime config submission.
additionally provides more info if handler/conversion failed (with double protection inside catch-case);
tests/utils.py: log handler "_MemHandler" of LogCaptureTestCase fixed now to be safe also (test-cases only);
tests/misctestcase.py: the safe logging of all possible constellations is covered in testSafeLogging now.
both should be additionally exception-safe, so avoid possible errors in log-handlers (concat, str. conversion, etc);
test cases extended to cover any possible variants (invalid chars in unicode, bytes, str + unterminated char-sequence) with both cases (with replace of chars, with and without errors inside adapter-handlers).
prevents sporadic tests-failures (too often by pypy) if waiting for some log-message (that was mistakenly swallowed up from cache-mechanism, so was logged but "dirty" flag is already reset)
Fixed bug with ignoreself option, which reached fail2ban-server as string 'False' (and not as boolean), so no matter which value was set - it results always in True (see gh-2078).
at least using both modes can ban port-scanners and prevent for other annoying "intruders", closing connection within preauth-stage (see gh-2085 for example).
revert sshd test case (better to use last given failure-id, so ipv6 instead ipv4, e. g. because of some wrong multi-line-id recognition);
improved output on AssertionError in samples-testcase factory.
intruder (as legitimate user) firstly tries to login with another user-name (brute-force), so hopes to reset failure counter by succeeded login;
this is fixed and covered in tests now;
sshd-filter extended to cover multiple-login attempts (also fully implements gh-2070);
The nftables framework replaces iptables. The fail2ban software already
includes support for nftables, so reflect that in the packaging.
Also, no need to `Recommends: iptables`, since is installed by default in every
Debian system. Instead, do `Recommends: nftables`.
Signed-off-by: Arturo Borrero Gonzalez <arturo@debian.org>
better error handling with error counting, differentiate special case ([Errno 24] Too many open files), with resulting stop of the server
(avoid flood the log file, closes gh-991 and similar issues);
restored auto-garbage, because of non-reference-counting python's (like pypy), otherwise it may leak there on objects like unix-socket, etc.
- try to acquire lock without blocking, if not possible - return cached/empty (max 5 times, otherwise do lock);
- minimized time of the lock of messages list;
- avoid sporadic dead-locking during cross lock together with lock within handling of self._strm.
- differentiate between unknown ban-time after upgrade (-2) and persistent ban-time (-1);
- better handling to correct current ban-time, uses `bantime.maxtime` (if increment allowed) or `bantime` of jail;
- extended test-cases in order to cover the change of `bantime.maxtime`.
* tag '0.10.2': (623 commits)
prepare release: bump version, update ChangeLog, man's and MANIFEST etc.
ChangeLog update
action.d/pf.conf: compatibility fix - recognizes that parameter `port` specified as empty, with or without braces (should be more backwards compatible to 0.9 now).
regex rewritten: a bit fewer vulnerable now and using non-capturing groups, test-cases extended in order to cover trying of injection on user name
nginx-http-auth: match usernames with spaces
regex updated using non-capturing groups
extended test-cases to cover new log-format (http_auth -> mod_auth)
Update lighttpd-auth.conf
file-filter's: provide stop function in order to explicitly delete/stop monitoring of each file.
Remove annoying error-message "rm_watch: cannot remove WD=2, Errno=Invalid argument (EINVAL)", logged from pyinotify-module if rm_watch called with non-existing watch file descriptor (probably multi-threaded issue by dual-remove). Closes gh-1865
should fix sporadic coverage decrease (don't cover "return", because too sporadic to get idle in pyinotify-callback);
fixed restoring sane environment (via stop/start) if invariant check failed: bypass possible errors in stop (if start/check succeeded hereafter); test cases extended to cover such situation. Closes gh-1997
action.d/hostsdeny.conf: actionunban rewritten using sed, also dots in IP were escaped now.
micro-fix: delete temporary file (forgotten in test-case `test_move_dir` by reassign to directory)
Update ChangeLog
stop ban of legitimate users with multiple public keys (e. g. git, etc), thereby differentiate between "invalid user" (going banned earlier) and valid users with public keys, for which the rejects of not valid public keys (failures) will be retarded up to "Too many authentication failures" resp. disconnect without success (accepted public key).
filter.d/sendmail-auth.conf - extended daemon for Fedora 24/RHEL - the daemon name is "sendmail" (gh-1632)
test cases extended in order to cover `firewallcmd-ipset` with `allports`
Update ChangeLog
firewallcmd-ipset-allports: implemented in `action.d/firewallcmd-ipset.conf` now (`action.d/firewallcmd-ipset-allports.conf` removed), usage:
...
algorithm of restore current bans after restart changed: update the restored ban-time (and therefore end of ban) of the ticket with ban-time of jail (as maximum), for all tickets with ban-time greater (or persistent); not affected if ban-time of the jail is unchanged between stop/start.
I have lighttpd 1.4.45 (Debian 9) and auth error log is different.
Now printing mod_auth and not http_auth.
I think that the change was in Lighttp 1.4.42
differentiate between "invalid user" (going banned earlier) and valid users with public keys, for which the rejects of not valid public keys (failures) will be retarded up to "Too many authentication failures" resp. disconnect without success (accepted public key).
allow to use dual parameter lists (coming through substitutions), e. g.: `name[p1=0, p2="..."][p3='...']`;
simplified explanation: `][` treats as `,` in new version.
cherry-picked from 0.10.
- provide waitForServerEnd method for decorator `with_foreground_server_thread`, to wait for real server stop if needed;
- accept any exit code in decorator `with_foreground_server_thread`, because multi-threaded, thus server can exit in-between;
- fix sporadic fail "AssertionError: 'Banned 5 / 5, 5 ticket(s)' was not found" (if some tickets will be processed earlier,
thus not as chunk but separately), so in case of:
Banned 1 / 1, 1 ticket(s) in 'nginx-blck-lst'
Banned 4 / 5, 5 ticket(s) in 'nginx-blck-lst'
fix-up for run_with_except_hook: avoid very sporadic error "'NoneType' object has no attribute 'exc_info'" (https://bugs.python.org/issue7336),
only extremely fast systems are affected ATM (2.x / 3.x), if thread ends nothing is available in .
Usage `logtarget = target[facility=..., datetime=on|off, format="..."]`:
- `facility` - specify syslog facility (default `daemon`, see https://docs.python.org/2/library/logging.handlers.html#sysloghandler
for the list of facilities);
- `datetime` - add date-time to the message (default on, ignored if `format` specified);
- `format` - specify own format how it will be logged, for example for short-log into STDOUT:
`fail2ban-server -f --logtarget 'stdout[format="%(relativeCreated)5d | %(message)s"]' start`;
Closes gh-1980
* introduces 2 new format directives (with corresponding `%Ex` prefix for more precise parsing):
- %k - one- or two-digit number giving the hour of the day (0-23) on a 24-hour clock,
(corresponds %H, but allows space if not zero-padded).
- %l - one- or two-digit number giving the hour of the day (12-11) on a 12-hour clock,
(corresponds %I, but allows space if not zero-padded).
* mysqld-auth test extended to cover new date-format in log.
Closes gh-1639
Added non-greedy fallback for quoted something (with lookahead simulated possessive greedy catch of non-quoted parts `[^"]*(?=")`).
Note that because host-info's are hereafter (with foreign input in-between), we would not use greedy or non-greedy catch-alls (`.*` or `.*?`) here (preventing performance losses).
When reporting the error `AUTH command used when not advertised`, Exim
starts with `SMTP protocol error in "........."`. Here, Exim logs the
SMTP command as it was provided by the connecting client.
https://github.com/Exim/exim/blob/exim-4_89+fixes/src/src/smtp_in.c#L2850
According to RFC 5321 (SMTP) "[..] a command verb [..] MAY be encoded
in upper case, lower case, or any mixture of upper and lower case with
no impact on its meaning."
https://tools.ietf.org/html/rfc5321#section-2.4
Lower case `auth login` brute-force attempts were seen in the wild and
were not caught by the current failregex.
This commit makes the failregex case-insensitive for the `AUTH`
command, so that lower case (`auth`) or mixed case (`aUtH`) now also
match. The failregex was already case-insensitive for the command
arguments (e.g. `AUTH login` already matched).
fix sporadic error by shutdown server in with_foreground_server_thread decorator (if shutdown too fast, but end-phase still does not reached the tester-thread);
Note the systemd-backend does not need the logpath at all;
Some defaults normalized (minimized configs, don't need to overwrite values in distribution-related path if equal).
add descriptions to stop syslog errors for extra_started_commands when running:
rc-service ipset describe
Oct 28 15:13:30 xxxx daemon.warn /etc/init.d/fail2ban[26446]: ^[[1m^[[36mreload^[[m: no description
Oct 28 15:13:30 xxxx daemon.warn /etc/init.d/fail2ban[26447]: ^[[1m^[[36mshowlog^[[m: no description
+ Added new user clarification for installing from source with git clone.
? Right now it is not clear where the fail2ban-0.11.0.tar.bz2 is available for download.
+ Added command for checking fail2ban version.
* add [Init?family=inet6] to nftables-common.conf and make nftable
expressions more modular
* change "ip protocol" to "meta l4proto" in nftables-allports.conf
since the former only works for IPv4
leave it end-anchored because variable part `user=<[^>]*>` (before `<HOST>`) to avoid injecting, but can be safe rewritten using `[^>]*` in opposite to "greedy" `user=<[^>]*>`.
- introduces mode `aggressive` and extends regex for this mode to match:
* no auth attempts (previously removed in gh-601, because of lots of false positives on misconfigured MTAs)
* disconnected before auth was ready
* client didn't finish SASL auth
- setup process generates `build/fail2ban.service` from `files/fail2ban.service.in` using distribution related bin-path;
- bug-fixing by running setup with option `--dry-run` (note: specify option `--dry-run` before `install`, like `python setup.py --dry-run install`);
- test cases extended to cover dry-run.
(especially critical if default value `encoding = auto` configured).
As PoC and coverage (this case fails without this "fix"):
$ env -i PATH="$PATH" bin/fail2ban-testcases --fast --no-network testAddBanInvalidEncoded
Variable `default_backend` switched to `%(default/backend)s`, so totally backwards compatible now, but now the setting of parameter `backend` in default section of `jail.local` can overwrite default backend also.
Test cases extended: test targeted section options "section/option" (default and cross sections options);
I got an issue like this:
[2016-05-15 22:53:00] SECURITY[26428] res_security_log.c: SecurityEvent="FailedACL",EventTV="2016-05-15T22:53:00.203+0300",Severity="Error",Service="AMI",EventVersion="1",AccountID="admin",SessionID="0x7fb580001518",LocalAddress="IPV4/TCP/0.0.0.0/5038",RemoteAddress="IPV4/TCP/78.129.227.4/62389",SessionTV="1970-01-01T03:00:00.000+0300"
# [sebres] rebased to current master and resolving conflicts.
introduced parameter `mode`: more (default, combines normal and rbl), auth, normal, rbl, ddos, extra or aggressive (combines all)
replacement for gh-1239, gh-1697, gh-1764; closes gh-1245, gh-1297.
Fixed mail.log path since in the default rsyslog configuration of debians the `mail.warn` is commented now (see `/etc/rsyslog.d/50-default.conf`: `#mail.warn -/var/log/mail.warn`).
Closes gh-1687
filter.d/roundcube-auth.conf: Fixed failregex when logging errors to journal instead to a local file.
Additionally fixes more complex injections on username.
Tokens `%z` and `%Z` are more precise now;
Introduced new tokens `%Exz` and `%ExZ` that fully support zone abbreviations and/or offset-based zones;
# TODO: because python currently does not support mixing of case-sensitive with case-insensitive matching,
# check how TZ (in uppercase) can be combined with %a/%b etc. (that are currently case-insensitive),
# to avoid invalid date-time recognition in strings like '11-Aug-2013 03:36:11.372 error ...'
# with wrong TZ "error", which is at least not backwards compatible.
# Hence %z currently match literal Z|UTC|GMT only (and offset-based), and %Exz - all zone abbreviations.
Commit 5678d08 of 2016-11-26 changed:
( user=<\S*>,)?
to:
( user=<[^>]+>,)?
The change from `*` (zero or more times) to `+` (one or more times) may
not have been intended. It will miss lines containing, for example:
Aborted login (tried to use disallowed plaintext auth): user=<>
This commit reverts the `+` back to `*`.
* Update Kerio Connect filter
Fixed regex for some log entries that did not get recognized and some additional error formats are added.
* Add missing colon, GitHub address
* Add filter tests
* Add missing test
This new option allows to force the time zone on log lines
that don't bear a time zone indication (GitHub issue #1773), so it behaves
actually with respect to log line contents as a default time zone.
For the time being, only fixed offset timezones (UTC or UTC[+-]hhmm) are
supported, but the implementation is designed to later on treat the case
of logical timezones with DST, e.g., Europe/Paris etc.
In particular, the timezone name gets passed all the way to the strptime
module, and the resulting offset is computed for the given log line, even
though for now, it doesn't actually depend on it.
Also, the DateTemplate subclass gets to choose whether to use it or not.
For instance, it doesn't make sense to apply a time zone offset to
Unix timestamps.
The drawback is to introduce an API change for DateTemplate. I hope it's
internal enough for that not being a problem.
normalize increment of ban-count or time (count increased in BanManager now, some dual increments fixed in the test-cases);
introduced new action-tag `<bancount>`, that is always incremented by each ban (starting by 1), opposite to tag `<bantime>` which can be prolonged retarded (up to 10 seconds)
- dynamic bantime: introduces new action-tag `<bantime>` corresponds to the current ban-time of the ticket;
Note: because it is dynamic, it should be normally removed from `jail.conf` (resp. `jail.local`).
- introduced new action command `actionprolong`, used for prolongation of the timeout (ban-time of the ticket);
- removed default `timeout` from `actionstart` of several actions;
- faster and safer function escapeTag (replacement at once in one run, '\n' and '\r' escaped also);
- Filter instance is already a thread;
- avoid stop pyinotify processing if an interim error occurs (and breaks main-loop, e. g. during multi-threaded processing by add/remove log-files)
If directory moved and the target is not watched path, so the monitoring of it could not be continued.
Now fixed with pending files await a monitoring if there (resp. its directories) appears again (respawn).
Closes gh-1769
ver. 0.9.7 (2017/05/11) - awaiting-victory
-----------
0.9.x line is no longer heavily developed. If you are interested in
new features (e.g. IPv6 support), please consider 0.10 branch and its
releases.
* Fixed a systemd-journal handling in fail2ban-regex (gh-1657)
* filter.d/sshd.conf
- Fixed non-anchored part of failregex (misleading match of colon inside
IPv6 address instead of `: ` in the reason-part by missing space, gh-1658)
(0.10th resp. IPv6 relevant only, amend for gh-1479)
* config/pathes-freebsd.conf
- Fixed filenames for apache and nginx log files (gh-1667)
* filter.d/exim.conf
- optional part `(...)` after host-name before `[IP]` (gh-1751)
- new reason "Unrouteable address" for "rejected RCPT" regex (gh-1762)
- match of complex time like `D=2m42s` in regex "no MAIL in SMTP connection" (gh-1766)
* filter.d/sshd.conf
- new aggressive rules (gh-864):
- Connection reset by peer (multi-line rule during authorization process)
- No supported authentication methods available
- single line and multi-line expression optimized, added optional prefixes
and suffix (logged from several ssh versions), according to gh-1206;
- fixed expression received disconnect auth fail (optional space after port
part, gh-1652)
and suffix (logged from several ssh versions), according to gh-1206;
* filter.d/suhosin.conf
- greedy catch-all before `<HOST>` fixed (potential vulnerability)
* filter.d/cyrus-imap.conf
- accept entries without login-info resp. hostname before IP address (gh-1707)
* Filter tests extended with check of all config-regexp, that contains greedy catch-all
before `<HOST>`, that is hard-anchored at end or precise sub expression after `<HOST>`
* New Actions:
- action.d/netscaler: Block IPs on a Citrix Netscaler ADC (gh-1663)
* New Filters:
- filter.d/domino-smtp: IBM Domino SMTP task (gh-1603)
* Introduced new log-level `MSG` (as INFO-2, equivalent to 18)
* tag '0.9.7': (33 commits)
Preparing for 0.9.7 release
Added newly added files to MANIFEST
update ChangeLog
filter.d/exim.conf: added new reason for "rejected RCPT" regex: Unrouteable address
try to fix travis integration of pypy3: setuptools recently dropped support for Python 3.0 - 3.2, but old pypy3 based on Python 3.2.5
filter.d/exim.conf: cherry-picked from 0.10, match complex time like `D=2m42s` (closes gh-1766)
Update ChangeLog #1757
filter.d/exim.conf: optional part `(...)` after host-name before `[IP]`, normalized over whole config file.
BF: specify explicit time offset not a time zone name to avoid needing tzdata during testing
Update ChangeLog
amend resp. restore of change from 59c35bc44a (gh-129): - logging of "Log rotation detected" with new MSG level - introduces new log-level MSG (as INFO-2, 18)
Update mysqld-auth.conf
Update ChangeLog
filter.d/cyrus-imap.conf: fixed `failregex` - accept entries without login-info resp. hostname before IP address
evil symlink removed: does not supported by some file systems (e. g. development over net share)
sshd-amend: optional space after port part
suhosin.conf: removed greedy match
sshd.conf: fixed expression "received disconnect ... auth fail" - optional space after port part (gh-1652)
change log update after rebase
sshd: additionally aggressive filter rules - no matching cipher resp. no matching key exchange method (gh-1545, gh-1117)
...
- `<fq-hostname>` - fully-qualified name of host (the same as `$(hostname -f)`)
- `<sh-hostname>` - short hostname (the same as `$(uname -n)`)
Execution of `uname -n` replaced in all mail actions with most interesting fully-qualified `<fq-hostname>`.
new action parameter "actionstart_on_demand" (bool) can be set to prevent/allow starting action on demand (default retrieved automatically, if some conditional parameter `param?family=...` presents in action properties);
It appears that, under Python3, on an IPv6 enabled machine,
the testing SMTP server on 'localhost' can turn out to listen on ::1 only,
which makes those tests break if the SMTP client part uses 127.0.0.1
directly. Using 'localhost' there as well makes the tests pass.
Removed warning "Mutliline regex set for jail ... but maxlines not greater than 1", because can be expected situation now:
non multi-line entry from systemd-filter containing new-lines (that should be ignored by anchors resp. entry parsed as single string);
small code review;
Regex will be compiled as MULTILINE only if needed (buffering with `maxlines` > 1), that enables:
- improve performance by the single line parsing;
- make regex more precise (because distinguish between anchors `^`/`$` for the begin/end of string and the new-line character '\n', e. g. if coming from filters (like systemd journal) that allow the parsing of log-entries contain new-line chars (as single entry);
Fail2ban will not ban a host which matches such addresses.
Option "ignoreip" affects additionally to "ignoreself" and don't need to include the DNS resp. IPs of the host self.
the situation details:
value of "_daemon" from default section "default" (with init section) falsely overwrites it from definition section "test" -
the resulting value of "_daemon" should be "test" in all 3 resulting failregex's (as specified in test.local),
fixed and covered now;
additionally more complex cases covered also (all filter parameters in jail via "%(known/...)s", dynamical interpolation across all, etc);
- the calling map contains normally dynamic values only (no recursive tags);
- recursive replacement can be vulnerable, because can contain foreign (user) input captured from log (will be replaced in the shell arguments);
- optional parameter `mode` rewritten: normal (default), ddos, extra or aggressive (combines all), see sshd for regex details);
test cases reformatted (since "filterOptions", we don't need multiple test log-files anymore);
- rewritten using `prefregex` and used MLFID-related multi-line parsing (by using tag `<F-MLFID>` instead of buffering with `maxlines`);
- optional parameter `mode` introduced: normal (default), extra or aggressive (see sendmail-reject for regex details);
test cases extended
Many times faster and fewer CPU-hungry because of parsing with `maxlines=1`, so without line buffering (scrolling of the buffer-window).
Combination of tags `<F-MLFID>` and `<F-NOFAIL>` can be used now to process multi-line logs using single-line expressions:
- tag `<F-MLFID>`: used to identify resp. store failure info for groups of log-lines with the same identifier (e. g. combined failure-info for the same conn-id by `<F-MLFID>(?:conn-id)</F-MLFID>`, see sshd.conf for example)
- tag `<F-NOFAIL>`: used as mark for no-failure (helper to accumulate common failure-info);
filter.d/sshd.conf: [sshd], [sshd-ddos], [sshd-aggressive] optimized with pre-filtering using new option `prefregex` and new multi-line handling.
Some filters extended with user name;
[filter.d/pam-generic.conf]: grave fix injection on user name to host fixed;
test-cases in testSampleRegexsFactory can now check the captured groups (using additionally fields in failJSON structure)
- dynamical string replacement within call of regexp.sub fixed with lambda-replacement (otherwise "sre_constants.error: bad escape \s at position");
- optional arguments (3.6 has more agrs by calling of SMTPServer.process_message);
- implicit convert byte to string, because python3.6 fails on binary data (test_smtp).
new ssh rule(s) added:
- Connection reset by peer (multi-line rule during authorization process);
- No supported authentication methods available;
Single line and multi-line expression optimized, added optional prefixes and suffix (logged from several ssh versions);
closes gh-864
Prevent executing of several actions (e.g. mail, send-mail etc) on restart (bans were already notified).
Test cases extended (smtp and by restart in ServerReloadTest).
Closes gh-1141
Closes gh-921
The fail2ban server can take several seconds to shut down. This can
make Gentoo's start-stop-service time out and decide that stopping has
failed, even if it actually succeeds a few seconds later.
The default timeout for start-stop-service if --retry is not specified
appears to be 5 seconds. Increase that to 30 seconds to be sure that if
fail2ban-server is going to be able to stop, it has time to do so.
* enh-rel0.9.6: (60 commits)
updated man pages
ENH: prep for 0.9.6 release (as of tomorrow)
BF: added missing entires into MANIFEST
Update ChangeLog
ChangeLog entry added + jail.conf review
code review, makes the test cases workable, added dev-notes
ChangeLog update
`filter.d/apache-modsecurity.conf` - fixed for newer version (one space, closes gh-1626) reviewed and optimized: - non-greedy catch-all replaced for safer match - unneeded catch-all anchoring removed - non-capturing groups
filter.d/dovecot.conf update: - fixes failregex, that ignores failures through some irrelevant info (closes#1623); - ignores whole additionally irrelevant info in anchored regex before fixed failure data `\((?:auth failed, \d+ attempts( in \d+ secs)?|tried to use (disabled|disallowed) \S+ auth)\)` - review, IPv6 compatibility fix, non-capturing groups
Update jail.conf
Use Fedora's backend-settings for openSUSE
amend after code review of merge gh-1581
Make changes and add test file
Add Mongodb-auth filter and jail
Update FILTERS
filter.d/sshd.conf: Match 'Invalid user' with 'port \d*'
ChangeLog entry added
filter.d/sendmail-reject.conf: double space (should be by missing dns-host only) Closes#1578
Update Changelog to reflect the new np.conf action
Create npf.conf for the NPF packet filter
...
- fixed for newer version (one space, closes gh-1626)
reviewed and optimized:
- non-greedy catch-all replaced for safer match
- unneeded catch-all anchoring removed
- non-capturing groups
- fixes failregex, that ignores failures through some irrelevant info (closes#1623);
- ignores whole additionally irrelevant info in anchored regex before fixed failure data `\((?:auth failed, \d+ attempts( in \d+ secs)?|tried to use (disabled|disallowed) \S+ auth)\)`
- review, IPv6 compatibility fix, non-capturing groups
added helper "action.d/helpers-common.conf", and `_grep_logs` part-command for actions needed grep logs from multiple log-files
test cases: executing of some complex actions covered
- starting service in normal mode (without forking)
- does not restart if service exited normally (exit-code 0, e.g. stopped via fail2ban-client)
- does not restart if service can not start (exit-code 255, e.g. wrong configuration, etc.)
- service can be additionally started/stopped with commands (fail2ban-client, fail2ban-server)
- normalize calling parameters (persistent parameters moved from function arguments to filter member variables)
- save last line as lambda instead of return it as string (lazy convert of process line tuple to string on demand, needed in fail2ban-regex only)
- datedetector rewritten more strict as earlier;
- default templates can be specified exacter using prefix/suffix syntax (via `datepattern`);
- more as one date pattern can be specified using option `datepattern` now (new-line separated);
- some default options like `datepattern` can be specified directly in section `[Definition]`, that avoids contrary usage of unnecessarily `[Init]` section, because of performance (each extra section costs time);
- option `datepattern` can be specified in jail also (jails without filters);
- if first group specified, only this will be cut out from search log-line (e. g.: `^date:[({DATE})]` will cut out only datetime match pattern, and leaves `date:[] failure ip...` for searching in filter);
- faster match and fewer searching of appropriate templates (DateDetector.matchTime calls rarer DateTemplate.matchDate now);
- standard filters extended with exact prefixed or anchored date templates;
template cache introduced (in opposition to default template cache, holds custom templates cached by pattern for possible common usage of same template/regex);
- `datepattern = {^LN-BEG}` - only line-begin anchored default patterns
(matches date only at begin of line, or with max distance up to 2 non-alphanumeric characters from line-begin);
- `datepattern = {*WD-BEG}` - only word-begin anchored default patterns;
- `datepattern = ^prefix{DATE}suffix` - exact specified default patterns (using prefix and suffix);
common filter configs gets a more precise, line-begin anchored (datepattern = {^LN-BEG}) resp. custom anchoring default date-patterns;
thereby follow rule "shortest distance to datetime should win", so possible collision causes search though all templates;
speedup it a little bit (not possible collision if distance <= 1 or if line-begin anchoring, so break search if such template found)
By change of the distance (e.g. another format found), the pattern with smallest distance will be always preferred now.
To speedup (template lookup) resp. minimize of list reorder counts, the distance will be used as divider factor of the template weight by the templates comparison.
* Misleading date patterns defined more precisely (using extended syntax %E[mdHMS]
for exact two-digit match)
* `filter.d/freeswitch.conf`
- Optional prefixes (server, daemon, dual time) if systemd daemon logs used (gh-1548)
- User part rewritten to accept IPv6 resp. domain after "@" (gh-1548)
- database duplicate code removed resp. merged with incr. version;
- ignores expired ban ticket directly in ban manager;
- don't change start of ban time for restored tickets in restoreCurrentBans (because of possible timing issues in the test-cases);
- small code review;
test cases could pass (so increase) verbosity to the client (and furthermore client to the server also), usable for debug purposes resp. simplifying read of the log-file;
custom and precise numeric log-levels can be given in test cases now;
JailThread get method `join` for safe usage of it, also for not started threads (test-cases or in case of error), that will be used for cleanup resp. wait purposes also (see join of pyinotify-filter);
waiting with `wait_for` extended with verifying of active flag;
implemented better error handling in some multi-threaded routines;
shutdown of jails rewritten (faster and safer, does not breaks shutdown process if some error occurred);
```bash
## start server with DEBUG log-level (ignore level read from fail2ban.conf):
fail2ban-client --loglevel DEBUG start
## or
fail2ban-server -c /cfg/path --loglevel DEBUG start
## keep server log-level by reload (without restart it)
fail2ban-client --loglevel DEBUG reload
## switch log-level back to INFO:
fail2ban-client set loglevel INFO
```
# [WARNING] TODO: be sure all banactions get a repair command, because otherwise stop/start will theoretically remove all the bans, but the tickets are still in BanManager, so in case of new failures it will not be banned, because "already banned" will happen.
- better ban/unban handling within actions (e.g. used dict instead of list)
- don't copy bans resp. its list on some operations;
- added new unbantime handling to relieve unBanList (prevent permanent searching for tickets to unban)
- prefer failure-ID as identifier of the ticket to its IP (most of the time the same, but it can be something else e.g. user name in some complex jails, as introduced in 0.10)
new replacement for `<ADDR>` in opposition to `<HOST>`, for separate usage of 2 address groups only (regardless of `usedns`), `ip4` and `ip6` together, without host (dns)
optimized FilterSystemd method `run`: better wait in idle (no busy-loop), better poll handling, the ban will executed anywhere (at least at 100th log-entry), also if we have never ending logging in this jail (e.g. extremely logging or too many failures)
systemd test cases extended
fixed pythonic filters and test scripts (running via "fail2ban-python" now);
fixed test case "testSetupInstallRoot" not for default python (also using direct call, out of virtualenv);
# Conflicts:
# config/filter.d/ignorecommands/apache-fakegooglebot
# fail2ban/tests/files/config/apache-auth/digest.py
# fail2ban/tests/files/ignorecommand.py
# fail2ban/tests/misctestcase.py
fixed pythonic filters and test scripts (running via "fail2ban-python" now);
fixed test case "testSetupInstallRoot" not for default python (also using direct call, out of virtualenv);
yoh: Squashed to ease cherry-picking into 0.9
* accept no space after "failed:"
fix issue #1497
* accept no space after "failed:"
* Update postfix-sasl
* Update postfix-sasl
* Update postfix-sasl
yoh: Squashed to ease cherry-picking into 0.9
* accept no space after "failed:"
fix issue #1497
* accept no space after "failed:"
* Update postfix-sasl
* Update postfix-sasl
* Update postfix-sasl
ASSP V1 development stopped at the end of 2014 and it is now deprecated.
All users were urged to upgrade to ASSP V2 which is still actively
developed.
fail2ban 0.9.5 (and trunk) still have code which only understands ASSP
V1 logs.
This means the filter ignores brute force attacks against ASSP. This fix
adds V2 support.
ASSP V1 development stopped at the end of 2014 and it is now deprecated.
All users were urged to upgrade to ASSP V2 which is still actively
developed. For some reason fail2ban 0.9.5 (and trunk) still have code
which only understands ASSP V1 logs. This means the filter ignores brute
force attacks against ASSP.
Now updated with anchored patterns tested against 6 months of log data.
ASSP V1 development stopped at the end of 2014 and it is now deprecated.
All users were urged to upgrade to ASSP V2 which is still actively
developed. For some reason fail2ban 0.9.5 (and trunk) still have code
which only understands ASSP V1 logs. This means the filter ignores brute
force attacks against ASSP.
* 'master' of git://github.com/fail2ban/fail2ban:
BF: do not rely on long relative path to upstairs config - symlink dereferenced copied during install (#1485)
ver. 0.9.5 (2016/07/15) - old-not-obsolete
-----------
0.9.x line is no longer heavily developed. If you are interested in
new features (e.g. IPv6 support), please consider 0.10 branch and its
releases.
* `filter.d/monit.conf`
- Extended failregex with new monit "access denied" version (gh-1355)
- failregex of previous monit version merged as single expression
* `filter.d/postfix.conf`, `filter.d/postfix-sasl.conf`
- Extended failregex daemon part, matching also `postfix/smtps/smtpd`
now (gh-1391)
* Fixed a grave bug within tags substitutions because of incorrect
detection of recursion in case of multiple inline substitutions
of the same tag (affected actions: `bsd-ipfw`, etc). Now tracks
the actual list of the already substituted tags (per tag instead
of single list)
* `filter.d/common.conf`
- Unexpected extra regex-space in generic `__prefix_line` (gh-1405)
- All optional spaces normalized in `common.conf`, test covered now
- Generic `__prefix_line` extended with optional brackets for the
date ambit (gh-1421), added new parameter `__date_ambit`
* `gentoo-initd` fixed `--pidfile` bug: `--pidfile` is option of
`start-stop-daemon`, not argument of fail2ban (see gh-1434)
* `filter.d/asterisk.conf`
- Fixed security log support for PJSIP and Asterisk 13+ (gh-1456)
- Improved log support for PJSIP and Asterisk 13+ with different
callID (gh-1458)
* New Actions:
- `action.d/firewallcmd-rich-rules` and `action.d/firewallcmd-rich-logging`
(gh-1367)
* New filters:
- slapd - ban hosts, that were failed to connect with invalid
credentials: error code 49 (gh-1478)
* Extreme speedup of all sqlite database operations (gh-1436),
by using of following sqlite options:
- (synchronous = OFF) write data through OS without syncing
- (journal_mode = MEMORY) use memory for the transaction logging
- (temp_store = MEMORY) temporary tables and indices are kept in memory
* journald journalmatch for pure-ftpd (gh-1362)
* Added additional regex filter for dovecot ldap authentication failures (gh-1370)
* `filter.d/exim*conf`
- Added additional regexes (gh-1371)
- Made port entry optional
* tag '0.9.5':
Added missing files to MANIFEST
BF: do not rely on long relative path to upstairs config - symlink common.conf
ver. 0.9.5 (2016/07/15) - old-not-obsolete
-----------
0.9.x line is no longer heavily developed. If you are interested in
new features (e.g. IPv6 support), please consider 0.10 branch and its
releases.
* `filter.d/monit.conf`
- Extended failregex with new monit "access denied" version (gh-1355)
- failregex of previous monit version merged as single expression
* `filter.d/postfix.conf`, `filter.d/postfix-sasl.conf`
- Extended failregex daemon part, matching also `postfix/smtps/smtpd`
now (gh-1391)
* Fixed a grave bug within tags substitutions because of incorrect
detection of recursion in case of multiple inline substitutions
of the same tag (affected actions: `bsd-ipfw`, etc). Now tracks
the actual list of the already substituted tags (per tag instead
of single list)
* `filter.d/common.conf`
- Unexpected extra regex-space in generic `__prefix_line` (gh-1405)
- All optional spaces normalized in `common.conf`, test covered now
- Generic `__prefix_line` extended with optional brackets for the
date ambit (gh-1421), added new parameter `__date_ambit`
* `gentoo-initd` fixed `--pidfile` bug: `--pidfile` is option of
`start-stop-daemon`, not argument of fail2ban (see gh-1434)
* `filter.d/asterisk.conf`
- Fixed security log support for PJSIP and Asterisk 13+ (gh-1456)
- Improved log support for PJSIP and Asterisk 13+ with different
callID (gh-1458)
* New Actions:
- `action.d/firewallcmd-rich-rules` and `action.d/firewallcmd-rich-logging`
(gh-1367)
* New filters:
- slapd - ban hosts, that were failed to connect with invalid
credentials: error code 49 (gh-1478)
* Extreme speedup of all sqlite database operations (gh-1436),
by using of following sqlite options:
- (synchronous = OFF) write data through OS without syncing
- (journal_mode = MEMORY) use memory for the transaction logging
- (temp_store = MEMORY) temporary tables and indices are kept in memory
* journald journalmatch for pure-ftpd (gh-1362)
* Added additional regex filter for dovecot ldap authentication failures (gh-1370)
* `filter.d/exim*conf`
- Added additional regexes (gh-1371)
- Made port entry optional
* tag '0.9.5': (70 commits)
DOC: preparations for 0.9.5 release
Added missing files to MANIFEST
another variant of regex
add trailing anchor to failregex
DOC: Reformatted ChangeLog into legit Markdown (Closes#962)
DOC: tuned up ChangeLog entries for 0.9.5
add PR id to ChangeLog
improved failregex according to @sebres recomendations
Improved changes of gh-1458: `[^']*` after callid was wrong, changed to `[^\)]*`; regexp anchored at the end; almost the same regex grouped to one;
Improve PJSIP log support for asterisk 13+ with different callID (Squash gh-1458) Change the asterisk pjsip filter to don't take the callId part Add optional part between "Request" and "from" Listed all log message from asterisk
* add `__prefix_line` to regex * fix time in log file
add info to log file
added sample log lines for slapd
adding openldap slapd filter
badip timeout option introduced, set to 30 seconds in our test cases (#1463)
DOC: changelog for recent exim filters tune up
Asterisk pjsip (#1456)
BF: finalize that sample log line for exim4
amend for new option of `usedns=raw` - forgotten validation fix inside setUseDns
RF: for consistency use (?:XXX)? instead of (?:|XXX)
...
* origin/master:
another variant of regex
add trailing anchor to failregex
add PR id to ChangeLog
improved failregex according to @sebres recomendations
* add `__prefix_line` to regex * fix time in log file
add info to log file
added sample log lines for slapd
adding openldap slapd filter
* commit '0.9.4-79-gaf8b650':
badip timeout option introduced, set to 30 seconds in our test cases (#1463)
DOC: changelog for recent exim filters tune up
Asterisk pjsip (#1456)
BF: finalize that sample log line for exim4
RF: for consistency use (?:XXX)? instead of (?:|XXX)
ENH: use non-capturing regex groups in exim-common and exim filters
ENH: exim filters -- make more use of %(host_info)s which in turn made more flexible
BF: make :port and I=[ip]:port optional for a "AUTH command used when not advertised"
* Improve PJSIP log support for Asterisk 13+
* Update changelog: filter.d/asterisk.conf - fix security log support for PJSIP and Asterisk 13+
* Change pjsip regexp with sebres observation, thanks to @nturcksin
- better recognition for usage of textToIp, expected or raw value should be used;
- separated failure id vs. host (if found use `fid` instead of `host` resp. `ip`);
- additional optional groups may be used in tags replacement by executing actions;
all optional spaces normalized in generic include `common.conf` + test cases are extended (using new example pseudo-filter and test log `zzz-generic-example`);
- client/server functionality moved away from bin and using now the common interface (introduced in fail2bancmdline);
- start in foreground fixed;
- server can act as client corresponding command line;
- command "restart" added: in opposite to "reload" in reality restarts the server (new process);
- several client/server bugs during starting process fixed.
respect standard spelling of IPv6 (with port) enclosed in brackets ([ipv6]:port), to prevent the necessarily changing of many filter definitions, comparison:
#IPv4:
127.0.0.1:55555
#IPv6:
[a:b:c:d::1]:55555
some small fixes (in test cases also) and code review and few optimizations;
implemented caching functionality for same substitutions inside replaceTag: very actual and extreme performance growth (up to 1000 times) for ban/unban because too slow substituteRecursiveTags by several tags and many includes, but totally unnecessary as long as parameters are not changing;
- introduced "conditional" sections, see for example `[Init?family=inet6]`;
- iptables-common and other iptables config(s) made IPv6 capable;
- several small code optimizations;
* all test cases passed (py3.x compatible);
SmartOS (and likely other Illumos platforms) enter log entries for failed sshd logins of the form:
`Authentication failed for USER from HOST`
The current sshd.conf regex matches `failure` -- add to this a match for `failed` to support Illumos
- differentiate between in loop and active (reset active only after really leaves a loop);
- differentiate between foreign and self loop-thread by stop/close connection;
- stops logging connection errors if too many errors;
- safe remove socket (no sporadic errors if too fast start/stop occurred);
ver. 0.9.4 (2016/03/08) - for-you-ladies
-----------
- Fixes:
* roundcube-auth jail typo for logpath
* Fix dnsToIp resolver for fqdn with large list of IPs (gh-1164)
* filter.d/apache-badbots.conf
- Updated useragent string regex adding escape for `+`
* filter.d/mysqld-auth.conf
- Updated "Access denied ..." regex for MySQL 5.6 and later (gh-1211, gh-1332)
* filter.d/sshd.conf
- Updated "Auth fail" regex for OpenSSH 5.9 and later
* Treat failed and killed execution of commands identically (only
different log messages), which addresses different behavior on different
exit codes of dash and bash (gh-1155)
* Fix jail.conf.5 man's section (gh-1226)
* Fixed default banaction for allports jails like pam-generic, recidive, etc
with new default variable `banaction_allports` (gh-1216)
* Fixed `fail2ban-regex` stops working on invalid (wrong encoded) character
for python version < 3.x (gh-1248)
* Use postfix_log logpath for postfix-rbl jail
* filters.d/postfix.conf - add 'Sender address rejected: Domain not found' failregex
* use `fail2ban_agent` as user-agent in actions badips, blocklist_de, etc (gh-1271)
* Fix ignoring the sender option by action_mw, action_mwl and action_c_mwl
* Changed filter.d/asterisk regex for "Call from ..." (few vulnerable now)
* Removed compression and rotation count from logrotate (inherit them from
the global logrotate config)
- New Features:
* New interpolation feature for definition config readers - `<known/parameter>`
(means last known init definition of filters or actions with name `parameter`).
This interpolation makes possible to extend a parameters of stock filter or
action directly in jail inside jail.local file, without creating a separately
filter.d/*.local file.
As extension to interpolation `%(known/parameter)s`, that does not works for
filter and action init parameters
* New actions:
- nftables-multiport and nftables-allports - filtering using nftables
framework. Note: it requires a pre-existing chain for the filtering rule.
* New filters:
- openhab - domotic software authentication failure with the
rest api and web interface (gh-1223)
- nginx-limit-req - ban hosts, that were failed through nginx by limit
request processing rate (ngx_http_limit_req_module)
- murmur - ban hosts that repeatedly attempt to connect to
murmur/mumble-server with an invalid server password or certificate.
- haproxy-http-auth - filter to match failed HTTP Authentications against a
HAProxy server
* New jails:
- murmur - bans TCP and UDP from the bad host on the default murmur port.
* sshd filter got new failregex to match "maximum authentication
attempts exceeded" (introduced in openssh 6.8)
* Added filter for Mac OS screen sharing (VNC) daemon
- Enhancements:
* Do not rotate empty log files
* Added new date pattern with year after day (e.g. Sun Jan 23 2005 21:59:59)
http://bugs.debian.org/798923
* Added openSUSE path configuration (Thanks Johannes Weberhofer)
* Allow to split ignoreip entries by ',' as well as by ' ' (gh-1197)
* Added a timeout (3 sec) to urlopen within badips.py action
(Thanks M. Maraun)
* Added check against atacker's Googlebot PTR fake records
(Thanks Pablo Rodriguez Fernandez)
* Enhance filter against atacker's Googlebot PTR fake records
(gh-1226)
* Nginx log paths extended (prefixed with "*" wildcard) (gh-1237)
* Added filter for openhab domotic software authentication failure with the
rest api and web interface (gh-1223)
* Add *_backend options for services to allow distros to set the default
backend per service, set default to systemd for Fedora as appropriate
* Performance improvements while monitoring large number of files (gh-1265).
Use associative array (dict) for monitored log files to speed up lookup
operations. Thanks @kshetragia
* Specified that fail2ban is PartOf iptables.service firewalld.service in
.service file -- would reload fail2ban if those services are restarted
* Provides new default `fail2ban_version` and interpolation variable
`fail2ban_agent` in jail.conf
* Enhance filter 'postfix' to ban incoming SMTP client with no fqdn hostname,
and to support multiple instances of postfix having varying suffix (gh-1331)
(Thanks Tom Hendrikx)
* files/gentoo-initd to use start-stop-daemon to robustify restarting the service
* tag '0.9.4': (138 commits)
MANIFEST RELEASE and man pages updates
Changes for the 0.9.4 release
datedetector: epoch time expression fix (now 10-11 chars, only whole number - anchored ^...\b or by special case within [], audit()) + test cases extended (positive/negative)
changelog about gentoo initd
added wp-admin
ENH(TST): a hypothetical example to show/test needing trailing anchoring
ENH: revert back to having detailed suffix anchored at the end for mysqld-auto.conf
Changelog for the recent PR and added Tom to THANKS
mysqld: failregex fixed (accepts different log level, more secure expression now); closes#1332
Add support for matching postfix multi-instance daemon names by default
DOC: removed Nick from listed as FreeBSD maintainer
DOC: adjusted ISSUE_TEMPLATE.md picking on @sebres's version
ENH: github templates for issues and PRs
ENH: add codecov support to travis.yml and bandge to README.md
gentoo-initd: Use start-stop-daemon in order to handle crashes better
regexp rewritten (few vulnerable as previous) + test case added
Update asterisk filter: changed regex for "Call from ...". Sometimes extension can have a plus symbol (+) because they can be phone number. Closes#1309
Add new regex into postfix filter. The new regexp is able to detect bad formatted SMTP EHLO command
Remove compression and count from logrotate
gentoo-initd: do not hide useful output
...
Currently, if fail2ban is killed (or crashes), its status will be
reported by '/etc/init.d/fail2ban status' as 'running' even though it
is not. Attempting to restart the service also fails, because Gentoo
unsuccessfully tries to stop the service.
By using start-stop-daemon and providing a pidfile, Gentoo will
instead report the status as 'crashed' and allow the service to be
restarted as normal.
- client/server functionality moved away from bin and using now the common interface (introduced in fail2bancmdline);
- start in foreground fixed;
- server can act as client corresponding command line;
- command "restart" added: in opposite to "reload" in reality restarts the server (new process);
- several client/server bugs during starting process fixed.
* pr/1232/head:
removed system.log
Removed old svn revision comment
removed false matches
Removed includes comment for screensharing jail
Now using a literal logpath for screensharing jail
Fixed blatant typo in regex
clarified comments on sample log format
Fixed name (again?)
Made screensharing jail off by default
Changed regex prequel
added entry for new screensharingd filter
name change & new sample data
Added json metadata
Sample log for test case
Replaced .* with literal
Update jail.conf
Added new path variable for system.log
Added in settings for screensharingd filter
Created file
Conflicts:
ChangeLog - moved to New Features
config/jail.conf - kept at the end
- default date templates cache + regexp compiling on the fly (if required);
- better usage of shared config over all test cases;
- obsolete test cases removed (simple test of memleak/gc);
- skip some slow test cases in fast mode (setup test, etc.);
prevents out of memory situation if many IP's makes extremely many failures (or very large files since last fail2ban run);
closes gh-1277
todo: parameter `maxentries` should be configurable (jail.conf resp. fail2ban.conf);
todo: adjust ban-time-incr branch by merge (table "bips").
```
in executeCmd
if retcode < 0:
TypeError: unorderable types: NoneType() < int()
```
2) prevents a rarely test case bug of testExecuteTimeoutWithNastyChildren, because no stdout (Resource temporarily unavailable), possible no flush by IO of the killing process;
definitely closes gh-161, also other usage of asyncore event loop (in test_smtp.py)
repair cache in ipToName (can returns None), precaching of invalid IPs (according to RFC 5737) to stop endless wait for resolving it in test cases.
- new options for "fail2ban-testcases" introduced: "-g" or "--no-gamin" and "-m" or "--memory-db", both are true also if "-f" or "--fast" specified,
for example: `fail2ban-testcases -ngm` will runs faster (because no network, no gamin, memory database)
but will use the same default sleep intervals as in production (in comparison to -nf);
- seekToTime rewritten, accuracy increased by seekToTime, extended for all FileFilter (PyInotify, Gamin also), test cases extended etc.
- common performance optimized and code reviewed;
-- cache dnsToIp, ipToName to prevent long wait during retrieving of ip/name for wrong dns or lazy dns-system;
-- instead of simple "sleep" used conditional wait "wait_for", that internal increases sleep interval up to sleeptime;
-- ticket / banmanager / failmanager modules are performance optimized;
-- api of filter (log files), jail, etc. rewritten and extended for performance purposes;
- performance of test cases optimized:
-- added option "--fast" to decrease wait intervals, avoid passive waiting, and skip few very slow test cases;
- code review after partially cherry pick of branch 'ban-time-incr' (see gh-716)
-- ticket module prepared to easy merge with newest version of 'ban-time-incr', now additionally holds banTime, banCount and json-data;
-- executeCmd partially moved from action to new module utils, etc.
-- python 2.6 compatibility;
- testExecuteTimeoutWithNastyChildren: test case repaired - wait for pid file inside bash, kill tree in any case (gh-1155);
- testSocket: test case repaired - wait for server thread starts a socket (listener)
addJail (executed before addLog) early uses a "INSERT OR REPLACE" statement to update "enabled" to 1 (and add jail the first time used at once), but this syntax in sqlite always deletes an entry (cause of constraint) and inserts it again, so because of CASCADE all log entries with this jail will be also deleted from logs table.
These fixes are pretty pedantic, but they do simplify the script a
little.
* Checking the existence of a file/directory before creating/deleting
it adds complexity and raciness. There are better options.
* mkdir -p does the job of making sure a directory exists. (It only
fails if there's a filesystem error or something.)
* Likewise, rm -f doesn't fail if the file doesn't exist.
* rm -r isn't neccessary because the socket shouldn't be a directory.
(If it is for some reason, that should be an error.)
However, still failing test, even though ```PYTHONPATH=. fail2ban-regex -v fail2ban/tests/files/logs/screensharingd /etc/fail2ban/filter.d/screensharingd.conf``` gives desired result
Previously, an attacker could fake a domain like
crawl-1-1-1-1.googlebot.com.fake.net and get resolved. This change
avoids to resolve fake Google domains.
Document the changes from 36919d9f in the ChangeLog and add myself to
the THANKS file (at @sebres suggestion).
Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
The regex for matching against "Auth fail" disconnect log message does
not match against current versions of ssh. OpenSSH 5.9 introduced
privilege separation of the pre-auth process, which included
[logging through monitor.c](http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/ssh/monitor.c.diff?r1=1.113&r2=1.114)
which adds " [preauth]" to the end of each message and causes the log
level to be prepended to each message.
It also fails to match against clients which send a disconnect message
with a description that is either empty or includes a space, since this
is the content in the log message after the disconnect code, per
[packet.c:1785](http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/ssh/packet.c?annotate=1.215),
which was matched by \S+. Although I have not observed this yet, I
couldn't find anything which would preclude it in [RFC
4253](https://tools.ietf.org/html/rfc4253#section-11.1) and since the
message is attacker-controlled it provides a way to avoid getting
banned.
This commit fixes both issues.
Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
* origin/master:
DOC: changelog for the timeout change
Set Timeout at urlopen to 3 seconds
README :: init/service example mentions debian based systems as the example
README :: fitted paragraph style
BF: disable testing on python 3.2 until coverage gets a fix
README :: Some style/grammar tweaks, and init/service script mention. Re: #1193
Set Timeout at urlopen to 3 seconds
* origin/master:
Comment spelling fixes
logrotate: Do not rotate empty logs
logrotate: Remove outdated Fedora comment
Updated Changelog
Bugfix for dnsToIp resolver for fqdn with large list of IPs; closes#1164
Added apache-badbots.conf
Update gen_badbots
Update apache-badbots.conf
changelog for freshly merged PR (roundcube-auth definition of logpath)
Fixed line suspected to be faulty
renamed <NAME> to correct <ACT> in protocol
Conflicts:
fail2ban/tests/actiontestcase.py -- due to comments fix up
As a useful side effect, prevents "Unable to contact server. Is it
running?" mails from cron when fail2ban hasn't been (intentionally)
running nor thus logging anything either.
ver. 0.9.3 (2015/08/01) - lets-all-stay-friends
----------
- IMPORTANT incompatible changes:
* filter.d/roundcube-auth.conf
- Changed logpath to 'errors' log (was 'userlogins')
* action.d/iptables-common.conf
- All calls to iptables command now use -w switch introduced in
iptables 1.4.20 (some distribution could have patched their
earlier base version as well) to provide this locking mechanism
useful under heavy load to avoid contesting on iptables calls.
If you need to disable, define 'action.d/iptables-common.local'
with empty value for 'lockingopt' in `[Init]` section.
* mail-whois-lines, sendmail-geoip-lines and sendmail-whois-lines
actions now include by default only the first 1000 log lines in
the emails. Adjust <grepopts> to augment the behavior.
- Fixes:
* reload in interactive mode appends all the jails twice (gh-825)
* reload server/jail failed if database used (but was not changed) and
some jail active (gh-1072)
* filter.d/dovecot.conf - also match unknown user in passwd-file.
Thanks Anton Shestakov
* Fix fail2ban-regex not parsing journalmatch correctly from filter config
* filter.d/asterisk.conf - fix security log support for Asterisk 12+
* filter.d/roundcube-auth.conf
- Updated regex to work with 'errors' log (1.0.5 and 1.1.1)
- Added regex to work with 'userlogins' log
* action.d/sendmail*.conf - use LC_ALL (superseeding LC_TIME) to override
locale on systems with customized LC_ALL
* performance fix: minimizes connection overhead, close socket only at
communication end (gh-1099)
* unbanip always deletes ip from database (independent of bantime, also if
currently not banned or persistent)
* guarantee order of dbfile to be before dbpurgeage (gh-1048)
* always set 'dbfile' before other database options (gh-1050)
* kill the entire process group of the child process upon timeout (gh-1129).
Otherwise could lead to resource exhaustion due to hanging whois
processes.
* resolve /var/run/fail2ban path in setup.py to help installation
on platforms with /var/run -> /run symlink (gh-1142)
- New Features:
* RETURN iptables target is now a variable: <returntype>
* New type of operation: pass2allow, use fail2ban for "knocking",
opening a closed port by swapping blocktype and returntype
* New filters:
- froxlor-auth - Thanks Joern Muehlencord
- apache-pass - filter Apache access log for successful authentication
* New actions:
- shorewall-ipset-proto6 - using proto feature of the Shorewall. Still requires
manual pre-configuration of the shorewall. See the action file for detail.
* New jails:
- pass2allow-ftp - allows FTP traffic after successful HTTP authentication
- Enhancements:
* action.d/cloudflare.conf - improved documentation on how to allow
multiple CF accounts, and jail.conf got new compound action
definition action_cf_mwl to submit cloudflare report.
* Check access to socket for more detailed logging on error (gh-595)
* fail2ban-testcases man page
* filter.d/apache-badbots.conf, filter.d/nginx-botsearch.conf - add
HEAD method verb
* Revamp of Travis and coverage automated testing
* Added a space between IP address and the following colon
in notification emails for easier text selection
* Character detection heuristics for whois output via optional setting
in mail-whois*.conf. Thanks Thomas Mayer.
Not enabled by default, if _whois_command is set to be
%(_whois_convert_charset)s (e.g. in action.d/mail-whois-common.local),
it
- detects character set of whois output (which is undefined by
RFC 3912) via heuristics of the file command
- converts whois data to UTF-8 character set with iconv
- sends the whois output in UTF-8 character set to mail program
- avoids that heirloom mailx creates binary attachment for input with
unknown character set
* tag '0.9.3': (99 commits)
Release changes (too much of manual "labor"! ;))
BF: realpath for /var/run/fail2ban Closes#1142
Changelog entry for killpg fix
Changelog entries for Serge's fixes
bug fix: option 'dbpurgeage' was never set (always default) by start of fail2ban, because of invalid sorting of options ('dbfile' should be always set before other database options) / closes#1048, closes#1050
BF: guarantee order of dbfile to be before dbpurgeage (Closes#1048)
DOC: Changelog for shorewall-ipset-proto6.conf + adjusted its description
DOC: moved and adjusted changelog entry from 0.9.2 within 0.9.3 to come
TST: test to verify killing stuck children processes
BF: kill the entire process group upon timeout (Close#1129)
Limit the number of log lines in *-lines.conf actions
ipjailmatches is on one line with its description in man jail.conf
DOC: Changelog for iptables -w change
Remove self.printlog() call
Remove literal "TODO" from method's name
BF: do not wrap iptables into itself. Thanks Lee
Added a space between IP address and the following colon
BF: symbiosis-blacklist-allports now also requires iptables-common.conf
RF: use <iptables> to take effect of it being a parameter
ENH: added lockingopt option for iptables actions, made iptables cmd itself a parameter
...
* 'grep-m1k' of github.com:szepeviktor/fail2ban:
Limit the number of log lines in *-lines.conf actions
Conflicts:
ChangeLog -- took both versions and adjusted the new one
for -n 1000 change
Reorganize .travis.yml
Separate coverage tests for Python 2 and Python 3
Execute setup.py install using the environment's Python exe
Sanitize Travis execution order
Event containing Challenge may come without ReceivedChallenge, but with
Response and ExpectedResponse.
Also Challenge now accepts '/' character, since it is used at least by PJSIP.
Unlike chan_sip and manager, PJSIP populates SessionID using
Call-Id header of a related SIP message.
As Call-Id of a SIP message can contain almost anything,
the regular expression for SessionID has been loosened.
Long delayed
ver. 0.9.2 (2015/04/29) - better-quick-now-than-later
----------
- Fixes:
* Fix ufw action commands
* infinite busy loop on _escapedTags match in substituteRecursiveTags gh-907.
Thanks TonyThompson
* port[s] typo in jail.conf/nginx-http-auth gh-913. Thanks Frederik Wagner
(fnerdwq)
* $ typo in jail.conf. Thanks Skibbi. Debian bug #767255
* grep'ing for IP in *mail-whois-lines.conf should now match also
at the beginning and EOL. Thanks Dean Lee
* jail.conf
- php-url-fopen: separate logpath entries by newline
* failregex declared direct in jail was joined to single line (specifying of
multiple expressions was not possible).
* filters.d/exim.conf - cover different settings of exim logs
details. Thanks bes.internal
* filter.d/postfix-sasl.conf - failregex is now case insensitive
* filters.d/postfix.conf - add 'Client host rejected error message' failregex
* fail2ban/__init__.py - add strptime thread safety hack-around
* recidive uses iptables-allports banaction by default now.
Avoids problems with iptables versions not understanding 'all' for
protocols and ports
* filter.d/dovecot.conf
- match pam_authenticate line from EL7
- match unknown user line from EL7
* Use use_poll=True for Python 2.7 and >=3.4 to overcome "Bad file
descriptor" msgs issue (gh-161)
* filter.d/postfix-sasl.conf - tweak failregex and add ignoreregex to ignore
system authentication issues
* fail2ban-regex reads filter file(s) completely, incl. '.local' file etc.
(gh-954)
* firewallcmd-* actions: split output into separate lines for grepping (gh-908)
* Guard unicode encode/decode issues while storing records in the database.
Fixes "binding parameter error (unsupported type)" (gh-973), thanks to kot
for reporting
* filter.d/sshd added regex for matching openSUSE ssh authentication failure
* filter.d/asterisk.conf:
- Dropped "Sending fake auth rejection" failregex since it incorrectly
targets the asterisk server itself
- match "hacking attempt detected" logs
- New Features:
- New filters:
- postfix-rbl Thanks Lee Clemens
- apache-fakegooglebot.conf Thanks Lee Clemens
- nginx-botsearch Thanks Frantisek Sumsal
- drupal-auth Thanks Lee Clemens
- New recursive embedded substitution feature added:
- `<<PREF>HOST>` becomes `<IPV4HOST>` for PREF=`IPV4`;
- `<<PREF>HOST>` becomes `1.2.3.4` for PREF=`IPV4` and IPV4HOST=`1.2.3.4`;
- New interpolation feature for config readers - `%(known/parameter)s`.
(means last known option with name `parameter`). This interpolation makes
possible to extend a stock filter or jail regexp in .local file
(opposite to simply set failregex/ignoreregex that overwrites it),
see gh-867.
- Monit config for fail2ban in files/monit/
- New actions:
- action.d/firewallcmd-multiport and action.d/firewallcmd-allports Thanks Donald Yandt
- action.d/sendmail-geoip-lines.conf
- action.d/nsupdate to update DNSBL. Thanks Andrew St. Jean
- New status argument for fail2ban-client -- flavor:
fail2ban-client status <jail> [flavor]
- empty or "basic" works as-is
- "cymru" additionally prints (ASN, Country RIR) per banned IP
(requires dnspython or dnspython3)
- Flush log at USR1 signal
- Enhancements:
* Enable multiport for firewallcmd-new action. Closes gh-834
* files/debian-initd migrated from the debian branch and should be
suitable for manual installations now (thanks Juan Karlo de Guzman)
* Define empty ignoreregex in filters which didn't have it to avoid
warnings (gh-934)
* action.d/{sendmail-*,xarf-login-attack}.conf - report local
timezone not UTC time/zone. Closes gh-911
* Conditionally log Ignore IP with reason (dns, ip, command). Closes gh-916
* Absorbed DNSUtils.cidr into addr2bin in filter.py, added unittests
* Added syslogsocket configuration to fail2ban.conf
* Note in the jail.conf for the recidive jail to increase dbpurgeage (gh-964)
* tag '0.9.2':
Hope for release tomorrow
BF: if no /dev/log on Linux -- don't expect setting syslog to work
Fix actions in ufw.conf
Add drupal-auth filter and jail
* commit '0f75ed5e2ab1159e45a7771a7a4e90c877ec848e':
Just use a system wide python in the tests digest.py
DOC: Slight tune up to RELEASE doc -- no need for PYTHONPATH to run tests
MANIFEST: updated for some new files, sorted all entries, removed some duplicates
Initial changes for the release -- simplified ChangeLog header etc
On Ubuntu 15.04 the ufw action was not working.
- With empty <application>, receiving errors:
2015-04-24 16:28:35,204 fail2ban.filter [8527]: INFO [sshd] Found 43.255.190.157
2015-04-24 16:28:35,695 fail2ban.actions [8527]: NOTICE [sshd] Ban 43.255.190.157
2015-04-24 16:28:35,802 fail2ban.action [8527]: ERROR [ -n "" ] && app="app " -- stdout: b''
2015-04-24 16:28:35,803 fail2ban.action [8527]: ERROR [ -n "" ] && app="app " -- stderr: b''
2015-04-24 16:28:35,803 fail2ban.action [8527]: ERROR [ -n "" ] && app="app " -- returned 1
- With action = ufw[application=OpenSSH], it was silently not doing
anything (no errors after "Ban x.x.x.x", but no IP addresses in ufw
status).
Re-arranged the bash commands on two lines, and it works with or without
<application>.
Long delayed and possibly incomplete 0.9.2 release:
ver. 0.9.2 (2015/04/26) - better-quick-now-than-later
----------
- Fixes:
* infinite busy loop on _escapedTags match in substituteRecursiveTags gh-907.
Thanks TonyThompson
* port[s] typo in jail.conf/nginx-http-auth gh-913. Thanks Frederik Wagner
(fnerdwq)
* $ typo in jail.conf. Thanks Skibbi. Debian bug #767255
* grep'ing for IP in *mail-whois-lines.conf should now match also
at the beginning and EOL. Thanks Dean Lee
* jail.conf
- php-url-fopen: separate logpath entries by newline
* failregex declared direct in jail was joined to single line (specifying of
multiple expressions was not possible).
* filters.d/exim.conf - cover different settings of exim logs
details. Thanks bes.internal
* filter.d/postfix-sasl.conf - failregex is now case insensitive
* filters.d/postfix.conf - add 'Client host rejected error message' failregex
* fail2ban/__init__.py - add strptime thread safety hack-around
* recidive uses iptables-allports banaction by default now.
Avoids problems with iptables versions not understanding 'all' for
protocols and ports
* filter.d/dovecot.conf
- match pam_authenticate line from EL7
- match unknown user line from EL7
* Use use_poll=True for Python 2.7 and >=3.4 to overcome "Bad file
descriptor" msgs issue (gh-161)
* filter.d/postfix-sasl.conf - tweak failregex and add ignoreregex to ignore
system authentication issues
* fail2ban-regex reads filter file(s) completely, incl. '.local' file etc.
(gh-954)
* firewallcmd-* actions: split output into separate lines for grepping (gh-908)
* Guard unicode encode/decode issues while storing records in the database.
Fixes "binding parameter error (unsupported type)" (gh-973), thanks to kot
for reporting
* filter.d/sshd added regex for matching openSUSE ssh authentication failure
* filter.d/asterisk.conf:
- Dropped "Sending fake auth rejection" failregex since it incorrectly
targets the asterisk server itself
- match "hacking attempt detected" logs
- New Features:
- New filters:
- postfix-rbl Thanks Lee Clemens
- apache-fakegooglebot.conf Thanks Lee Clemens
- nginx-botsearch Thanks Frantisek Sumsal
- New recursive embedded substitution feature added:
- `<<PREF>HOST>` becomes `<IPV4HOST>` for PREF=`IPV4`;
- `<<PREF>HOST>` becomes `1.2.3.4` for PREF=`IPV4` and IPV4HOST=`1.2.3.4`;
- New interpolation feature for config readers - `%(known/parameter)s`.
(means last known option with name `parameter`). This interpolation makes
possible to extend a stock filter or jail regexp in .local file
(opposite to simply set failregex/ignoreregex that overwrites it),
see gh-867.
- Monit config for fail2ban in files/monit/
- New actions:
- action.d/firewallcmd-multiport and action.d/firewallcmd-allports Thanks Donald Yandt
- action.d/sendmail-geoip-lines.conf
- action.d/nsupdate to update DNSBL. Thanks Andrew St. Jean
- New status argument for fail2ban-client -- flavor:
fail2ban-client status <jail> [flavor]
- empty or "basic" works as-is
- "cymru" additionally prints (ASN, Country RIR) per banned IP
(requires dnspython or dnspython3)
- Flush log at USR1 signal
- Enhancements:
* Enable multiport for firewallcmd-new action. Closes gh-834
* files/debian-initd migrated from the debian branch and should be
suitable for manual installations now (thanks Juan Karlo de Guzman)
* Define empty ignoreregex in filters which didn't have it to avoid
warnings (gh-934)
* action.d/{sendmail-*,xarf-login-attack}.conf - report local
timezone not UTC time/zone. Closes gh-911
* Conditionally log Ignore IP with reason (dns, ip, command). Closes gh-916
* Absorbed DNSUtils.cidr into addr2bin in filter.py, added unittests
* Added syslogsocket configuration to fail2ban.conf
* Note in the jail.conf for the recidive jail to increase dbpurgeage (gh-964)
* tag '0.9.2': (140 commits)
DOC: Slight tune up to RELEASE doc -- no need for PYTHONPATH to run tests
MANIFEST: updated for some new files, sorted all entries, removed some duplicates
Initial changes for the release -- simplified ChangeLog header etc
added \s after host
replaced .* before rhost with regex matching all the previous fields
Fixed typo in filter description authentification instead of authentication
Fixed the UTC -> CEST difference...
Added changes to ChangeLog & updated sample test cases
updated filter.d/sshd.conf
Do not run smtp tests if no_network set
BF: if install pypy -- come back to original directory
BF(OSX): apparently exceptions could not be compared for identity, use repr
very long time resolving IP for address "abcdef" on some PDC, under NAT etc. - replaced via "abcdef.abcdef" to prevent searching in local domains;
fix test for invalid IP (use TEST-NET-1 according to RFC 5737): since fef031b3cd failed, because on some platforms like vm:debian 10.0.0.0 returns 'localhost' (intern network).
Match hacking attempt IP instead of asterisk server IP (closes#1000)
BF: fixing up version comparison for pypy. Issue appeared in 2.5.0
ENH: minor formatting, no functional changes
BF: do not expect setting logtarget to SYSLOG to work on non-Linuxes
Added a comment about systemd backend for jails with logs outside of journal (Closes#959)
DOC: make a warning for recidive jail to increase dbpurgeage (Closes#964)
...
when set by user,
- detects character set of whois output (which is undefined by RFC 3912) via heuristics of the file command
- converts whois data to UTF-8 character set with iconv
- sends the whois output in UTF-8 character set to mail program
- avoids that heirloom mailx creates binary attachment for input with unknown character set
00:11 yoh: upgraded to 2.5.0 from debianexperimental and now reproduced locally
00:12 yoh: why pypy --version directs output to stderr? thereis no error
00:12 mattip: ok, can you try with a nightly from http://buildbot.pypy.org/nightly/trunk ?
00:15 * yoh running with 2.6.0-alpha0...
00:16 yoh: clean -- no crash
should avoid
File /usr/lib/python3.4/asyncore.py, line 208, in loop
poll_fun(timeout, map)
File /usr/lib/python3.4/asyncore.py, line 145, in poll
r, w, e = select.select(r, w, e, timeout)
OSError: [Errno 9] Bad file descriptor
Use ipset proto6 in shorewall. You must follow the rules to enable ipset in you blacklist
if you have a lot of spam (my case) is better use ipset rather than shorewall command line (is my firewall)
stop fail2ban with shorewall on one list of 1000 Ips takes 5 min with ipset in shorewall 10 sec.
* 'master' of https://github.com/rumple010/fail2ban:
Changed default TTL value to 60 seconds.
Added a reminder to create an nsupdate.local file to set required options.
Modified the ChangeLog and THANKS files to reflect the addition of action.d/nsupdate.conf.
add nsupdate action
Conflicts:
ChangeLog
As https://travis-ci.org/fail2ban/fail2ban/jobs/50609529 query might fail in
that response would not contain "categories". With this change we will handle
it explicitly and will spit out ValueError, providing information about
the response so it could be troubleshooted
Deleting not existent is not error.
Adding already present is not error.
Otherwise all those entries becomes stale forever, not removed and its number increases over time.
* 'enh/clarifyDnsUtilsMethods' of https://github.com/leeclemens/fail2ban:
Update ChangeLog
Clarify filter.DNSUtils functions' terminology and add unittests
Conflicts:
ChangeLog -- rephrased a bit as well
- Add : to match "SASL LOGIN authentication failed: Password:"
- Add ignoreregex to ignore system authentication issues:
"warning: unknown[1.1.1.1]: SASL LOGIN authentication failed: Connection lost to authentication server"
- Add test log messages for both
* 'enh/local_time_zone' of https://github.com/yarikoptic/fail2ban:
fixed typos, thanks szepeviktor for review
ENH: use non-UTC date invocation (without -u) and report offset for localzone (%z)
Conflicts:
ChangeLog
Adds a new action file that uses nsupdate to dynamically update a BIND
zone file with a TXT resource record representing a banned IP address.
Resource record is deleted from the zone when the ban expires.
test cases merged from remote-tracking branch 'yarikoptic:enh/embedded_tags' into issue/907
infinite busy loop on _escapedTags match in substituteRecursiveTags gh-907
* 'master' of git://github.com/fail2ban/fail2ban:
Update year in postfix logs test file
Add 'Client host rejected error message' regex Not sure if it was reworded (using Postfix 2.6) or a slightly different error, but I only have "Client host rejected: cannot find your hostname"
* commit '0.9.1-44-gd65c4f8': (31 commits)
moved debian's initd file to files/debian-initd from debian branch
Update ChangeLog
Monit config
BF: adjusted for new IP of example.com
downcase example
Added an item to "Fixes"
postfix-sasl failregex case insensitive
clean all after test setup (removes a build directory in current root of fail2ban)
exim filter: correct failregex for exim with extended log options
small fix: no cover for failed case
testSetupInstallRoot will be always skipped, because of "wrong" location of 'setup.py';
better and scalable solution for gh-867 (and gh-868), using only name convention like %(known/failregex)s to add custom expressions, so no interface changes in jail.conf are necessary (for example see test-known-interp in test cases);
Changelog entry for preceding fix
Separate php-url-fopen logpath by newline
python 2.6 compatibility: preventing RuntimeError: dictionary changed size during iteration.
interpolation of config readers extended with `%(known/parameter)s`. (means last known option with name `parameter`).
test cases extended (now correct)
BF: failregex declared direct in jail was joined to single line, (specifying of multiple expressions was not possible); feature request (gh-867): new options for jail introduced addfailregex/addignoreregex: extends regex specified in filter (opposite to failregex/ignoreregex that overwrites it);
Add ignoreregex to avoid warning on start
Add ignoreregex to avoid warning on start
...
feature request (gh-867): new options for jail introduced addfailregex/addignoreregex: extends regex specified in filter (opposite to failregex/ignoreregex that overwrites it);
• Do not install the following configuration files which are not used within the
Debian package of fail2ban:
/etc/fail2ban/paths-fedora.conf
/etc/fail2ban/paths-freebsd.conf
/etc/fail2ban/paths-osx.conf
Closes: Debian bug #767123 (https://bugs.debian.org/767123).
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
test cases: extended with test to check action together with database functionality (ex.: to verify lambdas in checkBan);
database: getBansMerged should work within lock, using reentrant lock (cause call of getBans inside of getBansMerged);
What aught to be a bugfix release delayed into a featured release 0.9.1
ver. 0.9.1 (2014/10/29) - better, faster, stronger
----------
- Refactoring (IMPORTANT -- Please review your setup and configuration):
* iptables-common.conf replaced iptables-blocktype.conf
(iptables-blocktype.local should still be read) and now also
provides defaults for the chain, port, protocol and name tags
- Fixes:
* start of file2ban aborted (on slow hosts, systemd considers the server has
been timed out and kills him), see gh-824
* UTF-8 fixes in pure-ftp thanks to Johannes Weberhofer. Closes gh-806.
* systemd backend error on bad utf-8 in python3
* badips.py action error when logging HTTP error raised with badips request
* fail2ban-regex failed to work in python3 due to space/tab mix
* recidive regex samples incorrect log level
* journalmatch for recidive incorrect PRIORITY
* loglevel couldn't be changed in fail2ban.conf
* Handle case when no sqlite library is available for persistent database
* Only reban once per IP from database on fail2ban restart
* Nginx filter to support missing server_name. Closes gh-676
* fail2ban-regex assertion error caused by miscount missed lines with
multiline regex
* Fix actions failing to execute for Python 3.4.0. Workaround for
http://bugs.python.org/issue21207
* Database now returns persistent bans on restart (bantime < 0)
* Recursive action tags now fully processed. Fixes issue with bsd-ipfw
action
* Fixed TypeError with "ipfailures" and "ipjailfailures" action tags.
Thanks Serg G. Brester
* Correct times for non-timezone date times formats during DST
* Pass a copy of, not original, aInfo into actions to avoid side-effects
* Per-distribution paths to the exim's main log
* Ignored IPs are no longer banned when being restored from persistent
database
* Manually unbanned IPs are now removed from persistent database, such they
wont be banned again when Fail2Ban is restarted
* Pass "bantime" parameter to the actions in default jail's action
definition(s)
* filters.d/sieve.conf - fixed typo in _daemon. Thanks Jisoo Park
* cyrus-imap -- also catch also failed logins via secured (imaps/pop3s).
Regression was introduced while strengthening failregex in 0.8.11 (bd175f)
Debian bug #755173
* postfix-sasl - added journalmatch. Thanks Luc Maisonobe
* postfix* - match with a new daemon string (postfix/submission/smtpd).
Closes gh-804 . Thanks Paul Traina
* apache - added filter for AH01630 client denied by server configuration.
- New features:
- New filters:
- monit Thanks Jason H Martin
- directadmin Thanks niorg
- apache-shellshock Thanks Eugene Hopkinson (SlowRiot)
- New actions:
- symbiosis-blacklist-allports for Bytemark symbiosis firewall
- fail2ban-client can fetch the running server version
- Added Cloudflare API action
- Enhancements
* Start performance of fail2ban-client (and tests) increased, start time
and cpu usage rapidly reduced. Introduced a shared storage logic, to
bypass reading lots of config files (see gh-824).
Thanks to Joost Molenaar for good catch (reported gh-820).
* Fail2ban-regex - add print-all-matched option. Closes gh-652
* Suppress fail2ban-client warnings for non-critical config options
* Match non "Bye Bye" disconnect messages for sshd locked account regex
* courier-smtp filter:
- match lines with user names
- match lines containing "535 Authentication failed" attempts
* Add <chain> tag to iptables-ipsets
* Realign fail2ban log output with white space to improve readability. Does
not affect SYSLOG output
* Log unhandled exceptions
* cyrus-imap: catch "user not found" attempts
* Add support for Portsentry
* tag '0.9.1': (36 commits)
ENH: additional versioning changes
Refreshed manpages
ENH: fail early in generate-man + provide PYTHONPATH upstairs
Changes for the 0.9.1 release versioning
Populated MANIFEST with more entries which were preiously missed or duplicated. Sorted within each "section"
Add portsentry to changelog
ConfigReader.touch renamed into protected _create_unshared
DOC: documentation about available vagrantfile setup
Added myself into THANKS
DOC: adjust docs in mytime to place docs into docstrings
ENH: do use @staticmethod (we are well beyond support of 2.4 now)
testExecuteTimeout fixed: give a test still 1 second, because system could be too busy
coverage: no cover (for failed except)
fix: fail2ban-regex with filter file failed (after merging #824, because test case missing); test case for 'readexplicit' added;
ENH: remove obsolete code for python < 2.6 (we support >= 2.6)
DOC: very minor (tabs/spaces)
We better check that installation doesn't cause any errors as well
code review, change log entries added;
reset share/cache storage (if we use 'reload' in client with interactive mode)
normalize tabs/spaces in docstrings;
...
* '_tent/cache-config-read' of https://github.com/sebres/fail2ban:
code review, change log entries added;
reset share/cache storage (if we use 'reload' in client with interactive mode)
normalize tabs/spaces in docstrings;
cache-config-read-v2 merged; logging normalized, set log level for loading (read or use shared) file(s) to INFO; prevent to read some files twice by read inside "_getIncludes" and by "read" self (occurred by only one file);
code review; more stable config sharing, configurator always shares it config readers now;
code review: use the same code (corresponding test cases - with sharing on and without it);
rewritten caching resp. sharing of ConfigReader and SafeConfigParserWithIncludes (v.2, first and second level cache, without fingerprinting etc.);
code review
ConfigReader/ConfigWrapper renamed as suggested from @yarikoptic; + code clarifying (suggested also);
Partially merge remote-tracking from 'sebres:cache-config-read-820': test cases extended, configurator.py adapted for test case.
ENH: keep spitting out logging to the screen in LogCaptureTestCases if HEAVYDEBUG
test case for check the read of config files will be cached;
more precise by test
ConfigWrapper class introduced: sharing of the same ConfigReader object between JailsReader and JailReader (don't read jail config each jail); sharing of the same DefinitionInitConfigReader (ActionReader, FilterReader) between all jails using that; cache of read a config files was optimized; test case extended for all types of config readers;
config cache optimized - prevent to read the same config file inside different resources multiple times; test case: read jail file only once;
test case for check the read of config files will be cached;
caching of read config files, to make start of fail2ban faster, see issue #820
* origin/master:
RF: moving logwatch setup/sample logs under files/logwatch
ENH: print rebans stats even if no "Failures" are logged, and reduce indentation in output
ENH: untabified and reindented entire script for sane formatting (no functional changes)
BF: logwatch -- fixing up regex for 'already banned'
Sample logfiles to test logwatch services script
Adjusting fail2ban logwatch script to match lines from 0.9 as well
* commit '0.9.0a2-814-g98dc084':
tests: define CONFIG_DIR in utils.
forgot to add test case to last commit
adding test case, changelog and thanks entries for apache shellshock filter
adding jail conf for shellshock filter
adding filter to detect Shellshock attack attempts against bash scripts through apache. See http://seclists.org/oss-sec/2014/q3/650
Add apache filter for AH01630 client denied by server configuration
RF: moving logwatch setup/sample logs under files/logwatch
DOC: Changelog and THANKS for previous changes
RF: remove those two additional failregexes for the postfix
ENH: add empty ignoreregex to avoid a warning (Close#805)
Update test cases and also suport smtps per request.
Add support for postfix/submission/smtpd matching.
ENH: print rebans stats even if no "Failures" are logged, and reduce indentation in output
ENH: untabified and reindented entire script for sane formatting (no functional changes)
BF: logwatch -- fixing up regex for 'already banned'
Sample logfiles to test logwatch services script
Adjusting fail2ban logwatch script to match lines from 0.9 as well
logging normalized, set log level for loading (read or use shared) file(s) to INFO;
prevent to read some files twice by read inside "_getIncludes" and by "read" self (occurred by only one file);
sharing of the same DefinitionInitConfigReader (ActionReader, FilterReader) between all jails using that;
cache of read a config files was optimized;
test case extended for all types of config readers;
Merge remote-tracking branch 'sebres:cache-config-read-820' into ban-time-incr:
config cache optimized - prevent to read the same config file inside different resources multiple times;
test case: read jail file only once;
+ optimized merge: use OrderedDict.update instead of merge in cycle;
addJail (executed before addLog) early uses a "INSERT OR REPLACE" statement to update "enabled" to 1 (and add jail the first time used at once), but this syntax in sqlite always deletes an entry (cause of constraint) and inserts it again, so because of CASCADE all log entries with this jail will be also deleted from logs table.
test cases: extended with test to check action together with database functionality (ex.: to verify lambdas in checkBan);
database: getBansMerged should work within lock, using reentrant lock (cause call of getBans inside of getBansMerged);
* commit '0.9.0a2-792-g1864f75': (113 commits)
Credits and notes from #806
fixed encoding
fixed encoding
ENH: Ignore errors while unbaning in symbiosis firewall
ENH: just a bit more descriptive exception ;-)
ENH/BF(TST): making permissions restrictive is not sufficient -- really remove file to test
changelog entry for postfix-sasl fix
added systemd configuration for postfix-sasl.conf
1.5 version of Fail2ban logwatch file
minor typo
Fxi jail.conf to use more syslog macros
ENH: symbiosis-blacklist-allports action
Fix typos.
changelog and thanks for the preceding fix
Added entry for Cloudflare action
ChangeLog Added and entry about Cloudflare action
Changed to Cloudflare JSON API
Fix sieve filter to use correct option
changelog entries for already merged and upcoming merge
Update courier-smtp.conf
...
Conflicts resolved:
fail2ban/server/database.py
fail2ban/tests/servertestcase.py
delBan modified (if manually unban):
delete from "bips" also (bad ips)
delete all tickets of this ip, also if currently not banned
Conflicts resolved:
ChangeLog
fail2ban/server/filter.py
fail2ban/server/jail.py
fail2ban/tests/actionstestcase.py
Test cases fixed:
testBanActionsAInfo - fail ticket with current time (otherwise ticket will be ignored - ban time too old)
testFail2BanExceptHook - use local sys.__excepthook__ to check was really executed and prevent write error in stderr.
- database_v1.db/bans/jail-name bug fixed - cause of different jail name in jails and bans, in test case (by updateDb): FOREIGN KEY constraint failed:
$ sqlite3 fail2ban/tests/files/database_v1.db
sqlite> select distinct jail from bans;
DummyJail #16244880 with 0 tickets
sqlite> select distinct name from jails;
DummyJail #29162448 with 0 tickets
sqlite> update bans set jail = (select distinct name from jails);
"bantimeextra.enabled" in jail.conf allows to use database for searching of previously banned ip's to increase a default ban time using special formula,
by default, each next ban it will be original banTime * 1, 2, 4, 8, 16, 32...
see "jail.conf" for some other options of "bantimeextra";
additional we can configure a little randomization of ban time, to prevent "clever" botnets calculate exact time IP can be unbanned.
WARNING: by first start the server upgrades sqlite database (table "bans" will recreated with another schema);
* origin/master:
Clean up setup.py
ENH: adding pruned with previous merge trailing \s* in nginx filter
nginx-http-auth: match when "referrer" is present
DOC: Update ChangeLog
DOC: Fix jail.conf(5) man page which had duplicate {fail,ignore}regex
Correct grammar
DOC: add thanks for last commit
DOC: nginx-http-auth filter
nginx-http-auth filter: match server_name = ""
ENH: Suppress configuration warnings if non-critical options are not set
Update comments in shorewall.conf for new settings
DOC: Improve error logging when specific backend set and fails
BF: Fix getting jail name from exceptions in beautifier for Python 3+
DOC: sphinx documentation
BF: On jail restart reinstatement of bans, fetch one ticket per IP
Fix 2 more typos that codespell didn't catch
Add documentation link to systemd service file
* commit '0.9.0-48-gabcab00':
Fix a few typos
BF: Ignored IPs no longer being banned from database on restart
ENH: Add iptables and firewalld to "After" for systemd service file.
DOC: Add database settings for fail2ban.conf to jail.conf(5) man page
TST: Skip badips.py test is no network option set
TST: Skip SYSLOG log target test if '/dev/log' not present
BF: fail2ban.conf reader expected "int" type for `loglevel`
DOC: Update ChangeLog fixes
BF: Handle case when no sqlite library is available for the database
* commit '0.9.0-654-gc8694d8':
Add news entry for 0.9.0
use upstream jail.conf, paths are now defined in paths-debian.conf
enable systemd integration
remove /var/lib/fail2ban/fail2ban.sqlite3 when purging
move renamed conffiles in maintscripts
refresh patch deb_manpages_reportbug
remove changeset_d4f6ca4f8531f332bcb7ce3a89102f60afaaa08e.diff, applied upstream
Update debian packaging to dh compat 9
Conflicts:
debian/jail.conf -- removed also in this branch
debian/postinst -- took changes for courier renames + adjusted version to 0.9.0-1~
debian/postrm
debian/preinst
* commit '0.9.0-37-gdeb5924': (449 commits)
Fix typo in comment
DOC: note to have release tagged by a signed/annotated tag
DOC: ChangeLog for gh-652
Add a --print-all-matched option.
DOC: Add DEVELOP documentation on use of pyflakes
TST: Unused import in test_smtp
DOC: Document recent changes
DOC: Document recent fixes
DOC: syntax
DOC: new ChangeLog header
TST: Fix test failing due to wrapping of long subject lines v2
TST: Fix test failing due to wrapping of log subject lines
BF: fail2ban-regex mix of tabs and spaces
Changelog entry
BF(PY2.4,2.5): provide bin() function (introduced in 2.6)
BF: python 2.4 and 2.5 compatibility -- no "except ... as ..." yet
BF: Remove unused imports and variables.
BF: badips.py action methods not static due to use of self._logSys
BF: FilterSystemd.formatJournalEntry is a classmethod, not staticmethod
BF: Remove "/var/run/fail2ban" from setup.py
...
* tag '0.8.13': (48 commits)
DOC: DEVELOP release note changes
PKG: version release
PKG: include nagios filter/log
DOC/ENH: update man pages for release
ENH: pull asterisk filter change to support syslog from 0.9 branch
Sanity-check print-all-* vs print-no-* options.
Add --print-no-{missed,ignored} and restore -all.
Only remember log lines we need to print later.
Fix the --print-all-{missed,ignored} options.
ENH: sendmail-reject for all smtp ports.
ENH: more sendmail-reject filter items thanks to fab23
BF: move to right location
ENH: rename sendmail-spam to sendmail-reject
BF: email address as arg1 in sendmail filters
ENH: wider regex for RBL and sendmail-spam
DOC: Add reference to action argument values which contain ","
BF: add jail.conf definitions for sendmail* filters
ENH: add filter for sendmail-{auth,spam}. Closes gh-20
ENH: Allow 255.255.255.0 style mask for ignoreip
BF: Fix complain action for multiple recipients and misplaced ";"
...
* tag '0.8.12': (177 commits)
DOC: version and release date to 0.8.12 on 2014-01-22
DOC/ENH: update man pages for release
DOC: ChangeLog dateing and normalisation
ENH: continue with rest of fail2ban config even if errors. Closes gh-585
DOC: ChangeLog. Note incompatible changes and group new filters and actions under New Features
BF: complete MANIFEST
DOC: note in man page about "last message repeated" syslog compression. Closes Debian bug #620364
DOC: credits for action.d/badips
BF: dovecot filters, session characters and order of session/tls in log messages
BF Dovecot auth failures
Matching any Exim authentication name
Update exim
ENH: add improper command pipelining postfix filter
Support ACL-events without AccountID. Typically happens when a registration from an unknown domain is performed.
BF: Add ejabberd-auth to jail.conf
ENH: ejabberd filter
TST: for tag substition, multiple on same line
DOC: spelling + GPL2+ for license
DOC: fixing formatting in the section names of the manpage - \fB to return into bold
DOC: some typos, fixes from Vincent Lefevre
...
* tag '0.8.11':
DOC: finalise 0.8.11 release
BF/ENH: DoS resistant roundcube-auth with test cases and more variation in IMAP error given
BF: exim filter to be DoS resistant
ENH: DoS resistant dropbear filter
BF/ENH: asterisk connection ID is a hex not decimal number. Add "Rejecting unknown SIP connection from <HOST>" regex thanks to Jonathan Lanning
ENH: apache-2.4 message IDs for filter apache-noscript
TST: change source URL
ENH: apache-overflow filter to have HTTP-2.4 message IDs and test samples
ENH: apache-overflows - more detail on "request failed: URI too long (longer than %d)" with test case
TST: end of json in sshd sample log
TST: fix space in sshd sample log
TST: test case that shows injection
DOC: more on filter regexes - DEVELOP
DOC: filter regex debugging
BF: anchor introduced nginx-http-auth at the end
* commit '0.8.11.pre1-29-gccd2657': (363 commits)
DOC: minor typos in ChangeLog
DOC: adding DEV Notes for for non-greedy matchin within sshd.conf
BF: disallow exploiting of non-greedy .* in previous fix by providing too long rhost -- do not impose length limits for user-provided input
BF: fixing injection for OpenSSH 6.3 -- making .* before <HOST> non-greedy
Changelog for prior changes (gen_buildbots)
ENH: condense asterisk regexs for speed
BF: missed action in nginx-http-auth
ENH: add filter.d/nginx-http-auth. Partially forfills #405
ENH: regenerated config/filter.d/apache-badbots.conf
NF: gen_badbots script to (re)generate/update config/filter.d/apache-badbots.conf
DOC: keeping Changelog release-phrases uniform, simplified intro, unified
DOC: Untabifying and reindenting a bit ChangeLog
DOC: few more links for DEVELOP
BF: fix dovecot filter for newer failure message. Closes Debian bug #709324
BF: fix to filter.d/wuftp to support pam authentication - Debian bug #665925
Add Fedora git repo of fail2ban package to DEVELOP
firewalld-0.3.8 release that support --remove-rules out so documenting this.
BF: remove duplication definition secion in webmin-auth
DOC: alter release notes a bit more and versions in README.md
BF/DOC: fix hopefully final MANIFEST and release instructions
...
Release 0.8.10 -- wanna-be-secure, addresses possible DoS with apache- filters
* tag '0.8.10': (25 commits)
DOC: add information on where to report vulnerabilities + pointer to HOWTO_Seek_Help
Changes for 0.8.10 release (changelog, version, etc)
BF: anchor apache- filters. Close#248
DOC: credits for gh-244
Filter Asterisk: Add sample log entry to testcase.
Filter Asterisk: Add AUTH_UNKNOWN_DOMAIN error to list
ENH: purge a few more .*
DOC: credits
DOC: how to do filter enhancements
TST: normalize logs to use example.com and 1.2.3.4 as IP
ENH/BF: constrain regex. Fix ACL error regex
ENH: port optional
Changelog for previous PR
DOC: Changelog entry fro preceeding merge from Terence
TST: Fix fail2ban.conf reader test for unreliable dictionary order
failregex when roundcube log driver is set to 'syslog'
fixed failregex line for roundcube 0.9+
TST: test all stock jails to have actions and correctly specifying blocktype
CFG: assure actions for all the jails
BF: blocktype must be defined within [Init] -- adding [Init] section. Close#232
...
Release 0.8.9
* tag '0.8.9':
BF: add missing files to MANIFEST (I think we shoult not rely on sdist anyways -- 'git tag' tarballs are more thorough ;) )
Releasing 0.8.9
* tag '0.8.9': (46 commits)
All the (version) updates for the release of 0.8.9
BF: (travis) relax the test for needed to be presented installed directories -- allow new
BF: (travis) if tests ran under coverage -- there is a traceback parts to report (thus > would be present)
ENH: also print the failing traceback line in case of failure
ENH: include explicit list of new files which should not be there upon "install --root"
ENH: now we know that logging handlers closing was still buggy in 2.6.2
ENH: issue a warning if jail name is longer than 19 symbols (Close#222)
DOC: inline commends with ';' are in effect only if ';' follows as space
BF: Fix for filterpoll incorrectly checking for jailless state
ENH: strengthen detection of working pyinotify
ENH: use the same python executable for setup.py test
ENH: actually tune up TraceBack to determine "unittest" portions of the stack across all python releases
TST: Some primarily smoke tests for tests utils
TST: cover few more lines in fail2banreader.py
ENH: basic test for setup.py itself (when applicable, should greatly improve coverage ;) )
ENH: consistent operation of formatExceptionInfo + unittest for it
ENH: point to the status of master branch on travis
ENH: close open file in a test
Previous coverage was 56% (without disregarding any pragma)
ENH: close files in _test_move_into_file
...
* commit '0.8.8-276-g5e1d8b0': (103 commits)
ENH: logs/sshd -- have ":" after [daemon] (other uses are uncommon)
ENH: logs/sshd -- use example.com as the resolved hostname in sample log lines
ENH: filter.d/sshd.conf -- allow for trailing "via IP" in logs
ENH: Removed unused log line
ENH: logrotate file
ENH: some form of logrotate based on what distros are doing
DOC: macports distribution of fail2ban
TST: test case for <br> tags in action
DOC: credits for BSD ssh filter changes
EHN: enhance sshd filter for bsd.
DOC: purge of svn tags
Slight tune up of changelog entry (treeish change due to squashing)
fixed test date thx to steven
tight control of the filter for ASSP
as yaroslav wishes
as daniel desires
changed from DateASSPlike class to DateStrptime
fixed little things
added new date format support for ASSP SMTP Proxy
Forgotten Changelog entry for "is None" changes
...
* commit '0.8.8-145-g72b0647': (114 commits)
ENH: Slight tune ups for fresh SOGo filter + comment into the sample log file
ENH: postfix filter -- react also on (450 4.7.1) with empty from/to. fixes#126
TST: basic testing of reading the shipped jail.conf (forcing all jails to be enabled)
ENH: allow to force enable all jails (for testing), do not crash for jails without actions (just warn)
ENH: minor -- add default value into the warning if option had none provided
ENH: _copy_lines_between_files -- read all needed, and only then write/flush at once
ENH: move pyinotify callback debug message into callback + delay string interpolations
ENH: adding ability to incorporate tracebacks into log lines while running tests
ENH: FailManager -- improve log message to report total # of detected failures as well
BF: allow to wait longer for FilterPoll in test_move_file
ENH: elaborated debug log message about already detected failures
BF: Remove custom __str__ for MonitorFailures and just adjust __name__ of the generated class
DOC: Added suggested by @beilber description of .d/ + added I formatting to all filenames
ENH: increase timeout to 20 sec from 10 sec in assert_correct_last_attempt
BF: fixing up for handling of TAI64N timestamps and adding some unittest for prev commit (not effective much though)
An example of failed logins against sogo
Update sogo-auth.conf
Added Daniel and Steven to THANKS
My improvements to manpages
PKG: change email that I want in RPMs
...
Primarily a bugfix release 0.8.8
* tag '0.8.8': (31 commits)
Getting ready for 0.8.8 release (changelog, version boost)
BF: guarantee that IP is stored as a base, non-unicode str (Closes gh-91)
ENH: BF (forgotten import) for prev commit + removed duplicate Author, adjusted __ fields for that in fail2ban-* scripts
ENH: until we make it proper module -- adjust sys.path only if system-wide run
ENH: fail2ban-testcases-all -- pass cmdline options to fail2ban-testcases
ENH: To help with gh-87 added hints into the log on some failure return codes (e.g. 0x7f00 for this one)
ENH: trying to go native travis-ci python way to take advantage of virtualenv's with older pythons
BF: typo
BF: added a little shell script to excercise tests against all available Python versions
ENH: travis -- try to run tests against all available python versions
NF: rudimentary .travis.yml for travis-ci.org service
BF: do not enable pyinotify backend if pyinotify is too old (Closes gh-80)
DOC: forgotten --help entry for " unban "
ENH: downgrade "already banned" from WARN to INFO level (Closes gh-79)
minor: added a note on now "negative" log entries on "POSSIBLE BREAK-IN ATTEMPT"
DOC: minor "fixes" in DEVELOP
Added in while loop to process the Fail Manager after the requested banned IP was added to its queue. This solves the issue of needing to touch the log file that is being monitored to get the IP to be banned accordingly. Added in import of FailManagerEmpty exception class.
ENH: refactored previous commit to make it more Pythonic (With prev commit closes gh-86, gh-81)
Added in command option to unban and IP, just like using 'banip'. Command looks like: fail2ban-client set <jail name> unbanip <ip>
BF: in code we should use MyTime wrapper instead of time module directly
...
Re-tagging 0.8.7 with hope to do that last time ;)
* tag '0.8.7':
ENH: match possibly present "pam_unix(sshd:auth):" portion for sshd (Closes: #648020)
ENH: add wu-ftpd failregex for use against syslog (Closes: #514239)
BF: anchor chain name in actioncheck's for iptables actions (Closes: #672228)
BF: inline comments must use ; not # -- recidive jail
* commit '0.8.6-100-gdca5634':
RF: reordered tests + enabled gamin now that its fix is pending in Debian
ENH+BF: filtergamin -- to be more inline with current design of filterinotify
ENH: 1 more sleep_4_poll to guarantee difference in time stamp
ENH: few more delays for cases relying on time stamps
* commit '0.8.6-95-gc0c1232':
ENH: tests much more robust now across pythons 2.4 -- 2.7
BF+RF: pyinotify refreshes watcher upon CREATE, unified/simplified *(add|del)LogPath among *Filters
Ask users to report bugs to github's issues
Replace "|" with "_" in ipmasq-ZZZzzz|fail2ban.rul (Closes gh-66)
* commit '0.8.6-90-g08564bd':
ENH: fail2ban-testcases -- custom logging format to ease debugging, non-0 exit code in case of failure
ENH: Filter's testcases -- rename, del + list again --- a bit unstable, might still fail from time to time
BF: pyinotify -- monitor the parent directory for IN_CREATE + process freshly added file (Closes gh-44)
ENH: first working unittest for checking polling and inotify backends
RF/BF: just use pyinotify.ThreadedNotifier thread in filterpyinotify
RF: filter.py -- single readline in a loop
ENH: FilterPoll -- adjusted some msgs + allowed to operate without jail (for testing)
Minor additional comment to DEVELOP
ENH: extended test LogfileMonitor
ENH: add more verbosity levels to be controlled while running unittests
Added few tests of FileFilter. yet to place them into a Jail-ed execution test
DOC: distilling some of server "design" into DEVELOP notes for common good
ENH: minor, just trailing spaces/tabs + reformated a string
ENH: added a basic test for FilterPoll for detection of modifications
clarified that the are existing test cases and the 'coming soon' is about creating new ones.
Added beginnings of documentation for developers
BF: usedns=no was not working at all
RF: filtertestcase.py to put common testing into a helping subroutine
ENH: be able to control verbosity from cmdline for fail2ban-testcases
* commit '0.8.6-69-gb4099da': (57 commits)
DOC: Adjusted header for config/*.conf to mention .local and way to comment
Fixed addBannedIP to add enough failures to trigger a ban, rather than just one failure.
Fix Gentoo initd script (drop extra_commands)
ENH: catch failed ssh logins due to being listed in DenyUsers. Close gh-47 (Closes: #669063)
DOC: comment in jail.conf for the need of multiple jails for asterisk
Add the INCLUDE section to use __pid_re feature
Disable asterisk jail by default
Change jail for asterisk, add support for SIP and SIP-TLS on TCP and UDP ports
Change NOTICE by NOTICE%(__pid_re)s
Added a warning for the recidive jail
BF: fail2ban-regex -- adding forgotten char for -v
Remove custom bantime
Add sample log file for asterisk
Add $ at the end of the failregex
ENH: fail2ban-regex -- quieter by default and added --verbose mode
ENH: minor, just pythonized some parts of fail2ban-regex summary
ENH: rudimentary __repr__ for Filter and Jail + moved usedns into set method
BF: allow trailing whitespace in few missing it regexes for sshd.conf
BF+ENH: added %m-%d-%Y pattern + do not add %Y for Feb 29 fix if already present in the pattern
minor comment into TODO
...
* commit '0.8.6':
Changelog and version changes for 0.8.6
Revert "ENH: server.py -- addLogPath with tail=True"
DOC: updated contact information to direct to github and mailing list
ENH: added custom timeformat with '.' as separator. Close gh-1
BF: stop all communications before stopping the jails (Close gh-7)
ENH: added logging while stopping the jails
Add Date: header for sendmail*.conf actions
BF: gentoo-initd assure /var/run dir + remove stale sock file
ENH: fix of syntax for compatibility with Python 2.4
ENH: stay compatible with python < 2.5 (use md5 if hashlib is N/A)
BF: use hashlib instead of deprecated md5
ENH: Remove obsolete code from gentoo init script. Bug gentoo#367819.
* 'master' (early part):
Adding log samples accumulated in Debian branch
BF: removing minor bashism in ipmasq example file (closes: #530078). Thanks Raphael Geissert
Added ipmasq rule file to restart fail2ban when iptables are wiped out (closes: #461417). Thanks Guido Bozzetto
added rudimentary .gitignore
BF: be able to detect time for VNC recording only 2 letters of year (closes: #537610)
* master: (27 commits)
ENH: server.py -- addLogPath with tail=True
ENH: set/getFile for ticket.py -- found in source distribution of 0.8.4
NF: adding unittests for previous commit
ENH: removed expansion for few Date and Revision SVN keywords
ENH: sshd.conf -- allow user names to have spaces and trailing spaces in the line
Removed duplicate entry for DataCha0s/2\.0 in badbots (closes: #519557)
BF: return declaration of TABLE back (thanks to michaelberg79)
Update server/datetemplate.py
format output of matches data.
ENH: minor unittest to see if tickets carry correct 'matches'
ENH: introduced usa of Ticket.__matches throughout
ENH: added 'matches' to the Ticket(s) and deprecated "custom" constructors for derived *Tickets
ENH: modelines for emacs and vim to assure consistent indentation scheme (tabs)
ENH: failmanager -- additional debug message about # of known failures
ENH: rudimentary __str__ for the ticket
ENH: more human-accessible printout of the dates if any comparison fails
ENH: few debug messages and use MyTime.localtime instead of straight time.time
ENH: Added localtime() to MyTime
BF: set TZ to CEST while unittesting so dates matching would work
ENH: added a .pylintrc to help with consistent appearance and catch obvious problems
...
Conflicts:
MANIFEST -- wasn't present before due to base on source distribution
server/ticket.py -- strange conflict -- should be benign
* upstream:
for 0.8.5 release -- changelog + version
BF: use addfailregex instead of failregex while processing per-jail "failregex" parameter (Closes: #635830) (LP: #635036)
BF: use os.path.join to generate full path - fixes includes in configs given local filename
very minor -- uniform indentation in example
BF: use standard/reserved example.com instead of mail.com
ENH: Adding author for dovecot filter and prunning unneeded space in the regexp
* commit 'remotes/upstream-repo/tags/FAIL2BAN-0_8_5^':
for 0.8.5 release -- changelog + version
BF: use addfailregex instead of failregex while processing per-jail "failregex" parameter (Closes: #635830) (LP: #635036)
BF: use os.path.join to generate full path - fixes includes in configs given local filename
very minor -- uniform indentation in example
BF: use standard/reserved example.com instead of mail.com
ENH: Adding author for dovecot filter and prunning unneeded space in the regexp
Conflicts:
common/version.py -- my added copyright
* debian: (21 commits)
debian/jail.conf: got 'chain' parameter to be specified for iptables actions (Closes: #515599)
debian/jail.conf: closing " for protocol specification
BF: proftpd filter -- if login failed -- count regardless of the reason for failure
BF: Allow for trailing spaces in proftpd logs
BF: escaping () in pure-ftpd filter. Thanks Teodor
BF: allow space in the trailing of failregex for sasl.conf: see http://bugs.debian.org/573314
ENH: add <chain> to action.d/iptables*. Thanks Matthijs Kooijman: see http://bugs.debian.org/515599
NF: Adding found on a drive filter.d/dovecot.conf
ENH: make filter.d/apache-overflows.conf catch more: see http://bugs.debian.org/574182
ENH: dropbear filter: see http://bugs.debian.org/546913
BF: Use /var/run/fail2ban instead of /tmp for temp files in actions: see http://bugs.debian.org/544232
ENH: adjusted description for sasl jail (Closes: #615952)
ENH: slight rewordings of the long description (Closes: #588176)
debian/copyright: updated copyright years
Boosted policy compliance version to 3.9.1 (no changes seems to be due)
spellcheck jail.conf. Thanks Christoph Anton Mitterer
spellcheck debian/jail.conf (Closes: #598206). Thanks Christoph Anton Mitterer
debian: default ignoreip to ignore entire loopback zone (/8): see http://bugs.debian.org/598200
default ignoreip to ignore entire loopback zone (/8): see http://bugs.debian.org/598200
Tai64N stores time in GMT, we need to convert to local time before returning
...
+ trailing whitespaces were removed
Thanks to Christoph Anton Mitterer for the original bugreport raising the
concern and Matthijs Kooijman for giving 'chains parameter' idea
* commit 'upstream/0.8.4+svn20110323': (37 commits)
BF: proftpd filter -- if login failed -- count regardless of the reason for failure
BF: Allow for trailing spaces in proftpd logs
BF: escaping () in pure-ftpd filter. Thanks Teodor
BF: allow space in the trailing of failregex for sasl.conf: see http://bugs.debian.org/573314
ENH: add <chain> to action.d/iptables*. Thanks Matthijs Kooijman: see http://bugs.debian.org/515599
NF: Adding found on a drive filter.d/dovecot.conf
ENH: make filter.d/apache-overflows.conf catch more: see http://bugs.debian.org/574182
ENH: dropbear filter: see http://bugs.debian.org/546913
BF: Use /var/run/fail2ban instead of /tmp for temp files in actions: see http://bugs.debian.org/544232
spellcheck jail.conf. Thanks Christoph Anton Mitterer
default ignoreip to ignore entire loopback zone (/8): see http://bugs.debian.org/598200
Tai64N stores time in GMT, we need to convert to local time before returning
debug entry for lines ignored due to falling below findtime (v2)
disabling entirely named-refused-udp jail with a big fat warning
added time module. bug reported in buanzo's blog at http://blogs.buanzo.com.ar/2009/04/fail2ban-patch-ban-ip-address-manually.html
- Patch to make log file descriptors cloexec to stop leaking file descriptors on fork/exec. Thanks to Jonathan Underwood. https://bugzilla.redhat.com/show_bug.cgi?id=230191#c24
- Changed to SVN version.
- Release 0.8.4.
- Oups... Forgot the ChangeLog...
- Check the inode number for rotation in addition to checking the first line of the file. Thanks to Jonathan Kamens. - Red Hat Bugzilla - Bug 503852 - SF.net Bug #2800279.
...
* upstream-0.8:
BF: proftpd filter -- if login failed -- count regardless of the reason for failure
BF: Allow for trailing spaces in proftpd logs
BF: escaping () in pure-ftpd filter. Thanks Teodor
BF: allow space in the trailing of failregex for sasl.conf: see http://bugs.debian.org/573314
ENH: add <chain> to action.d/iptables*. Thanks Matthijs Kooijman: see http://bugs.debian.org/515599
NF: Adding found on a drive filter.d/dovecot.conf
ENH: make filter.d/apache-overflows.conf catch more: see http://bugs.debian.org/574182
ENH: dropbear filter: see http://bugs.debian.org/546913
BF: Use /var/run/fail2ban instead of /tmp for temp files in actions: see http://bugs.debian.org/544232
* upstream-0.8:
spellcheck jail.conf. Thanks Christoph Anton Mitterer
default ignoreip to ignore entire loopback zone (/8): see http://bugs.debian.org/598200
Tai64N stores time in GMT, we need to convert to local time before returning
debug entry for lines ignored due to falling below findtime (v2)
disabling entirely named-refused-udp jail with a big fat warning
added time module. bug reported in buanzo's blog at http://blogs.buanzo.com.ar/2009/04/fail2ban-patch-ban-ip-address-manually.html
Should have been done long before -- debian/ branch tracks source
releases of fail2ban, and there was 0.8.4 which was at once merged into
debian-release (debian branch + patches/fixes) but not into debian
* commit 'upstream/0.8.4':
Imported Upstream version 0.8.4
- Use 80 columns.
- Fixed maxretry/findtime rate. Many thanks to Christos Psonis. Tracker #2019714.
- Made the named-refused regex a bit less restrictive in order to match logs with "view". Thanks to Stephen Gildea.
- Use timetuple instead of utctimetuple for ISO 8601. Maybe not a 100% correct fix but seems to work. Tracker #2500276.
- Changed <HOST> template to be more restrictive. Debian bug #514163.
- Added cyrus-imap and sieve filters. Thanks to Jan Wagner. Debian bug #513953.
- Pull a commit from Yaroslav git repo. BF: addressing added bang to ssh log (closes: #512193).
- Added missing semi-colon in the bind9 example. Thanks to Yaroslav Halchenko.
- Added NetBSD ipfilter (ipf command) action. Thanks to Ed Ravin. Tracker #2484115.
- Improved SASL filter. Thanks to Loic Pefferkorn. Tracker #2310410.
- Added CPanel date format. Thanks to David Collins. Tracker #1967610.
- Added nagios script. Thanks to Sebastian Mueller.
- Removed print.
- Removed begin-line anchor for "standard" timestamp. Fixed Debian bug #500824.
- Remove socket file on startup is fail2ban crashed. Thanks to Detlef Reichelt.
* debian:
Replacing word of caution with big fat warning and commenting out named-refused-udp completely (Closes: #583364)
Adding arno-iptables-firewall (no deprecation of ipmasq per Joey Hess mentioning, which still could be used on lenny systems)
* up/fixes:
BF: proftpd filter -- if login failed -- count regardless of the reason for failure
BF: Allow for trailing spaces in proftpd logs (closes: #507986)
BF: be able to detect time for VNC recording only 2 letters of year (closes: #537610)
BF: escaping (). Thanks Teodor (Closes: #544744)
Conflicts:
config/filter.d/proftpd.conf
* debian:
actually boosting policy
BF: use "set logtartet" instead of "reload" while logrotate. Thanks J.M.Roth (Closes: #537773)
BF: adjusted README.Debian - multiport is default (closes: #545971)
* up/log_examples:
added proftpd examples from wiki
NF: few examples for pure-ftpd
* upstream: (21 commits)
Imported Upstream version 0.8.4
- Release 0.8.4.
- Oups... Forgot the ChangeLog...
- Check the inode number for rotation in addition to checking the first line of the file. Thanks to Jonathan Kamens.
- Fixed typo. Thanks to Dudi Goldenberg.
added traceback to asyncserver.py's import.
Added item about logging subsystem shutdown being moved, to Changelog.
moved logging shutdown out of quit(), into end of start() in server.py
Disabled jail lighttpd-fastcgi by default.
- Added entry for "Ban IP" command.
added "Ban IP" command to fail2ban branch 0.8
- Added two new filters: lighttpd-fastcgi and php-url-fopen.
- Moved last entries in the config/ part.
added two new filter files (PHP url_fopen, lighttpd fastcgi alerts), updated MANIFEST and jail.conf accordingly
- Added svn:keywords property.
- Added helper module in common.
added 'unexpected communication error' fix to ChangeLog. Added formatExceptionInfo to server/asyncserver.py
added missing import sys to asyncserver.py
more readable code for python version comparison
added python version detection to asyncore.loop(use_poll=True|False)
...
* commit '0_8_4':
- Release 0.8.4.
- Oups... Forgot the ChangeLog...
- Check the inode number for rotation in addition to checking the first line of the file. Thanks to Jonathan Kamens.
- Fixed typo. Thanks to Dudi Goldenberg.
added traceback to asyncserver.py's import.
Added item about logging subsystem shutdown being moved, to Changelog.
moved logging shutdown out of quit(), into end of start() in server.py
Disabled jail lighttpd-fastcgi by default.
- Added entry for "Ban IP" command.
added "Ban IP" command to fail2ban branch 0.8
- Added two new filters: lighttpd-fastcgi and php-url-fopen.
- Moved last entries in the config/ part.
added two new filter files (PHP url_fopen, lighttpd fastcgi alerts), updated MANIFEST and jail.conf accordingly
- Added svn:keywords property.
- Added helper module in common.
added 'unexpected communication error' fix to ChangeLog. Added formatExceptionInfo to server/asyncserver.py
added missing import sys to asyncserver.py
more readable code for python version comparison
added python version detection to asyncore.loop(use_poll=True|False)
Fix for python 2.6 / 3.0 incompatibility
* debian:
BF: thanks lintian -- removing stale /var/run/fail2ban from dirs -- should be created by init script
Thanks lintian for catching a misspell in README.Debian
boosted debhelper compatibility to 5
* up/fixes:
Removed duplicate entry for DataCha0s/2\.0 in badbots (closes: #519557)
BF: Allow for trailing spaces in proftpd logs (closes: #507986)
* up/log_examples:
added sasl example log file
* debian:
Added a comment into Debian-shipped jail.conf about sasl logpath -- it might preferable to monitor warn.log in case of postfix
* up/ipmasq:
BF: removing minor bashism in ipmasq example file (closes: #530078). Thanks Raphael Geissert
* upstream:
- Use 80 columns.
- Fixed maxretry/findtime rate. Many thanks to Christos Psonis. Tracker #2019714.
- Made the named-refused regex a bit less restrictive in order to match logs with "view". Thanks to Stephen Gildea.
- Use timetuple instead of utctimetuple for ISO 8601. Maybe not a 100% correct fix but seems to work. Tracker #2500276.
- Changed <HOST> template to be more restrictive. Debian bug #514163.
- Added cyrus-imap and sieve filters. Thanks to Jan Wagner. Debian bug #513953.
- Pull a commit from Yaroslav git repo. BF: addressing added bang to ssh log (closes: #512193).
- Added missing semi-colon in the bind9 example. Thanks to Yaroslav Halchenko.
- Added NetBSD ipfilter (ipf command) action. Thanks to Ed Ravin. Tracker #2484115.
- Improved SASL filter. Thanks to Loic Pefferkorn. Tracker #2310410.
- Added CPanel date format. Thanks to David Collins. Tracker #1967610.
- Added nagios script. Thanks to Sebastian Mueller.
- Removed print.
- Removed begin-line anchor for "standard" timestamp. Fixed Debian bug #500824.
- Remove socket file on startup is fail2ban crashed. Thanks to Detlef Reichelt.
Conflicts:
config/filter.d/sshd.conf
server/filter.py
* commit 'upstream-repo/FAIL2BAN-0_8':
- Use 80 columns.
- Fixed maxretry/findtime rate. Many thanks to Christos Psonis. Tracker #2019714.
- Made the named-refused regex a bit less restrictive in order to match logs with "view". Thanks to Stephen Gildea.
- Use timetuple instead of utctimetuple for ISO 8601. Maybe not a 100% correct fix but seems to work. Tracker #2500276.
- Changed <HOST> template to be more restrictive. Debian bug #514163.
- Added cyrus-imap and sieve filters. Thanks to Jan Wagner. Debian bug #513953.
- Pull a commit from Yaroslav git repo. BF: addressing added bang to ssh log (closes: #512193).
- Added missing semi-colon in the bind9 example. Thanks to Yaroslav Halchenko.
- Added NetBSD ipfilter (ipf command) action. Thanks to Ed Ravin. Tracker #2484115.
- Improved SASL filter. Thanks to Loic Pefferkorn. Tracker #2310410.
- Added CPanel date format. Thanks to David Collins. Tracker #1967610.
- Added nagios script. Thanks to Sebastian Mueller.
- Removed print.
- Removed begin-line anchor for "standard" timestamp. Fixed Debian bug #500824.
- Remove socket file on startup is fail2ban crashed. Thanks to Detlef Reichelt.
Conflicts:
MANIFEST
TODO
to forbid matching IPs encoded in the hostname (like
1.2.3.4.xxx.yyyyy.zzz) prior doing actual DNS lookup.
It is quite important and actually security hazard: DoS is easy to
perform...
Since now we don't do \S matching in <HOST> regexp, in the development
branches we should also allow for unicode characters by adding 're.U' to HOST's
regexp
* added example for BREAK-IN in ssh
* Syncing current debian revision to FAIL2BAN-0_8@717 of upstream,
since it includes fixes to some forwarded bugs. Total list of
functional changes
- Added actions to report abuse to ISP, DShield and myNetWatchman.
Thanks to Russell Odom.
- Added apache-nohome.conf. Thanks to Yaroslav Halchenko.
- Added new time format. No idea from where it comes...
- Added new regex. Thanks to Tobias Offermann.
- Try to match the regex even if the line does not contain a valid
date/time. Described in Debian #491253. Thanks to Yaroslav
Halchenko.
- Removed "timeregex" and "timepattern" stuff that is not needed
anymore.
- Added date template for Day-Month-Year Hour:Minute:Second
(closes: #491253)
- Added date pattern for Hour:Minute:Second. Thanks to Andreas
Itzchak Rehberg.
- Use current day and month instead of Jan 1st if both are not
available in the log. Thanks to Andreas Itzchak Rehberg.
- Improved pattern. Thanks to Yaroslav Halchenko.
- Merged patches from Debian package. Thanks to Yaroslav Halchenko.
* debian:
- Added actions to report abuse to ISP, DShield and myNetWatchman. Thanks to Russell Odom.
- Added svn:keywords property.
- Added apache-nohome.conf. Thanks to Yaroslav Halchenko.
- Added new time format. No idea from where it comes...
- Added new regex. Thanks to Tobias Offermann.
- Try to match the regex even if the line does not contain a valid date/time. Described in Debian #491253. Thanks to Yaroslav Halchenko.
- Removed "timeregex" and "timepattern" stuff that is not needed anymore.
- Added date template for Day-Month-Year Hour:Minute:Second.
- Added date pattern for Hour:Minute:Second. Thanks to Andreas Itzchak Rehberg.
- Use current day and month instead of Jan 1st if both are not available in the log. Thanks to Andreas Itzchak Rehberg.
- Improved pattern. Thanks to Yaroslav Halchenko.
- Merged patches from Debian package. Thanks to Yaroslav Halchenko.
- Changed to SVN version.
Conflicts:
config/filter.d/sshd.conf
* upstream:
- Added actions to report abuse to ISP, DShield and myNetWatchman. Thanks to Russell Odom.
- Added svn:keywords property.
- Added apache-nohome.conf. Thanks to Yaroslav Halchenko.
- Added new time format. No idea from where it comes...
- Added new regex. Thanks to Tobias Offermann.
- Try to match the regex even if the line does not contain a valid date/time. Described in Debian #491253. Thanks to Yaroslav Halchenko.
- Removed "timeregex" and "timepattern" stuff that is not needed anymore.
- Added date template for Day-Month-Year Hour:Minute:Second.
- Added date pattern for Hour:Minute:Second. Thanks to Andreas Itzchak Rehberg.
- Use current day and month instead of Jan 1st if both are not available in the log. Thanks to Andreas Itzchak Rehberg.
- Improved pattern. Thanks to Yaroslav Halchenko.
- Merged patches from Debian package. Thanks to Yaroslav Halchenko.
- Changed to SVN version.
* commit 'remotes/upstream-repo/FAIL2BAN-0_8':
- Added actions to report abuse to ISP, DShield and myNetWatchman. Thanks to Russell Odom.
- Added svn:keywords property.
- Added apache-nohome.conf. Thanks to Yaroslav Halchenko.
- Added new time format. No idea from where it comes...
- Added new regex. Thanks to Tobias Offermann.
- Try to match the regex even if the line does not contain a valid date/time. Described in Debian #491253. Thanks to Yaroslav Halchenko.
- Removed "timeregex" and "timepattern" stuff that is not needed anymore.
- Added date template for Day-Month-Year Hour:Minute:Second.
- Added date pattern for Hour:Minute:Second. Thanks to Andreas Itzchak Rehberg.
- Use current day and month instead of Jan 1st if both are not available in the log. Thanks to Andreas Itzchak Rehberg.
- Improved pattern. Thanks to Yaroslav Halchenko.
- Merged patches from Debian package. Thanks to Yaroslav Halchenko.
- Changed to SVN version.
Conflicts:
MANIFEST
* up/sshd_fixes:
BF: Specify explicitely facilities in "Failed .. for". Thanks Dean Gaudet. (closes: #481760)
Added failregex for "User not known" in sshd.conf. thanks Alexander Gerasiov (closes: #479966)
* debian: (23 commits)
Imported Upstream version 0.8.3
- Prepared for 0.8.3.
Adjusted vcs paths
- Prepared for 0.8.3
- Send file if the number of lines is greater or equal and not only equal to the limit.
- Use poll instead of select in asyncore.loop. This should solve the "Unknown error 514". Thanks to Michael Geiger and Klaus Lehmann.
- Added missing ignoreregex to filters. Thanks to Klaus Lehmann.
- Added and changed some logging level and messages.
- Added svn:keywords.
- Added ISO 8601 date/time format.
- Better (correct) fix for ignoreregex in jail.[conf|local].
- Fixed ignoreregex processing in fail2ban-client. Thanks to René Berber.
- Added "Day/Month/Year Hour:Minute:Second" date template. Thanks to Dennis Winter.
- Added svn:keywords.
- Added gssftpd filter. Thanks to Kevin Zembower.
- Changed some log level.
- Fixed "fail2ban-client get <jail> logpath". Bug #1916986.
- Fixed PID file while started in daemon mode. Thanks to Christian Jobic who submitted a similar patch.
- Fixed socket path in redhat and suse init script. Thanks to Jim Wight.
- Create /var/run/fail2ban during install.
...
Conflicts:
ChangeLog
config/filter.d/gssftpd.conf
config/filter.d/pam-generic.conf
debian/changelog
* upstream: (22 commits)
Imported Upstream version 0.8.3
- Prepared for 0.8.3.
- Prepared for 0.8.3
- Send file if the number of lines is greater or equal and not only equal to the limit.
- Use poll instead of select in asyncore.loop. This should solve the "Unknown error 514". Thanks to Michael Geiger and Klaus Lehmann.
- Added missing ignoreregex to filters. Thanks to Klaus Lehmann.
- Added and changed some logging level and messages.
- Added svn:keywords.
- Added ISO 8601 date/time format.
- Better (correct) fix for ignoreregex in jail.[conf|local].
- Fixed ignoreregex processing in fail2ban-client. Thanks to René Berber.
- Added "Day/Month/Year Hour:Minute:Second" date template. Thanks to Dennis Winter.
- Added svn:keywords.
- Added gssftpd filter. Thanks to Kevin Zembower.
- Changed some log level.
- Fixed "fail2ban-client get <jail> logpath". Bug #1916986.
- Fixed PID file while started in daemon mode. Thanks to Christian Jobic who submitted a similar patch.
- Fixed socket path in redhat and suse init script. Thanks to Jim Wight.
- Create /var/run/fail2ban during install.
- Added "pam-generic" filter and more configuration fixes. Thanks to Yaroslav Halchenko.
...
* commit '0_8_3': (21 commits)
- Prepared for 0.8.3.
- Prepared for 0.8.3
- Send file if the number of lines is greater or equal and not only equal to the limit.
- Use poll instead of select in asyncore.loop. This should solve the "Unknown error 514". Thanks to Michael Geiger and Klaus Lehmann.
- Added missing ignoreregex to filters. Thanks to Klaus Lehmann.
- Added and changed some logging level and messages.
- Added svn:keywords.
- Added ISO 8601 date/time format.
- Better (correct) fix for ignoreregex in jail.[conf|local].
- Fixed ignoreregex processing in fail2ban-client. Thanks to René Berber.
- Added "Day/Month/Year Hour:Minute:Second" date template. Thanks to Dennis Winter.
- Added svn:keywords.
- Added gssftpd filter. Thanks to Kevin Zembower.
- Changed some log level.
- Fixed "fail2ban-client get <jail> logpath". Bug #1916986.
- Fixed PID file while started in daemon mode. Thanks to Christian Jobic who submitted a similar patch.
- Fixed socket path in redhat and suse init script. Thanks to Jim Wight.
- Create /var/run/fail2ban during install.
- Added "pam-generic" filter and more configuration fixes. Thanks to Yaroslav Halchenko.
- Process failtickets as long as failmanager is not empty.
...
* debian:
2 new jails: xinetd-fail, apache-overflows added to jails.conf
minor: adjusted comment for named jails to come closer to upstream
BF: apache-* jails -- authentication failures are recorded in *error.log files, thus paths were adjusted
* debian:
Confirms to policy 3.7.3 (no changes)
Bye Bye dpatch: now everything is handled in git branches
removing patches from dpatch system since they are in branches now
added a comment to README.Debian and to the list of examples for ipmasq example file
Fixed == bashism (Closes: #464647). Thanks Raphael Geisser
* deb/specifics:
slight tune ups in upstream sources destined only for debian are kept in this branch
* up/0.9-0.8:
* up/apache_noscript_extend:
Extended apache-noscript filter with more file extensions and to react to "script not found or unable to stat" log message (closes: #456565). Thanks Tim Connors
* up/ipmasq:
Added ipmasq rule file to restart fail2ban when iptables are wiped out (closes: #461417). Thanks Guido Bozzetto
* up/log_examples:
up/log_examples: moved vsftpd log from up/vsftpd_optional_user
added examples of log lines (for named-refused, pam-generic, sshd) under files/logs for easy testing
* up/mail_whois_lines:
mail-whois-lines: moved fix for proper names from dpatch
* up/named_refused_fixed:
named_refused: moved fix for proper config+filters from dpatch
* up/pam_generic:
added pam-generic from dpatch
* up/proftpd_fix+extend:
Fix/extension of proftpd failrexes (Closes: #461412). Thanks Guido Bozzetto
* up/sshd_refused_connect:
* up/vsftpd_optional_user:
up/vsftpd_optional_user: moving examples into up/examples branch
BF: vsftp anchoring
- anchored properly at the end of line, and source code has .examples
files to perform testing of the rules.
- added new explicit rule for users not in the AllowUsers lists
- Removed obsolete Build-Depends-Indep on help2man, python-dev
- Explicit removal of *.pyc files compiled during build
- Invoke 'python setup.py clean' in clean target, which required also
to move python into Build-Depends
* Minor clean up of debian/rules
with multiport module. That is to address the fact that most services
listen on multiple port (for encrypted and non-encrypted connections)
* Added [courierauth] jail (First 2 items are to partially address #407404
non-released versions (which were suggested to the users to overcome
problems reported in bug reports). In particular attention should be paid
to upstream changelog entries
- Several "failregex" and "ignoreregex" are now accepted.
Creation of rules should be easier now.
This is an alternative solution to 'multiple <HOST>' entries fix,
which is not applied to this shipped version - pay cautios if upgrading
from 0.7.5-3~pre?
- Allow comma in action options. The value of the option must
be escaped with " or '.
That allowed to implement requested ability to ban multiple ports
at once (See 373592). README.Debian and jail.conf adjusted to reflect
possible use of iptables-mport
- Now Fail2ban goes in /usr/share/fail2ban instead of
/usr/lib/fail2ban. This is more compliant with FHS.
Patch 00_share_insteadof_lib no longer applied
* Refactored installed by debian package jail.conf:
- Added option banaction which is to incorporate banning agent
(usually some flavor of iptables rule), which can then be easily
overriden globally or per section
- Multiple actions are defined as action_* to serve as shortcuts
rephrased (Closes: #402350)
* Added mail-whois-lines action, which emails log lines containing abuser
IP. Those lines are often required for proper abuse reports sent to the
Internet providers. Forwarding of such received emails to the email
addresses of abuse departments present in the output of whois is a
tentative solution for semi-automatic abuse reporting (Closes: #358810)
from jail.{conf,local} into an action or a filter definition (closes:
#398739)
* Removed Uploaded field from control since I am a DD now. Big thanks to
Barak Pearlmutter for being the sponsor of my packages for few
years.
invoking mail actions (closes: #396668)
* Removed obsolete entries in TODO and README
* README.Debian describes a bit issue of interpolations vs parameters
passed from jail.{conf,local} into an action or a filter definition
(closes: #398739)
* Initial version of postfix filter (closes: #377711)
react to client command to stop. Handling of status was adjusted as well
* Added apache-noscript to jail.conf
* Default action does not send emails to be inline with previous (0.6.x)
behavior
* Robust startup: if iptables module gets fully initialized after
startup of fail2ban, fail2ban will do "maxreinit" attempts to
initialize its own firewall. It will sleep between attempts for
"polltime" number of seconds (closes: #334272).
* To overcome possible conflict with other firewall solutions and as a
secondary solution for the bug 334272, fail2ban startup is moved
during bootup to the latest (S99) sequenece position. That should not
cause any discomfort I believe.
* Fixed handling of SYSLOG logging target. Now it can log
to any SYSLOG target and facility as directed by the
config
* Format of SYSLOG entries fixed to look closer to standard
* Fixed errata in config/gentoo-confd
* Introduced findtime configuration variable to control the
lifetime of caught "failed" log entries
git-svn-id: https://fail2ban.svn.sourceforge.net/svnroot/fail2ban/branches/FAIL2BAN-0_5@195 a942ae1a-1317-0410-a47c-b1dcaea8d605
## Fail2Ban: ban hosts that cause multiple authentication errors
Fail2Ban scans log files like /var/log/pwdfail and bans IP that makes too many
password failures. It updates firewall rules to reject the IP address. These
rules can be defined by the user. Fail2Ban can read multiple log files such as
sshd or Apache web server ones.
Fail2Ban scans log files like `/var/log/auth.log` and bans IP addresses conducting
too many failed login attempts. It does this by updating system firewall rules
to reject new connections from those IP addresses, for a configurable amount
of time. Fail2Ban comes out-of-the-box ready to read many standard log files,
such as those for sshd and Apache, and is easily configured to read any log
file of your choosing, for any error you wish.
Fail2Ban is able to reduce the rate of incorrect authentications attempts
however it cannot eliminate the risk that weak authentication presents.
Configure services to use only two factor or public/private authentication
Though Fail2Ban is able to reduce the rate of incorrect authentication
attempts, it cannot eliminate the risk presented by weak authentication.
Set up services to use only two factor, or public/private authentication
mechanisms if you really want to protect services.
<img src="http://www.worldipv6launch.org/wp-content/themes/ipv6/downloads/World_IPv6_launch_logo.svg" height="52pt"/> | Since v0.10 fail2ban supports the matching of IPv6 addresses.
------|------
This README is a quick introduction to Fail2ban. More documentation, FAQ, HOWTOs
are available in fail2ban(1) manpage and on the website http://www.fail2ban.org
This README is a quick introduction to Fail2Ban. More documentation, FAQ, and HOWTOs
to be found on fail2ban(1) manpage, [Wiki](https://github.com/fail2ban/fail2ban/wiki),
Alternatively, you can clone the source from GitHub to a directory of Your choice, and do the install from there. Pick the correct branch, for example, 0.11
# Notes.: command executed once at the start of Fail2Ban.
# Notes.: command executed on demand at the first ban (or at the start of Fail2Ban if actionstart_on_demand is set to false).
# Values: CMD
#
actionstart = ipfw show | fgrep -q 'table(<table>)' || ( ipfw show | awk 'BEGIN { b = 1 } { if ($1 <= b) { b = $1 + 1 } else { e = b } } END { if (e) exit e <br> else exit b }'; num=$?; ipfw -q add $num <blocktype> <block> from table\(<table>\) to me <port>; echo $num > "<startstatefile>" )
actionstart = ipfw show | fgrep -c -m 1 -s 'table(<table>)' > /dev/null 2>&1 || ( ipfw show | awk 'BEGIN { b = <lowest_rule_num> } { if ($1 < b) {} else if ($1 == b) { b = $1 + 1 } else { e = b } } END { if (e) exit e <br> else exit b }'; num=$?; ipfw -q add $num <blocktype> <block> from table\(<table>\) to me <port>; echo $num > "<startstatefile>" )
# Option: actionstop
# Notes.: command executed once at the end of Fail2Ban
# Notes.: command executed at the stop of jail (or at the end of Fail2Ban)
message = Dear Sir/Madam,\n\nWe have detected abuse from the IP address $IP, which according to a abusix.com is on your network. We would appreciate if you would investigate and take action as appropriate.\n\nLog lines are given below, but please ask if you require any further information.\n\n(If you are not the correct person to contact about this please accept our apologies - your e-mail address was extracted from the whois record by an automated process.)\n\n This mail was generated by Fail2Ban.\nThe recipient address of this report was provided by the Abuse Contact DB by abusix.com. abusix.com does not maintain the content of the database. All information which we pass out, derives from the RIR databases and is processed for ease of use. If you want to change or report non working abuse contacts please contact the appropriate RIR. If you have any further question, contact abusix.com directly via email (info@abusix.com). Information about the Abuse Contact Database can be found here: https://abusix.com/global-reporting/abuse-contact-db\nabusix.com is neither responsible nor liable for the content or accuracy of this message.\n
# Path to the log files which contain relevant lines for the abuser IP
# Because rich rules can only handle single or a range of ports we must split ports and execute the command for each port. Ports can be single and ranges separated by a comma or space for an example: http, https, 22-60, 18 smtp
actionban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --add-rich-rule="rule family='<family>' source address='<ip>' port port='$p' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <rich-blocktype>"; done
actionunban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --remove-rich-rule="rule family='<family>' source address='<ip>' port port='$p' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <rich-blocktype>"; done
[Init]
# log levels are "emerg", "alert", "crit", "error", "warning", "notice", "info" or "debug"
# firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' service name='<service>' <rich-blocktype>"
#
# Because rich rules can only handle single or a range of ports we must split ports and execute the command for each port. Ports can be single and ranges separated by a comma or space for an example: http, https, 22-60, 18 smtp
actionban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --add-rich-rule="rule family='<family>' source address='<ip>' port port='$p' protocol='<protocol>' <rich-blocktype>"; done
actionunban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --remove-rich-rule="rule family='<family>' source address='<ip>' port port='$p' protocol='<protocol>' <rich-blocktype>"; done
message = Dear Sir/Madam,\n\nWe have detected abuse from the IP address $IP, which according to abusix.com is on your network. We would appreciate if you would investigate and take action as appropriate.\n\nLog lines are given below, but please ask if you require any further information.\n\n(If you are not the correct person to contact about this please accept our apologies - your e-mail address was extracted from the whois record by an automated process.)\n\n This mail was generated by Fail2Ban in a X-ARF format! You can find more information about x-arf at http://www.x-arf.org/specification.html.\n\nThe recipient address of this report was provided by the Abuse Contact DB by abusix.com. abusix.com does not maintain the content of the database. All information which we pass out, derives from the RIR databases and is processed for ease of use. If you want to change or report non working abuse contacts please contact the appropriate RIR. If you have any further question, contact abusix.com directly via email (info@abusix.com). Information about the Abuse Contact Database can be found here: https://abusix.com/global-reporting/abuse-contact-db\nabusix.com is neither responsible nor liable for the content or accuracy of this message.\n
# Option: loglines
@@ -97,7 +116,7 @@ mailargs = -f <sender>
# Option: tlp
# Notes.: Traffic light protocol defining the sharing of this information.
# http://www.trusted-introducer.org/ISTLPv11.pdf
# green is share to those involved in network security but it is not
# green is share to those involved in network security but it is not
# to be released to the public.
tlp = green
@@ -116,7 +135,7 @@ logpath = /dev/null
# Option: sender
# Notes.: This is the sender that is included in the XARF report
sender = fail2ban@`uname -n`
sender = fail2ban@<fq-hostname>
# Option: port
# Notes.: This is the port number that received the login-attack
# This filter is aimed at blocking specific URLs that don't exist. This
# could be a set of URLs places in a Disallow: directive in robots.txt or
# just some web services that don't exist caused bots are searching for
# exploitable content. This filter is designed to have a low false postitive
# exploitable content. This filter is designed to have a low false positive
# rate due.
#
# An alternative to this is the apache-noscript filter which blocks all
@@ -17,30 +17,21 @@
[INCLUDES]
# overwrite with apache-common.local if _apache_error_client is incorrect.
# Load regexes for filtering from botsearch-common.conf
before = apache-common.conf
botsearch-common.conf
[Definition]
failregex = ^%(_apache_error_client)s ((AH001(28|30): )?File does not exist|(AH01264: )?script not found or unable to stat): <webroot><block>(, referer: \S+)?\s*$
^%(_apache_error_client)s script '<webroot><block>' not found or unable to stat(, referer: \S+)?\s*$
failregex = ^%(_apache_error_client)s ((AH001(28|30): )?File does not exist|(AH01264: )?script not found or unable to stat): /\S*(php([45]|[.-]cgi)?|\.asp|\.exe|\.pl)(, referer: \S+)?\s*$
^%(_apache_error_client)s script '/\S*(php([45]|[.-]cgi)?|\.asp|\.exe|\.pl)\S*' not found or unable to stat(, referer: \S+)?\s*$
failregex = ^%(_apache_error_client)s ((AH0013[456]: )?Invalid (method|URI) in request .*( - possible attempt to establish SSL connection on non-SSL port)?|(AH00565: )?request failed: URI too long \(longer than \d+\)|request failed: erroneous characters after protocol string: .*|AH00566: request failed: invalid characters in URI)(, referer: \S+)?$
failregex = ^%(_apache_error_client)s (?:(?:AH0013[456]: )?Invalid (method|URI) in request\b|(?:AH00565: )?request failed: URI too long \(longer than \d+\)|request failed: erroneous characters after protocol string:|(?:AH00566: )?request failed: invalid characters in URI\b)
ignoreregex =
# DEV Notes:
#
# [sebres] Because this apache-log could contain very long URLs (and/or referrer),
# the parsing of it anchored way may be very vulnerable (at least as regards
# the system resources, see gh-1790). Thus rewritten without end-anchor ($).
#
# fgrep -r 'URI too long' httpd-2.*
# httpd-2.2.25/server/protocol.c: "request failed: URI too long (longer than %d)", r->server->limit_req_line);
# Note: First three failregex matches below are for ASSP V1 with the remaining being designed for V2. Deleting the V1 regex is recommended but I left it in for compatibility reasons.
^\s*(?:[\w\-]+\s+)*(?:\[\S+\]\s+)*<HOST> \[SMTP Error\] 535 5\.7\.8 Error: authentication failed:\s+(?:\S+|Connection lost to authentication server|Invalid authentication mechanism|Invalid base64 data in continued response)?$
ignoreregex =
datepattern = {^LN-BEG}%%b-%%d-%%Exy %%H:%%M:%%S
{^LN-BEG}
# DEV Notes:
# V1 Examples matches:
# Apr-27-13 02:33:09 Blocking 217.194.197.97 - too much AUTH errors (41);
# Dec-29-12 17:10:31 [SSL-out] 200.247.87.82 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol;
failregex = ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?' - (Wrong password|Username/auth name mismatch|No matching peer found|Not a local domain|Device does not match ACL|Peer is not supposed to register|ACL error \(permit/deny\)|Not a local domain)$
^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Call from '[^']*' \(<HOST>:\d+\) to extension '\d+' rejected because extension not found in context 'default'\.$
^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Host <HOST> failed to authenticate as '[^']*'$
^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s No registration for peer '[^']*' \(from <HOST>\)$
^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Host <HOST> failed MD5 authentication for '[^']*' \([^)]+\)$
^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Failed to authenticate (user|device) [^@]+@<HOST>\S*$
^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s (?:handle_request_subscribe: )?Sending fake auth rejection for (device|user) \d*<sip:[^@]+@<HOST>>;tag=\w+\S*$
failregex = ^Registration from '[^']*' failed for '<HOST>(:\d+)?' - (?:Wrong password|Username/auth name mismatch|No matching peer found|Not a local domain|Device does not match ACL|Peer is not supposed to register|ACL error \(permit/deny\)|Not a local domain)$
^Call from '[^']*' \(<HOST>:\d+\) to extension '[^']*' rejected because extension not found in context
^(?:Host )?<HOST> (?:failed (?:to authenticate\b|MD5 authentication\b)|tried to authenticate with nonexistent user\b)
^No registration for peer '[^']*' \(from <HOST>\)$
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
sebres