* tag '0.10.2': (623 commits)
prepare release: bump version, update ChangeLog, man's and MANIFEST etc.
ChangeLog update
action.d/pf.conf: compatibility fix - recognizes that parameter `port` specified as empty, with or without braces (should be more backwards compatible to 0.9 now).
regex rewritten: a bit fewer vulnerable now and using non-capturing groups, test-cases extended in order to cover trying of injection on user name
nginx-http-auth: match usernames with spaces
regex updated using non-capturing groups
extended test-cases to cover new log-format (http_auth -> mod_auth)
Update lighttpd-auth.conf
file-filter's: provide stop function in order to explicitly delete/stop monitoring of each file.
Remove annoying error-message "rm_watch: cannot remove WD=2, Errno=Invalid argument (EINVAL)", logged from pyinotify-module if rm_watch called with non-existing watch file descriptor (probably multi-threaded issue by dual-remove). Closes gh-1865
should fix sporadic coverage decrease (don't cover "return", because too sporadic to get idle in pyinotify-callback);
fixed restoring sane environment (via stop/start) if invariant check failed: bypass possible errors in stop (if start/check succeeded hereafter); test cases extended to cover such situation. Closes gh-1997
action.d/hostsdeny.conf: actionunban rewritten using sed, also dots in IP were escaped now.
micro-fix: delete temporary file (forgotten in test-case `test_move_dir` by reassign to directory)
Update ChangeLog
stop ban of legitimate users with multiple public keys (e. g. git, etc), thereby differentiate between "invalid user" (going banned earlier) and valid users with public keys, for which the rejects of not valid public keys (failures) will be retarded up to "Too many authentication failures" resp. disconnect without success (accepted public key).
filter.d/sendmail-auth.conf - extended daemon for Fedora 24/RHEL - the daemon name is "sendmail" (gh-1632)
test cases extended in order to cover `firewallcmd-ipset` with `allports`
Update ChangeLog
firewallcmd-ipset-allports: implemented in `action.d/firewallcmd-ipset.conf` now (`action.d/firewallcmd-ipset-allports.conf` removed), usage:
...
I have lighttpd 1.4.45 (Debian 9) and auth error log is different.
Now printing mod_auth and not http_auth.
I think that the change was in Lighttp 1.4.42
differentiate between "invalid user" (going banned earlier) and valid users with public keys, for which the rejects of not valid public keys (failures) will be retarded up to "Too many authentication failures" resp. disconnect without success (accepted public key).
allow to use dual parameter lists (coming through substitutions), e. g.: `name[p1=0, p2="..."][p3='...']`;
simplified explanation: `][` treats as `,` in new version.
cherry-picked from 0.10.
- provide waitForServerEnd method for decorator `with_foreground_server_thread`, to wait for real server stop if needed;
- accept any exit code in decorator `with_foreground_server_thread`, because multi-threaded, thus server can exit in-between;
- fix sporadic fail "AssertionError: 'Banned 5 / 5, 5 ticket(s)' was not found" (if some tickets will be processed earlier,
thus not as chunk but separately), so in case of:
Banned 1 / 1, 1 ticket(s) in 'nginx-blck-lst'
Banned 4 / 5, 5 ticket(s) in 'nginx-blck-lst'
fix-up for run_with_except_hook: avoid very sporadic error "'NoneType' object has no attribute 'exc_info'" (https://bugs.python.org/issue7336),
only extremely fast systems are affected ATM (2.x / 3.x), if thread ends nothing is available in .
Usage `logtarget = target[facility=..., datetime=on|off, format="..."]`:
- `facility` - specify syslog facility (default `daemon`, see https://docs.python.org/2/library/logging.handlers.html#sysloghandler
for the list of facilities);
- `datetime` - add date-time to the message (default on, ignored if `format` specified);
- `format` - specify own format how it will be logged, for example for short-log into STDOUT:
`fail2ban-server -f --logtarget 'stdout[format="%(relativeCreated)5d | %(message)s"]' start`;
Closes gh-1980
* introduces 2 new format directives (with corresponding `%Ex` prefix for more precise parsing):
- %k - one- or two-digit number giving the hour of the day (0-23) on a 24-hour clock,
(corresponds %H, but allows space if not zero-padded).
- %l - one- or two-digit number giving the hour of the day (12-11) on a 12-hour clock,
(corresponds %I, but allows space if not zero-padded).
* mysqld-auth test extended to cover new date-format in log.
Closes gh-1639
Added non-greedy fallback for quoted something (with lookahead simulated possessive greedy catch of non-quoted parts `[^"]*(?=")`).
Note that because host-info's are hereafter (with foreign input in-between), we would not use greedy or non-greedy catch-alls (`.*` or `.*?`) here (preventing performance losses).
When reporting the error `AUTH command used when not advertised`, Exim
starts with `SMTP protocol error in "........."`. Here, Exim logs the
SMTP command as it was provided by the connecting client.
https://github.com/Exim/exim/blob/exim-4_89+fixes/src/src/smtp_in.c#L2850
According to RFC 5321 (SMTP) "[..] a command verb [..] MAY be encoded
in upper case, lower case, or any mixture of upper and lower case with
no impact on its meaning."
https://tools.ietf.org/html/rfc5321#section-2.4
Lower case `auth login` brute-force attempts were seen in the wild and
were not caught by the current failregex.
This commit makes the failregex case-insensitive for the `AUTH`
command, so that lower case (`auth`) or mixed case (`aUtH`) now also
match. The failregex was already case-insensitive for the command
arguments (e.g. `AUTH login` already matched).
fix sporadic error by shutdown server in with_foreground_server_thread decorator (if shutdown too fast, but end-phase still does not reached the tester-thread);
Note the systemd-backend does not need the logpath at all;
Some defaults normalized (minimized configs, don't need to overwrite values in distribution-related path if equal).
add descriptions to stop syslog errors for extra_started_commands when running:
rc-service ipset describe
Oct 28 15:13:30 xxxx daemon.warn /etc/init.d/fail2ban[26446]: ^[[1m^[[36mreload^[[m: no description
Oct 28 15:13:30 xxxx daemon.warn /etc/init.d/fail2ban[26447]: ^[[1m^[[36mshowlog^[[m: no description
* add [Init?family=inet6] to nftables-common.conf and make nftable
expressions more modular
* change "ip protocol" to "meta l4proto" in nftables-allports.conf
since the former only works for IPv4
leave it end-anchored because variable part `user=<[^>]*>` (before `<HOST>`) to avoid injecting, but can be safe rewritten using `[^>]*` in opposite to "greedy" `user=<[^>]*>`.
- introduces mode `aggressive` and extends regex for this mode to match:
* no auth attempts (previously removed in gh-601, because of lots of false positives on misconfigured MTAs)
* disconnected before auth was ready
* client didn't finish SASL auth
- setup process generates `build/fail2ban.service` from `files/fail2ban.service.in` using distribution related bin-path;
- bug-fixing by running setup with option `--dry-run` (note: specify option `--dry-run` before `install`, like `python setup.py --dry-run install`);
- test cases extended to cover dry-run.
(especially critical if default value `encoding = auto` configured).
As PoC and coverage (this case fails without this "fix"):
$ env -i PATH="$PATH" bin/fail2ban-testcases --fast --no-network testAddBanInvalidEncoded
Variable `default_backend` switched to `%(default/backend)s`, so totally backwards compatible now, but now the setting of parameter `backend` in default section of `jail.local` can overwrite default backend also.
Test cases extended: test targeted section options "section/option" (default and cross sections options);
I got an issue like this:
[2016-05-15 22:53:00] SECURITY[26428] res_security_log.c: SecurityEvent="FailedACL",EventTV="2016-05-15T22:53:00.203+0300",Severity="Error",Service="AMI",EventVersion="1",AccountID="admin",SessionID="0x7fb580001518",LocalAddress="IPV4/TCP/0.0.0.0/5038",RemoteAddress="IPV4/TCP/78.129.227.4/62389",SessionTV="1970-01-01T03:00:00.000+0300"
# [sebres] rebased to current master and resolving conflicts.
introduced parameter `mode`: more (default, combines normal and rbl), auth, normal, rbl, ddos, extra or aggressive (combines all)
replacement for gh-1239, gh-1697, gh-1764; closes gh-1245, gh-1297.
Fixed mail.log path since in the default rsyslog configuration of debians the `mail.warn` is commented now (see `/etc/rsyslog.d/50-default.conf`: `#mail.warn -/var/log/mail.warn`).
Closes gh-1687
filter.d/roundcube-auth.conf: Fixed failregex when logging errors to journal instead to a local file.
Additionally fixes more complex injections on username.
Tokens `%z` and `%Z` are more precise now;
Introduced new tokens `%Exz` and `%ExZ` that fully support zone abbreviations and/or offset-based zones;
# TODO: because python currently does not support mixing of case-sensitive with case-insensitive matching,
# check how TZ (in uppercase) can be combined with %a/%b etc. (that are currently case-insensitive),
# to avoid invalid date-time recognition in strings like '11-Aug-2013 03:36:11.372 error ...'
# with wrong TZ "error", which is at least not backwards compatible.
# Hence %z currently match literal Z|UTC|GMT only (and offset-based), and %Exz - all zone abbreviations.
Commit 5678d08 of 2016-11-26 changed:
( user=<\S*>,)?
to:
( user=<[^>]+>,)?
The change from `*` (zero or more times) to `+` (one or more times) may
not have been intended. It will miss lines containing, for example:
Aborted login (tried to use disallowed plaintext auth): user=<>
This commit reverts the `+` back to `*`.
* Update Kerio Connect filter
Fixed regex for some log entries that did not get recognized and some additional error formats are added.
* Add missing colon, GitHub address
* Add filter tests
* Add missing test
This new option allows to force the time zone on log lines
that don't bear a time zone indication (GitHub issue #1773), so it behaves
actually with respect to log line contents as a default time zone.
For the time being, only fixed offset timezones (UTC or UTC[+-]hhmm) are
supported, but the implementation is designed to later on treat the case
of logical timezones with DST, e.g., Europe/Paris etc.
In particular, the timezone name gets passed all the way to the strptime
module, and the resulting offset is computed for the given log line, even
though for now, it doesn't actually depend on it.
Also, the DateTemplate subclass gets to choose whether to use it or not.
For instance, it doesn't make sense to apply a time zone offset to
Unix timestamps.
The drawback is to introduce an API change for DateTemplate. I hope it's
internal enough for that not being a problem.
- Filter instance is already a thread;
- avoid stop pyinotify processing if an interim error occurs (and breaks main-loop, e. g. during multi-threaded processing by add/remove log-files)
If directory moved and the target is not watched path, so the monitoring of it could not be continued.
Now fixed with pending files await a monitoring if there (resp. its directories) appears again (respawn).
Closes gh-1769
ver. 0.9.7 (2017/05/11) - awaiting-victory
-----------
0.9.x line is no longer heavily developed. If you are interested in
new features (e.g. IPv6 support), please consider 0.10 branch and its
releases.
* Fixed a systemd-journal handling in fail2ban-regex (gh-1657)
* filter.d/sshd.conf
- Fixed non-anchored part of failregex (misleading match of colon inside
IPv6 address instead of `: ` in the reason-part by missing space, gh-1658)
(0.10th resp. IPv6 relevant only, amend for gh-1479)
* config/pathes-freebsd.conf
- Fixed filenames for apache and nginx log files (gh-1667)
* filter.d/exim.conf
- optional part `(...)` after host-name before `[IP]` (gh-1751)
- new reason "Unrouteable address" for "rejected RCPT" regex (gh-1762)
- match of complex time like `D=2m42s` in regex "no MAIL in SMTP connection" (gh-1766)
* filter.d/sshd.conf
- new aggressive rules (gh-864):
- Connection reset by peer (multi-line rule during authorization process)
- No supported authentication methods available
- single line and multi-line expression optimized, added optional prefixes
and suffix (logged from several ssh versions), according to gh-1206;
- fixed expression received disconnect auth fail (optional space after port
part, gh-1652)
and suffix (logged from several ssh versions), according to gh-1206;
* filter.d/suhosin.conf
- greedy catch-all before `<HOST>` fixed (potential vulnerability)
* filter.d/cyrus-imap.conf
- accept entries without login-info resp. hostname before IP address (gh-1707)
* Filter tests extended with check of all config-regexp, that contains greedy catch-all
before `<HOST>`, that is hard-anchored at end or precise sub expression after `<HOST>`
* New Actions:
- action.d/netscaler: Block IPs on a Citrix Netscaler ADC (gh-1663)
* New Filters:
- filter.d/domino-smtp: IBM Domino SMTP task (gh-1603)
* Introduced new log-level `MSG` (as INFO-2, equivalent to 18)
* tag '0.9.7': (33 commits)
Preparing for 0.9.7 release
Added newly added files to MANIFEST
update ChangeLog
filter.d/exim.conf: added new reason for "rejected RCPT" regex: Unrouteable address
try to fix travis integration of pypy3: setuptools recently dropped support for Python 3.0 - 3.2, but old pypy3 based on Python 3.2.5
filter.d/exim.conf: cherry-picked from 0.10, match complex time like `D=2m42s` (closes gh-1766)
Update ChangeLog #1757
filter.d/exim.conf: optional part `(...)` after host-name before `[IP]`, normalized over whole config file.
BF: specify explicit time offset not a time zone name to avoid needing tzdata during testing
Update ChangeLog
amend resp. restore of change from 59c35bc44a (gh-129): - logging of "Log rotation detected" with new MSG level - introduces new log-level MSG (as INFO-2, 18)
Update mysqld-auth.conf
Update ChangeLog
filter.d/cyrus-imap.conf: fixed `failregex` - accept entries without login-info resp. hostname before IP address
evil symlink removed: does not supported by some file systems (e. g. development over net share)
sshd-amend: optional space after port part
suhosin.conf: removed greedy match
sshd.conf: fixed expression "received disconnect ... auth fail" - optional space after port part (gh-1652)
change log update after rebase
sshd: additionally aggressive filter rules - no matching cipher resp. no matching key exchange method (gh-1545, gh-1117)
...
- `<fq-hostname>` - fully-qualified name of host (the same as `$(hostname -f)`)
- `<sh-hostname>` - short hostname (the same as `$(uname -n)`)
Execution of `uname -n` replaced in all mail actions with most interesting fully-qualified `<fq-hostname>`.
new action parameter "actionstart_on_demand" (bool) can be set to prevent/allow starting action on demand (default retrieved automatically, if some conditional parameter `param?family=...` presents in action properties);
It appears that, under Python3, on an IPv6 enabled machine,
the testing SMTP server on 'localhost' can turn out to listen on ::1 only,
which makes those tests break if the SMTP client part uses 127.0.0.1
directly. Using 'localhost' there as well makes the tests pass.
Removed warning "Mutliline regex set for jail ... but maxlines not greater than 1", because can be expected situation now:
non multi-line entry from systemd-filter containing new-lines (that should be ignored by anchors resp. entry parsed as single string);
small code review;
Regex will be compiled as MULTILINE only if needed (buffering with `maxlines` > 1), that enables:
- improve performance by the single line parsing;
- make regex more precise (because distinguish between anchors `^`/`$` for the begin/end of string and the new-line character '\n', e. g. if coming from filters (like systemd journal) that allow the parsing of log-entries contain new-line chars (as single entry);
Fail2ban will not ban a host which matches such addresses.
Option "ignoreip" affects additionally to "ignoreself" and don't need to include the DNS resp. IPs of the host self.
the situation details:
value of "_daemon" from default section "default" (with init section) falsely overwrites it from definition section "test" -
the resulting value of "_daemon" should be "test" in all 3 resulting failregex's (as specified in test.local),
fixed and covered now;
additionally more complex cases covered also (all filter parameters in jail via "%(known/...)s", dynamical interpolation across all, etc);
- the calling map contains normally dynamic values only (no recursive tags);
- recursive replacement can be vulnerable, because can contain foreign (user) input captured from log (will be replaced in the shell arguments);
- optional parameter `mode` rewritten: normal (default), ddos, extra or aggressive (combines all), see sshd for regex details);
test cases reformatted (since "filterOptions", we don't need multiple test log-files anymore);
- rewritten using `prefregex` and used MLFID-related multi-line parsing (by using tag `<F-MLFID>` instead of buffering with `maxlines`);
- optional parameter `mode` introduced: normal (default), extra or aggressive (see sendmail-reject for regex details);
test cases extended
Many times faster and fewer CPU-hungry because of parsing with `maxlines=1`, so without line buffering (scrolling of the buffer-window).
Combination of tags `<F-MLFID>` and `<F-NOFAIL>` can be used now to process multi-line logs using single-line expressions:
- tag `<F-MLFID>`: used to identify resp. store failure info for groups of log-lines with the same identifier (e. g. combined failure-info for the same conn-id by `<F-MLFID>(?:conn-id)</F-MLFID>`, see sshd.conf for example)
- tag `<F-NOFAIL>`: used as mark for no-failure (helper to accumulate common failure-info);
filter.d/sshd.conf: [sshd], [sshd-ddos], [sshd-aggressive] optimized with pre-filtering using new option `prefregex` and new multi-line handling.
Some filters extended with user name;
[filter.d/pam-generic.conf]: grave fix injection on user name to host fixed;
test-cases in testSampleRegexsFactory can now check the captured groups (using additionally fields in failJSON structure)
- dynamical string replacement within call of regexp.sub fixed with lambda-replacement (otherwise "sre_constants.error: bad escape \s at position");
- optional arguments (3.6 has more agrs by calling of SMTPServer.process_message);
- implicit convert byte to string, because python3.6 fails on binary data (test_smtp).
new ssh rule(s) added:
- Connection reset by peer (multi-line rule during authorization process);
- No supported authentication methods available;
Single line and multi-line expression optimized, added optional prefixes and suffix (logged from several ssh versions);
closes gh-864
Prevent executing of several actions (e.g. mail, send-mail etc) on restart (bans were already notified).
Test cases extended (smtp and by restart in ServerReloadTest).
Closes gh-1141
Closes gh-921
The fail2ban server can take several seconds to shut down. This can
make Gentoo's start-stop-service time out and decide that stopping has
failed, even if it actually succeeds a few seconds later.
The default timeout for start-stop-service if --retry is not specified
appears to be 5 seconds. Increase that to 30 seconds to be sure that if
fail2ban-server is going to be able to stop, it has time to do so.
* enh-rel0.9.6: (60 commits)
updated man pages
ENH: prep for 0.9.6 release (as of tomorrow)
BF: added missing entires into MANIFEST
Update ChangeLog
ChangeLog entry added + jail.conf review
code review, makes the test cases workable, added dev-notes
ChangeLog update
`filter.d/apache-modsecurity.conf` - fixed for newer version (one space, closes gh-1626) reviewed and optimized: - non-greedy catch-all replaced for safer match - unneeded catch-all anchoring removed - non-capturing groups
filter.d/dovecot.conf update: - fixes failregex, that ignores failures through some irrelevant info (closes#1623); - ignores whole additionally irrelevant info in anchored regex before fixed failure data `\((?:auth failed, \d+ attempts( in \d+ secs)?|tried to use (disabled|disallowed) \S+ auth)\)` - review, IPv6 compatibility fix, non-capturing groups
Update jail.conf
Use Fedora's backend-settings for openSUSE
amend after code review of merge gh-1581
Make changes and add test file
Add Mongodb-auth filter and jail
Update FILTERS
filter.d/sshd.conf: Match 'Invalid user' with 'port \d*'
ChangeLog entry added
filter.d/sendmail-reject.conf: double space (should be by missing dns-host only) Closes#1578
Update Changelog to reflect the new np.conf action
Create npf.conf for the NPF packet filter
...
- fixed for newer version (one space, closes gh-1626)
reviewed and optimized:
- non-greedy catch-all replaced for safer match
- unneeded catch-all anchoring removed
- non-capturing groups
- fixes failregex, that ignores failures through some irrelevant info (closes#1623);
- ignores whole additionally irrelevant info in anchored regex before fixed failure data `\((?:auth failed, \d+ attempts( in \d+ secs)?|tried to use (disabled|disallowed) \S+ auth)\)`
- review, IPv6 compatibility fix, non-capturing groups
added helper "action.d/helpers-common.conf", and `_grep_logs` part-command for actions needed grep logs from multiple log-files
test cases: executing of some complex actions covered
- starting service in normal mode (without forking)
- does not restart if service exited normally (exit-code 0, e.g. stopped via fail2ban-client)
- does not restart if service can not start (exit-code 255, e.g. wrong configuration, etc.)
- service can be additionally started/stopped with commands (fail2ban-client, fail2ban-server)
- normalize calling parameters (persistent parameters moved from function arguments to filter member variables)
- save last line as lambda instead of return it as string (lazy convert of process line tuple to string on demand, needed in fail2ban-regex only)
- datedetector rewritten more strict as earlier;
- default templates can be specified exacter using prefix/suffix syntax (via `datepattern`);
- more as one date pattern can be specified using option `datepattern` now (new-line separated);
- some default options like `datepattern` can be specified directly in section `[Definition]`, that avoids contrary usage of unnecessarily `[Init]` section, because of performance (each extra section costs time);
- option `datepattern` can be specified in jail also (jails without filters);
- if first group specified, only this will be cut out from search log-line (e. g.: `^date:[({DATE})]` will cut out only datetime match pattern, and leaves `date:[] failure ip...` for searching in filter);
- faster match and fewer searching of appropriate templates (DateDetector.matchTime calls rarer DateTemplate.matchDate now);
- standard filters extended with exact prefixed or anchored date templates;
template cache introduced (in opposition to default template cache, holds custom templates cached by pattern for possible common usage of same template/regex);
- `datepattern = {^LN-BEG}` - only line-begin anchored default patterns
(matches date only at begin of line, or with max distance up to 2 non-alphanumeric characters from line-begin);
- `datepattern = {*WD-BEG}` - only word-begin anchored default patterns;
- `datepattern = ^prefix{DATE}suffix` - exact specified default patterns (using prefix and suffix);
common filter configs gets a more precise, line-begin anchored (datepattern = {^LN-BEG}) resp. custom anchoring default date-patterns;
thereby follow rule "shortest distance to datetime should win", so possible collision causes search though all templates;
speedup it a little bit (not possible collision if distance <= 1 or if line-begin anchoring, so break search if such template found)
By change of the distance (e.g. another format found), the pattern with smallest distance will be always preferred now.
To speedup (template lookup) resp. minimize of list reorder counts, the distance will be used as divider factor of the template weight by the templates comparison.
* Misleading date patterns defined more precisely (using extended syntax %E[mdHMS]
for exact two-digit match)
* `filter.d/freeswitch.conf`
- Optional prefixes (server, daemon, dual time) if systemd daemon logs used (gh-1548)
- User part rewritten to accept IPv6 resp. domain after "@" (gh-1548)
test cases could pass (so increase) verbosity to the client (and furthermore client to the server also), usable for debug purposes resp. simplifying read of the log-file;
custom and precise numeric log-levels can be given in test cases now;
JailThread get method `join` for safe usage of it, also for not started threads (test-cases or in case of error), that will be used for cleanup resp. wait purposes also (see join of pyinotify-filter);
waiting with `wait_for` extended with verifying of active flag;
implemented better error handling in some multi-threaded routines;
shutdown of jails rewritten (faster and safer, does not breaks shutdown process if some error occurred);
```bash
## start server with DEBUG log-level (ignore level read from fail2ban.conf):
fail2ban-client --loglevel DEBUG start
## or
fail2ban-server -c /cfg/path --loglevel DEBUG start
## keep server log-level by reload (without restart it)
fail2ban-client --loglevel DEBUG reload
## switch log-level back to INFO:
fail2ban-client set loglevel INFO
```
# [WARNING] TODO: be sure all banactions get a repair command, because otherwise stop/start will theoretically remove all the bans, but the tickets are still in BanManager, so in case of new failures it will not be banned, because "already banned" will happen.
- better ban/unban handling within actions (e.g. used dict instead of list)
- don't copy bans resp. its list on some operations;
- added new unbantime handling to relieve unBanList (prevent permanent searching for tickets to unban)
- prefer failure-ID as identifier of the ticket to its IP (most of the time the same, but it can be something else e.g. user name in some complex jails, as introduced in 0.10)
new replacement for `<ADDR>` in opposition to `<HOST>`, for separate usage of 2 address groups only (regardless of `usedns`), `ip4` and `ip6` together, without host (dns)
optimized FilterSystemd method `run`: better wait in idle (no busy-loop), better poll handling, the ban will executed anywhere (at least at 100th log-entry), also if we have never ending logging in this jail (e.g. extremely logging or too many failures)
systemd test cases extended
fixed pythonic filters and test scripts (running via "fail2ban-python" now);
fixed test case "testSetupInstallRoot" not for default python (also using direct call, out of virtualenv);
# Conflicts:
# config/filter.d/ignorecommands/apache-fakegooglebot
# fail2ban/tests/files/config/apache-auth/digest.py
# fail2ban/tests/files/ignorecommand.py
# fail2ban/tests/misctestcase.py
fixed pythonic filters and test scripts (running via "fail2ban-python" now);
fixed test case "testSetupInstallRoot" not for default python (also using direct call, out of virtualenv);
yoh: Squashed to ease cherry-picking into 0.9
* accept no space after "failed:"
fix issue #1497
* accept no space after "failed:"
* Update postfix-sasl
* Update postfix-sasl
* Update postfix-sasl
yoh: Squashed to ease cherry-picking into 0.9
* accept no space after "failed:"
fix issue #1497
* accept no space after "failed:"
* Update postfix-sasl
* Update postfix-sasl
* Update postfix-sasl
ASSP V1 development stopped at the end of 2014 and it is now deprecated.
All users were urged to upgrade to ASSP V2 which is still actively
developed.
fail2ban 0.9.5 (and trunk) still have code which only understands ASSP
V1 logs.
This means the filter ignores brute force attacks against ASSP. This fix
adds V2 support.
ASSP V1 development stopped at the end of 2014 and it is now deprecated.
All users were urged to upgrade to ASSP V2 which is still actively
developed. For some reason fail2ban 0.9.5 (and trunk) still have code
which only understands ASSP V1 logs. This means the filter ignores brute
force attacks against ASSP.
Now updated with anchored patterns tested against 6 months of log data.
ASSP V1 development stopped at the end of 2014 and it is now deprecated.
All users were urged to upgrade to ASSP V2 which is still actively
developed. For some reason fail2ban 0.9.5 (and trunk) still have code
which only understands ASSP V1 logs. This means the filter ignores brute
force attacks against ASSP.
* 'master' of git://github.com/fail2ban/fail2ban:
BF: do not rely on long relative path to upstairs config - symlink dereferenced copied during install (#1485)
ver. 0.9.5 (2016/07/15) - old-not-obsolete
-----------
0.9.x line is no longer heavily developed. If you are interested in
new features (e.g. IPv6 support), please consider 0.10 branch and its
releases.
* `filter.d/monit.conf`
- Extended failregex with new monit "access denied" version (gh-1355)
- failregex of previous monit version merged as single expression
* `filter.d/postfix.conf`, `filter.d/postfix-sasl.conf`
- Extended failregex daemon part, matching also `postfix/smtps/smtpd`
now (gh-1391)
* Fixed a grave bug within tags substitutions because of incorrect
detection of recursion in case of multiple inline substitutions
of the same tag (affected actions: `bsd-ipfw`, etc). Now tracks
the actual list of the already substituted tags (per tag instead
of single list)
* `filter.d/common.conf`
- Unexpected extra regex-space in generic `__prefix_line` (gh-1405)
- All optional spaces normalized in `common.conf`, test covered now
- Generic `__prefix_line` extended with optional brackets for the
date ambit (gh-1421), added new parameter `__date_ambit`
* `gentoo-initd` fixed `--pidfile` bug: `--pidfile` is option of
`start-stop-daemon`, not argument of fail2ban (see gh-1434)
* `filter.d/asterisk.conf`
- Fixed security log support for PJSIP and Asterisk 13+ (gh-1456)
- Improved log support for PJSIP and Asterisk 13+ with different
callID (gh-1458)
* New Actions:
- `action.d/firewallcmd-rich-rules` and `action.d/firewallcmd-rich-logging`
(gh-1367)
* New filters:
- slapd - ban hosts, that were failed to connect with invalid
credentials: error code 49 (gh-1478)
* Extreme speedup of all sqlite database operations (gh-1436),
by using of following sqlite options:
- (synchronous = OFF) write data through OS without syncing
- (journal_mode = MEMORY) use memory for the transaction logging
- (temp_store = MEMORY) temporary tables and indices are kept in memory
* journald journalmatch for pure-ftpd (gh-1362)
* Added additional regex filter for dovecot ldap authentication failures (gh-1370)
* `filter.d/exim*conf`
- Added additional regexes (gh-1371)
- Made port entry optional
* tag '0.9.5':
Added missing files to MANIFEST
BF: do not rely on long relative path to upstairs config - symlink common.conf
ver. 0.9.5 (2016/07/15) - old-not-obsolete
-----------
0.9.x line is no longer heavily developed. If you are interested in
new features (e.g. IPv6 support), please consider 0.10 branch and its
releases.
* `filter.d/monit.conf`
- Extended failregex with new monit "access denied" version (gh-1355)
- failregex of previous monit version merged as single expression
* `filter.d/postfix.conf`, `filter.d/postfix-sasl.conf`
- Extended failregex daemon part, matching also `postfix/smtps/smtpd`
now (gh-1391)
* Fixed a grave bug within tags substitutions because of incorrect
detection of recursion in case of multiple inline substitutions
of the same tag (affected actions: `bsd-ipfw`, etc). Now tracks
the actual list of the already substituted tags (per tag instead
of single list)
* `filter.d/common.conf`
- Unexpected extra regex-space in generic `__prefix_line` (gh-1405)
- All optional spaces normalized in `common.conf`, test covered now
- Generic `__prefix_line` extended with optional brackets for the
date ambit (gh-1421), added new parameter `__date_ambit`
* `gentoo-initd` fixed `--pidfile` bug: `--pidfile` is option of
`start-stop-daemon`, not argument of fail2ban (see gh-1434)
* `filter.d/asterisk.conf`
- Fixed security log support for PJSIP and Asterisk 13+ (gh-1456)
- Improved log support for PJSIP and Asterisk 13+ with different
callID (gh-1458)
* New Actions:
- `action.d/firewallcmd-rich-rules` and `action.d/firewallcmd-rich-logging`
(gh-1367)
* New filters:
- slapd - ban hosts, that were failed to connect with invalid
credentials: error code 49 (gh-1478)
* Extreme speedup of all sqlite database operations (gh-1436),
by using of following sqlite options:
- (synchronous = OFF) write data through OS without syncing
- (journal_mode = MEMORY) use memory for the transaction logging
- (temp_store = MEMORY) temporary tables and indices are kept in memory
* journald journalmatch for pure-ftpd (gh-1362)
* Added additional regex filter for dovecot ldap authentication failures (gh-1370)
* `filter.d/exim*conf`
- Added additional regexes (gh-1371)
- Made port entry optional
* tag '0.9.5': (70 commits)
DOC: preparations for 0.9.5 release
Added missing files to MANIFEST
another variant of regex
add trailing anchor to failregex
DOC: Reformatted ChangeLog into legit Markdown (Closes#962)
DOC: tuned up ChangeLog entries for 0.9.5
add PR id to ChangeLog
improved failregex according to @sebres recomendations
Improved changes of gh-1458: `[^']*` after callid was wrong, changed to `[^\)]*`; regexp anchored at the end; almost the same regex grouped to one;
Improve PJSIP log support for asterisk 13+ with different callID (Squash gh-1458) Change the asterisk pjsip filter to don't take the callId part Add optional part between "Request" and "from" Listed all log message from asterisk
* add `__prefix_line` to regex * fix time in log file
add info to log file
added sample log lines for slapd
adding openldap slapd filter
badip timeout option introduced, set to 30 seconds in our test cases (#1463)
DOC: changelog for recent exim filters tune up
Asterisk pjsip (#1456)
BF: finalize that sample log line for exim4
amend for new option of `usedns=raw` - forgotten validation fix inside setUseDns
RF: for consistency use (?:XXX)? instead of (?:|XXX)
...
* origin/master:
another variant of regex
add trailing anchor to failregex
add PR id to ChangeLog
improved failregex according to @sebres recomendations
* add `__prefix_line` to regex * fix time in log file
add info to log file
added sample log lines for slapd
adding openldap slapd filter
* commit '0.9.4-79-gaf8b650':
badip timeout option introduced, set to 30 seconds in our test cases (#1463)
DOC: changelog for recent exim filters tune up
Asterisk pjsip (#1456)
BF: finalize that sample log line for exim4
RF: for consistency use (?:XXX)? instead of (?:|XXX)
ENH: use non-capturing regex groups in exim-common and exim filters
ENH: exim filters -- make more use of %(host_info)s which in turn made more flexible
BF: make :port and I=[ip]:port optional for a "AUTH command used when not advertised"
* Improve PJSIP log support for Asterisk 13+
* Update changelog: filter.d/asterisk.conf - fix security log support for PJSIP and Asterisk 13+
* Change pjsip regexp with sebres observation, thanks to @nturcksin
- better recognition for usage of textToIp, expected or raw value should be used;
- separated failure id vs. host (if found use `fid` instead of `host` resp. `ip`);
- additional optional groups may be used in tags replacement by executing actions;
all optional spaces normalized in generic include `common.conf` + test cases are extended (using new example pseudo-filter and test log `zzz-generic-example`);
- client/server functionality moved away from bin and using now the common interface (introduced in fail2bancmdline);
- start in foreground fixed;
- server can act as client corresponding command line;
- command "restart" added: in opposite to "reload" in reality restarts the server (new process);
- several client/server bugs during starting process fixed.
respect standard spelling of IPv6 (with port) enclosed in brackets ([ipv6]:port), to prevent the necessarily changing of many filter definitions, comparison:
#IPv4:
127.0.0.1:55555
#IPv6:
[a:b:c:d::1]:55555
some small fixes (in test cases also) and code review and few optimizations;
implemented caching functionality for same substitutions inside replaceTag: very actual and extreme performance growth (up to 1000 times) for ban/unban because too slow substituteRecursiveTags by several tags and many includes, but totally unnecessary as long as parameters are not changing;
- introduced "conditional" sections, see for example `[Init?family=inet6]`;
- iptables-common and other iptables config(s) made IPv6 capable;
- several small code optimizations;
* all test cases passed (py3.x compatible);
SmartOS (and likely other Illumos platforms) enter log entries for failed sshd logins of the form:
`Authentication failed for USER from HOST`
The current sshd.conf regex matches `failure` -- add to this a match for `failed` to support Illumos
- differentiate between in loop and active (reset active only after really leaves a loop);
- differentiate between foreign and self loop-thread by stop/close connection;
- stops logging connection errors if too many errors;
- safe remove socket (no sporadic errors if too fast start/stop occurred);
ver. 0.9.4 (2016/03/08) - for-you-ladies
-----------
- Fixes:
* roundcube-auth jail typo for logpath
* Fix dnsToIp resolver for fqdn with large list of IPs (gh-1164)
* filter.d/apache-badbots.conf
- Updated useragent string regex adding escape for `+`
* filter.d/mysqld-auth.conf
- Updated "Access denied ..." regex for MySQL 5.6 and later (gh-1211, gh-1332)
* filter.d/sshd.conf
- Updated "Auth fail" regex for OpenSSH 5.9 and later
* Treat failed and killed execution of commands identically (only
different log messages), which addresses different behavior on different
exit codes of dash and bash (gh-1155)
* Fix jail.conf.5 man's section (gh-1226)
* Fixed default banaction for allports jails like pam-generic, recidive, etc
with new default variable `banaction_allports` (gh-1216)
* Fixed `fail2ban-regex` stops working on invalid (wrong encoded) character
for python version < 3.x (gh-1248)
* Use postfix_log logpath for postfix-rbl jail
* filters.d/postfix.conf - add 'Sender address rejected: Domain not found' failregex
* use `fail2ban_agent` as user-agent in actions badips, blocklist_de, etc (gh-1271)
* Fix ignoring the sender option by action_mw, action_mwl and action_c_mwl
* Changed filter.d/asterisk regex for "Call from ..." (few vulnerable now)
* Removed compression and rotation count from logrotate (inherit them from
the global logrotate config)
- New Features:
* New interpolation feature for definition config readers - `<known/parameter>`
(means last known init definition of filters or actions with name `parameter`).
This interpolation makes possible to extend a parameters of stock filter or
action directly in jail inside jail.local file, without creating a separately
filter.d/*.local file.
As extension to interpolation `%(known/parameter)s`, that does not works for
filter and action init parameters
* New actions:
- nftables-multiport and nftables-allports - filtering using nftables
framework. Note: it requires a pre-existing chain for the filtering rule.
* New filters:
- openhab - domotic software authentication failure with the
rest api and web interface (gh-1223)
- nginx-limit-req - ban hosts, that were failed through nginx by limit
request processing rate (ngx_http_limit_req_module)
- murmur - ban hosts that repeatedly attempt to connect to
murmur/mumble-server with an invalid server password or certificate.
- haproxy-http-auth - filter to match failed HTTP Authentications against a
HAProxy server
* New jails:
- murmur - bans TCP and UDP from the bad host on the default murmur port.
* sshd filter got new failregex to match "maximum authentication
attempts exceeded" (introduced in openssh 6.8)
* Added filter for Mac OS screen sharing (VNC) daemon
- Enhancements:
* Do not rotate empty log files
* Added new date pattern with year after day (e.g. Sun Jan 23 2005 21:59:59)
http://bugs.debian.org/798923
* Added openSUSE path configuration (Thanks Johannes Weberhofer)
* Allow to split ignoreip entries by ',' as well as by ' ' (gh-1197)
* Added a timeout (3 sec) to urlopen within badips.py action
(Thanks M. Maraun)
* Added check against atacker's Googlebot PTR fake records
(Thanks Pablo Rodriguez Fernandez)
* Enhance filter against atacker's Googlebot PTR fake records
(gh-1226)
* Nginx log paths extended (prefixed with "*" wildcard) (gh-1237)
* Added filter for openhab domotic software authentication failure with the
rest api and web interface (gh-1223)
* Add *_backend options for services to allow distros to set the default
backend per service, set default to systemd for Fedora as appropriate
* Performance improvements while monitoring large number of files (gh-1265).
Use associative array (dict) for monitored log files to speed up lookup
operations. Thanks @kshetragia
* Specified that fail2ban is PartOf iptables.service firewalld.service in
.service file -- would reload fail2ban if those services are restarted
* Provides new default `fail2ban_version` and interpolation variable
`fail2ban_agent` in jail.conf
* Enhance filter 'postfix' to ban incoming SMTP client with no fqdn hostname,
and to support multiple instances of postfix having varying suffix (gh-1331)
(Thanks Tom Hendrikx)
* files/gentoo-initd to use start-stop-daemon to robustify restarting the service
* tag '0.9.4': (138 commits)
MANIFEST RELEASE and man pages updates
Changes for the 0.9.4 release
datedetector: epoch time expression fix (now 10-11 chars, only whole number - anchored ^...\b or by special case within [], audit()) + test cases extended (positive/negative)
changelog about gentoo initd
added wp-admin
ENH(TST): a hypothetical example to show/test needing trailing anchoring
ENH: revert back to having detailed suffix anchored at the end for mysqld-auto.conf
Changelog for the recent PR and added Tom to THANKS
mysqld: failregex fixed (accepts different log level, more secure expression now); closes#1332
Add support for matching postfix multi-instance daemon names by default
DOC: removed Nick from listed as FreeBSD maintainer
DOC: adjusted ISSUE_TEMPLATE.md picking on @sebres's version
ENH: github templates for issues and PRs
ENH: add codecov support to travis.yml and bandge to README.md
gentoo-initd: Use start-stop-daemon in order to handle crashes better
regexp rewritten (few vulnerable as previous) + test case added
Update asterisk filter: changed regex for "Call from ...". Sometimes extension can have a plus symbol (+) because they can be phone number. Closes#1309
Add new regex into postfix filter. The new regexp is able to detect bad formatted SMTP EHLO command
Remove compression and count from logrotate
gentoo-initd: do not hide useful output
...
Currently, if fail2ban is killed (or crashes), its status will be
reported by '/etc/init.d/fail2ban status' as 'running' even though it
is not. Attempting to restart the service also fails, because Gentoo
unsuccessfully tries to stop the service.
By using start-stop-daemon and providing a pidfile, Gentoo will
instead report the status as 'crashed' and allow the service to be
restarted as normal.
* pr/1232/head:
removed system.log
Removed old svn revision comment
removed false matches
Removed includes comment for screensharing jail
Now using a literal logpath for screensharing jail
Fixed blatant typo in regex
clarified comments on sample log format
Fixed name (again?)
Made screensharing jail off by default
Changed regex prequel
added entry for new screensharingd filter
name change & new sample data
Added json metadata
Sample log for test case
Replaced .* with literal
Update jail.conf
Added new path variable for system.log
Added in settings for screensharingd filter
Created file
Conflicts:
ChangeLog - moved to New Features
config/jail.conf - kept at the end
- default date templates cache + regexp compiling on the fly (if required);
- better usage of shared config over all test cases;
- obsolete test cases removed (simple test of memleak/gc);
- skip some slow test cases in fast mode (setup test, etc.);
prevents out of memory situation if many IP's makes extremely many failures (or very large files since last fail2ban run);
closes gh-1277
todo: parameter `maxentries` should be configurable (jail.conf resp. fail2ban.conf);
todo: adjust ban-time-incr branch by merge (table "bips").
```
in executeCmd
if retcode < 0:
TypeError: unorderable types: NoneType() < int()
```
2) prevents a rarely test case bug of testExecuteTimeoutWithNastyChildren, because no stdout (Resource temporarily unavailable), possible no flush by IO of the killing process;
definitely closes gh-161, also other usage of asyncore event loop (in test_smtp.py)
repair cache in ipToName (can returns None), precaching of invalid IPs (according to RFC 5737) to stop endless wait for resolving it in test cases.
- new options for "fail2ban-testcases" introduced: "-g" or "--no-gamin" and "-m" or "--memory-db", both are true also if "-f" or "--fast" specified,
for example: `fail2ban-testcases -ngm` will runs faster (because no network, no gamin, memory database)
but will use the same default sleep intervals as in production (in comparison to -nf);
- seekToTime rewritten, accuracy increased by seekToTime, extended for all FileFilter (PyInotify, Gamin also), test cases extended etc.
- common performance optimized and code reviewed;
-- cache dnsToIp, ipToName to prevent long wait during retrieving of ip/name for wrong dns or lazy dns-system;
-- instead of simple "sleep" used conditional wait "wait_for", that internal increases sleep interval up to sleeptime;
-- ticket / banmanager / failmanager modules are performance optimized;
-- api of filter (log files), jail, etc. rewritten and extended for performance purposes;
- performance of test cases optimized:
-- added option "--fast" to decrease wait intervals, avoid passive waiting, and skip few very slow test cases;
- code review after partially cherry pick of branch 'ban-time-incr' (see gh-716)
-- ticket module prepared to easy merge with newest version of 'ban-time-incr', now additionally holds banTime, banCount and json-data;
-- executeCmd partially moved from action to new module utils, etc.
-- python 2.6 compatibility;
- testExecuteTimeoutWithNastyChildren: test case repaired - wait for pid file inside bash, kill tree in any case (gh-1155);
- testSocket: test case repaired - wait for server thread starts a socket (listener)
addJail (executed before addLog) early uses a "INSERT OR REPLACE" statement to update "enabled" to 1 (and add jail the first time used at once), but this syntax in sqlite always deletes an entry (cause of constraint) and inserts it again, so because of CASCADE all log entries with this jail will be also deleted from logs table.
These fixes are pretty pedantic, but they do simplify the script a
little.
* Checking the existence of a file/directory before creating/deleting
it adds complexity and raciness. There are better options.
* mkdir -p does the job of making sure a directory exists. (It only
fails if there's a filesystem error or something.)
* Likewise, rm -f doesn't fail if the file doesn't exist.
* rm -r isn't neccessary because the socket shouldn't be a directory.
(If it is for some reason, that should be an error.)
However, still failing test, even though ```PYTHONPATH=. fail2ban-regex -v fail2ban/tests/files/logs/screensharingd /etc/fail2ban/filter.d/screensharingd.conf``` gives desired result
Previously, an attacker could fake a domain like
crawl-1-1-1-1.googlebot.com.fake.net and get resolved. This change
avoids to resolve fake Google domains.
Document the changes from 36919d9f in the ChangeLog and add myself to
the THANKS file (at @sebres suggestion).
Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
The regex for matching against "Auth fail" disconnect log message does
not match against current versions of ssh. OpenSSH 5.9 introduced
privilege separation of the pre-auth process, which included
[logging through monitor.c](http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/ssh/monitor.c.diff?r1=1.113&r2=1.114)
which adds " [preauth]" to the end of each message and causes the log
level to be prepended to each message.
It also fails to match against clients which send a disconnect message
with a description that is either empty or includes a space, since this
is the content in the log message after the disconnect code, per
[packet.c:1785](http://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/ssh/packet.c?annotate=1.215),
which was matched by \S+. Although I have not observed this yet, I
couldn't find anything which would preclude it in [RFC
4253](https://tools.ietf.org/html/rfc4253#section-11.1) and since the
message is attacker-controlled it provides a way to avoid getting
banned.
This commit fixes both issues.
Signed-off-by: Kevin Locke <kevin@kevinlocke.name>
* origin/master:
DOC: changelog for the timeout change
Set Timeout at urlopen to 3 seconds
README :: init/service example mentions debian based systems as the example
README :: fitted paragraph style
BF: disable testing on python 3.2 until coverage gets a fix
README :: Some style/grammar tweaks, and init/service script mention. Re: #1193
Set Timeout at urlopen to 3 seconds
* origin/master:
Comment spelling fixes
logrotate: Do not rotate empty logs
logrotate: Remove outdated Fedora comment
Updated Changelog
Bugfix for dnsToIp resolver for fqdn with large list of IPs; closes#1164
Added apache-badbots.conf
Update gen_badbots
Update apache-badbots.conf
changelog for freshly merged PR (roundcube-auth definition of logpath)
Fixed line suspected to be faulty
renamed <NAME> to correct <ACT> in protocol
Conflicts:
fail2ban/tests/actiontestcase.py -- due to comments fix up
As a useful side effect, prevents "Unable to contact server. Is it
running?" mails from cron when fail2ban hasn't been (intentionally)
running nor thus logging anything either.
ver. 0.9.3 (2015/08/01) - lets-all-stay-friends
----------
- IMPORTANT incompatible changes:
* filter.d/roundcube-auth.conf
- Changed logpath to 'errors' log (was 'userlogins')
* action.d/iptables-common.conf
- All calls to iptables command now use -w switch introduced in
iptables 1.4.20 (some distribution could have patched their
earlier base version as well) to provide this locking mechanism
useful under heavy load to avoid contesting on iptables calls.
If you need to disable, define 'action.d/iptables-common.local'
with empty value for 'lockingopt' in `[Init]` section.
* mail-whois-lines, sendmail-geoip-lines and sendmail-whois-lines
actions now include by default only the first 1000 log lines in
the emails. Adjust <grepopts> to augment the behavior.
- Fixes:
* reload in interactive mode appends all the jails twice (gh-825)
* reload server/jail failed if database used (but was not changed) and
some jail active (gh-1072)
* filter.d/dovecot.conf - also match unknown user in passwd-file.
Thanks Anton Shestakov
* Fix fail2ban-regex not parsing journalmatch correctly from filter config
* filter.d/asterisk.conf - fix security log support for Asterisk 12+
* filter.d/roundcube-auth.conf
- Updated regex to work with 'errors' log (1.0.5 and 1.1.1)
- Added regex to work with 'userlogins' log
* action.d/sendmail*.conf - use LC_ALL (superseeding LC_TIME) to override
locale on systems with customized LC_ALL
* performance fix: minimizes connection overhead, close socket only at
communication end (gh-1099)
* unbanip always deletes ip from database (independent of bantime, also if
currently not banned or persistent)
* guarantee order of dbfile to be before dbpurgeage (gh-1048)
* always set 'dbfile' before other database options (gh-1050)
* kill the entire process group of the child process upon timeout (gh-1129).
Otherwise could lead to resource exhaustion due to hanging whois
processes.
* resolve /var/run/fail2ban path in setup.py to help installation
on platforms with /var/run -> /run symlink (gh-1142)
- New Features:
* RETURN iptables target is now a variable: <returntype>
* New type of operation: pass2allow, use fail2ban for "knocking",
opening a closed port by swapping blocktype and returntype
* New filters:
- froxlor-auth - Thanks Joern Muehlencord
- apache-pass - filter Apache access log for successful authentication
* New actions:
- shorewall-ipset-proto6 - using proto feature of the Shorewall. Still requires
manual pre-configuration of the shorewall. See the action file for detail.
* New jails:
- pass2allow-ftp - allows FTP traffic after successful HTTP authentication
- Enhancements:
* action.d/cloudflare.conf - improved documentation on how to allow
multiple CF accounts, and jail.conf got new compound action
definition action_cf_mwl to submit cloudflare report.
* Check access to socket for more detailed logging on error (gh-595)
* fail2ban-testcases man page
* filter.d/apache-badbots.conf, filter.d/nginx-botsearch.conf - add
HEAD method verb
* Revamp of Travis and coverage automated testing
* Added a space between IP address and the following colon
in notification emails for easier text selection
* Character detection heuristics for whois output via optional setting
in mail-whois*.conf. Thanks Thomas Mayer.
Not enabled by default, if _whois_command is set to be
%(_whois_convert_charset)s (e.g. in action.d/mail-whois-common.local),
it
- detects character set of whois output (which is undefined by
RFC 3912) via heuristics of the file command
- converts whois data to UTF-8 character set with iconv
- sends the whois output in UTF-8 character set to mail program
- avoids that heirloom mailx creates binary attachment for input with
unknown character set
* tag '0.9.3': (99 commits)
Release changes (too much of manual "labor"! ;))
BF: realpath for /var/run/fail2ban Closes#1142
Changelog entry for killpg fix
Changelog entries for Serge's fixes
bug fix: option 'dbpurgeage' was never set (always default) by start of fail2ban, because of invalid sorting of options ('dbfile' should be always set before other database options) / closes#1048, closes#1050
BF: guarantee order of dbfile to be before dbpurgeage (Closes#1048)
DOC: Changelog for shorewall-ipset-proto6.conf + adjusted its description
DOC: moved and adjusted changelog entry from 0.9.2 within 0.9.3 to come
TST: test to verify killing stuck children processes
BF: kill the entire process group upon timeout (Close#1129)
Limit the number of log lines in *-lines.conf actions
ipjailmatches is on one line with its description in man jail.conf
DOC: Changelog for iptables -w change
Remove self.printlog() call
Remove literal "TODO" from method's name
BF: do not wrap iptables into itself. Thanks Lee
Added a space between IP address and the following colon
BF: symbiosis-blacklist-allports now also requires iptables-common.conf
RF: use <iptables> to take effect of it being a parameter
ENH: added lockingopt option for iptables actions, made iptables cmd itself a parameter
...
* 'grep-m1k' of github.com:szepeviktor/fail2ban:
Limit the number of log lines in *-lines.conf actions
Conflicts:
ChangeLog -- took both versions and adjusted the new one
for -n 1000 change
Reorganize .travis.yml
Separate coverage tests for Python 2 and Python 3
Execute setup.py install using the environment's Python exe
Sanitize Travis execution order
Event containing Challenge may come without ReceivedChallenge, but with
Response and ExpectedResponse.
Also Challenge now accepts '/' character, since it is used at least by PJSIP.
Unlike chan_sip and manager, PJSIP populates SessionID using
Call-Id header of a related SIP message.
As Call-Id of a SIP message can contain almost anything,
the regular expression for SessionID has been loosened.
Long delayed
ver. 0.9.2 (2015/04/29) - better-quick-now-than-later
----------
- Fixes:
* Fix ufw action commands
* infinite busy loop on _escapedTags match in substituteRecursiveTags gh-907.
Thanks TonyThompson
* port[s] typo in jail.conf/nginx-http-auth gh-913. Thanks Frederik Wagner
(fnerdwq)
* $ typo in jail.conf. Thanks Skibbi. Debian bug #767255
* grep'ing for IP in *mail-whois-lines.conf should now match also
at the beginning and EOL. Thanks Dean Lee
* jail.conf
- php-url-fopen: separate logpath entries by newline
* failregex declared direct in jail was joined to single line (specifying of
multiple expressions was not possible).
* filters.d/exim.conf - cover different settings of exim logs
details. Thanks bes.internal
* filter.d/postfix-sasl.conf - failregex is now case insensitive
* filters.d/postfix.conf - add 'Client host rejected error message' failregex
* fail2ban/__init__.py - add strptime thread safety hack-around
* recidive uses iptables-allports banaction by default now.
Avoids problems with iptables versions not understanding 'all' for
protocols and ports
* filter.d/dovecot.conf
- match pam_authenticate line from EL7
- match unknown user line from EL7
* Use use_poll=True for Python 2.7 and >=3.4 to overcome "Bad file
descriptor" msgs issue (gh-161)
* filter.d/postfix-sasl.conf - tweak failregex and add ignoreregex to ignore
system authentication issues
* fail2ban-regex reads filter file(s) completely, incl. '.local' file etc.
(gh-954)
* firewallcmd-* actions: split output into separate lines for grepping (gh-908)
* Guard unicode encode/decode issues while storing records in the database.
Fixes "binding parameter error (unsupported type)" (gh-973), thanks to kot
for reporting
* filter.d/sshd added regex for matching openSUSE ssh authentication failure
* filter.d/asterisk.conf:
- Dropped "Sending fake auth rejection" failregex since it incorrectly
targets the asterisk server itself
- match "hacking attempt detected" logs
- New Features:
- New filters:
- postfix-rbl Thanks Lee Clemens
- apache-fakegooglebot.conf Thanks Lee Clemens
- nginx-botsearch Thanks Frantisek Sumsal
- drupal-auth Thanks Lee Clemens
- New recursive embedded substitution feature added:
- `<<PREF>HOST>` becomes `<IPV4HOST>` for PREF=`IPV4`;
- `<<PREF>HOST>` becomes `1.2.3.4` for PREF=`IPV4` and IPV4HOST=`1.2.3.4`;
- New interpolation feature for config readers - `%(known/parameter)s`.
(means last known option with name `parameter`). This interpolation makes
possible to extend a stock filter or jail regexp in .local file
(opposite to simply set failregex/ignoreregex that overwrites it),
see gh-867.
- Monit config for fail2ban in files/monit/
- New actions:
- action.d/firewallcmd-multiport and action.d/firewallcmd-allports Thanks Donald Yandt
- action.d/sendmail-geoip-lines.conf
- action.d/nsupdate to update DNSBL. Thanks Andrew St. Jean
- New status argument for fail2ban-client -- flavor:
fail2ban-client status <jail> [flavor]
- empty or "basic" works as-is
- "cymru" additionally prints (ASN, Country RIR) per banned IP
(requires dnspython or dnspython3)
- Flush log at USR1 signal
- Enhancements:
* Enable multiport for firewallcmd-new action. Closes gh-834
* files/debian-initd migrated from the debian branch and should be
suitable for manual installations now (thanks Juan Karlo de Guzman)
* Define empty ignoreregex in filters which didn't have it to avoid
warnings (gh-934)
* action.d/{sendmail-*,xarf-login-attack}.conf - report local
timezone not UTC time/zone. Closes gh-911
* Conditionally log Ignore IP with reason (dns, ip, command). Closes gh-916
* Absorbed DNSUtils.cidr into addr2bin in filter.py, added unittests
* Added syslogsocket configuration to fail2ban.conf
* Note in the jail.conf for the recidive jail to increase dbpurgeage (gh-964)
* tag '0.9.2':
Hope for release tomorrow
BF: if no /dev/log on Linux -- don't expect setting syslog to work
Fix actions in ufw.conf
Add drupal-auth filter and jail
* commit '0f75ed5e2ab1159e45a7771a7a4e90c877ec848e':
Just use a system wide python in the tests digest.py
DOC: Slight tune up to RELEASE doc -- no need for PYTHONPATH to run tests
MANIFEST: updated for some new files, sorted all entries, removed some duplicates
Initial changes for the release -- simplified ChangeLog header etc
On Ubuntu 15.04 the ufw action was not working.
- With empty <application>, receiving errors:
2015-04-24 16:28:35,204 fail2ban.filter [8527]: INFO [sshd] Found 43.255.190.157
2015-04-24 16:28:35,695 fail2ban.actions [8527]: NOTICE [sshd] Ban 43.255.190.157
2015-04-24 16:28:35,802 fail2ban.action [8527]: ERROR [ -n "" ] && app="app " -- stdout: b''
2015-04-24 16:28:35,803 fail2ban.action [8527]: ERROR [ -n "" ] && app="app " -- stderr: b''
2015-04-24 16:28:35,803 fail2ban.action [8527]: ERROR [ -n "" ] && app="app " -- returned 1
- With action = ufw[application=OpenSSH], it was silently not doing
anything (no errors after "Ban x.x.x.x", but no IP addresses in ufw
status).
Re-arranged the bash commands on two lines, and it works with or without
<application>.
Long delayed and possibly incomplete 0.9.2 release:
ver. 0.9.2 (2015/04/26) - better-quick-now-than-later
----------
- Fixes:
* infinite busy loop on _escapedTags match in substituteRecursiveTags gh-907.
Thanks TonyThompson
* port[s] typo in jail.conf/nginx-http-auth gh-913. Thanks Frederik Wagner
(fnerdwq)
* $ typo in jail.conf. Thanks Skibbi. Debian bug #767255
* grep'ing for IP in *mail-whois-lines.conf should now match also
at the beginning and EOL. Thanks Dean Lee
* jail.conf
- php-url-fopen: separate logpath entries by newline
* failregex declared direct in jail was joined to single line (specifying of
multiple expressions was not possible).
* filters.d/exim.conf - cover different settings of exim logs
details. Thanks bes.internal
* filter.d/postfix-sasl.conf - failregex is now case insensitive
* filters.d/postfix.conf - add 'Client host rejected error message' failregex
* fail2ban/__init__.py - add strptime thread safety hack-around
* recidive uses iptables-allports banaction by default now.
Avoids problems with iptables versions not understanding 'all' for
protocols and ports
* filter.d/dovecot.conf
- match pam_authenticate line from EL7
- match unknown user line from EL7
* Use use_poll=True for Python 2.7 and >=3.4 to overcome "Bad file
descriptor" msgs issue (gh-161)
* filter.d/postfix-sasl.conf - tweak failregex and add ignoreregex to ignore
system authentication issues
* fail2ban-regex reads filter file(s) completely, incl. '.local' file etc.
(gh-954)
* firewallcmd-* actions: split output into separate lines for grepping (gh-908)
* Guard unicode encode/decode issues while storing records in the database.
Fixes "binding parameter error (unsupported type)" (gh-973), thanks to kot
for reporting
* filter.d/sshd added regex for matching openSUSE ssh authentication failure
* filter.d/asterisk.conf:
- Dropped "Sending fake auth rejection" failregex since it incorrectly
targets the asterisk server itself
- match "hacking attempt detected" logs
- New Features:
- New filters:
- postfix-rbl Thanks Lee Clemens
- apache-fakegooglebot.conf Thanks Lee Clemens
- nginx-botsearch Thanks Frantisek Sumsal
- New recursive embedded substitution feature added:
- `<<PREF>HOST>` becomes `<IPV4HOST>` for PREF=`IPV4`;
- `<<PREF>HOST>` becomes `1.2.3.4` for PREF=`IPV4` and IPV4HOST=`1.2.3.4`;
- New interpolation feature for config readers - `%(known/parameter)s`.
(means last known option with name `parameter`). This interpolation makes
possible to extend a stock filter or jail regexp in .local file
(opposite to simply set failregex/ignoreregex that overwrites it),
see gh-867.
- Monit config for fail2ban in files/monit/
- New actions:
- action.d/firewallcmd-multiport and action.d/firewallcmd-allports Thanks Donald Yandt
- action.d/sendmail-geoip-lines.conf
- action.d/nsupdate to update DNSBL. Thanks Andrew St. Jean
- New status argument for fail2ban-client -- flavor:
fail2ban-client status <jail> [flavor]
- empty or "basic" works as-is
- "cymru" additionally prints (ASN, Country RIR) per banned IP
(requires dnspython or dnspython3)
- Flush log at USR1 signal
- Enhancements:
* Enable multiport for firewallcmd-new action. Closes gh-834
* files/debian-initd migrated from the debian branch and should be
suitable for manual installations now (thanks Juan Karlo de Guzman)
* Define empty ignoreregex in filters which didn't have it to avoid
warnings (gh-934)
* action.d/{sendmail-*,xarf-login-attack}.conf - report local
timezone not UTC time/zone. Closes gh-911
* Conditionally log Ignore IP with reason (dns, ip, command). Closes gh-916
* Absorbed DNSUtils.cidr into addr2bin in filter.py, added unittests
* Added syslogsocket configuration to fail2ban.conf
* Note in the jail.conf for the recidive jail to increase dbpurgeage (gh-964)
* tag '0.9.2': (140 commits)
DOC: Slight tune up to RELEASE doc -- no need for PYTHONPATH to run tests
MANIFEST: updated for some new files, sorted all entries, removed some duplicates
Initial changes for the release -- simplified ChangeLog header etc
added \s after host
replaced .* before rhost with regex matching all the previous fields
Fixed typo in filter description authentification instead of authentication
Fixed the UTC -> CEST difference...
Added changes to ChangeLog & updated sample test cases
updated filter.d/sshd.conf
Do not run smtp tests if no_network set
BF: if install pypy -- come back to original directory
BF(OSX): apparently exceptions could not be compared for identity, use repr
very long time resolving IP for address "abcdef" on some PDC, under NAT etc. - replaced via "abcdef.abcdef" to prevent searching in local domains;
fix test for invalid IP (use TEST-NET-1 according to RFC 5737): since fef031b3cd failed, because on some platforms like vm:debian 10.0.0.0 returns 'localhost' (intern network).
Match hacking attempt IP instead of asterisk server IP (closes#1000)
BF: fixing up version comparison for pypy. Issue appeared in 2.5.0
ENH: minor formatting, no functional changes
BF: do not expect setting logtarget to SYSLOG to work on non-Linuxes
Added a comment about systemd backend for jails with logs outside of journal (Closes#959)
DOC: make a warning for recidive jail to increase dbpurgeage (Closes#964)
...
when set by user,
- detects character set of whois output (which is undefined by RFC 3912) via heuristics of the file command
- converts whois data to UTF-8 character set with iconv
- sends the whois output in UTF-8 character set to mail program
- avoids that heirloom mailx creates binary attachment for input with unknown character set
00:11 yoh: upgraded to 2.5.0 from debianexperimental and now reproduced locally
00:12 yoh: why pypy --version directs output to stderr? thereis no error
00:12 mattip: ok, can you try with a nightly from http://buildbot.pypy.org/nightly/trunk ?
00:15 * yoh running with 2.6.0-alpha0...
00:16 yoh: clean -- no crash
should avoid
File /usr/lib/python3.4/asyncore.py, line 208, in loop
poll_fun(timeout, map)
File /usr/lib/python3.4/asyncore.py, line 145, in poll
r, w, e = select.select(r, w, e, timeout)
OSError: [Errno 9] Bad file descriptor
Use ipset proto6 in shorewall. You must follow the rules to enable ipset in you blacklist
if you have a lot of spam (my case) is better use ipset rather than shorewall command line (is my firewall)
stop fail2ban with shorewall on one list of 1000 Ips takes 5 min with ipset in shorewall 10 sec.
* 'master' of https://github.com/rumple010/fail2ban:
Changed default TTL value to 60 seconds.
Added a reminder to create an nsupdate.local file to set required options.
Modified the ChangeLog and THANKS files to reflect the addition of action.d/nsupdate.conf.
add nsupdate action
Conflicts:
ChangeLog
As https://travis-ci.org/fail2ban/fail2ban/jobs/50609529 query might fail in
that response would not contain "categories". With this change we will handle
it explicitly and will spit out ValueError, providing information about
the response so it could be troubleshooted
Deleting not existent is not error.
Adding already present is not error.
Otherwise all those entries becomes stale forever, not removed and its number increases over time.
* 'enh/clarifyDnsUtilsMethods' of https://github.com/leeclemens/fail2ban:
Update ChangeLog
Clarify filter.DNSUtils functions' terminology and add unittests
Conflicts:
ChangeLog -- rephrased a bit as well
- Add : to match "SASL LOGIN authentication failed: Password:"
- Add ignoreregex to ignore system authentication issues:
"warning: unknown[1.1.1.1]: SASL LOGIN authentication failed: Connection lost to authentication server"
- Add test log messages for both
* 'enh/local_time_zone' of https://github.com/yarikoptic/fail2ban:
fixed typos, thanks szepeviktor for review
ENH: use non-UTC date invocation (without -u) and report offset for localzone (%z)
Conflicts:
ChangeLog
Adds a new action file that uses nsupdate to dynamically update a BIND
zone file with a TXT resource record representing a banned IP address.
Resource record is deleted from the zone when the ban expires.
test cases merged from remote-tracking branch 'yarikoptic:enh/embedded_tags' into issue/907
infinite busy loop on _escapedTags match in substituteRecursiveTags gh-907
* 'master' of git://github.com/fail2ban/fail2ban:
Update year in postfix logs test file
Add 'Client host rejected error message' regex Not sure if it was reworded (using Postfix 2.6) or a slightly different error, but I only have "Client host rejected: cannot find your hostname"
* commit '0.9.1-44-gd65c4f8': (31 commits)
moved debian's initd file to files/debian-initd from debian branch
Update ChangeLog
Monit config
BF: adjusted for new IP of example.com
downcase example
Added an item to "Fixes"
postfix-sasl failregex case insensitive
clean all after test setup (removes a build directory in current root of fail2ban)
exim filter: correct failregex for exim with extended log options
small fix: no cover for failed case
testSetupInstallRoot will be always skipped, because of "wrong" location of 'setup.py';
better and scalable solution for gh-867 (and gh-868), using only name convention like %(known/failregex)s to add custom expressions, so no interface changes in jail.conf are necessary (for example see test-known-interp in test cases);
Changelog entry for preceding fix
Separate php-url-fopen logpath by newline
python 2.6 compatibility: preventing RuntimeError: dictionary changed size during iteration.
interpolation of config readers extended with `%(known/parameter)s`. (means last known option with name `parameter`).
test cases extended (now correct)
BF: failregex declared direct in jail was joined to single line, (specifying of multiple expressions was not possible); feature request (gh-867): new options for jail introduced addfailregex/addignoreregex: extends regex specified in filter (opposite to failregex/ignoreregex that overwrites it);
Add ignoreregex to avoid warning on start
Add ignoreregex to avoid warning on start
...
feature request (gh-867): new options for jail introduced addfailregex/addignoreregex: extends regex specified in filter (opposite to failregex/ignoreregex that overwrites it);
• Do not install the following configuration files which are not used within the
Debian package of fail2ban:
/etc/fail2ban/paths-fedora.conf
/etc/fail2ban/paths-freebsd.conf
/etc/fail2ban/paths-osx.conf
Closes: Debian bug #767123 (https://bugs.debian.org/767123).
Signed-off-by: Christoph Anton Mitterer <mail@christoph.anton.mitterer.name>
test cases: extended with test to check action together with database functionality (ex.: to verify lambdas in checkBan);
database: getBansMerged should work within lock, using reentrant lock (cause call of getBans inside of getBansMerged);
What aught to be a bugfix release delayed into a featured release 0.9.1
ver. 0.9.1 (2014/10/29) - better, faster, stronger
----------
- Refactoring (IMPORTANT -- Please review your setup and configuration):
* iptables-common.conf replaced iptables-blocktype.conf
(iptables-blocktype.local should still be read) and now also
provides defaults for the chain, port, protocol and name tags
- Fixes:
* start of file2ban aborted (on slow hosts, systemd considers the server has
been timed out and kills him), see gh-824
* UTF-8 fixes in pure-ftp thanks to Johannes Weberhofer. Closes gh-806.
* systemd backend error on bad utf-8 in python3
* badips.py action error when logging HTTP error raised with badips request
* fail2ban-regex failed to work in python3 due to space/tab mix
* recidive regex samples incorrect log level
* journalmatch for recidive incorrect PRIORITY
* loglevel couldn't be changed in fail2ban.conf
* Handle case when no sqlite library is available for persistent database
* Only reban once per IP from database on fail2ban restart
* Nginx filter to support missing server_name. Closes gh-676
* fail2ban-regex assertion error caused by miscount missed lines with
multiline regex
* Fix actions failing to execute for Python 3.4.0. Workaround for
http://bugs.python.org/issue21207
* Database now returns persistent bans on restart (bantime < 0)
* Recursive action tags now fully processed. Fixes issue with bsd-ipfw
action
* Fixed TypeError with "ipfailures" and "ipjailfailures" action tags.
Thanks Serg G. Brester
* Correct times for non-timezone date times formats during DST
* Pass a copy of, not original, aInfo into actions to avoid side-effects
* Per-distribution paths to the exim's main log
* Ignored IPs are no longer banned when being restored from persistent
database
* Manually unbanned IPs are now removed from persistent database, such they
wont be banned again when Fail2Ban is restarted
* Pass "bantime" parameter to the actions in default jail's action
definition(s)
* filters.d/sieve.conf - fixed typo in _daemon. Thanks Jisoo Park
* cyrus-imap -- also catch also failed logins via secured (imaps/pop3s).
Regression was introduced while strengthening failregex in 0.8.11 (bd175f)
Debian bug #755173
* postfix-sasl - added journalmatch. Thanks Luc Maisonobe
* postfix* - match with a new daemon string (postfix/submission/smtpd).
Closes gh-804 . Thanks Paul Traina
* apache - added filter for AH01630 client denied by server configuration.
- New features:
- New filters:
- monit Thanks Jason H Martin
- directadmin Thanks niorg
- apache-shellshock Thanks Eugene Hopkinson (SlowRiot)
- New actions:
- symbiosis-blacklist-allports for Bytemark symbiosis firewall
- fail2ban-client can fetch the running server version
- Added Cloudflare API action
- Enhancements
* Start performance of fail2ban-client (and tests) increased, start time
and cpu usage rapidly reduced. Introduced a shared storage logic, to
bypass reading lots of config files (see gh-824).
Thanks to Joost Molenaar for good catch (reported gh-820).
* Fail2ban-regex - add print-all-matched option. Closes gh-652
* Suppress fail2ban-client warnings for non-critical config options
* Match non "Bye Bye" disconnect messages for sshd locked account regex
* courier-smtp filter:
- match lines with user names
- match lines containing "535 Authentication failed" attempts
* Add <chain> tag to iptables-ipsets
* Realign fail2ban log output with white space to improve readability. Does
not affect SYSLOG output
* Log unhandled exceptions
* cyrus-imap: catch "user not found" attempts
* Add support for Portsentry
* tag '0.9.1': (36 commits)
ENH: additional versioning changes
Refreshed manpages
ENH: fail early in generate-man + provide PYTHONPATH upstairs
Changes for the 0.9.1 release versioning
Populated MANIFEST with more entries which were preiously missed or duplicated. Sorted within each "section"
Add portsentry to changelog
ConfigReader.touch renamed into protected _create_unshared
DOC: documentation about available vagrantfile setup
Added myself into THANKS
DOC: adjust docs in mytime to place docs into docstrings
ENH: do use @staticmethod (we are well beyond support of 2.4 now)
testExecuteTimeout fixed: give a test still 1 second, because system could be too busy
coverage: no cover (for failed except)
fix: fail2ban-regex with filter file failed (after merging #824, because test case missing); test case for 'readexplicit' added;
ENH: remove obsolete code for python < 2.6 (we support >= 2.6)
DOC: very minor (tabs/spaces)
We better check that installation doesn't cause any errors as well
code review, change log entries added;
reset share/cache storage (if we use 'reload' in client with interactive mode)
normalize tabs/spaces in docstrings;
...
* '_tent/cache-config-read' of https://github.com/sebres/fail2ban:
code review, change log entries added;
reset share/cache storage (if we use 'reload' in client with interactive mode)
normalize tabs/spaces in docstrings;
cache-config-read-v2 merged; logging normalized, set log level for loading (read or use shared) file(s) to INFO; prevent to read some files twice by read inside "_getIncludes" and by "read" self (occurred by only one file);
code review; more stable config sharing, configurator always shares it config readers now;
code review: use the same code (corresponding test cases - with sharing on and without it);
rewritten caching resp. sharing of ConfigReader and SafeConfigParserWithIncludes (v.2, first and second level cache, without fingerprinting etc.);
code review
ConfigReader/ConfigWrapper renamed as suggested from @yarikoptic; + code clarifying (suggested also);
Partially merge remote-tracking from 'sebres:cache-config-read-820': test cases extended, configurator.py adapted for test case.
ENH: keep spitting out logging to the screen in LogCaptureTestCases if HEAVYDEBUG
test case for check the read of config files will be cached;
more precise by test
ConfigWrapper class introduced: sharing of the same ConfigReader object between JailsReader and JailReader (don't read jail config each jail); sharing of the same DefinitionInitConfigReader (ActionReader, FilterReader) between all jails using that; cache of read a config files was optimized; test case extended for all types of config readers;
config cache optimized - prevent to read the same config file inside different resources multiple times; test case: read jail file only once;
test case for check the read of config files will be cached;
caching of read config files, to make start of fail2ban faster, see issue #820
* origin/master:
RF: moving logwatch setup/sample logs under files/logwatch
ENH: print rebans stats even if no "Failures" are logged, and reduce indentation in output
ENH: untabified and reindented entire script for sane formatting (no functional changes)
BF: logwatch -- fixing up regex for 'already banned'
Sample logfiles to test logwatch services script
Adjusting fail2ban logwatch script to match lines from 0.9 as well
* commit '0.9.0a2-814-g98dc084':
tests: define CONFIG_DIR in utils.
forgot to add test case to last commit
adding test case, changelog and thanks entries for apache shellshock filter
adding jail conf for shellshock filter
adding filter to detect Shellshock attack attempts against bash scripts through apache. See http://seclists.org/oss-sec/2014/q3/650
Add apache filter for AH01630 client denied by server configuration
RF: moving logwatch setup/sample logs under files/logwatch
DOC: Changelog and THANKS for previous changes
RF: remove those two additional failregexes for the postfix
ENH: add empty ignoreregex to avoid a warning (Close#805)
Update test cases and also suport smtps per request.
Add support for postfix/submission/smtpd matching.
ENH: print rebans stats even if no "Failures" are logged, and reduce indentation in output
ENH: untabified and reindented entire script for sane formatting (no functional changes)
BF: logwatch -- fixing up regex for 'already banned'
Sample logfiles to test logwatch services script
Adjusting fail2ban logwatch script to match lines from 0.9 as well
logging normalized, set log level for loading (read or use shared) file(s) to INFO;
prevent to read some files twice by read inside "_getIncludes" and by "read" self (occurred by only one file);
sharing of the same DefinitionInitConfigReader (ActionReader, FilterReader) between all jails using that;
cache of read a config files was optimized;
test case extended for all types of config readers;
* commit '0.9.0a2-792-g1864f75': (113 commits)
Credits and notes from #806
fixed encoding
fixed encoding
ENH: Ignore errors while unbaning in symbiosis firewall
ENH: just a bit more descriptive exception ;-)
ENH/BF(TST): making permissions restrictive is not sufficient -- really remove file to test
changelog entry for postfix-sasl fix
added systemd configuration for postfix-sasl.conf
1.5 version of Fail2ban logwatch file
minor typo
Fxi jail.conf to use more syslog macros
ENH: symbiosis-blacklist-allports action
Fix typos.
changelog and thanks for the preceding fix
Added entry for Cloudflare action
ChangeLog Added and entry about Cloudflare action
Changed to Cloudflare JSON API
Fix sieve filter to use correct option
changelog entries for already merged and upcoming merge
Update courier-smtp.conf
...
* origin/master:
Clean up setup.py
ENH: adding pruned with previous merge trailing \s* in nginx filter
nginx-http-auth: match when "referrer" is present
DOC: Update ChangeLog
DOC: Fix jail.conf(5) man page which had duplicate {fail,ignore}regex
Correct grammar
DOC: add thanks for last commit
DOC: nginx-http-auth filter
nginx-http-auth filter: match server_name = ""
ENH: Suppress configuration warnings if non-critical options are not set
Update comments in shorewall.conf for new settings
DOC: Improve error logging when specific backend set and fails
BF: Fix getting jail name from exceptions in beautifier for Python 3+
DOC: sphinx documentation
BF: On jail restart reinstatement of bans, fetch one ticket per IP
Fix 2 more typos that codespell didn't catch
Add documentation link to systemd service file
2014-04-12 11:32:02 -04:00
341 changed files with 23408 additions and 7694 deletions
## Fail2Ban: ban hosts that cause multiple authentication errors
Fail2Ban scans log files like /var/log/pwdfail and bans IP that makes too many
password failures. It updates firewall rules to reject the IP address. These
rules can be defined by the user. Fail2Ban can read multiple log files such as
sshd or Apache web server ones.
Fail2Ban scans log files like `/var/log/auth.log` and bans IP addresses having
too many failed login attempts. It does this by updating system firewall rules
to reject new connections from those IP addresses, for a configurable amount
of time. Fail2Ban comes out-of-the-box ready to read many standard log files,
such as those for sshd and Apache, and is easy to configure to read any log
file you choose, for any error you choose.
Fail2Ban is able to reduce the rate of incorrect authentications attempts
however it cannot eliminate the risk that weak authentication presents.
Though Fail2Ban is able to reduce the rate of incorrect authentications
attempts, it cannot eliminate the risk that weak authentication presents.
Configure services to use only two factor or public/private authentication
mechanisms if you really want to protect services.
<img src="http://www.worldipv6launch.org/wp-content/themes/ipv6/downloads/World_IPv6_launch_logo.svg" height="52pt"/> | Since v0.10 fail2ban supports the matching of the IPv6 addresses.
------|------
This README is a quick introduction to Fail2ban. More documentation, FAQ, HOWTOs
are available in fail2ban(1) manpage and on the website http://www.fail2ban.org
are available in fail2ban(1) manpage, [Wiki](https://github.com/fail2ban/fail2ban/wiki)
# Notes.: command executed once at the start of Fail2Ban.
# Values: CMD
#
actionstart = ipfw show | fgrep -q 'table(<table>)' || ( ipfw show | awk 'BEGIN { b = 1 } { if ($1 <= b) { b = $1 + 1 } else { e = b } } END { if (e) exit e <br> else exit b }'; num=$?; ipfw -q add $num <blocktype> <block> from table\(<table>\) to me <port>; echo $num > "<startstatefile>" )
actionstart = ipfw show | fgrep -c -m 1 -s 'table(<table>)' > /dev/null 2>&1 || ( ipfw show | awk 'BEGIN { b = <lowest_rule_num> } { if ($1 < b) {} else if ($1 == b) { b = $1 + 1 } else { e = b } } END { if (e) exit e <br> else exit b }'; num=$?; ipfw -q add $num <blocktype> <block> from table\(<table>\) to me <port>; echo $num > "<startstatefile>" )
# Option: actionstop
@@ -38,7 +38,7 @@ actioncheck =
# Values: CMD
#
# requires an ipfw rule like "deny ip from table(1) to me"
message = Dear Sir/Madam,\n\nWe have detected abuse from the IP address $IP, which according to a abusix.com is on your network. We would appreciate if you would investigate and take action as appropriate.\n\nLog lines are given below, but please ask if you require any further information.\n\n(If you are not the correct person to contact about this please accept our apologies - your e-mail address was extracted from the whois record by an automated process.)\n\n This mail was generated by Fail2Ban.\nThe recipient address of this report was provided by the Abuse Contact DB by abusix.com. abusix.com does not maintain the content of the database. All information which we pass out, derives from the RIR databases and is processed for ease of use. If you want to change or report non working abuse contacts please contact the appropriate RIR. If you have any further question, contact abusix.com directly via email (info@abusix.com). Information about the Abuse Contact Database can be found here: https://abusix.com/global-reporting/abuse-contact-db\nabusix.com is neither responsible nor liable for the content or accuracy of this message.\n
# Path to the log files which contain relevant lines for the abuser IP
# Because rich rules can only handle single or a range of ports we must split ports and execute the command for each port. Ports can be single and ranges separated by a comma or space for an example: http, https, 22-60, 18 smtp
actionban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --add-rich-rule="rule family='<family>' source address='<ip>' port port='$p' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <rich-blocktype>"; done
actionunban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --remove-rich-rule="rule family='<family>' source address='<ip>' port port='$p' protocol='<protocol>' log prefix='f2b-<name>' level='<level>' limit value='<rate>/m' <rich-blocktype>"; done
[Init]
# log levels are "emerg", "alert", "crit", "error", "warning", "notice", "info" or "debug"
# firewall-cmd --zone=<zone> --add-rich-rule="rule family='ipv4' source address='<ip>' service name='<service>' <rich-blocktype>"
#
# Because rich rules can only handle single or a range of ports we must split ports and execute the command for each port. Ports can be single and ranges separated by a comma or space for an example: http, https, 22-60, 18 smtp
actionban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --add-rich-rule="rule family='<family>' source address='<ip>' port port='$p' protocol='<protocol>' <rich-blocktype>"; done
actionunban = ports="<port>"; for p in $(echo $ports | tr ", " " "); do firewall-cmd --remove-rich-rule="rule family='<family>' source address='<ip>' port port='$p' protocol='<protocol>' <rich-blocktype>"; done
message = Dear Sir/Madam,\n\nWe have detected abuse from the IP address $IP, which according to abusix.com is on your network. We would appreciate if you would investigate and take action as appropriate.\n\nLog lines are given below, but please ask if you require any further information.\n\n(If you are not the correct person to contact about this please accept our apologies - your e-mail address was extracted from the whois record by an automated process.)\n\n This mail was generated by Fail2Ban in a X-ARF format! You can find more information about x-arf at http://www.x-arf.org/specification.html.\n\nThe recipient address of this report was provided by the Abuse Contact DB by abusix.com. abusix.com does not maintain the content of the database. All information which we pass out, derives from the RIR databases and is processed for ease of use. If you want to change or report non working abuse contacts please contact the appropriate RIR. If you have any further question, contact abusix.com directly via email (info@abusix.com). Information about the Abuse Contact Database can be found here: https://abusix.com/global-reporting/abuse-contact-db\nabusix.com is neither responsible nor liable for the content or accuracy of this message.\n
# Option: loglines
@@ -97,7 +100,7 @@ mailargs = -f <sender>
# Option: tlp
# Notes.: Traffic light protocol defining the sharing of this information.
# http://www.trusted-introducer.org/ISTLPv11.pdf
# green is share to those involved in network security but it is not
# green is share to those involved in network security but it is not
# to be released to the public.
tlp = green
@@ -116,7 +119,7 @@ logpath = /dev/null
# Option: sender
# Notes.: This is the sender that is included in the XARF report
sender = fail2ban@`uname -n`
sender = fail2ban@<fq-hostname>
# Option: port
# Notes.: This is the port number that received the login-attack
# This filter is aimed at blocking specific URLs that don't exist. This
# could be a set of URLs places in a Disallow: directive in robots.txt or
# just some web services that don't exist caused bots are searching for
# exploitable content. This filter is designed to have a low false postitive
# exploitable content. This filter is designed to have a low false positive
# rate due.
#
# An alternative to this is the apache-noscript filter which blocks all
@@ -17,30 +17,21 @@
[INCLUDES]
# overwrite with apache-common.local if _apache_error_client is incorrect.
# Load regexes for filtering from botsearch-common.conf
before = apache-common.conf
botsearch-common.conf
[Definition]
failregex = ^%(_apache_error_client)s ((AH001(28|30): )?File does not exist|(AH01264: )?script not found or unable to stat): <webroot><block>(, referer: \S+)?\s*$
^%(_apache_error_client)s script '<webroot><block>' not found or unable to stat(, referer: \S+)?\s*$
failregex = ^%(_apache_error_client)s ((AH0013[456]: )?Invalid (method|URI) in request .*( - possible attempt to establish SSL connection on non-SSL port)?|(AH00565: )?request failed: URI too long \(longer than \d+\)|request failed: erroneous characters after protocol string: .*|AH00566: request failed: invalid characters in URI)(, referer: \S+)?$
failregex = ^%(_apache_error_client)s (?:(?:AH0013[456]: )?Invalid (method|URI) in request\b|(?:AH00565: )?request failed: URI too long \(longer than \d+\)|request failed: erroneous characters after protocol string:|(?:AH00566: )?request failed: invalid characters in URI\b)
ignoreregex =
# DEV Notes:
#
# [sebres] Because this apache-log could contain very long URLs (and/or referrer),
# the parsing of it anchored way may be very vulnerable (at least as regards
# the system resources, see gh-1790). Thus rewritten without end-anchor ($).
#
# fgrep -r 'URI too long' httpd-2.*
# httpd-2.2.25/server/protocol.c: "request failed: URI too long (longer than %d)", r->server->limit_req_line);
# Note: First three failregex matches below are for ASSP V1 with the remaining being designed for V2. Deleting the V1 regex is recommended but I left it in for compatibility reasons.
^\s*(?:[\w\-]+\s+)*(?:\[\S+\]\s+)*<HOST> \[SMTP Error\] 535 5\.7\.8 Error: authentication failed:\s+(?:\S+|Connection lost to authentication server|Invalid authentication mechanism|Invalid base64 data in continued response)?$
ignoreregex =
datepattern = {^LN-BEG}%%b-%%d-%%Exy %%H:%%M:%%S
{^LN-BEG}
# DEV Notes:
# V1 Examples matches:
# Apr-27-13 02:33:09 Blocking 217.194.197.97 - too much AUTH errors (41);
# Dec-29-12 17:10:31 [SSL-out] 200.247.87.82 SSL negotiation with client failed: SSL accept attempt failed with unknown errorerror:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol;
failregex = ^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Registration from '[^']*' failed for '<HOST>(:\d+)?' - (Wrong password|Username/auth name mismatch|No matching peer found|Not a local domain|Device does not match ACL|Peer is not supposed to register|ACL error \(permit/deny\)|Not a local domain)$
^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Call from '[^']*' \(<HOST>:\d+\) to extension '\d+' rejected because extension not found in context 'default'\.$
^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Host <HOST> failed to authenticate as '[^']*'$
^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s No registration for peer '[^']*' \(from <HOST>\)$
^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Host <HOST> failed MD5 authentication for '[^']*' \([^)]+\)$
^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s Failed to authenticate (user|device) [^@]+@<HOST>\S*$
^(%(__prefix_line)s|\[\]\s*)%(log_prefix)s (?:handle_request_subscribe: )?Sending fake auth rejection for (device|user) \d*<sip:[^@]+@<HOST>>;tag=\w+\S*$
failregex = ^Registration from '[^']*' failed for '<HOST>(:\d+)?' - (?:Wrong password|Username/auth name mismatch|No matching peer found|Not a local domain|Device does not match ACL|Peer is not supposed to register|ACL error \(permit/deny\)|Not a local domain)$
^Call from '[^']*' \(<HOST>:\d+\) to extension '[^']*' rejected because extension not found in context
^(?:Host )?<HOST> (?:failed (?:to authenticate\b|MD5 authentication\b)|tried to authenticate with nonexistent user\b)
^No registration for peer '[^']*' \(from <HOST>\)$
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Yaroslav Halchenko