ModSecurity is available under two licenses. Users can choose to use the software under the terms of the GNU General Public License (http://www.gnu.org/licenses/gpl.html), as an Open Source / Free Software product. Alternatively, a variety of commercial licenses is available: end-user licenses for individual or site-wide deployment, OEM licenses for closed-source distribution with applications, web servers, or security appliances. For more information on commercial licensing please contact Thinking Stone.

This module would not be possible without the fine people who have created the Apache Web server, and the fine people who have spent many hours building the Apache modules I used to learn Apache module programming from.

ModSecurity is developed by Ivan Ristic and Thinking Stone. Comments and feature requests are welcome. Please send your emails to <ivanr@webkreator.com>.

If you want to access the latest version of the module you need to get it from the CVS repository. The list of changes made since the last stable release is normally available on the web site (and in the file CHANGES). The CVS repository for ModSecurity is hosted by SourceForge (http://www.sf.net). You can access it directly or view if through web using this address: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/mod-security/ 

To download the source code to your computer you need to execute the following two commands:

$ cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/mod-security login
$ cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/mod-security \
> co mod_security

The first line will log you in as an anonymous user, and the second will download all files available in the repository.

If you don't like CVS but you still want the latest version you can download the latest nightly tarball from the following address:

http://www.modsecurity.org/download/snapshot/mod_security-snapshot.tar.gz

New features are added to mod_security one by one, with regression tests being run after each change. This should ensure that the version available from CVS is always usable.

To download the stable release go to http://www.modsecurity.org/download/. Binary distributions are sometimes available. If they are, they are listed on the download page. If not download the source code distribution.

When installing from source you have two choices: to install the module into the web server itself, or to compile mod_security.c into a dynamic shared object (DSO).

# /apachehome/bin/apxs -cia mod_security.c

# /apachehome/bin/apachectl stop
# /apachehome/bin/apachectl start

$ cd <apache1-source>
$ cp <modsecurity-source>/apache1/mod_security.c ./src/modules/extra
$ ./configure \
> --activate-module=src/modules/extra/mod_security \
> -–enable-module=security

$ cd <apache2-source>
$ cp <modsecurity-source>/apache2/mod_security.c ./modules/proxy
$ ./configure \
> -enable-security \
> --with-module=proxy:mod_security.c

$ cd <modsecurity-source>/apache2
$ mkdir -r <apache2-source>/modules/security
$ cp mod_security.c Makefile.in config.m4 <apache2-source>/modules/security
$ cd <apache2-source>
$ ./buildconf

$ ./configure --enable-security

In some circumstances, you will want to install the module as a binary. At the moment I only make Windows binaries available for download. When installing from binary you are likely to have two DSO libraries in the distribution, one for each major Apache branch. Choose the file appropriate for the version you are using. Then proceed as described below:

LoadModule security_module    libexec/mod_security.so

AddModule mod_security.c

LoadModule security_module    modules/mod_security.so

The filtering engine is disabled by default. To start monitoring requests add the following to your configuration file:

SecFilterEngine On

Supported parameter values for this parameter are:

Request body payload (or POST payload) scanning is disabled by default. To use it, you need to turn it on:

SecFilterScanPOST On

mod_security supports two encoding types for the request body:

Other encodings are not used by most web applications. To make sure that only requests with these two encoding types are accepted by the web server, add the following line to your configuration file:

SecFilterSelective HTTP_Content-Type \
"!(^$|^application/x-www-form-urlencoded$|^multipart/form-data;)"

It is possible to turn POST payload scanning off on per-request basis. If ModSecurity sees that an environment variable MODSEC_NOPOSTBUFFERING is defined it will not perform POST payload buffering. For example, to turn POST payload buffering off for file uploads use the following:

SetEnvIfNoCase Content-Type \
"^multipart/form-data;" "MODSEC_NOPOSTBUFFERING=Do not buffer file uploads"

The value assigned to the MODSEC_NOPOSTBUFFERING variable will be written to the debug log, so you can put in there something that will tell you why was buffering turned off.

It is also possible to enable or disable ModSecurity on the per-request basis. This is done via the MODSEC_ENABLE environment variable, in combination with the SetEnvIf and SetEnvIfNoCase directives. If MODSEC_ENABLE is not set the configuration specified with SecFilterEngine will be used. If MODSEC_ENABLE is set the value of SecFilterEngine will be ignored. To values for MODSEC_ENABLE are the same as for the SecFilterEngine directive: On, Off, or DynamicOnly.

The HTTP protocol supports a method of request transfer where the size of the payload is not known in advance. The body of the request is delivered in chunks. Hence the name chunked transfer encoding. ModSecurity does not support chunked requests at this time; when a request is made with chunked encoding it will ignore the body of the request. As far as I am aware no browser uses chunked encoding to send requests. Although Apache does support this encoding for some operations most modules (e.g. the PHP module with Apache 1.3.x) don't.

Left unattended this may present an opportunity for an attacker to sneak malicious payload. Add the following line to your configuration to prevent attackers to exploit this weakness:

SecFilterSelective HTTP_Transfer-Encoding "!^$"

This will not affect your ability to send responses using the chunked transfer encoding.

Whenever a rule is matched against a request, one or more actions are performed. Individual filters can each have their own actions but it is easier to define a default set of actions for all filters. (You can always have per-rule actions if you want.) You define default actions with the configuration directive SecFilterDefaultAction. For example, the following will configure the engine to log each rule match, and reject the request with status code 404:

SecFilterDefaultAction "deny,log,status:404"

The SecFilterDefaultAction directive accepts only one parameter, a comma-separated list of actions separated. The actions you specify here will be performed on every filter match, except for rules that have their own action lists.

As of 1.8.6 implicit request validation (if configured) will be performed only at the beginning of request processing. Implicit validation consists of the checks of the request line, and the headers.

Filters defined in parent folders are normally inherited by nested Apache configuration contexts. This is behaviour is acceptable (and required) in most cases, but not all the time. Sometimes you need to relax checks in some part of the site. By using the SecFilterInheritance directive:

SecFilterInheritance Off

you can instruct ModSecurity to disregard parent filters so that you can start with rules from the scratch. This directive affects rules only. The configuration is always inherited from the parent context but you can override it as you are pleased using the appropriate configuration directives.

When you choose not to inherit the rules from the parent context you can either write new rules for the new context, or simply use the Include directive to include the same rules into many different contexts.

Sometimes only a small change to the rule set is required in the child context. In such cases you may choose to use the selective inheritance option. You can do this with the help of the following two directives:

The SecFilterImport and SecFilterRemove directives both accept a list rule IDs. The target rules must have IDs associated with them (this is done using the id action). The directives will be executed in the order they appear in the configuration file. Therefore, it is possible to remove a rule with SecFilterRemove and then add it again with SecFilterImport. Below you can find two examples that arrive at the same rule configuration, but take different routes to get there.

Example 1: the rules from the parent context are not inherited, but a single rule is imported.

SecFilter XXX id:1001
SecFilter YYY id:1002
SecFilter ZZZ id:1003

<Location /subcontext/>
    SecFilterInheritance Off
    SecFilterImport 1003
</Location>

Example 2: the rules from the parent context are inherited, with two rules removed.

SecFilter XXX id:1001
SecFilter YYY id:1002
SecFilter ZZZ id:1003

<Location /subcontext/>
    SecFilterRemove 1001 1002
</Location>

When you are deploying ModSecurity in multi-user environments, and your users are allowed to use the rules in their .htaccess files, you may not wish to allow them to not inherit the rules from the parent context. There are two ways to achieve this.

First, you can mark certain rules mandatory using the mandatory action. Such rules will always be inherited in the child context.

The other way is to use the SecFilterInheritanceMandatory directive to simply make all rules in the context mandatory for all child contexts.

SecFilterInheritanceMandatory On

You may be wondering what happens in a situation like this one:

SecFilter XXX id:1001
SecFilterInheritanceMandatory On
<Location /subcontext/>
    SecFilterInheritance Off
    SecFilter YYY id:1002
    SecFilter ZZZ id:1003,mandatory
</Location>

<Location /subcontext/another/>
    SecFilterRemove 1001 1002 1003
    SecFilter QQQ id:1004
</Location>

Since rule inheritance is mandatory in the main context, the /subcontext/ context will inherit rule 1001 in spite of an attempt not to (using SecFilterInheritance Off). This subcontext will first run rule 1001, followed by the rules 1002 and 1003.

The mandatory rule 1001 from the main context will also propagate to context /subcontext/another/, in spite of an attempt to remove it. This is also true for the rule 1003, which was made mandatory for inheritance using the mandatory action. The SecFilterRemove 1001 1002 1003 directive will, however, succeed in removing rule 1002 because inheritance was not mandatory in /subcontext/. This context will therefore first run rule 1001 and 1003, followed by the rule 1004.

Special characters need to be encoded before they can be transmitted in the URL. Any character can be replaced using the three character combination %XY, where XY represents an hexadecimal character code (see http://www.rfc-editor.org/rfc/rfc1738.txt for more details). Hexadecimal numbers only allow letters A to F, but attackers sometimes use other letters in order to trick the decoding algorithm. ModSecurity checks all supplied encodings in order to verify they are valid.

You can turn URL encoding validation on with the following line:

SecFilterCheckURLEncoding On

Like many other features Unicode encoding validation is disabled by default. You should turn it on if your application or the underlying operating system accept/understand Unicode.

SecFilterCheckUnicodeEncoding On

This feature will assume UTF-8 encoding and check for three types of errors:

You can force requests to consist only of bytes from a certain byte range. This can be useful to avoid stack overflow attacks (since they usually contain "random" binary content). To only allow bytes from 32 to 126 (inclusive), use the following directive:

SecFilterForceByteRange 32 126

Default range values are 0 and 255, i.e. all byte values are allowed.

Prior to 1.9 ModSecurity supported the SecServerResponseToken directive. When used, this directive exposed the presence of the module (with the version) in the web server signature. This directive no longer works in 1.9. If used, it will emit a warning message to the error log.

The most simplest form of filtering is, well, simple. It looks like this:

SecFilter KEYWORD

For each simple filter like this, ModSecurity will look for the keyword in the request. The search is pretty broad; it will be applied to the first line of the request (the one that looks like this GET /index.php?parameter=value HTTP/1.0). In case of POST requests, the body of the request will be searched too (provided the request body buffering is enabled, of course).

Filters are not applied to raw request data, but on a normalised copy instead. We do this because attackers can (and do) apply various evasion techniques to avoid detection. For example, you might want to setup a filter that detects shell command execution:

SecFilter /bin/sh

But the attacker may use a string /bin/./sh (which has the same meaning) in order to avoid the filter.

ModSecurity automatically applies the following transformations:

You can choose whether to enable or disable the following checks:

Null byte attacks try to confuse C/C++ based software and trick it into thinking that a string ends before it actually does. This type of an attack is typically rejected with a proper SecFilterByteRange filter. However, if you do not do this a null byte can interfere with ModSecurity processing. To fight this, ModSecurity looks for null bytes during the decoding phase and converts them into spaces. So, where before this filter:

SecFilter hidden

would not detect the word hidden in this request:

GET /one/two/three?p=visible%00hidden HTTP/1.0

it now works as expected.

The simplest method of filtering I discussed earlier is actually slightly more complex. Its full syntax is as follows:

SecFilter KEYWORD [ACTIONS]

First of all, the keyword is not a simple text. It is an regular expression. A regular expression is a mini programming language designed to pattern matching in text. To make most out of this (now) powerful tool you need to understand regular expressions well. I recommend that you start with one of the following resources:

The second parameter is an action list definition, which specifies what will happen if the filter matches. Actions are explained later in this manual.

If exclamation mark is the first character of a regular expression, the filter will treat that regular expression as inverted. For example, the following:

SecFilter !php

will reject all requests that do not contain the word php.

While SecFilter allows you to start quickly, you will soon discover that the search it performs is too broad, and doesn't work very well. Another directive:

SecFilterSelective LOCATION KEYWORD [ACTIONS]

allows you to choose exactly where you want the search to be performed. The KEYWORD and the ACTIONS bits are the same as in SecFilter. The LOCATION bit requires further explanation.

The LOCATION parameter consist of a series of location identifiers separated with a pipe.

Examine the following example:

SecFilterSelective "REMOTE_ADDR|REMOTE_HOST" KEYWORD

It will apply the regular expression only the IP address of the client and the host name. The list of possible location identifiers includes all CGI variables, and some more. Here is the full list:

There are some special locations:

And even more special:

A limited number of output-specific variables are also available for Apache 2 (only when output buffering is enabled):

The ARG_variable location names support inverted usage when used together with the ARG location. For example:

SecFilterSelective "ARGS|!ARG_param" KEYWORD

will search all arguments except the one named param.

ModSecurity provides full support for Cookies. By default cookies will be treated as if they were in version 0 format (Netscape-style cookies). However, version 1 cookies (as defined in RFC 2965) are also supported. To enable version 1 cookie support use the SecFilterCookieFormat directive:

# enable version 1 (RFC 2965) cookies
SecFilterCookieFormat 1

By default, ModSecurity will not try to normalise cookie names and values. However, since some applications and platforms (e.g. PHP) do encode cookie content you can choose to apply normalisation techniques to cookies. This is done using the SecFilterNormalizeCookies directive.

SecFilterNormalizeCookies On

ModSecurity supports output filtering in the version for Apache 2. It is disabled by default so you need to enable it first:

SecFilterScanOutput On

After that, simply add selective filters using a special variable OUTPUT:

SecFilterSelective OUTPUT "credit card numbers"

Those who have perhaps followed my columns at http://www.webkreator.com/php/ know that I am somewhat obsessed with the inability of PHP to prevent fatal errors. I have gone to great lengths to prevent fatal errors from spilling to end users (see http://www.webkreator.com/php/configuration/handling-fatal-and-parse-errors.html) but now, finally, I don't have to worry any more about that. The following will catch PHP output error in the response body, replace the response with an error, and execute a custom PHP script (so that the application administrator can be notified):

SecFilterSelective OUTPUT "Fatal error:" deny,status:500
ErrorDocument 500 /php-fatal-error.html

You should note that although you can mix output filters with input filters, they are not executed at the same time. Input filters are executed before a request is processed by Apache, while the output filters are executed after Apache completes request processing.

Output filtering is only useful for plain text and HTML output. Applying regular expressions to binary content (for example images) will only slow down the server. By default ModSecurity will scan output in responses that have no content type, or whose content type is text/plan or text/html. You can change this using the SecFilterOutputMimeTypes directive:

SecFilterOutputMimeTypes "(null) text/html text/plain"

Configured as in example above ModSecurity will apply output filters to plain text files, HTML files, and files where the mime type is not specified "(null)".

While output monitoring is a useful feature in some circumstances you should be aware that it isn't foolproof. If an attacker is in a full control of request processing she can evade output monitoring in two ways:

As of 1.9 another output variable is supported - OUTPUT_STATUS. This variable contains the status code of the response.

There are three places where you can put actions. One is the SecFilterDefaultAction directive, where you define actions you want executed for the rules that follow the directive:

SecFilterDefaultAction "deny,log,status:500"

This example defines an action list that consists of three actions. Commas are used to separate actions in a list. The first two actions consist of a single word. But the third action requires a parameter. Use double colon to separate the parameter from the action name. Action parameters must not contain whitespace unless you surround them in single quotes (escape a single quote in the parameter with a backslash):

SecFilterDefaultAction "deny,log,status:'Hello World!'"

You can also specify per-filter actions. Both filtering directives (SecFilter and SecFilterSelective) accept a set of actions as an optional parameter. Per-rule actions are merged with the actions specified in the most recent SecFilterSignatureAction directive (the default value is log,deny,status:403). The following rules apply to the merging process:

SecFilterDefaultAction log,deny,status:500

# The rule below will respond with actions
# specified in the context it is executed in.
# You should note that the context a rule is
# executed in is not necessarily the context
# that rule was created in. Through inheritance
# one rule can be executed in many different
# contexts.
SecFilter 000

# Warning rules
SecFilterSignatureAction log,pass
SecFilter 111 id:1
SecFilter 222 id:2

# Error rules
SecFilterSignatureAction log,deny,status:403
SecFilter 333 id:3
SecFilter 444 id:4
# Rule below, too, will reject with status 403
SecFilter 555

Sometimes, when you want to include third-party rules in your configuration, you may want to appear what actions will be allowed to appear in them. You can do this with the help of the SecFilterActionsRestricted directive:

SecFilterSignatureAction log,deny,status:403
SecFilterActionsRestricted On
Include conf/third-party-rules.conf

The only actions allowed in the per-rule configuration when the restricted mode is enabled are the meta-data rules id, msg, rev, and severity. Other rules will be silently ignored.

Wherever possible, ModSecurity will add information to the request headers, thus allowing your scripts to find and use them. Obviously, you will have to configure ModSecurity not to reject requests in order for your scripts to be executed at all. At a first glance it may be strange that I'm using the request headers for this purpose instead of, for example, environment variables. Although environment variables would be more elegant, input headers are always visible to scripts executed using an ErrorDocument directive (see below) while environment variables are not.

This is the list of headers added:

ModSecurty will export a request body through the mod_security-body note. You can use this for logging:

LogFormat "%h %l %u %t \"%r\" %>s %{mod_security-body}n 

If your configuration returns a HTTP status code 500, and you configure Apache to execute a custom script whenever this code occurs (for example: ErrorDocument 500 /error500.php) you will be able to use your favourite scripting engine to respond to errors. The information on the error will be in the environment variables REDIRECT_* and HTTP_MOD_SECURITY_* (as described here: http://httpd.apache.org/docs-2.0/custom-error.html).

In some cases, after detecting a particularly dangerous attack or a series of attacks you will want to prevent further attacks coming from the same source. You can do this by modifying the firewall to reject all traffic coming from a particular IP address (I have written a helper script that works with iptables, download it from here: http://www.apachesecurity.net).

This method can be very dangerous since it can result in a denial of service (DOS) attack. For example, an attacker can use a proxy to launch attacks. Rejecting all requests from a proxy server can be very dangerous since all legitimate users will be affected too.

Since most proxies send information describing the original client (some information on this is available here http://www.webkreator.com/cms/view.php/1685.html, under the "Stop hijacking" header), we can try to be smart and find the real IP address. While this can work, consider the following scenario:

Therefore this method can be useful only if you do not allow access to the application through proxies, or allow access only through proxies that are well known and, more importantly, trusted.

If you still want to ban requests based on IP address (in spite of all our warnings), you will need to write a small script that will executed on a filter match. The script should extract the IP address of the attacker from environment variables, and then make a call to iptables or ipchains to ban the IP address. We will include a sample script doing this with a future version of mod_security.

ModSecurity is capable of intercepting files uploaded through POST requests and multipart/form-data encoding or (as of 1.9) through PUT requests.

SecUploadDir /tmp

SecUploadApproveScript /full/path/to/the/script.sh

SecUploadKeepFiles On

SecUploadKeepFiles RelevantOnly

# mkdir /tmp/webfiles
# chown httpd:clamav /tmp/webfiles
# chmod 2750 /tmp/webfiles

#!/usr/bin/perl
#
# modsec-clamscan.pl
# mod_security, http://www.modsecurity.org
# Copyright (c) 2002-2004 Ivan Ristic <ivanr@webkreator.com>
#
# $Id: modsecurity-manual.xml,v 1.7 2005/11/01 13:59:55 ivanr Exp $
#
# This script is an interface between mod_security and its
# ability to intercept files being uploaded through the
# web server, and ClamAV

# by default use the command-line version of ClamAV,
# which is slower but more likely to work out of the
# box
$CLAMSCAN = "/usr/bin/clamscan";

# using ClamAV in daemon mode is faster since the
# anti-virus engine is already running, but you also
# need to configure file permissions to allow ClamAV,
# usually running as a user other than the one Apache
# is running as, to access the files
# $CLAMSCAN = "/usr/bin/clamdscan";

if (@ARGV != 1) {
    print "Usage: modsec-clamscan.pl <filename>\n";
    exit;
}

my ($FILE) = @ARGV;

$cmd = "$CLAMSCAN --stdout --disable-summary $FILE";
$input = `$cmd`;
$input =~ m/^(.+)/;
$error_message = $1;

$output = "0 Unable to parse clamscan output [$1]";

if ($error_message =~ m/: Empty file\.$/) {
    $output = "1 empty file";
}
elsif ($error_message =~ m/: (.+) ERROR$/) {
    $output = "0 clamscan: $1";
}
elsif ($error_message =~ m/: (.+) FOUND$/) {
    $output = "0 clamscan: $1";
}
elsif ($error_message =~ m/: OK$/) {
    $output = "1 clamscan: OK";
}

print "$output\n";

SecUploadInMemoryLimit 125000

One technique that often helps slow down and confuse attackers is the web server identity change. Web servers typically send their identity with every HTTP response in the Server header. Apache is particularly helpful here, not only sending its name and full version by default, but it also allows server modules to append their versions too.

To change the identity of the Apache web server you would have to go into the source code, find where the name "Apache" is hard-coded, change it, and recompile the server. The same effect can be achieved using the SecServerSignature directive:

SecServerSignature "Microsoft-IIS/5.0"

It should be noted that although this works quite well, skilled attackers (and tools) may use other techniques to "fingerprint" the web server. For example, default files, error message, ordering of the outgoing headers, the way the server responds to certain requests and similar - can all give away the true identity. I will look into further enhancing the support for identity masking in the future releases of mod_security.

If you change Apache signature but you are annoyed by the strange message in the error log (some modules are still visible - this only affects the error log, from the outside it still works as expected):

[Fri Jun 11 04:02:28 2004] [notice] Microsoft-IIS/5.0 mod_ssl/2.8.12 OpenSSL/0.9.6b \
configured -- resuming normal operations

Then you should re-arrange the modules loading order to allow mod_security to run last, exactly as explained for chrooting.

When the SecServerSignature directive is used to change the public server signature, ModSecurity will start writing the real signature to the error log, to allow you to identify the web server and the modules used.

[Fri Jun 11 04:02:28 2004] [notice] mod_security/1.9dev1 configured - Apache/2.0.52 \
(Unix) PHP/4.3.10 proxy_html/2.4

Use the SecFilterDebugLog directive to choose a file where debug output will be written. If the parameter does not start with a forward slash, Apache home path will be prepended to it.

SecFilterDebugLog logs/modsec_debug_log

You can control how detailed the debug log is with SecFilterDebugLevel:

SecFilterDebugLevel 4

Possible log values are:

Standard Apache logging will not help much if you need to trace back steps of a particular user or an attacker. The problem is that only a very small subset of each request is written to a log file. This problem can be remedied with the audit logging feature of ModSecurity. These two directives:

SecAuditEngine On
SecAuditLog logs/audit_log

will let mod_security know that you want a full audit log stored into the log file audit log. Here is an example of how a request is logged:

========================================
Request: 192.168.0.2 - - [[18/May/2003:11:20:43 +0100]] "GET /cgi-bin/printenv?p1=666 \
HTTP/1.0" 406 822
Handler: cgi-script
----------------------------------------
GET /cgi-bin/printenv?p1=666 HTTP/1.0
Host: wkx.dyndns.org:8080
User-Agent: mod_security regression test utility
Connection: Close
mod_security-message: Access denied with code 406. Pattern match "666" at \
ARGS_SELECTIVE
mod_security-action: 406

HTTP/1.0 406 Not Acceptable
========================================

You can see that on the first line you get what you normally get from Apache. The second line contains the name of the handler that was supposed to handle the request. Full request (with additional mod_security headers) is given after the separator, and the response headers (in this case there is only one line) is given after one empty line.

When the POST filtering is on, the POST payload will always be included in the audit log. Actual response will never be included (at least not in this version).

At this time, the audit logging part of the module will log Apache 1.x error messages, on the line below the Handler: line. The line will always begin with Error:. This functionality will be added to the Apache 2.x version of the module if possible.

SecAuditLogRelevantStatus ^5

AddType application/x-httpd-php .php

AddHandler application/x-httpd-php .php

# Yes, we want to use the new format
SecAuditLogType Concurrent

# Directory where the files will be stored
# MUST NOT BE THE SAME AS THE APACHE LOGS FOLDER
SecAuditLogStorageDir /var/www/audit_log/

# The index of all files created
SecAuditLog /var/www/audit_log/index

# Choose what to log – everything (default is ABCFHZ)
SecAuditLogParts ABCDEFGHZ

192.168.2.101 192.168.2.11 - - [15/Jul/2005:11:56:52 +0100] \
"POST /form.php HTTP/1.1" 403 3 "http://192.168.2.101:8080/form.php" \
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.8) Gecko/20050511 \
Firefox/1.0.4" G3yTd38AAAEAAAM7BLwAAAAA "-" \
/20050715/20050715-1156/20050715-115652-G3yTd38AAAEAAAM7BLwAAAAA 0 1031 \
md5:dc910f6d647d47b32ae6e47326f0ca42

--67458b6b-A--
[15/Jul/2005:11:56:52 +0100] G3yTd38AAAEAAAM7BLwAAAAA \
192.168.2.11 4236 192.168.2.101 8080
--67458b6b-B--
POST /form.php HTTP/1.1
Host: 192.168.2.101:8080
User-Agent: Mozilla/5.0
Accept: */*
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 300
Connection: keep-alive
Referer: http://192.168.2.101:8080/form.php
Content-Type: application/x-www-form-urlencoded
Content-Length: 5

--67458b6b-C--
f=111
--67458b6b-E--
403 (Response body)
--67458b6b-F--
HTTP/1.1 403 Forbidden
Last-Modified: Fri, 08 Jul 2005 14:25:30 GMT
ETag: "decb4-3-34b96a80"
Accept-Ranges: bytes
Content-Length: 19
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Content-Type: text/html

--67458b6b-H--
Message: Pattern match "111" at POST_PAYLOAD \
[id "1"] [rev "2"] [msg "3"] [severity "4"]
Apache-Handler: application/x-httpd-php
Stopwatch: 1126536042708000 11024 (7276* 7375 9842)

--67458b6b-Z--

Since 1.9 ModSecurity supports a new directive, SecGuardianLog, that is designed to send all access data to another program using the piped logging feature. Since Apache is typically deployed in a multi-process fashion, making information sharing difficult, the idea is to deploy a single external process to observe all requests in a stateful manner, providing additional protection.

Development of a state of the art external protection tool will be a focus of subsequent ModSecurity releases. However, a fully functional tool is already available as part of the Apache httpd tools project (http://www.apachesecurity.net/tools/). The tool is called httpd-guardian and can be used to defend against Denial of Service attacks. It uses the blacklist tool (from the same project) to interact with an iptables-based firewall, dynamically blacklisting the offending IP addresses. Assuming httpd-guardian is already configured (look into the source code for the detailed instructions) you only need to add one line to your Apache configuration to deploy it:

SecGuardianLog |/path/to/httpd-guardian

By default httpd-guardian will defend against clients that send more 120 requests in a minute, or more than 360 requests in five minutes.

Since 1.8 it is possible to use Apache custom logging to log only those requests where mod_security was involved. This is because ModSecurity now defines an environment variable mod_security-relevant whenever it performs an action. To use a custom log file, add the following (or similar) to your configuration:

CustomLog logs/modsec_custom_log \
"%h %l %u %t \"%r\" %>s %b %{mod_security-message}i" \
env=mod_security-relevant

Web application firewalls have a difficult job trying to make sense of data that passes by, without any knowledge of the application and its business logic. The protection they provide comes from having an independent layer of security on the outside. Because data validation is done twice, security can be increased without having to touch the application. In some cases, however, the fact that everything is done twice brings problems. Problems can arise in the areas where the communication protocols are not well specified, or where either the device or the application do things that are not in the specification.

The worst offender is the cookie specification. (Actually all four of them: http://wp.netscape.com/newsref/std/cookie_spec.html, http://www.ietf.org/rfc/rfc2109.txt, http://www.ietf.org/rfc/rfc2964.txt, http://www.ietf.org/rfc/rfc2965.txt.) For many of the cases, possible in real life, there is no mention in the specification - leaving the programmers to do what they think is appropriate. For the largest part this is not a problem when the cookies are well formed, as most of them are. The problem is also not evident because most applications parse cookies they themselves send. It becomes a problem when you think from a point of view of a web application firewall, and a determined adversary trying to get past it. In the 1.8.x branch and until 1.8.6, ModSecurity (changes were made to 1.8.7) used a v1 cookie parser. However, the differences between v0 and v1 formats could be exploited to make a v1 parser see one cookie where a v0 parser would see more. Consider the following:

Cookie: innocent="; nasty=payload; third="

A v0 parser does not understand  double quotes. It typically only looks for semi-colons and splits the header accordingly. Such a parser sees cookies innocent, nasty, and third. A v1 parser, on the other hand, sees only one cookie - innocent.

How is the impedance mismatch affecting the web application firewall users and developers? It certainly makes our lives more difficult but that’s all right - it’s a part of the game. Developers will have to work to incorporate better and smarter parsing routines. For example, there are two cookie parsers in ModSecurity 1.8.7 and the user can choose which one to use. (A v0 format parser is now used by default.) But such improvements, since they cannot be automated, only make using the firewall more difficult - one more thing for the users to think about and configure.

On the other hand, the users, if they don’t want to think about cookie parsers, can always fall back to use those parts of HTTP that are much better defined. Headers, for example. Instead of using COOKIE_innocent to target an individual cookie they can just use HTTP_Cookie to target the whole cookie header. Other variables, such as ARGS, will look at all variables at once no matter how hard adversaries try to mask them.

A small HTTP testing utility was developed as part of the ModSecurity effort. It provides a simple and easy way to send crafted HTTP requests to a server, and to determine whether the attack was successfully detected or not.

Calling the utility without parameters will result in its usage printed:

$ ./run-test.pl
Usage: ./run-test.pl host[:port] testfile1, testfile2, ...

First parameter is the host name of the server, with port being optional. All other parameters are filenames of files containing crafted HTTP requests.

To make your life a little bit easier, the utility will generate certain request headers automatically:

You can include them in the request if you need to. The utility will not add them if they are already there.

Here is how an HTTP request looks like:

# 01 Simple keyword filter
#
# mod_security is configured not to allow
# the "/cgi-bin/keyword" pattern
#
GET /cgi-bin/keyword HTTP/1.0

This request consists only of the first line, with no additional headers. You can create as complicated requests as you wish. Here is one example of a POST method usage:

# 10 Keyword in POST
#
POST /cgi-bin/printenv HTTP/1.0
Content-Type: application/x-www-form-urlencoded
Content-Length: 5

p=333

Lines that are at the beginning of the file and begin with # will be treated as comments. The first line is special, and it should contain the name of the test.

The utility expects status 200 as a result and will treat such responses as successes. If you want some other response you need to tell it by writing the expected response code on the first line (anywhere on the line). Like this:

# 14 Redirect action (requires 302)
GET /cgi-bin/test.cgi?p=xxx HTTP/1.0

The brackets and the "requires" keyword are not required but are recommended for better readability.

As an example of ModSecurity capabilities we will demonstrate how you can use it to detect and prevent the most common security problems. We won't go into detail here about problems themselves but a very good description is available in the Open Web Application Security Project's guide, available at http://www.owasp.org.

SecFilter "\.\./"

SecFilter "<script"
SecFilter "<.+>"

<Location /cms/article-update.php>
    SecFilterInheritance Off
    # other filters here ...
    SecFilterSelective "ARGS|!ARG_body" "<.+>"
</Location>

SecFilter "delete[[:space:]]+from"
SecFilter "insert[[:space:]]+into"
SecFilter "select.+from"

SecFilterSelective ARGS "bin/"

SecFilterByteRange 32 126

SecFilterSelective THE_REQUEST "!^[\x0a\x0d\x20-\x7f]+$"

The protection provided by ModSecurity comes at a cost, but the cost is generally very low. Your web server becomes a little bit slower and uses more memory.

Please read the following notes:

By default mod_security will try to run at the last possible moment in Apache request pre-processing, but just before the request is actually run (for example, processed by mod_php). I have chosen this approach because the most important function of mod_security is to protect the application. On the other hand by doing this we are leaving certain parts of Apache unprotected although there are things we could do about it. For those who wish to experiment, as of 1.9dev3 mod_security can be compiled to run at the earliest possible moment. Just compile it with -DENABLE_EARLY_HOOK. Bear in mind that this is an experimental feature. Some of the differences you will discover are:

Subsequent releases of ModSecurity are likely to allow rule processing to be split into two phases. One to run as early as possible, and another, to run as late as possible.

Regular expressions can be pretty powerful. Here is how you can check whether a parameter is an integer between 0 and 99999:

SecFilterSelective ARG_parameter "!^[0-9]{1,5}$"

Forbid file upload for the application as a whole, but allow it in a subfolder:

# Reject requests with header "Content-Type" set
# to "multipart/form-data"
SecFilterSelective HTTP_CONTENT_TYPE multipart/form-data

# Only for the script that performs upload
<Location /upload.php>
    # Do not inherit filters from the parent folder
    SecFilterInheritance Off
</Location>

Earlier versions of FormMail could be abused to send email to any recipient (I've been told that there is a new version that can be secured properly).

# Only for the FormMail script
<Location /cgi-bin/FormMail>
    # Reject request where the value of parameter "recipient"
    # does not end with "@webkreator.com"
    SecFilterSelective ARG_recipient "![a-zA-Z0-9]+@webkreator\.com$">
</Location>