L7-filter Kernel Version HOWTO

Important links on this page:

What You Need To Get

The 2.4 or 2.6 Linux kernel source (2.6 strongly preferred) from kernel.org
The iptables source from netfilter.org
Our "l7-filter kernel version" package (netfilter-layer7-vX.Y.tar.gz)
Our "Protocol definitions" package (l7-protocols-YYYY-MM-DD.tar.gz)

Kernel Patch

Short version for experts: Apply our kernel patch. Enable the new match option in Netfilter.
Check our kernel compatibility list to see if the Linux version you want to use has been tested.
Use the appropriate kernel patch from the "Layer 7 patches" package to patch^[1] the kernel (read the README in the package to determine which patch to use). Set up your kernel as you would otherwise. Now enable the following options (these are correct for Linux 2.6.21.1, but they tend to move around a lot, so you may have to go hunting if you have a different kernel version):

"Prompt for development and/or incomplete code/drivers" (under "Code maturity level options")
"Network packet filtering framework" (Networking → Networking support → Networking Options)
"Netfilter Xtables support" (on the same screen)
"Netfilter connection tracking support" (... → Network packet filtering framework → Core Netfilter Configuration), select "Layer 3 Independent Connection tracking"
"Connection tracking flow accounting" (on the same screen)
And finally, "Layer 7 match support"
Optional but highly recommended: Lots of other Netfilter options, notably "FTP support" and other matches. If you don't know what you're doing, go ahead and enable all of them.

Warning: Some users have reported kernel crashes when they using SMP with l7-filter. (Some have also reported that their SMP systems run fine.) If you have a multi-CPU machine, test carefully before putting it into production with l7-filter.
Compile and install the kernel as usual. (Our code may generate warnings about "initialization from incompatible pointer type", ignore them.) Reboot.

¹How to patch a source tree

Suppose you have a patch called happy.patch. To apply it, go into the root directory of the source tree you want to patch and run "patch -p1 < happy.patch"

Iptables Setup

First read the README in the package "Layer 7 patches". Depending on your version of iptables, the instructions are different.

iptables 1.4.0 and older

Use the appropriate iptables patch to to patch^[1] iptables. Compile iptables, pointing it at your patched kernel source:

Run "chmod +x extensions/.layer7-test" (information about file permissions can't be contained in the patch)
Then "make KERNEL_DIR=/path/to/patched/kernel_source" (you must have configured your kernel source before this step)
And install (as root): "make install KERNEL_DIR=/path/to/patched/kernel_source"

iptables 1.4.1

Don't use this version. There's no reason to and it's difficult to compile.

iptables 1.4.1.1 and newer

Copy libxt_layer7.c and libxt_layer7.man (from the subdirectory of the "Layer 7 patches" package that the README points you to) to the extensions/ directory of your iptables source. Then:

"./configure --with-ksource=/path/to/patched/kernel_source" (use the full path)
"make"
(as root) "make install"

Protocol Definitions (Pattern Files)

These files tell iptables and the kernel how protocol names correspond to regular expressions, e.g. "ftp" means "^220[\x09-\x0d -~]*ftp".
Uncompress the "Protocol Definitions" package and make the resulting directory /etc/l7-protocols.^[2]
You should now be ready to actually do stuff.

²Notes for non-conformists

You can also install the patterns in a custom location. If you do this, you need to specify --l7dir before --l7proto when you use l7-filter:

iptables [...] -m layer7 --l7dir /home/bob/patterns --l7proto http [...]

Actually doing stuff

There are three things you may be interested in doing: (1) blocking certain protocols (2) controlling bandwidth use (3) accounting. We cover each of these cases below.
First, a reminder: Just because you're using l7-filter, you don't need to do all of your packet classification using it. It's likely that what you want to accomplish can be at least partially done with less demanding classifiers, such as port matching. For instance, you can probably assume that traffic on TCP port 80 that isn't matched by any P2P patterns is HTTP; you don't need to actually use the HTTP pattern.
l7-filter uses the standard iptables extension syntax. (If you are not familiar with this, it's time to read the documentation at netfilter.org or at least "man iptables".)
iptables [specify table & chain] -m layer7 --l7proto [protocol name] -j [action]
(Or, if you're just interested in accounting, omit "-j [action]".)
For a list of valid protocol names, see the protocols page. You can also add your own protocols.
The only trick is that, in order to do its classification, l7-filter must be able to see all of the relevant traffic. It only sees packets if they go through an l7-filter rule. One way of ensuring this is to use the POSTROUTING chain of the mangle table:
iptables -t mangle -A POSTROUTING -m layer7 --l7proto [etc.]
See this packet flow diagram for details. In some cases, l7-filter can sucessfully match even if it can only see one side of the connection, but in general, this won't work.
If you are using a version of l7-filter earlier than 2.7, you must manually load the ip_conntrack module kernel for l7-filter to work. Newer versions do this automatically.

1. Blocking

Don't. Here's why:

l7-filter matching isn't foolproof: there may be both false positives (one protocol can look like another) and false negatives (applications can do obscure things that we didn't count on). Patterns that are known to regularly generate false positives are marked "overmatching" on the protocols page, but others may also do so occasionally.
Almost every type of Internet traffic has legitimate uses. For instance, P2P protocols, while widely used to violate copyright, are also an efficient way to distribute open source software and legally free music.
Programs can respond to being blocked by port-hopping, switching between TCP and UDP, opening a new connection for every trivial operation, using encryption, or employing other evasion tactics. Trying to block such protocols has consequences on two levels:
1. In the case of port/protocol-hopping, you make it harder for yourself to identify protocols that already act this way.
2. You encourage programmers to include these "features" in new programs, making it harder for everyone in the future. For example: In early 2006, Bittorrent started moving towards end-to-end encryption because many networks were either blocking it or severely restricting its bandwidth.
l7-filter patterns are not generally designed with blocking in mind. We consider a protocol to be well identified if the identification is useful for controlling its bandwidth. This means, for instance, that for P2P applications, we do not focus on catching connections that are not downloads.
Blocking with l7-filter provides no security, since any reasonably determined person can easily circumvent it.

Instead of dropping packets you don't like, we recommend using Linux QoS to restrict their bandwidth usage. If you insist on using l7-filter to drop packets, make sure you have investigated other options first, such as the features of your HTTP proxy (useful for worms).

2. Bandwidth Restriction

To control the bandwidth that a protocol uses, you can use Netfilter to "mark" the packets and QoS to filter on that mark. To mark:
iptables -t mangle -A POSTROUTING -m layer7 --l7proto imap -j MARK --set-mark 3
The number "3" is arbitrary. It can be any integer. Then use tc to filter on that mark (tc is "traffic control", the userspace tool for Linux QoS, part of the iproute2 package):
tc filter add dev eth0 protocol ip parent 1:0 prio 1 handle 3 fw flowid 1:3
Did you understand that last command? You can try reading The Linux Advanced Routing and Traffic Control HOWTO for enlightenment. You should do this so that you have some idea what you're doing, but unfortunately, tc is incredibly obtuse and you're likely to wish you just had a canned script. Well, we can help:

These may need to be modified if your setup is significantly different than mine, but it should provide a much better starting point than most other things you are likely to find.
Be prudent when choosing the amount of bandwidth you allow each protocol. Restricting a protocol to an unusably low bandwidth can have similar consequences to blocking it.

3. Accouting

If you just want to keep track of what's in use on your network, simply use the above command without any -j option. For example:

iptables -t mangle -A POSTROUTING -m layer7 --l7proto imap

You can then get statistics by using iptables -L. (See "man iptables" for details.)

More Information

Dealing with FTP, IRC, etc.

Some protocols open child connections to transfer data. FTP is the most familiar example. If you have loaded the ip_conntrack_ftp or nf_conntrack_ftp kernel module, l7-filter will classify FTP and all its child connections as FTP. The same goes for IRC/IRC-DCC, etc.
If you wish to classify the children differently, use the standard iptables "helper" match. You can use "-m --helper ftp" to match ftp child connections. Of course, once you've done this, it's silly to involve l7-filter, at least for the children.

The "unset" and "unknown" matches

l7-filter marks unmatched connections that it is still trying to match as "unset". The first few packets of all TCP connections as well as those of some UDP connections will match this. Similarly, l7-filter marks connections that it has given up trying to match as "unknown". These are matched just like normal protocols:

iptables -A FORWARD -m layer7 --l7proto unset
iptables -A FORWARD -m layer7 --l7proto unknown

The "unset" match is only supported by l7-filter 2.9 and up.

Upgrading the protocol definitions

The protocol definitions are simple text files with a format described in the Pattern-HOWTO. They can be updated as a package or individually.
If you update the protocol definitions, you need to clear the relevant iptables rules and re-enter them. This is because the pattern files are only read by iptables, not directly by the kernel.

Other things to know

By default, l7-filter looks at the first 10 packets or 2kB, whichever is smaller. These limits are somewhat conservative. It is well known that some HTTP connections (those that involve large cookies), for instance, need more packets to be matched.
- You can alter the number of packets at any time through /proc/net/layer7_numpackets. (i.e. "echo 16 > /proc/net/layer7_numpackets".)
- In l7-filter versions 2.0 and forward, you can alter the number of bytes at module load time: "modprobe xt_layer7 maxdatalen=N" (ipt_layer7 in old versions), where N is in bytes. This should be used cautiously, since performance may decrease drastically with larger data sizes. To prevent you from accidentally bringing down your network, there is an artificial limit of 65536 imposed. If you're sure you know what you're doing, you can remove this limit by editing ipt_layer7.c or xt_layer7.c in the kernel source.
It's possible (although rare) for a connection to be matchable by more than one pattern. The patterns are tested in the order you specified with iptables. After a match is made, l7-filter does not continue testing that connection, so changing the order of your rules may change what happens.
Sometimes important messages go only to the system log, not the terminal you are working at. Such messages include notifications that regular expressions failed to compile and various things that tc generates. A useful command is "tail -f /var/log/messages".

Please see the FAQ for more information.

Tech黑手 - 工作雜記

2012年2月6日星期一