Filtering chains¶

Filtering rules are organized in filter chains. These chains can be thought of as having a tree structure with packet filtering rules as entries in individual chains (branches).

Packets start their filter evaluation in the root chain and can then continue their evaluation in other chains, return from those chains back into the root chain or be dropped or accepted by a filtering rule in one of the traversed chains.

Libvirt's network filtering system automatically creates individual root chains for every virtual machine's network interface on which the user chooses to activate traffic filtering. The user may write filtering rules that are either directly instantiated in the root chain or may create protocol-specific filtering chains for efficient evaluation of protocol-specific rules. The following chains exist:

• root

• mac (since 0.9.8)

• stp (spanning tree protocol) (since 0.9.8)

• vlan (802.1Q) (since 0.9.8)

• arp, rarp

• ipv4

• ipv6

Since 0.9.8 multiple chains evaluating the mac, stp, vlan, arp, rarp, ipv4, or ipv6 protocol can be created using the protocol name only as a prefix in the chain's name. This for examples allows chains with names arp-xyz or arp-test to be specified and have ARP protocol packets evaluated in those chains.

The following filter shows an example of filtering ARP traffic in the arp chain.

<filter name='no-arp-spoofing' chain='arp' priority='-500'>
  <uuid>f88f1932-debf-4aa1-9fbe-f10d3aa4bc95</uuid>
  <rule action='drop' direction='out' priority='300'>
    <mac match='no' srcmacaddr='$MAC'/>
  </rule>
  <rule action='drop' direction='out' priority='350'>
    <arp match='no' arpsrcmacaddr='$MAC'/>
  </rule>
  <rule action='drop' direction='out' priority='400'>
    <arp match='no' arpsrcipaddr='$IP'/>
  </rule>
  <rule action='drop' direction='in' priority='450'>
    <arp opcode='Reply'/>
    <arp match='no' arpdstmacaddr='$MAC'/>
  </rule>
  <rule action='drop' direction='in' priority='500'>
    <arp match='no' arpdstipaddr='$IP'/>
  </rule>
  <rule action='accept' direction='inout' priority='600'>
    <arp opcode='Request'/>
  </rule>
  <rule action='accept' direction='inout' priority='650'>
    <arp opcode='Reply'/>
  </rule>
  <rule action='drop' direction='inout' priority='1000'/>
</filter>

The consequence of putting ARP-specific rules in the arp chain, rather than for example in the root chain, is that packets for any other protocol than ARP do not need to be evaluated by ARP protocol-specific rules. This improves the efficiency of the traffic filtering. However, one must then pay attention to only put filtering rules for the given protocol into the chain since any other rules will not be evaluated, i.e., an IPv4 rule will not be evaluated in the ARP chain since no IPv4 protocol packets will traverse the ARP chain.

Filtering chain priorities¶

All chains are connected to the root chain. The order in which those chains are accessed is influenced by the priority of the chain. The following table shows the chains that can be assigned a priority and their default priorities.

A chain with a lower priority value is accessed before one with a higher value.

Since 0.9.8 the above listed chains can be assigned custom priorities by writing a value in the range [-1000, 1000] into the priority (XML) attribute in the filter node. The above example filter shows the default priority of -500 for arp chains.

Usage of variables in filters¶

Two variables names have so far been reserved for usage by the network traffic filtering subsystem: MAC and IP.

MAC is the MAC address of the network interface. A filtering rule that references this variable will automatically be instantiated with the MAC address of the interface. This works without the user having to explicitly provide the MAC parameter. Even though it is possible to specify the MAC parameter similar to the IP parameter above, it is discouraged since libvirt knows what MAC address an interface will be using.

The parameter IP represents the IP address that the operating system inside the virtual machine is expected to use on the given interface. The IP parameter is special in so far as the libvirt daemon will try to determine the IP address (and thus the IP parameter's value) that is being used on an interface if the parameter is not explicitly provided but referenced. For current limitations on IP address detection, consult the section on Limitations on how to use this feature and what to expect when using it.

The above-shown network filer no-arp-spoofing is an example of a network filter XML referencing the MAC and IP variables.

Note that referenced variables are always prefixed with the $ (dollar) sign. The format of the value of a variable must be of the type expected by the filter attribute in the XML. In the above example, the IP parameter must hold a dotted IP address in decimal numbers format. Failure to provide the correct value type will result in the filter not being instantiatable and will prevent a virtual machine from starting or the interface from attaching when hotplugging is used. The types that are expected for each XML attribute are shown below.

Since 0.9.8 variables can contain lists of elements, e.g., the variable IP can contain multiple IP addresses that are valid on a particular interface. The notation for providing multiple elements for the IP variable is:

...
<devices>
  <interface type='bridge'>
    <mac address='00:16:3e:5d:c7:9e'/>
    <filterref filter='clean-traffic'>
      <parameter name='IP' value='10.0.0.1'/>
      <parameter name='IP' value='10.0.0.2'/>
      <parameter name='IP' value='10.0.0.3'/>
    </filterref>
  </interface>
</devices>
...

This then allows filters to enable multiple IP addresses per interface. Therefore, with the list of IP address shown above, the following rule will create 3 individual filtering rules, one for each IP address.

...
<rule action='accept' direction='in' priority='500'>
  <tcp srpipaddr='$IP'/>
</rule>
...

Since 0.9.10 it is possible to access individual elements of a variable holding a list of elements. A filtering rule like the following accesses the 2nd element of the variable DSTPORTS.

...
<rule action='accept' direction='in' priority='500'>
  <udp dstportstart='$DSTPORTS[1]'/>
</rule>
...

Since 0.9.10 it is possible to create filtering rules that instantiate all combinations of rules from different lists using the notation of $VARIABLE[@<iterator ID>]. The following rule allows a virtual machine to receive traffic on a set of ports, which are specified in DSTPORTS, from the set of source IP address specified in SRCIPADDRESSES. The rule generates all combinations of elements of the variable DSTPORT with those of SRCIPADDRESSES by using two independent iterators to access their elements.

...
<rule action='accept' direction='in' priority='500'>
  <ip srcipaddr='$SRCIPADDRESSES[@1]' dstportstart='$DSTPORTS[@2]'/>
</rule>
...

In an example we assign concrete values to SRCIPADDRESSES and DSTPORTS

SRCIPADDRESSES = [ 10.0.0.1, 11.1.2.3 ]
DSTPORTS = [ 80, 8080 ]

Accessing the variables using $SRCIPADDRESSES[@1] and $DSTPORTS[@2] would then result in all combinations of addresses and ports being created:

10.0.0.1, 80
10.0.0.1, 8080
11.1.2.3, 80
11.1.2.3, 8080

Accessing the same variables using a single iterator, for example by using the notation $SRCIPADDRESSES[@1] and $DSTPORTS[@1], would result in parallel access to both lists and result in the following combinations:

10.0.0.1, 80
11.1.2.3, 8080

Further, the notation of $VARIABLE is short-hand for $VARIABLE[@0]. The former notation always assumes the iterator with Id '0'.

Automatic IP address detection¶

The detection of IP addresses used on a virtual machine's interface is automatically activated if the variable IP is referenced but no value has been assigned to it. Since 0.9.13 the variable CTRL_IP_LEARNING can be used to specify the IP address learning method to use. Valid values are any, dhcp, or none.

The value any means that libvirt may use any packet to determine the address in use by a virtual machine, which is the default behavior if the variable CTRL_IP_LEARNING is not set. This method will only detect a single IP address on an interface. Once a VM's IP address has been detected, its IP network traffic will be locked to that address, if for example IP address spoofing is prevented by one of its filters. In that case the user of the VM will not be able to change the IP address on the interface inside the VM, which would be considered IP address spoofing. When a VM is migrated to another host or resumed after a suspend operation, the first packet sent by the VM will again determine the IP address it can use on a particular interface.

A value of dhcp specifies that libvirt should only honor DHCP server-assigned addresses with valid leases. This method supports the detection and usage of multiple IP address per interface. When a VM is resumed after a suspend operation, still valid IP address leases are applied to its filters. Otherwise the VM is expected to again use DHCP to obtain new IP addresses. The migration of a VM to another physical host requires that the VM again runs the DHCP protocol.

Use of CTRL_IP_LEARNING=dhcp (DHCP snooping) provides additional anti-spoofing security, especially when combined with a filter allowing only trusted DHCP servers to assign addresses. To enable this, set the variable DHCPSERVER to the IP address of a valid DHCP server and provide filters that use this variable to filter incoming DHCP responses.

When DHCP snooping is enabled and the DHCP lease expires, the VM will no longer be able to use the IP address until it acquires a new, valid lease from a DHCP server. If the VM is migrated, it must get a new valid DHCP lease to use an IP address (e.g., by bringing the VM interface down and up again).

Note that automatic DHCP detection listens to the DHCP traffic the VM exchanges with the DHCP server of the infrastructure. To avoid denial-of-service attacks on libvirt, the evaluation of those packets is rate-limited, meaning that a VM sending an excessive number of DHCP packets per second on an interface will not have all of those packets evaluated and thus filters may not get adapted. Normal DHCP client behavior is assumed to send a low number of DHCP packets per second. Further, it is important to setup appropriate filters on all VMs in the infrastructure to avoid them being able to send DHCP packets. Therefore VMs must either be prevented from sending UDP and TCP traffic from port 67 to port 68 or the DHCPSERVER variable should be used on all VMs to restrict DHCP server messages to only be allowed to originate from trusted DHCP servers. At the same time anti-spoofing prevention must be enabled on all VMs in the subnet.

If CTRL_IP_LEARNING is set to none, libvirt does not do IP address learning and referencing IP without assigning it an explicit value is an error.

The following XML provides an example for the activation of IP address learning using the DHCP snooping method:

<interface type='bridge'>
  <source bridge='virbr0'/>
  <filterref filter='clean-traffic'>
    <parameter name='CTRL_IP_LEARNING' value='dhcp'/>
  </filterref>
</interface>

Reserved Variables¶

The following table lists reserved variables in use by libvirt.

References to other filters¶

Any filter may hold references to other filters. Individual filters may be referenced multiple times in a filter tree but references between filters must not introduce loops (directed acyclic graph).

The following shows the XML of the clean-traffic network filter referencing several other filters.

<filter name='clean-traffic'>
  <uuid>6ef53069-ba34-94a0-d33d-17751b9b8cb1</uuid>
  <filterref filter='no-mac-spoofing'/>
  <filterref filter='no-ip-spoofing'/>
  <filterref filter='allow-incoming-ipv4'/>
  <filterref filter='no-arp-spoofing'/>
  <filterref filter='no-other-l2-traffic'/>
  <filterref filter='qemu-announce-self'/>
</filter>

To reference another filter, the XML node filterref needs to be provided inside a filter node. This node must have the attribute filter whose value contains the name of the filter to be referenced.

New network filters can be defined at any time and may contain references to network filters that are not known to libvirt, yet. However, once a virtual machine is started or a network interface referencing a filter is to be hotplugged, all network filters in the filter tree must be available. Otherwise the virtual machine will not start or the network interface cannot be attached.

Filter rules¶

The following XML shows a simple example of a network traffic filter implementing a rule to drop traffic if the IP address (provided through the value of the variable IP) in an outgoing IP packet is not the expected one, thus preventing IP address spoofing by the VM.

<filter name='no-ip-spoofing' chain='ipv4'>
  <uuid>fce8ae33-e69e-83bf-262e-30786c1f8072</uuid>
  <rule action='drop' direction='out' priority='500'>
    <ip match='no' srcipaddr='$IP'/>
  </rule>
</filter>

A traffic filtering rule starts with the rule node. This node may contain up to three attributes

• action -- mandatory; must either be drop (matching the rule silently discards the packet with no further analysis), reject (since 0.9.0; matching the rule generates an ICMP reject message with no further analysis), accept (matching the rule accepts the packet with no further analysis), return (since 0.9.7; matching the rule passes this filter, but returns control to the calling filter for further analysis), or continue (since 0.9.7; matching the rule goes on to the next rule for further analysis).

• direction -- mandatory; must either be in, out or inout if the rule is for incoming, outgoing or incoming-and-outgoing traffic

• priority -- optional; the priority of the rule controls the order in which the rule will be instantiated relative to other rules. Rules with lower value will be instantiated before rules with higher values. Valid values are in the range of 0 to 1000. Since 0.9.8 this has been extended to cover the range of -1000 to 1000. If this attribute is not provided, priority 500 will automatically be assigned.

  Note that filtering rules in the root chain are sorted with filters connected to the root chain following their priorities. This allows to interleave filtering rules with access to filter chains. (See also section on Filtering chain priorities .)

• statematch -- optional; possible values are '0' or 'false' to turn the underlying connection state matching off; default is 'true'

  Also read the section on Advanced Filter Configuration Topics.

The above example indicates that the traffic of type ip will be associated with the chain 'ipv4' and the rule will have priority 500. If for example another filter is referenced whose traffic of type ip is also associated with the chain 'ipv4' then that filter's rules will be ordered relative to the priority 500 of the shown rule.

A rule may contain a single rule for filtering of traffic. The above example shows that traffic of type ip is to be filtered.

[...]
<rule action='drop' direction='in'>
  <protocol match='no' attribute1='value1' attribute2='value2'/>
  <protocol attribute3='value3'/>
</rule>
[...]

[...]
<mac match='no' srcmacaddr='$MAC'/>
[...]

Advanced Filter Configuration Topics¶

The following sections discuss advanced filter configuration topics.

[...]
<rule direction='in' action='accept' statematch='false'>
  <tcp dstportstart='12345'/>
</rule>
[...]

[...]
<rule action='drop' direction='in' priority='400'>
  <tcp connlimit-above='1'/>
</rule>
<rule action='accept' direction='in' priority='500'>
  <tcp dstportstart='22'/>
</rule>
<rule action='drop' direction='out' priority='400'>
  <icmp connlimit-above='1'/>
</rule>
<rule action='accept' direction='out' priority='500'>
  <icmp/>
</rule>
<rule action='accept' direction='out' priority='500'>
  <udp dstportstart='53'/>
</rule>
<rule action='drop' direction='inout' priority='1000'>
  <all/>
</rule>
[...]

echo 3 > /proc/sys/net/netfilter/nf_conntrack_icmp_timeout

Example custom filter¶

As an example we want to now build a filter that fulfills the following list of requirements:

• prevents a VM's interface from MAC, IP and ARP spoofing

• opens only TCP ports 22 and 80 of a VM's interface

• allows the VM to send ping traffic from an interface but not let the VM be pinged on the interface

• allows the VM to do DNS lookups (UDP towards port 53)

The requirement to prevent spoofing is fulfilled by the existing clean-traffic network filter, thus we will reference this filter from our custom filter.

To enable traffic for TCP ports 22 and 80 we will add 2 rules to enable this type of traffic. To allow the VM to send ping traffic we will add a rule for ICMP traffic. For simplicity reasons we allow general ICMP traffic to be initiated from the VM, not just ICMP echo request and response messages. To then disallow all other traffic to reach or be initiated by the VM we will then need to add a rule that drops all other traffic. Assuming our VM is called test and the interface we want to associate our filter with is called eth0, we name our filter test-eth0. The result of these considerations is the following network filter XML:

<filter name='test-eth0'>
  <!-- reference the clean traffic filter to prevent
       MAC, IP and ARP spoofing. By not providing
       and IP address parameter, libvirt will detect the
       IP address the VM is using. -->
  <filterref filter='clean-traffic'/>

  <!-- enable TCP ports 22 (ssh) and 80 (http) to be reachable -->
  <rule action='accept' direction='in'>
    <tcp dstportstart='22'/>
  </rule>

  <rule action='accept' direction='in'>
    <tcp dstportstart='80'/>
  </rule>

  <!-- enable general ICMP traffic to be initiated by the VM;
       this includes ping traffic -->
  <rule action='accept' direction='out'>
    <icmp/>
  </rule>

  <!-- enable outgoing DNS lookups using UDP -->
  <rule action='accept' direction='out'>
    <udp dstportstart='53'/>
  </rule>

  <!-- drop all other traffic -->
  <rule action='drop' direction='inout'>
    <all/>
  </rule>

</filter>

Note that none of the rules in the above XML contain the IP address of the VM as either source or destination address, yet the filtering of the traffic works correctly. The reason is that the evaluation of the rules internally happens on a per-interface basis and the rules are evaluated based on the knowledge about which (tap) interface has sent or will receive the packet rather than what their source or destination IP address may be.

An XML fragment for a possible network interface description inside the domain XML of the test VM could then look like this:

[...]
<interface type='bridge'>
  <source bridge='mybridge'/>
  <filterref filter='test-eth0'/>
</interface>
[...]

To more strictly control the ICMP traffic and enforce that only ICMP echo requests can be sent from the VM and only ICMP echo responses be received by the VM, the above ICMP rule can be replaced with the following two rules:

<!-- enable outgoing ICMP echo requests-->
<rule action='accept' direction='out'>
  <icmp type='8'/>
</rule>

<!-- enable incoming ICMP echo replies-->
<rule action='accept' direction='in'>
  <icmp type='0'/>
</rule>

Second example custom filter¶

In this example we now want to build a similar filter as in the example above, but extend the list of requirements with an ftp server located inside the VM. Further, we will be using features that have been available since 0.8.5. The requirements for this filter are:

• prevents a VM's interface from MAC, IP and ARP spoofing

• opens only TCP ports 22 and 80 of a VM's interface

• allows the VM to send ping traffic from an interface but not let the VM be pinged on the interface

• allows the VM to do DNS lookups (UDP towards port 53)

• enable an ftp server (in active mode) to be run inside the VM

The additional requirement of allowing an ftp server to be run inside the VM maps into the requirement of allowing port 21 to be reachable for ftp control traffic as well as enabling the VM to establish an outgoing tcp connection originating from the VM's TCP port 20 back to the ftp client (ftp active mode). There are several ways of how this filter can be written and we present 2 solutions.

The 1st solution makes use of the state attribute of the TCP protocol that gives us a hook into the connection tracking framework of the Linux host. For the VM-initiated ftp data connection (ftp active mode) we use the RELATED state that allows us to detect that the VM-initiated ftp data connection is a consequence of ( or 'has a relationship with' ) an existing ftp control connection, thus we want to allow it to let packets pass the firewall. The RELATED state, however, is only valid for the very first packet of the outgoing TCP connection for the ftp data path. Afterwards, the state to compare against is ESTABLISHED, which then applies equally to the incoming and outgoing direction. All this is related to the ftp data traffic originating from TCP port 20 of the VM. This then leads to the following solution (since 0.8.5 (QEMU, KVM)) :

<filter name='test-eth0'>
  <!-- reference the clean traffic filter to prevent
       MAC, IP and ARP spoofing. By not providing
       and IP address parameter, libvirt will detect the
       IP address the VM is using. -->
  <filterref filter='clean-traffic'/>

  <!-- enable TCP port 21 (ftp-control) to be reachable -->
  <rule action='accept' direction='in'>
    <tcp dstportstart='21'/>
  </rule>

  <!-- enable TCP port 20 for VM-initiated ftp data connection
       related to an existing ftp control connection -->
  <rule action='accept' direction='out'>
    <tcp srcportstart='20' state='RELATED,ESTABLISHED'/>
  </rule>

  <!-- accept all packets from client on the ftp data connection -->
  <rule action='accept' direction='in'>
    <tcp dstportstart='20' state='ESTABLISHED'/>
  </rule>

  <!-- enable TCP ports 22 (ssh) and 80 (http) to be reachable -->
  <rule action='accept' direction='in'>
    <tcp dstportstart='22'/>
  </rule>

  <rule action='accept' direction='in'>
    <tcp dstportstart='80'/>
  </rule>

  <!-- enable general ICMP traffic to be initiated by the VM;
       this includes ping traffic -->
  <rule action='accept' direction='out'>
    <icmp/>
  </rule>

  <!-- enable outgoing DNS lookups using UDP -->
  <rule action='accept' direction='out'>
    <udp dstportstart='53'/>
  </rule>

  <!-- drop all other traffic -->
  <rule action='drop' direction='inout'>
    <all/>
  </rule>

</filter>

Before trying out a filter using the RELATED state, you have to make sure that the appropriate connection tracking module has been loaded into the host's kernel. Depending on the version of the kernel, you must run either one of the following two commands before the ftp connection with the VM is established.

modprobe nf_conntrack_ftp   # where available  or

modprobe ip_conntrack_ftp   # if above is not available

If other protocols than ftp are to be used in conjunction with the RELATED state, their corresponding module must be loaded. Modules exist at least for the protocols ftp, tftp, irc, sip, sctp, and amanda.

The 2nd solution makes uses the state flags of connections more than the previous solution did. In this solution we take advantage of the fact that the NEW state of a connection is valid when the very first packet of a traffic flow is seen. Subsequently, if the very first packet of a flow is accepted, the flow becomes a connection and enters the ESTABLISHED state. This allows us to write a general rule for allowing packets of ESTABLISHED connections to reach the VM or be sent by the VM. We write specific rules for the very first packets identified by the NEW state and for which ports they are acceptable. All packets for ports that are not explicitly accepted will be dropped and therefore the connection will not go into the ESTABLISHED state and any subsequent packets be dropped.

<filter name='test-eth0'>
  <!-- reference the clean traffic filter to prevent
       MAC, IP and ARP spoofing. By not providing
       and IP address parameter, libvirt will detect the
       IP address the VM is using. -->
  <filterref filter='clean-traffic'/>

  <!-- let the packets of all previously accepted connections reach the VM -->
  <rule action='accept' direction='in'>
    <all state='ESTABLISHED'/>
  </rule>

  <!-- let the packets of all previously accepted and related connections be sent from the VM -->
  <rule action='accept' direction='out'>
    <all state='ESTABLISHED,RELATED'/>
  </rule>

  <!-- enable traffic towards port 21 (ftp), 22 (ssh) and 80 (http) -->
  <rule action='accept' direction='in'>
    <tcp dstportstart='21' dstportend='22' state='NEW'/>
  </rule>

  <rule action='accept' direction='in'>
    <tcp dstportstart='80' state='NEW'/>
  </rule>

  <!-- enable general ICMP traffic to be initiated by the VM;
       this includes ping traffic -->
  <rule action='accept' direction='out'>
    <icmp state='NEW'/>
  </rule>

  <!-- enable outgoing DNS lookups using UDP -->
  <rule action='accept' direction='out'>
    <udp dstportstart='53' state='NEW'/>
  </rule>

  <!-- drop all other traffic -->
  <rule action='drop' direction='inout'>
    <all/>
  </rule>

</filter>

VM Migration¶

VM migration is only supported if the whole filter tree that is referenced by a virtual machine's top level filter is also available on the target host. The network filter clean-traffic for example should be available on all libvirt installations of version 0.8.1 or later and thus enable migration of VMs that for example reference this filter. All other custom filters must be migrated using higher layer software. It is outside the scope of libvirt to ensure that referenced filters on the source system are equivalent to those on the target system and vice versa.

Migration must occur between libvirt installations of version 0.8.1 or later in order not to lose the network traffic filters associated with an interface.

VLAN filtering on Linux¶

VLAN (802.1Q) packets, if sent by a virtual machine, cannot be filtered with rules for protocol IDs arp, rarp, ipv4 and ipv6 but only with protocol IDs mac and vlan. Therefore, the example filter clean-traffic will not work as expected.