Today I received a question about how to distribute the outgoing connections between several IP addresses attached to an interface. Suppose that you have 3 IPs in the eth0 interface and you want to do round robin between that IPs for outgoing connections. With regular iproute commands you can’t. Doing some tricks with fwmarks, ip rule and ip route neither.
The only way that I’ve found to it is using SNAT and statistics to get a real Round Robin balance:
iptables -t nat -A POSTROUTING -m statistic --mode nth --every 3 -j SNAT --to 192.168.1.201
iptables -t nat -A POSTROUTING -m statistic --mode nth --every 2 -j SNAT --to 192.168.1.202
iptables -t nat -A POSTROUTING -m statistic --mode nth --every 1 -j SNAT --to 192.168.1.203
The IPs described in the example should be local IPs.
Today chatting via IRC I remembered a problem that I had some years ago with virtualization, iptables, nat and bridge. The situation of the guy asking was pretty similar. He has a one virtual machine (Qemu/KVM) connected to the world using a bridge and its default gateway is the virtualization host. He was trying to apply destination NAT to the VM in the host machine but it didn’t work. The rule was simple:
iptables -t nat -A POSTROUTING -s 10.0.3.11 -o eth0 -j MASQUERADE
It is perfect, there is nothing wrong there but he never saw the packet in the POSTROUTING chain. Why? The quick answer is “packets don’t cross nat table twice”. There is a flag in the Linux bridge to enable filtering with Iptables. Packets go to Iptables in the kernel when they are forwarded by the bridge. This includes the NAT table.
In the bridging process, you don’t know the outgoing interface so the previous rule doesn’t work. He needs the interface because he’s using MASQUERADE. In the routing process, the packets go to iptables but they never cross NAT tables because the packet already crossed the table in the bridging process.
How can we fix this? There are two options I think:
- echo 0 > /proc/sys/net/bridge/bridge-nf-call-iptables #To disable Iptables in the bridge.
- Raw table: Some years ago appeared a new tables in Iptables. This table can be used to avoid packets (connection really) to enter the NAT table: iptables -t raw -I PREROUTING -i BRIDGE -s x.x.x.x -j NOTRACK.
If you still don’t understand why this happens, I’ll try to explain one more thing. If you have an scenario with Virtualization and you host is your gateway, the packets follow this steps: [VM]->[bridge]->[virtual interface]->[host]->[physical interface]->[net]. When they cross the host, you have the routing process there.