Routing public IP addresses across via virtual tunnel

A recent situation came up which required that I move a group of servers off-site without changing their public IP address. The off-site location is behind my cable modem at my house. To solve the problem I enlisted the help of an open source application called vtun or Virtual Tunnel and IP tables on a standard CentOS 4.4 installation. Below is a outline of the steps I took to solve the problem.

-> Build two CentOS 4.4 Linux routers. Minimal installation plus development tools. In my situation I used a dual 350Mhz PII and tunnel router #2 was an old Pentium Pro 200Mhz. The dual 350Mhz is located at the main data center and the older Linux box is located at my house.

-> Download and configure vtund from www.vtund.info. There are a number of example configurations available on their website. Below is what I have used.


# IP Tunnel Server Configuration
cobra {
passwd XXXXXXX; # Password
type tun; # IP tunnel
proto udp; # UDP protocol
compress lzo:9; # LZO compression level 9
encrypt no; # Encryption
keepalive yes; # Keep connection alive

up {
# Connection is Up
# XXX.XXX.XXX.XXX - local, XXX.XXX.XXX.XXX - remote
ifconfig “%% XXX.XXX.XXX.XXX pointopoint XXX.XXX.XXX.XXX mtu 1450″;
};
down {
# Connection is Down
# Shutdown interface
ifconfig “%% down”;
};
}

# IP Tunnel Client Configuration
cobra {
passwd XXXXXXX; # Password
device tun0; # Device tun0
persist yes; # Persist mode
up {
# Connection is Up

# Assign IP addresses.
ifconfig “%% XXX.XXX.XXX.XXX pointopoint XXX.XXX.XXX.XXX mtu 1450″;
};
}
The configuration above is very simlar to the example configurations provided by vtund. There is more you can do here to make this connection more robust but for the purpose of this blog entry I’ll show it works in more detail. To make the connection between to the two servers you run the following command on the client machine.

vtund cobra

This should create a new network interface called tun0 on both servers. This will allow you to connect across the tunnel. To test this i recommend using the following commands.


On the client
tcpdump -i tun0
On the server
ping


You should see packets come across the interface and reply back. Once you have established the tunnel connection there is a bit more required to route public IP address both in and out of the tunnel interface. For the sake of this example we’ll use 10.10.10.0/24 as the network block your routing. You’ll want to be using a public IP block for this to work correctly not the interal block from the example!

-> Assuming 10.10.10.0/24 is already routed to the tunnel server you’ll now need to route to packets from the tunnel server to the client. To do this you simply type:

ip route add 10.10.10.0/24 dev tun0


This will push packets from the server -> client. Once the packets are on the client end of the tunnel however they do not have a correct route back to the sender. In my situation the tunnel client is located on a cable modem with a default gateway on a different network. My block of 10.10.10.0/24 cannot be routed across someone elses network. The solution is to flag packets as they enter into the tunnel client router. Once the packets get flagged we can apply an alternate route table to them. The alternate route cable is identical to the standard table except the default gateway is tun0 rather then the cable modem interface. Below is a simple script to generate a duplicate route table and add the make the default gateway the tunnel server:


ip route show table main | grep -Ev ^default | while read ROUTE ; do
ip route add table 200 $ROUTE
done
ip route add default via XXX.XXX.XXX.XXX table 200


Now that the alternate table has been created you can verify it works by typing:

ip route list table 200


This should be a simlar table to your main route table except the default gateway should be the IP address of the server-side of the tunnel interface. Now that this table is in place we need to start flagging packets. In order to do this we need to make some special rules in ip tables. Below are the commands to create the flagging rules:

iptables -t mangle -A PREROUTING -i eth2 -j MARK --set-mark=200
iptables -t mangle -A PREROUTING -s 10.10.10.0/24 -j MARK --set-mark=200
iptables -t mangle -A OUTPUT -s 10.10.10.0/24 -j MARK --set-mark=200

Once the iptable rules are in place you can check them by issuing the ‘iptables -nvL -t mangle’ command. In the output you should see packets hitting your new iptable rules. Once you have active hits you’ll need to add a rule that all packets with a flag need to use the alternate table. To create this rule you need the following

ip rule add fwmark 200 table 200

To verify the rule was added you can issue the ‘ip rule list’ command.
Once you have does this your ready to route packets in and out of the tunnel. In my situation I added a third network interface card running NAT for my home computers.