Tag Archives: linux

Centos7, Rackspace and kswap0

I’ve just started to test an application from one my clients in CentOS 7. A Python app that runs on Python 2.7 that is the default version in the latest version of CentOS/RHEL. As usual, all the cloud providers don’t configure the swap space by default, so I have to configure Chef to create it when the deploy starts. When I started to test the deployment, I discover that the instance crashed compiling some of the Python modules. Debugging I see a problem that is reported a very frequently if you search in Google: “kswapd0 using all the CPU”.

I switch my tests to Digital Ocean, and there it worked perfectly. After a while, I discovered some differences in /etc/sysctl.conf. Rackspace guys setup vm.swappiness=0,  thing that triggers a bug in kswap0 (there is information about this on the net). I’ve disabled that configuration and the variable took the value of 30, the default in the kernel.

And everything works again…

A little history from the our daily DevOps engineer life .


Post to Twitter

Convert Rackspace Cloud Server images to OVA

I wrote an script to convert Rackspace Cloud Server Images to OVA files. This file can be imported to Vmware and Virtualbox (and may be other hypervisors).

You have to get a copy of .tgz files generated from Cloud Servers snapshots and then provide it as first argument of this script.

The script is here.

In the comments you can see the requirements and how to use it.

Feedback is welcome.

Post to Twitter

Load balance between source IPs in Linux

Today I received a question about how to distribute the outgoing connections between several IP addresses attached to an interface. Suppose that you have 3 IPs in the eth0 interface and you want to do round robin between that IPs for outgoing connections. With regular iproute commands you can’t. Doing some tricks with fwmarks, ip rule and ip route neither.

The only way that I’ve found to it is using SNAT and statistics to get a real Round Robin balance:


iptables -t nat -A POSTROUTING  -m statistic --mode nth --every 3 -j SNAT --to

iptables -t nat -A POSTROUTING  -m statistic --mode nth --every 2 -j SNAT --to

iptables -t nat -A POSTROUTING  -m statistic --mode nth --every 1 -j SNAT --to

The IPs described in the example should be local IPs.

Post to Twitter

My two commits to the Linux Kernel

Four years ago I had nothing to do and started to read some pieces of the kernel code. I read about the BLK, a feature that kernel developers have been trying to remove the last years (I’m not sure if they have finished). My small contribution to that task was this two patches that I sent at the end of 2007.  It’s not a big thing but it required some understanding of the kernel internals or I would have broken something 🙂

commit 1

commit 2


BLK or the Big Kernel Lock is a basic locking mechanism introduced in the kernel when SMP support was included. It’s a global lock, lock_kernel() and unlock_kernel() are the functions to use it. This lock is bad because is global, if the networking code wants to protect something calls it. If the filesystem code wants to protect other fs related data calls it too. Two unrelated data structures are protected and two code paths are blocked without any sense. Was an easy implementation in the first years of SMP support, but it had been spreading without control. In the last years, kernel developers have been trying to remove it. New kernel code has independent locks and block the data structures, so unrelated code paths are not blocked.


Post to Twitter

Source routing with Squid

This is a small HOWTO about doing source routing with Squid and Linux. With Squid you can specifiy the outgoing IP address using ACLs. That means that you can select the outgoing IP using the information inside HTTP messages, thing that you can’t do with a firewall. The syntax is simple:

acl somedomain dstdomain somedomain.com tcp_outgoing_address somedomain

Those two lines say: “If the request is asking for somedomain.com, go to the world using IP”.

Now the Linux part. If you have more than one public IP address and you want to make the Squid configuration to work you need some iproute lines.

ip rule add table 10 from ip route add table 10 default via GW

And those two lines says: “If the source IP of the packaet is, go via GW”.

Source routing with Linux is simple. What you do is to create a new table. This table will be used by that packets that match the criteria specified in “ip rule”. The “default table” is the table main, everything goes there if there is no rule. Is the table that you see with “ip route” or “route -n” (please, don’t use the last command anymore).


Post to Twitter

Zimbra archiving with compression, the numbers

Today I calculated the space saved in one of the stores thanks to archiving+compression in one of the Zimbra servers that I’ve installed more than one ayer ago. The archiving volume has 273GB of email that uses 159GB of disk after compression. That’s 42% of saving.

I’m using a script to archive mails in the Open Source Edition that I’ve developed last year, running without any problems for more that 12 months.

It’s in my toolbox: https://github.com/diegows/toolbox

Post to Twitter