You are here: Home / Software / embedded linux notes / DistCC

DistCC

When playing with embedded systems (sheevaplug, raspberry, etc) at one point there comes the point that you need to compile something.

The embedded systems are rather slow. If a faster system is available on the same network, it can be used to do a lot of the heavy lifting.

One way to do this is full cross compiling. Compile and log the complete code on the fast system. Transfer the binary to the embedded system. Done.

This get’s problematic once autoconf wants to detect which libraries are available, etc.

To avoid setting up a complete build environment, only the compile phase should be done on the fast system. Everything else is done on the embedded system.

The key for this is using DistCC.

The follwing notes assume a fast system running archlinux x86_64, the server. And a raspberry Pi, first generation, as embeded system, the client

Server Setup

First add a crosscompiler suite. Either you compile your own (bootstrap your own or use something like crosstools-ng). Or you just download the one archlinux is using anyway. (http://archlinuxarm.org/builder/xtools/x-tools6h.tar.xz). :

mkdir /mnt/scratch/distcc
cd /mnt/scratch/distcc
wget http://archlinuxarm.org/builder/xtools/x-tools6h.tar.xz
tar -Jxf x-tools6h.tar.xz
mv x-tools6h/arm-unknown-linux-gnueabi armv6h

install distcc (pacman -S distcc). Edit /etc/conf.d/distcc. Configure the environment if distccd to keep the crosscompiler gcc path in front of the normal gcc. Also add the hosts that are allowed to access this server:

PATH=/mnt/scratch/distcc/armv6h/bin:$PATH
DISTCC_ARGS="--user nobody --allow 127.0.0.1 --allow 192.168.1.10"

Here the path as stated in the extraction above and access for two hosts. You could also allow the complete network by using 192.168.1.0/24.

Now start distcc and add it to startup.

Client Setup

Edit /etc/makepkg.conf and set:

MAKEFLAGS="-j2"
BUILDENV=(fakeroot distcc color !ccache check !sign)
DISTCC_HOSTS="192.168.1.2"

Here 192.168.1.2 is the server we just set up. In the buildenv we removed the ! from distcc.

Thats it. Now any makepkg run on the client (raspberry) will compile on the server and is much faster. For example squeezelite took 17 seconds to build with distcc and 116 seconds without.