VM From Scratch

I have spent a lot of time installing various Linux distributions, automating all the complex things, trying out things like puppet, writing my own management scripts. Well, in the end of the day, the goal was to run a very simple application. Usually these applications run inside a hypervisor, and we don't really want to spend time with the operating system's management. All I want is a Virtual Machine that has my application and I don't want to know more about it. I think I'm even not interested in sshing into that box. I want to build VMs quickly.

There are several tools to build a VM, and I believe all do their job perfectly. But I think it's time for me to try to build a VM from scratch. The components, that I will use are:

• linux
• crosstool-ng
• buildroot

Getting the kernel

At the time of writing the linux version that I picked, is 3.10.32 - I believe, it has a long term support. So let's dive in, and get the source code

cd /data/matelakat/kernel
wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.10.32.tar.xz
tar -xJf linux-3.10.32.tar.xz

Now that we have the sources, we can create a cross compiler.

crosstool-ng

This tool will help me to build a cross compiler. At the initial phase, my target architecture is 686 32bit. I picked this one, as I would like to test the VM with nested virtualization (ESXi inside VirtualBox), and you can't run 64 bit guests there. So let's get started:

first, download the tools, and check out a specific revision. This makes it easier for anyone to reproduce.

mkdir /data/matelakat/xtool
cd /data/matelakat/xtool
hg clone http://crosstool-ng.org/hg/crosstool-ng
cd crosstool-ng/
hg checkout 9321d9d7af9b

Now comes the crosstool-ng specific part. First, we need to actually install crosstool-ng:

./bootstrap
./configure --prefix=$(cd .. && pwd)/ctng
make
make install
export PATH=$PATH:$(cd .. && pwd)/ctng/bin

The next step is to generate a configuration for the crosstool. I will start by getting a sample configuration file, and overwriting it with my config:

cd /data/matelakat/xtool/
mkdir configuration
cd configuration/
ct-ng x86_64-unknown-linux-gnu
cp /data/matelakat/vmfromscratch/crosstool-ng/.config .config
ct-ng menuconfig

If you are happy with the config, it's time to build the cross compiler. My config will put the downloaded sources and the built tools to ../src and ../x-tools, so I need to create those directories in advance. Also, as I have a laptop with 4 cores, I will use 6 jobs simultaneously:

mkdir ../src ../x-tools
time ct-ng build.6

I can definitely tell: my CPU cores are busy - some hot air flows out of my good old Lenovo R500. After around an hour, the crosscompiler was ready:

du -shc ../x-tools/

364M../x-tools/
364Mtotal

Build the Kernel

Now that we have a crosscompiler, we can get back to the kernel directory, configure the tool paths, and compile a new kernel:

cd /data/matelakat/kernel/linux-3.10.32/
cp /data/matelakat/vmfromscratch/linux/.config ./
make oldconfig
make menuconfig
make -j 4

Unfortunately, I didn't measure the time it took, but it was quite fast

Build the Root Filesystem

Now that I have a nice kernel I also need a root filesystem. Buildroot will help me to build that:

mkdir /data/matelakat/br
cd /data/matelakat/br
wget -qO - http://buildroot.uclibc.org/downloads/buildroot-2014.02.tar.gz | tar -xzf -
cd buildroot-2014.02/
cp /data/matelakat/vmfromscratch/buildroot/config .config
make menuconfig
make

Putting it all together

Now that I have everything, I only need to create an iso cd with my new system

mkdir isoroot
cp ../br/buildroot-2014.02/output/images/rootfs.cpio.gz isoroot/initrd.img
cp ../kernel/linux-3.10.32/arch/i386/boot/bzImage isoroot/
cp ../br/buildroot-2014.02/output/images/isolinux.bin isoroot/isolinux/
cp isolinux.cfg isoroot/isolinux/
mkisofs -J -o minsys.iso -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table isoroot/
qemu-system-i386 -M pc -cdrom minsys.iso -net nic,model=e1000 -net user

Next steps

My next step will be to automate all these things.