Debootstrap as a single user

fakeroot -s fakechroot.save fakechroot debootstrap --variant=fakechroot --components=main,universe precise precise
fakeroot -i fakechroot.save -s fakechroot.save fakechroot chroot precise apt-get update
fakeroot -i fakechroot.save -s fakechroot.save fakechroot chroot precise apt-get -qy install git

Ubuntu installer - Unmount partitions that are in use?

I am working on the unattended-iso project, and seeing this message:

for the bots and search engines

The installer has detected that the following disks have mounted
partitions:

/dev/sda

Do you want the installer to try to unmount the partitions on these
disks before continuing? If you leave them mounted, you will not be
able to create, delete, or resize partitions on these disks, but you
may be able to install to existing partitions there.

Unmount partitions that are in use?

It happens whenever you already have a filesystem on say sda1. Workaround:

d-i preseed/early_command string umount /media || true

• Fix
• Link to the original solution

Sherlocking Mock based builds

Jenkins is frozen, waiting for something. One useful link is to look at ${JENKINS_URL}/threadDump. That reveals which thread is doing what.

Mine is stuck like this:

waiting for hudson.remoting.Channel@f7f725

Okay, I logged onto the other machine, and yes, it still has an open ssh connection to the guest. So log on to the guest, and look at the open files:

sudo lsof -i 4

Okay, I now know some PIDs: 775 and 790

Let's see the file descriptors:

sudo ls -l /proc/790/fd/

lrwx------ 1 root root 64 May 16 19:51 0 -> /dev/null
lrwx------ 1 root root 64 May 16 19:51 1 -> /dev/null
lr-x------ 1 root root 64 May 16 19:51 10 -> pipe:[7800]
lrwx------ 1 root root 64 May 16 19:51 2 -> /dev/null
lrwx------ 1 root root 64 May 16 19:51 3 -> socket:[7744]
lrwx------ 1 root root 64 May 16 19:51 4 -> socket:[7781]
lrwx------ 1 root root 64 May 16 19:51 5 -> socket:[7791]
lr-x------ 1 root root 64 May 16 19:51 6 -> pipe:[7798]
l-wx------ 1 root root 64 May 16 19:51 7 -> pipe:[7798]

Nice, I guess that pipe is the bad guy, but how do I know, what is that?

sudo lsof  | grep 7800

sshd        790         ubuntu   10r     FIFO                0,8      0t0       7800 pipe
udevd     30043           root    4w     FIFO                0,8      0t0       7800 pipe

All right, that means that I know a new PID...

sudo ls -l /proc/30043/fd/

Okay, that's nice, but what was the command line?

sudo cat /proc/30043/cmdline
udevd --daemon

Nice, now I need to look at mock and udevd.

Handwriting to SVG: potrace and image magick

#!/bin/bash
set -eux

INPUT="$1"
OUTPUT="$2"

WORKDIR="$(mktemp -d)"
convert \
    -format bmp -colorspace gray -normalize \
    -threshold 90% "$INPUT" "$WORKDIR/bitmap.bmp"

potrace -b svg -r 300 -t 5 -o "$OUTPUT" "$WORKDIR/bitmap.bmp"

Machine Callback Design

The Problem

Being able to remotely execute commands on stateless executors. The problem is that the controller does not know how to connect to the executor. Such a situation could be when a VM is started, and the hypervisor is unable to detect its IP information, or when the VM is behind a firewall, so the controller cannot access the executor directly.

To make things a bit more easier, let me define some things for the rest of the document:

executor:	The virtual machine that will execute the code
controller:	The machine that orchestrates the whole process

There is a 1:1 relationship between the controller instance and the executor instance

Step 1 - Create Configuration

On the controller, some files and settings need to exist:

• generate CONTROLLER_KEY
• allocate two ports on the system: CONTROLLER_PORT and CALLBACK_PORT. CONTROLLER_PORT has to be free on localhost, CALLBACK_PORT needs to be available on an externally accessible IP.

Step 2 - Create AFTER_INSTALL Script for the Specific Distribution

This script's responsibilities are:

• Configure the new system, so that eth0 will work with DHCP
• Configure a CALLBACK service on the new system, that will be executed on each boot, and will:
• Establish an SSH connection to the controller's IP address, with a specific CALLBACK_USER on a specific CALLBACK_PORT with a specific CALLBACK_KEY and forward it's ssh to a CONTROLLER_PORT
• Authenticate CONTROLLER_KEY for CONTROLLER_USER
• Shut down the machine

Step 3 - Create an Install Media

To install an operating system on a VM using some virtualisation platform. I think the Greatest Common Divisior of virtualisation platforms that I know so far is an ISO file. The project unattended-ubuntu-iso creates an ISO that will install Ubuntu, and shut down afterwards. It is also possible to inject a custom AFTER_INSTALL script. The installer should also create CONTROLLER_USER.

Step 4 - Install Operating System

A VM has to be created with a virtual hard drive, and a virtual cdrom - with the Install Media inserted. At this point the VM does not need network connection. It only uses the CDROM. The created VM has to be started, and have to wait until it's halted. The AFTER_INSTALL script will perform the specified steps, and shut the VM down.

Step 5 - Start a Controller

On the controller, an ssh server has to be started, allowing CALLBACK_USER to connect with CALLBACK_KEY.

Step 6 - Save the Image

The installer VM that was used to run the install process can be destroyed, only the hard disk needs to be kept - this is the image.

Step 7 - Start a new Instance

With the image created in the previous step, the executor could be created. This time it needs network connection as well. The virtualisation platform has to start the executor. As the AFTER_INSTALL script has already configured the networking, the executor will be configured by DHCP. As the executor started, it will start the CALLBACK script, which will establish a secure channel on CONTROLLER_PORT.

Step 8 - Execute commands

The controller can now access the executor's ssh server on the CONTROLLER_PORT. It will use CONTROLLER_USER and CONTROLLER_KEY to execute commands on the remote machine.