Mate's Blog

Tuesday, 18 March 2014

Sunday, 2 March 2014

This weekend started with friday. I took a day off to relax my mind. This relaxation ended up sitting in front of my laptop, experimenting wit vsphere's opensource java api, while trying to build an embedded vm that can run java - just with buildroot. On saturday, I managed to find out, what osgi is, and I think it will be a useful ground for running java bundles on my emvms (embedded vms). The same day I learned, that a proper libc is needed, so I experimented with the linux kernel - crosstool-ng - buildroot trinity. On sunday we went to cinema to watch the rather old movie: gravity, re done and posted yesterday's build efforts with some extra deployment script, and closed the day with some control theory, learning about zenos, and how to deal with type 1 zenos. Good night.

First post from phone

This is my first post created with my phone. The picture shows a thermapen, we used it during the weekend.

VM From Scratch

I have spent a lot of time installing various Linux distributions, automating all the complex things, trying out things like puppet, writing my own management scripts. Well, in the end of the day, the goal was to run a very simple application. Usually these applications run inside a hypervisor, and we don't really want to spend time with the operating system's management. All I want is a Virtual Machine that has my application and I don't want to know more about it. I think I'm even not interested in sshing into that box. I want to build VMs quickly.

There are several tools to build a VM, and I believe all do their job perfectly. But I think it's time for me to try to build a VM from scratch. The components, that I will use are:

linux
crosstool-ng
buildroot

Getting the kernel

At the time of writing the linux version that I picked, is 3.10.32 - I believe, it has a long term support. So let's dive in, and get the source code

cd /data/matelakat/kernel
wget https://www.kernel.org/pub/linux/kernel/v3.x/linux-3.10.32.tar.xz
tar -xJf linux-3.10.32.tar.xz

Now that we have the sources, we can create a cross compiler.

crosstool-ng

This tool will help me to build a cross compiler. At the initial phase, my target architecture is 686 32bit. I picked this one, as I would like to test the VM with nested virtualization (ESXi inside VirtualBox), and you can't run 64 bit guests there. So let's get started:

first, download the tools, and check out a specific revision. This makes it easier for anyone to reproduce.

mkdir /data/matelakat/xtool
cd /data/matelakat/xtool
hg clone http://crosstool-ng.org/hg/crosstool-ng
cd crosstool-ng/
hg checkout 9321d9d7af9b

Now comes the crosstool-ng specific part. First, we need to actually install crosstool-ng:

./bootstrap
./configure --prefix=$(cd .. && pwd)/ctng
make
make install
export PATH=$PATH:$(cd .. && pwd)/ctng/bin

The next step is to generate a configuration for the crosstool. I will start by getting a sample configuration file, and overwriting it with my config:

cd /data/matelakat/xtool/
mkdir configuration
cd configuration/
ct-ng x86_64-unknown-linux-gnu
cp /data/matelakat/vmfromscratch/crosstool-ng/.config .config
ct-ng menuconfig

If you are happy with the config, it's time to build the cross compiler. My config will put the downloaded sources and the built tools to ../src and ../x-tools, so I need to create those directories in advance. Also, as I have a laptop with 4 cores, I will use 6 jobs simultaneously:

mkdir ../src ../x-tools
time ct-ng build.6

I can definitely tell: my CPU cores are busy - some hot air flows out of my good old Lenovo R500. After around an hour, the crosscompiler was ready:

du -shc ../x-tools/

364M../x-tools/
364Mtotal

Build the Kernel

Now that we have a crosscompiler, we can get back to the kernel directory, configure the tool paths, and compile a new kernel:

cd /data/matelakat/kernel/linux-3.10.32/
cp /data/matelakat/vmfromscratch/linux/.config ./
make oldconfig
make menuconfig
make -j 4

Unfortunately, I didn't measure the time it took, but it was quite fast

Build the Root Filesystem

Now that I have a nice kernel I also need a root filesystem. Buildroot will help me to build that:

mkdir /data/matelakat/br
cd /data/matelakat/br
wget -qO - http://buildroot.uclibc.org/downloads/buildroot-2014.02.tar.gz | tar -xzf -
cd buildroot-2014.02/
cp /data/matelakat/vmfromscratch/buildroot/config .config
make menuconfig
make

Putting it all together

Now that I have everything, I only need to create an iso cd with my new system

mkdir isoroot
cp ../br/buildroot-2014.02/output/images/rootfs.cpio.gz isoroot/initrd.img
cp ../kernel/linux-3.10.32/arch/i386/boot/bzImage isoroot/
cp ../br/buildroot-2014.02/output/images/isolinux.bin isoroot/isolinux/
cp isolinux.cfg isoroot/isolinux/
mkisofs -J -o minsys.iso -b isolinux/isolinux.bin -c isolinux/boot.cat -no-emul-boot -boot-load-size 4 -boot-info-table isoroot/
qemu-system-i386 -M pc -cdrom minsys.iso -net nic,model=e1000 -net user

Next steps

My next step will be to automate all these things.

Sunday, 9 February 2014

PDF - Two pages per page to one pages per page

So we had a PDF, it's size was 297 x 215 mm. Each page of this PDF contained two pages, like this:

 Page 1 of the input file                       Page 1 of output   Page 2 of output
    +------------------------------+            +---------------+  +---------------+
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    | Page 1        | Page 2       |            |   Page 1      |  |   Page 2      |
    |  of the book  |  of the book |            |    of the book|  |    of the book|
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    +------------------------------+            +---------------+  +---------------+
                                     +------->
 Page 2 of the input file            +------->  Page 3 of output   Page 4 of output
    +------------------------------+            +---------------+  +---------------+
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    | Page 3        | Page 4       |            |   Page 3      |  |   Page 4      |
    |  of the book  |  of the book |            |    of the book|  |    of the book|
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    |               |              |            |               |  |               |
    +------------------------------+            +---------------+  +---------------+

So to do this, I extracted the left sides as page-0.pdf, and the right sides as page-1.pdf to the folder out, and the n joined these together:

#!/bin/bash
set -eux

# Default resolution is 720 dpi
# For points, use 72 dpi

# The actual pdf was 297 x 215 mm --(720 dpi)----> 8418 x 6094 pixels
# I only want to extract half of it -------------> 4209 x 6094
# For the right page, I need an offset -(72 dpi)-> -421.0 0

INPUT_FILE="$1"

rm -rf ./out
mkdir out

ghostscript -o "out/%d-0.pdf" -sDEVICE=pdfwrite -g4209x6094 \
-c "<</PageOffset [0 0]>> setpagedevice" -f "$INPUT_FILE"

ghostscript -o "out/%d-1.pdf" -sDEVICE=pdfwrite -g4209x6094 \
-c "<</PageOffset [-421.0 0]>> setpagedevice" -f "$INPUT_FILE"

ghostscript -o "singlepage-$INPUT_FILE" -sDEVICE=pdfwrite -dPDFSETTINGS=/ebook \
    $(ls out/* | sort -V)

rm -rf ./out

References:
http://superuser.com/questions/54054/convert-pdf-2-sides-per-page-to-1-side-per-page

Sunday, 2 February 2014

Back to school - Coursera - Crypto, Control Theory, Android

After KR told me that he signed up for a Cryptography course on Coursera, I decided to do the same. I learned crypto at Uni, but that was a long time ago, so I thought, it's time to refresh my memories. The story does not end here, as this weekend, I signed up to two more courses: an Android course, and a Robot Control course. I'm a bit behind at the moment for each of the courses. I find this Coursera amazing, and I hope my wife won't kill me before I finish these courses. Should you be interested, here is a list of the courses that I started:

https://class.coursera.org/crypto-009
https://class.coursera.org/conrob-002
https://class.coursera.org/android-001

I just started to watch the video lectures for the Android class, and watching this video atm as recommended by the professor:

Sunday, 25 August 2013

Ubuntu - change dns server for proxychains

This is really a not for myself, so that I don't forget it. So if you are using proxychains to access some servers through a proxy, you might want to use that system's dns server. In that case, presuming that you are running ubuntu, please modify the file:

/usr/lib/proxychains3/proxyresolv

When you open the file, it will be obvious what needs to be changed.