Running Scratchbox in KVM

Contents

[edit] Preface

This is currently (mostly) a personal notes page, but you're welcome to edit, comment and make corrections or additions to it. Hopefully (and after some editing) it might become a complete HOWTO guide at some point.

This is still a work in progress, meaning that I don't have a working installation yet. (I don't expect any problems in the setup though, I just have had limited time to put into this.)

I'm also a KVM newbie, so I could be doing some things wrong or in an unnecessarily complicated way. (Being new to this is also the reason why I decided to start writing down what I do; otherwise I'd have no way to backtrack and repeat my steps if I screw up somewhere along the way...)

[edit] Why run scratchbox in KVM?

There are several reasons why you might want to do this:

  • You might be running a 64-bit distribution, and be uncomfortable force-installing 32-bit packages on it
  • You might not like installing software that hasn't gone through your distro's usual package management and QA process
  • You might want to be able to snapshot your environment when it's ready, to be able to reinstall easily and/or migrate it to another machine.

There are several alternatives which might also work for you:

  • The official VMware image, which is cross-platform and works out of the box. However, you might not like the audit rights clause of the license.
  • Other virtualization solutions such as Xen. Xen is open source but (unlike KVM) not part of the mainline linux kernel.
  • User-mode linux might work - I think there was some discussion about this on maemo-developers, but I'm not sure what's the status of that.
  • A simple chroot environment with ia32-libs, but then we're already somewhat drifting from the "why" part above.
  • Or using a separate, standalone, non-virtualized installation would of course work too, if you have a machine to spare for that or don't mind rebooting, but that's beside the point of this article.

[edit] Requirements

Firstly, your CPU must support virtualization. This can be checked with: 

$ egrep '^flags.*(vmx|svm)' /proc/cpuinfo

Second, you need the modules kvm and either kvm-amd or kvm-intel available and loaded.

Third, and assuming a Debian system here, install the following: 

# apt-get install kvm virt-manager

(YMMV; there are other ways to setup KVM, and other distros might package the tools differently, but this is what we're doing here.)

Fourth, and finally, add yourself to the 'libvirt' group: 

# adduser yourusername libvirt

and either log out of your X session and re-login, or (as I prefer to do), 

$ ssh -X localhost

and do rest of your user-privileged setup in that terminal.

[edit] Installation

Also refer to the KVM documentation and FAQ.

TODO: Check the Management tools listed on the KVM pages, to see if there's some better way for the setup. At least AQEMU looks promising.

[edit] Network

As we're not setting up a virtual server farm here, you probably won't want to bridge your physical network interface; a virtual network will do just fine.

At least Debian kvm comes with a default virtual network predefined; start it with 

# virsh net-start default

This will create a virtual virbr0 device and setup iptables forwarding rules for it. Those should be mostly fine, but you may want to check them.

Next, you'll probably want to run a DHCP server to provide an address for your virtual machine(s). Assuming the server in the 'dhcp' package is used, add virbr0 to the served interfaces in /etc/default/dhcp, and add a subnet definition to /etc/dhcpd.conf - e.g. 

subnet 192.168.122.0 netmask 255.255.255.0 {
        range 192.168.122.2 192.168.122.254;
        option routers 192.168.122.1;
        option subnet-mask 255.255.255.0;
        option broadcast-address 192.168.122.255;
        option domain-name-servers 192.168.122.1;
        option netbios-name-servers 192.168.122.1;
}

(you can check the proper values with virsh net-dumpxml default). Oh, and yes I run my own DNS server too; you might try leaving out the domain-name-servers line if you don't. I guess probably the virbr0 forwarding rules would handle that with the outside world. The netbios-name-servers line is there just in case I add Win* virtual machines to the network later, and is not really needed here.

Now we're all set, proceed to:

[edit] Creating the virtual machine, option one

There are at least two ways to initialize the virtual machine. The first one I tried was - if my memory (er, shell history) serves me right: 

$ virt-install --connect qemu:///system -n sb -r 1024 --arch i686 --vcpus 1 --os-type linux --os-variant debianlenny -v --virt-type kvm --accelerate --disk path=/home/juha/kvm/sb.qcow2,size=20,sparse=true --disk path=/dev/cdrom,device=cdrom,bus=ide,perms=ro --network network=default --keymap fi --vnc --noautoconsole

These options are somewhat superfluous (some of them would be the default anyway, but I was working through the man page and learning this stuff at the same time), but anyway this creates a 32-bit (i686) virtual machine named 'sb', with one virtual cpu, 1024 megs of ram, and 20 gigs of disk. I'm not sure what the os-type and os-variant options do exactly, but I assume they may enable some settings to be optimized for the particular OS flavor.

This also sets up vnc for the machine, with Finnish keymap (you may want to change that), but doesn't open the console automatically (I prefer to view it with virt-manager).

Note that we defined the cdrom with --disk instead of the usual --cdrom. This is to be able to give a bus=ide option instead of the usual virtio driver. For some reason kvm fails to boot from cdrom unless emulating the ide bus. Some quick googling revealed that it would - apparently - expect a 'boot=on' parameter which the current Debian virt-install fails to provide. You could try to patch the scripts to fix this, or you can just use 'ide' as above. Virtio is expected to be faster, but you probably won't be using the cdrom for much else than installation anyway.

Also note that you could as well give a path to a bootable ISO image, but I happened to have a Deian-Lenny install CD lying around, so I used that.

[edit] Creating the virtual machine, option two

I actually didn't finish the installation with the above command line, deciding instead to install newer versions of kvm and the tools from unstable - to check if the booting issue would be fixed. It wasn't, but with virt-manager 0.8.0-2 it's possible to create virtual machines through the GUI. (Actually I'm not sure if the version I had from 'testing' might have included that as well, but if it did, I didn't notice. Like said these are my installation notes, please comment or fix if what I'm saying is not correct.)

If you already created the virtual machine before and want to destroy and redefine it, this can be done with 

virsh destroy sb
virsh undefine sb

So off to the virt-manager GUI. Most of the setup is pretty straightforward and won't be covered here in detail, but unfortunately this has a couple of gotchas too:

  • The GUI insists on creating your disk image in /var/lib/libvirt/images. This may be a problem if your /var is on a separate partition, and doesn't have enough space free.
    • To work around this, I bind-mounted $HOME/kvm there (a symlink didn't seem to do the trick, for some reason). Unfortunately the GUI still tries to be a bit too smart and limits the maximum image size to the space available on /var, so additionally you'll have to grep the python scripts included in the '*virt*' packages for the above path, and replace it with the real physical path in your homedir. I think there were two such files. (I could be wrong, but only editing them also didn't seem to work without the bind-mount.)
    • A better workaround could be selecting to use an existing image file, if you have one from a previous virt-install attempt or can figure out the right command to create it otherwise, but I didn't try that...
  • Unless the cdrom booting issue has been fixed by the time you're doing this, your virtual machine will start but you will be greeted with the "No bootable device" message on the vnc console.
    • To fix this, shut down (or force off) the virtual machine, and enter "virsh edit sb". Find the cdrom definition in the XML and change its bus attribute from 'virtio' to 'ide'. While you're at it, you might replace the path for the disk image with the real location, so you won't need to keep the bind-mount around. Save the file and you're all set.

You should now have a working virtual machine installation. Insert the installation cdrom, check that the Boot Device in virt-manager "Details" page is set to CDROM (don't forget to Apply the change if it's not), and proceed to the next step:

[edit] Installing the OS

There shouldn't be anything special or noteworthy here. Just a plain normal Debian installation (you may need to change the virt-manager Boot Device back to Hard Disk after the installation is complete).

I realized afterwards that I probably allocated way too much disk space to the virtual machine. My intention is to keep all my source code on the host machine and do all editing there, just mounting the directories to the virtual machine using some network filesystem, so I won't be needing much space on the virtual side.

This time I also installed the full Debian system in the virtual machine, complete with the GNOME desktop, OpenOffice, GIMP, Iceweasel, etc. which won't be needed on the virtual development box. So if I do this again, I'll probably use a much smaller virtual disk, and only install a minimal Debian system (and use ssh to complete the setup, instead of using the full desktop environment over vnc).

One further question: I chose the default single-partition setup, which also seemed to create swap on the virtual disk. Does that ever make sense, or would it be better to just give the virtual machine "enough" ram and let the host handle swapping? How much would be enough for building stuff in the SDK? How much would be too much (with respect to the physical ram installed on the host)?

[edit] Installing the SDK

With the OS in place, there shouldn't be anything unusual about the SDK installation either, but I didn't get that far yet. Up to now I had enough to do just learning to use KVM and working around the various bugs in the virtualization tools, and decided to stop here for today and write those steps down. If someone else has time to continue this before I do, please go ahead :-)

EDIT: No problems with SDK installation, but didn't have time to continue with this page yet.

[edit] Setting up a network filesystem

To be able to edit sources on the host side (and not have to replicate my entire emacs setup). Could be done in either direction: sources on host, and exported to vm so scratchbox can access them, or sources on vm and exported to host.

TODO, the biggest piece still missing from a fully functional setup. I'm not so familiar with network filesystems, any ideas what's the best way to do this?

[edit] Setting up Xephyr

In principle you could install Xephyr either on your host or in the virtual machine. I'm currently running it in the virtual machine over ssh, e.g. 

ssh -X ip.of.virtual.machine Xephyr :2 -host-cursor -screen 800x480x16 -dpi 96 -ac -kb

and then the usual "export DISPLAY=:2" and "af-sb-init.sh start" in scratchbox.

[edit] OpenGL with the virtual setup

TODO. Possible?

[edit] Finetuning

Putting the necessary steps from above into scripts so that the setup is easy to restart after a reboot. Maybe other scripts or tools to make the initial setup easier, or even a ready-to-run SDK image, if some kind person at Nokia or in the community decides to create such things? TODO.

Retrieved from "http://wiki.maemo.org/Running_Scratchbox_in_KVM"