The Xen technology allows you to run multiple operating systems on a single physical system, govern resource consumption and even migrate domains (which are the virtual environments in which a guest operating system runs) from one Xen-powered system to another. Xen requires the host operating system to support Xen (which, in this case, will be a Linux kernel) but guest operating systems can run unmodified if your hardware supports Intel Virtualization Technology (VT-x) or AMD Virtualization Technology (SVM). Otherwise your guest operating systems must also support Xen.

This guide will talk you through the configuration steps necessary to get Xen up and running on Gentoo Linux. We will not discuss Xen itself (the Xen project has decent documentation available) nor will we talk about specialized setups that might be very interesting for Xen setups but are not Xen-related (like exporting Portage through NFS, booting Linux using PXE, etc.)

Domain0 is the primary domain under Xen, hosting the host operating system which governs all other domains. In this chapter we will prepare an existing Gentoo installation to become the host operating system in this domain and build the Xen-powered kernel so that Gentoo is ready to host other Xen domains.

A drastic change that might be necessary is to rebuild the entire Gentoo installation with a different CFLAGS setting. Guest operating systems running under Xen might otherwise see major performance degradation. If you, however, are planning on checking out Xen rather than installing it for production use and are not terribly fond of rebuilding all programs, you can skip this step. In this case you will notice performance degradation but you will still be able to use Xen.

~# nano -w /etc/make.conf
(Add -mno-tls-direct-seg-refs)
CFLAGS="-O2 -march=pentium4 -pipe -mno-tls-direct-seg-refs"

~# emerge -e world

If you boot your system using an initial ramdisk (initrd) you need to rebuild the initrd as well (which is best done by running all steps you would do when you rebuild your kernel).

Xen actually contains many components, so you'll need to install a couple of packages. Because it is still ~arch masked you first need to unmask it by adding the necessary lines to /etc/portage/package.keywords and then install them.

~# nano -w /etc/portage/package.keywords
app-emulation/xen
app-emulation/xen-tools
sys-kernel/xen-sources

~# emerge xen xen-tools xen-sources

Next we'll build the Linux kernel with Xen support. This kernel, whose sources are available at /usr/src/linux-2.6.x.z-xen, will be our main running kernel (i.e. the one running domain 0). In the XEN section you'll find drivers for all kinds of input/output, each driver having a backend and frontend implementation available. For the domain 0 kernel you need to select the backend implementation: these are used by the other domains (who use the frontend drivers) to communicate directly with the hardware.

Of course, don't forget to select Xen-compatible at Processor type and features. If you're wondering about networking: each interface in a domain has a point-to-point link to an interface on domain 0 (called vifX.Y where X is the domain number and Y the Yth interface of that domain), so you can configure your network the way you want (bridging, NAT, etc.)

A nice hint is to have the kernel make process store its intermediate object files elsewhere so that you can reuse the same kernel tree to build different configurations:

~# mkdir -p ~/build/dom0 ~/build/domU
~# make O=~/build/dom0 menuconfig
(Configure the kernel)
~# make O=~/build/dom0 && make O=~/build/dom0 modules_install

Once the kernel is built you'll find the kernel image immediately in the build directory (not inside arch/ or any other directory) called vmlinuz. Copy it to /boot and then configure your bootloader to use the Xen hypervisor (one of the components installed previously) which is stored as /boot/xen.gz. In the bootloader configuration, add your newly built kernel as the kernel that Xen should boot. For instance, for GRUB:

title Xen 3.0 / Gentoo Linux 2.6.x.y
root (hd0,0)
kernel /boot/xen.gz
module /boot/kernel-2.6.x.y-xen0 root=/dev/hda3

Now reboot your system into Xen. Once you are booted, you need to load the Xen daemon:

~# /etc/init.d/xend start

Now check if you can do whatever you normally do on your system. If this is the case, you can edit your bootloader configuration to always boot into Xen and add the Xen deamon to the default runlevel so that it is started automatically next time you boot.

Go to the Xen-powered Linux kernel source and update the configuration. It is wise to keep as many topics as possible similar to the main kernel except the XEN settings where drivers should now have their frontend implementation selected instead of the backend. Then build the kernel and place the resulting vmlinuz file where you want (we assume this is /mnt/data/xen/kernel):

~# make O=~/build/domU
~# cp ~/build/vmlinuz /mnt/data/xen/kernel/kernel-2.6.x.y-xen

It is also possible to create a single kernel image for both the administrative domain and the unpriviledged domain. More information about this can be found in the Xen user manual.

For best performance, it is best to dedicate a partition (or logical volume) to a domain rather than a file based filesystem. However, if you are going to use Xen primarily for tests using a file based filesystem does have its advantages (especially regarding maintenance).

You can create a file based filesystem using dd and mke2fs (or any other file system creation tool). For instance, to create a 2Gbyte ext3 filesystem:

~# dd if=/dev/zero of=/mnt/data/xen/disks/ext3root.img bs=1M count=2048
~# mke2fs -j /mnt/data/xen/disks/ext3root.img

Next we create a Xen configuration file for a domain. You can store these configuration files where you want, for instance at /mnt/data/xen/configs. As an example, we create a configuration file for a small Gentoo environment which uses the disk image we created previously:

~# nano -w /mnt/data/xen/configs/gentoo

kernel = "/mnt/data/xen/kernel/kernel-2.6.x.y-xen"
memory = 512
name   = "gentoo"
(Map the disk image to the virtual /dev/sda1)
disk   = ['file:/mnt/data/xen/disks/ext3root.img,sda1,w']
root   = "/dev/sda1 ro"

You can find example configuration files in /etc/xen.

Now we're all set and we can launch the new domain. If the disk image contained an operating system, we could just create and attach the domain using the xm command (Xen manager):

~# xm create /mnt/data/xen/configs/gentoo -c

The domain would be booted inside the terminal in which you executed the command. However, in our case, the disk image is empty so the domain wouldn't be able to do much. To fix this, you can loop-mount the image and install Gentoo as you're used to.

If you want to disconnect from the domain, press Ctrl+]. You can always reconnect to the domains' console using xm console gentoo. However, there is only one console per domain, so only use it when you can't access the domain otherwise (for instance, through SSH).

Xen supports at least two ways of configuring your (virtual) network: routed and bridged.

When selecting the routed approach, the interface inside your unpriviledged domain is connected to the virtual interface on your administrative domain. On your administrative domain (domain 0), the virtual interface is linked together with eth0. The interface inside your unpriviledged domain should have an IP address on the same network as the interface on the administrative domain. Any communication to that IP address can only occur from the administrative domain, unless you set up specific routing rules.

When selecting the bridged approach, TODO.

Before you set up the interface on your unpriviledged domain, make sure that Xen's netloop and netbk drivers are loaded. A quick hint: if you have netloop as a module, load it with nloopbacks=0 so that it doesn't create pointless interfaces to the loopback device. Then, edit your domain configuration file and add a vif instruction to it.

~# nano -w /mnt/data/xen/configs/gentoo

(Add the vif instruction)
vif    = [ 'ip=192.168.1.101, vifname=veth1' ]

In the above example, the interface will be created for the unpriviledged domain (in which it will be called eth0) and Xen will ensure that address 192.168.1.101 will be reachable from the administrative domain through interface veth1.

This doesn't mean that the virtual eth0 interface will automatically have IP 192.168.1.101 assigned to it, but rather that, if you don't give it that IP, it will not be connected with the administrative domain and thus cannot be reached.

Now edit /etc/xen/xend-config.xsp as follows to select routed network configuration:

~# nano -w /etc/xen/xend-config.xsp

(Comment out the following lines)
#(network-script network-bridge)
#(vif-script vif-bridge)

(Enable the following lines)
(network-script network-route)
(vif-script vif-route)