Bug 70161 - Ebuild to be used with the source and or binaries of Xen
|
Bug#:
70161
|
Product: Gentoo Linux
|
Version: unspecified
|
Platform: x86
|
|
OS/Version: Linux
|
Status: CLOSED
|
Severity: enhancement
|
Priority: P2
|
|
Resolution: TEST-REQUEST
|
Assigned To: chrb@gentoo.org
|
Reported By: stefan@konink.de
|
|
Component: Ebuilds
|
|
|
URL:
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/index.html
|
|
Summary: Ebuild to be used with the source and or binaries of Xen
|
|
Keywords:
|
|
Status Whiteboard:
|
|
Opened: 2004-11-05 06:28 0000
|
Xen is a virtual machine monitor for x86 that supports execution of multiple
guest operating systems with unprecedented levels of performance and resource
isolation. Xen is Open Source software, released under the terms of the GNU
General Public License. We have a fully functional ports of Linux 2.4 and 2.6
running over Xen, and regularly use it for running demanding applications like
MySQL, Apache and PostgreSQL. Any Linux distribution (RedHat, SuSE, Debian,
Mandrake) should run unmodified over the ported OS.
Reproducible: Always
Steps to Reproduce:
'We' should be read as 'They'.
Are you requesting an ebuild, or do you already have one to share?
Best regards,
Stu
Requesting, to download the latest sources bitkeeper tools are needed. Plus,
XEN start downloading a kernel which is allready found on most systems in
distfiles.
I've got some almost-working ebuilds to build and install Xen (tentatively
named sys-apps/xen and sys-kernels/xen-sources); if nobody else is already
working on this, I should (hopefully) be able to finish them to a
generally-working state in the next several days.
I've uploaded my current attempts to http://people.pwf.cam.ac.uk/pjt47/xentoo/
- I wouldn't be surprised if I've done various things wrongly, but it appears
to work for me (as far as booting Gentoo using a XenLinux kernel and running
xend; I haven't tried actually running a virtual Gentoo machine).
Current problems include being limited to the 2.6 kernel, not handling
unprivileged domains (unless you manually use a different default kernel config
file), confusing etc-update by putting binary files in /etc (since I didn't
want to automatically mount /boot, and wasn't sure where else to put them), and
possibly other things that I've not noticed or remembered.
To build the docs, tgif (see bug 36185) is required.
inherit mount-boot should sort out the auto mounting of /boot for the ebuild.
Aha, thanks - I've updated it to use mount-boot and copy the files directly
into /boot. (Updated ebuilds at http://people.pwf.cam.ac.uk/pjt47/xentoo/
again.)
I'm not sure what the most convenient way of installing an unprivileged kernel
is; I currently provide manual instructions to cp /usr/src/linux-2.6.9-xen0 to
linux-2.6.9-xenU and then to switch the default config file. It seems a little
wasteful to devote half a gigabyte to two identical copies of the kernel
sources, but I don't know what else to do.
Use flag (xen0only)? Seperate package?
I'd say unpack to two different directories in this ebuild. A user who is low on space could always unmerge the sources once they've built the two kernels they require?
if someone can create a Gentoo-ified version of the xendomains init.d script,
i'll add this package to portage ...
xen looks very promising, btw the developers have released version 2.0.1, is
anyone able to create a Gentoo-ified version of the xendomains init.d script so
that this package can be added to the tree?
Created an attachment (id=44962) [details]
xen-2.0.1.ebuild
Tested version :)
Added the cam.ac.uk mirror as sourceforge does not seem to have it on the
mirrors at present, and one of the mirrors hung for me. Added the cam.ac.uk
webpage also.
Had to use some 'trickery' to get it to compile, as the package is zipped in a
xen-2.0 directory, so I have a mv command. Hopefully this is a fair way to do
it, and should also mean that it is easy to update just by copying and
remnaming the ebuild as the version is bumped.
Will look at the init.d script to see if I can Gentoo-ify it so it can be
merged mainstream!
Created an attachment (id=44963) [details]
xendomains
Gentoo-ified xendomains script.
There is plenty more that could be done to this script - restart and reload are
commented out from the original script.
Haven't played with Xen too much yet, but wondering if it might be worth
setting individual scripts for each domain, cf net.* scripts (eg xen.dom0,
xen.dom1 etc)
Created an attachment (id=44965) [details]
xend
/etc/init.d/xend script, modified to represent that it is starting dom0 as
opposed to Xen in general. Should help remove confusion during boot with the
xenddomains script.
> /etc/init.d/xend script, modified to represent that it is starting dom0 as
opposed to Xen in general
you can't start dom0 as it is started by xen when it boots. stopping dom0 means halting the machine. xend is afaik xen control daemon for performing management tasks like starting/stopping other domains, resource usage control,...
and you'll need another daemon for live migration support.
Good point. Now that I look closer at the script (Why didn't I read it earlier)
it is for starting the Control Daemon for the domains. Is it worth modifying
the xend script for that?
you'll have to compile glibc without nptl in USE flags, otherwise you'll get
this warning:
***************************************************************
***************************************************************
** WARNING: Currently emulating unsupported memory accesses **
** in /lib/tls libraries. The emulation is very **
** slow, and may not work correctly with all **
** programs (e.g., some may 'Segmentation fault'). **
** TO ENSURE FULL PERFORMANCE AND CORRECT FUNCTION, **
** YOU MUST EXECUTE THE FOLLOWING AS ROOT: **
** mv /lib/tls /lib/tls.disabled **
***************************************************************
***************************************************************
despite what it says, /lib/tls doesn't exist on my system, but see this message
from Ian Pratt (Rik is probably Rik van Riel from Red Hat):
NPTL is supported, but we're forced to take traps for almost
every attempt to access thread local storage, which is slow (I
think Rik said 12-15% on a kernel compile).
The only way to improve on this would be to have our own very
slightly modified version of libc that could be installed to
restore full performance. We're hoping to look into this at some
point.
Disabling NPTL works for most people.
Ian
ok, then if i understand this correctly, the xen init.d scripts are really only
meant to be used from inside of xen ?
if so, i'll have the ebuild stick them in /usr/share/xen/ instead of
/etc/init.d/ ...
It depends what you think "inside of xen" is. You need two types of a kernel; a
host kernel (Xen calls this domain 0) and guest kernel (you can assign any
number/name to this domain). When you boot your system with a Xen host kernel
you are already inside Xen (domain 0), however to start a guest kernel you need
to have xend running. I thinks /etc/init.d is the correct location for both
scripts.
ok, but the only files that should go into /etc/init.d/ are scripts that have
been 'gentoo-ified'
yep, only two scripts need to go to /etc/init.d; xend (to start/stop the
xen-interface) and xendomains (to start/stop the quest systems during
boot/shutdown).
if you use xen, you need at least two things - xen kernel itself and patched
linux kernel (aka "domain0" in xen terminology). these two thing will provide
(almost) the same services as normal vanilla linux kernel, plus some xen stuff,
so you can run any distribution (including gentoo :-)) on the top of such
system.
but this setup is useless if you don't run any virtual domains (you don't have
to bother with xen just to run "one instance" of OS), so you'll probably want
to create another virtual machines. in order to do it, you'll need xend running
inside of domain0 (`/etc/init.d/xend start` is a good way, it is only shell
script to actually start xend daemon) and have prepared special kernel (linux,
netbsd, freebsd, winxp (well, not available due to licensing issues)) and root
fs. then you can start any number of other "machines", or, in Xen terminoology,
domains.
as you probably want to start all (or some of) these virtual machines at
bootup, xen has provided the "xendomains" init script, it looks into
/etc/xen/auto/ and starts domains specified there.
Michiel is right.
Hi All,
Thanks for the ebuild. Testing it now.
I got the following error:
cc1: warning: -fprefetch-loop-arrays not supported for this target (try -march switches)
I remember that it's possible to disable certain useflags and cflags for ebuilds (but forgot how). Might be wise to disable this one ?
Grtz Ramon
Just wanted to let you know that version 2.0.2 has been released.
Just wanted to let you know that version 2.0.3 has been released.
Question for the devs:
I don't want to be the impatient user but isn't it time to put these ebuilds in Portage? There are a lot of people interested if you look at the CC list but almost nothing happens here. I fear that people willing to try Xen might jump over to RHEL or SuSE.
Just my 2 euro
Question for the devs:
I don't want to be the impatient user but isn't it time to put these ebuilds in Portage? There are a lot of people interested if you look at the CC list but almost nothing happens here. I fear that people willing to try Xen might jump over to RHEL or SuSE.
Just my 2 euroçents of course, let's not start a flamewar on Gentoo VS ... please.
I don't know. I would think not. I have started to use those ebuilds, but I'm
ending up righting an ebuild of my own because these ones had hickups.
A link to this bug from the forums or elsewhere would help much more in getting
this tested and find a proper implementation for it. Then release it to
Portage, unless it worked out of the box for everybody elses?!?
Ok, what is everybody's general status about this?
I have succesfuly builded and implemented Xen on a 2.4 kernel box here. There's still issues to get this done properly thrue a ebuild, but I think I have a sufficient basic grasp of this to generate an ebuild.
The biggest issue I had was how to make it download a 2.4 or a 2.6 kernel. As such, I think patching the kernels upstream would be a better approach (I'd even hope it would be directly included into the kernels at www.kernel.org). Because of that, i had problems integrating my .config inside xen0. But I think I can manage it on my next iteration.
My other issue is the fact the the vanilla Xen script will install the xenU kernel into /boot. This is somewhet inappropriate because then it requires you to mount it to access the kernel. As a simplification, I simply made the configuration file point directly at the bzImage inside the kernel's source hierarchy (arch/xen/boot/bzImage).
Another issue I encounter is the fact that xenU cannot absolutely not work properly if started with devfs. So this caused problems when using Gentoo as a guest, which will complain if devfs is not enabled. Insights about this would be appreciated. I guess I should try to guest something else like Knoppix.
What about you guys? Where are you at with this?
>Another issue I encounter is the fact that xenU cannot absolutely not work properly if started with devfs. So this caused problems when using Gentoo as a guest, which will complain if devfs is not enabled. Insights about this would be appreciated.
emerge udev
devfs at boot is a requirement of Gentoo. In a Xen point of view, udev would
probably cause the same issue. A Xen booted guest seems to need static device
nodes in /dev.
On monday, I will extract the build steps from the Makefiles to incoporate them
directly in the ebuild instead. This will give the flexibility to use a
distinct xen patched kernel, instead of having to use "make kernel-bla" from
xen.
So, I guess that by the end of next week, I'll have a ebuild to submit here. I
mainly wanted to know if others did work on this and avoid duplicating stuffs.
udev is supported as well, as a mather of fact it will be the default very soon
for architectures that support it, same goes for the 2.6 kernel which you need
when using udev, we use it on all our Xen-servers, I suggest you take a look at
Philip's webpage, he already made an ebuild for the Xen-sources.
We need to put this on 2.4 systems. So the limitation is either because of
Gentoo's inhability to run with a static /dev, or xen's inhability to run with
a dynamic one.
I'm doing my second iteration install to check if the ebuilds works well, and I
always seem to have issues with the xenU kernel. I'm unsure if xen is what we
really want. For now, I'll admit I way prefer the vmware paradigm of
virtualization.
I mean, shouldnt it be the guest kernel who should get xenified, instead of
having a bunch of xenU kernels that are completely unrelated to the guest
kernel (that could have various patches applied). I don't get it :(
I wasn't able to get the ebuild in #37 to go. First it would not find the file
to download. I manually downloaded it from http://www.cl.cam.ac.uk and created
the digest. When I went to install it I received the following error.
>>> emerge (1 of 1) sys-kernel/xen-2.0.4 to /
>>> md5 src_uri ;-) xen-2.0.4-src.tgz
>>> Unpacking source...
>>> Unpacking xen-2.0.4-src.tgz to /var/tmp/portage/xen-2.0.4/work
/usr/lib/portage/bin/ebuild.sh: line 33: cd:
/var/tmp/portage/xen-2.0.4/work/xen-2.0.4: No such file or directory
sed: can't read tools/libxc/Makefile: No such file or directory
sed: can't read tools/libxutil/Makefile: No such file or directory
sed: can't read tools/misc/miniterm/Makefile: No such file or directory
sed: can't read tools/misc/nsplitd/Makefile: No such file or directory
sed: can't read tools/misc/Makefile: No such file or directory
sed: can't read tools/xentrace/Makefile: No such file or directory
sed: can't read xen/arch/x86/Rules.mk: No such file or directory
!!! ERROR: sys-kernel/xen-2.0.4 failed.
!!! Function src_unpack, Line 39, Exitcode 2
!!! sed cflags
Apparently, ${S} and the contents of the tgz don't agree.
when trying the latest ebuild, I get the following error. as I posted on the
xen mailing list about an earlier version of this ebuild, I suspect the problem
is due to makefiles using $(DESTDIR} which is the getting wrong value from the
ebuild environment:
ACCESS DENIED mkdir: /usr/portage/distfiles/install
install: cannot create directory `/usr/portage/distfiles/install': Permission
denied
unset DISTDIR is present just to address that problem
tried the suggestion and the build process got a little further:
[ -d /usr/lib ] || install -d -m0755 -p /usr/lib
install -m0755 libxutil.so.2.0.0 /usr/lib
ACCESS DENIED open_wr: /usr/lib/libxutil.so.2.0.0
install: cannot create regular file `/usr/lib/libxutil.so.2.0.0': Permission denied
make[1]: *** [install] Error 1
make[1]: Leaving directory `/var/tmp/portage/xen-2.0.4/work/xen-2.0/tools/libxutil'
looked inside xen-2.0/tools/libxutil/Makefile and saw that it used $(DESTDIR) on installation. I set DESTDIR to ${WORKDIR} but I got the same failure. any suggestions for what I should try next?
Would it help if somebody (I) provide a rsync mirror for a portage overlay
including xen (and tgif)?
It seems that no dev wants that in the official tree at the moment...
Just wanted to let you know that version 2.0.5 has been released.
*** Bug 85211 has been marked as a duplicate of this bug. ***
The build fails for me for some reason. I get these 'asm' errors..
make[2]: Entering directory `/mnt/data/Gentoo-Tmp/portage/xen-2.0.4-r1/work/xen-2.0/xen/common'
gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include -Wall -Werror -pipe -I/mnt/data/Gentoo-Tmp/portage/xen-2.0.4-r1/work/xen-2.0/xen/include -Wno-pointer-arith -Wredundant-decls -O2 -march=pentium3 -fomit-frame-pointer -pipe -falign-functions=64 -mmmx -msse -msse2 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -c ac_timer.c -o ac_timer.o
gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include -Wall -Werror -pipe -I/mnt/data/Gentoo-Tmp/portage/xen-2.0.4-r1/work/xen-2.0/xen/include -Wno-pointer-arith -Wredundant-decls -O2 -march=pentium3 -fomit-frame-pointer -pipe -falign-functions=64 -mmmx -msse -msse2 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -c dom0_ops.c -o dom0_ops.o
gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include -Wall -Werror -pipe -I/mnt/data/Gentoo-Tmp/portage/xen-2.0.4-r1/work/xen-2.0/xen/include -Wno-pointer-arith -Wredundant-decls -O2 -march=pentium3 -fomit-frame-pointer -pipe -falign-functions=64 -mmmx -msse -msse2 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -c dom_mem_ops.c -o dom_mem_ops.o
gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include -Wall -Werror -pipe -I/mnt/data/Gentoo-Tmp/portage/xen-2.0.4-r1/work/xen-2.0/xen/include -Wno-pointer-arith -Wredundant-decls -O2 -march=pentium3 -fomit-frame-pointer -pipe -falign-functions=64 -mmmx -msse -msse2 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -c domain.c -o domain.o
gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include -Wall -Werror -pipe -I/mnt/data/Gentoo-Tmp/portage/xen-2.0.4-r1/work/xen-2.0/xen/include -Wno-pointer-arith -Wredundant-decls -O2 -march=pentium3 -fomit-frame-pointer -pipe -falign-functions=64 -mmmx -msse -msse2 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -c elf.c -o elf.o
dom_mem_ops.c: In function `free_dom_mem':
/mnt/data/Gentoo-Tmp/portage/xen-2.0.4-r1/work/xen-2.0/xen/include/asm/mm.h:160: error: can't find a register in class `BREG' while reloading `asm'
I am rather suprised that it is taking so long to get xen ebuilds into portage.
I guess the lack of any gentoo specific document explaining what is involved
in setting it up would be part of the problem. These ebuilds are based on
Philip Taylor original ebuilds with some modifications. They have been updated
since version 2.0 and are currently running on internal production servers.
They will not work with hardened patches (i.e. -pie).
probably obvious but.
the xen ebuild goes into
apps-sys/xen
all associated files go into
apps-sys/xen/files
the xen-sources ebuild goes into
sys-kernel/xen-sources
Xen will not just work out of the box. You need to create the environment for
the operating system instances (i.e. filesystems). In that respect it is very
much like UML. To install xen you will need to read the manual
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/readmes/user/user.html
Question: does Xen run on any architectures other than x86 and x86-64? Does it
work with 2.4.25?
> Question: does Xen run on any architectures other than x86 and x86-64?
No. Even the ia64 support is not very mature - see the xen-devel mailing list for details.
> Does it work with 2.4.25?
IIRC, dom0 (the domain which can controll another "virtual" domains) must be 2.6. Other domains could run 2.4 kernels or OpenBSD (maybe other BSD derivates, Kip Macy is developing FreeBSD port - consult the mailing list).
BTW, why does this bug depend on bug 36185? AFAIK the tgif is required only for building of docs.
I've just tried removing the tgif DEPEND line in the ebuild, then compile it
with USE=doc : it compiles just fine, and the docs are generated.
Is this dependency _really_ needed ?
Is there any status update available to this bug? Other distros already use
Xen, a lot of users want Xen but nothing seems to happen here.
I hope this is the right "noise" to make this bug active again...
I'm interested in this too, do we need testers?
It seems to me we have too much commotion and are lacking some verified working ebuilds, maybe if someone already au fait with this bug could write a short how-to, with links to what file(s) are needed, and we could go about testing them to help iron out the wrinkles.
I think if it was a little clearer and we had sane ebuilds it would be a good candidate for portage.
(From update of attachment 57185 [details])
#!/sbin/runscript
# Copyright 1999-2005 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
opts="start stop status restart"
depend() {
before xendomain
}
await_daemons_up() {
for ((i=0; i<5; i++)); do
sleep 1
xend status && return 0
done
return 1
}
stop_all_xendomains(){
einfo "Shutting down all Xen domains"
xm list | cut -d' ' -f1 | grep -Ev "^(Name|Domain-0)$" |
while read dom; do
# Stop all running domains.
ebegin " Stopping domain $dom"
xm shutdown --halt --wait $dom >/dev/null
eend $?
done
}
start() {
ebegin "Starting Xen control daemon"
xend start
xend status || await_daemons_up
eend $?
if [ "$XENSV" = YES ] && xend status; then
ebegin "Starting Xensv"
xensv start
eend $?
fi
}
stop() {
if [ "$(xm list | wc -l)" -gt 2 ]; then
stop_all_xendomains
fi
if [ "$XENSV" = YES ]; then
ebegin "Stopping Xensv"
xensv stop
eend $?
fi
ebegin "Stopping Xen control daemon"
xend stop
eend $?
}
status() {
xend status
}
(From update of attachment 57185 [details])
#!/sbin/runscript
# Copyright 1999-2005 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
opts="start stop status restart"
depend() {
before xendomain
}
await_daemons_up() {
for ((i=0; i<5; i++)); do
sleep 1
xend status && return 0
done
return 1
}
stop_all_xendomains(){
einfo "Shutting down all Xen domains"
xm list | cut -d' ' -f1 | grep -Ev "^(Name|Domain-0)$" |
while read dom; do
# Stop all running domains.
ebegin " Stopping domain $dom"
xm shutdown --halt --wait $dom >/dev/null
eend $?
done
}
start() {
ebegin "Starting Xen control daemon"
xend start
xend status || await_daemons_up
eend $?
if [ "$XENSV" = YES ] && xend status; then
ebegin "Starting Xensv"
xensv start
eend $?
fi
}
stop() {
if [ "$(xm list | wc -l)" -gt 2 ]; then
stop_all_xendomains
fi
if [ "$XENSV" = YES ]; then
ebegin "Stopping Xensv"
xensv stop
eend $?
fi
ebegin "Stopping Xen control daemon"
xend stop
eend $?
}
status() {
xend status
}
(From update of attachment 57185 [details])
Sorry about the last two comments. I'm about to learn how to submit files to
bugzilla :-(
new Attachments:
xend-init:
- support for starting xensv (via xend-conf)
- now shuts down all running domains before stopping xend
- removed lots of "
new Attachments:
xend-init:
- support for starting xensv (via xend-conf)
- now shuts down all running domains before stopping xend
- removed lots of "¦¦ die" statements (makes no sense in init scripts)
xend-conf:
- new file, used to enable xensv
xendomains-init:
- simplified a lot
- xendomains-conf removed, as it wasn't really neccessary
xen-2.0.5.ebuild:
- use emake instead of make
- newinitd & newconfd functions used instead of cp
all:
- gentoo conforming headers added
I would really like to see this in portage ASAP.
Could someone please test this new ebuild and init-files?
Created an attachment (id=57231) [details]
sys-apps/xen/xen-2.0.5.ebuild
sys-apps/xen-2.0.5.ebuild
* removed tgif which does not seem to be necessary to compile the documentation
* added debug flag (needed for verbose debugging messages).
* re-added xendomains-conf which is needed, as explained below.
Created an attachment (id=57232) [details]
sys-apps/xen/files/xendomains-init
/files/xendomains-init
* moved AUTODIR back into the config script. This is needed for running groups
of domains, or domains with specific options based on the runlevel.
* added ${} around variables
* changed starting from auto -> ${AUTODIR}
Created an attachment (id=57234) [details]
sys-apps/xen/files/xend-init
/files/xend-init -> /etc/init.d/xend
* added net dependency because xend needs network setup before starting. The
default setup specifically require eth0 but as this is a setting it seemed like
a bad idea putting it in the init script.
* removed stop_all_xendomains function. It is not necessary to step though all
domains when shutting down in xend. This is just a safety/sanity check incase
a domain has been created outside the init scripts.
these are modifications to Christoph Gysin files above. You will need the
conf.d/xend file above to use them. The AUTODIR has been moved into a
configuration file because this allows the use of multiple groups of domains.
This works in a similar way to /etc/init.d/net.eth0.
Old attachments made obsolete.
conf.d/xend renamed to sys-apps/xen/files/xend-conf
I've just tested Edward Middleton's modifications. Everything works as
expected.
Can anybody else confirm this, so we can put it into portage?
i've used the ebuild xen-sources-2.6.10.ebuild (id=53759) and did this steps:
make ARCH=xen menuconfig
make ARCH=xen
make ARCH=xen modules_install
make ARCH=xen install
but no /boot/xen.gz
something missing in the ebuild?
there's only xen.gz in the xen-2.0-install.tgz
* /boot/xen.gz is the hypervisor (mediates operating system access to the
hardware) installed with sys-apps/xen
* /boot/vmlinuz is a domain0-N kernel created from the sys-kernel/xen-sources
* these are not in the same ebuild because there are a number of operating
systems other then linux (various bsd's, plan9 etc) that will run under the
hypervisor so it does not make sense packaging them all in one ebuild.
Created an attachment (id=57444) [details]
sys-apps/xen/files/xendomains-init
* If you try and start/stop a domain that is allready running an error occurs.
I have added a check is_running to see weather the domain is running before
starting or stopping.
spanky, seems to be gentooified, etc -- you want this?
no, i dont use it ;)
i found qemu to be an easier solution ...
Apples and oranges, SpanKY. :) The files seem to be good enough for getting it
into portage!
When I run /etc/init.d/xend start I get the following error:
* Starting Xen control daemon...
Traceback (most recent call last):
File "/usr/sbin/xend", line 27, in ?
from xen.xend.server import SrvDaemon
File "/usr/lib/python/xen/xend/server/SrvDaemon.py", line 35, in ?
from xen.xend.server import SrvServer
File "/usr/lib/python/xen/xend/server/SrvServer.py", line 28,
in ?
from twisted.web import server, static
ImportError: No module named web
And shouldn't there be an ebuild to create the xen-sources?
This problem is caused by the twisted 2.0.0 packages being unmasked. In
twisted 2.0.0 the package has been split into a number of smaller packages. I
have not tested with 2.0.0 at the moment. The simple solution is to change the
sys-apps/xen/xen-2.0.5.ebuild dependency from >=dev-python/twisted-1.3.0 to
=dev-python/twisted-1.3.0 so that only the 1.3.0 package meets the dependency.
Created an attachment (id=59355) [details]
/app-emulation/xen/xen-2.0.5.ebuild
Based on Edward Middleton's latest ebuild (57231).
Changes:
* Added pkg_postinst() instructions describing where to look for help and what
to do next to get Xen up and running.
* Specified a specific version of dev-python/twisted (1.3.0) since 2.0.0 seems
to break Xen.
* Put needed script files in /xen/files/${PV}/ directory so that version bumps
will be easy (recommended method from Gentoo Ebuild HOWTO).
* Suggesting it be put into portage in /app-emulation/ (actually, I'm torn
about this as it doesn't feel quite right; maybe we should create an
/app-virtualization/ :) )
Required ./files:
/xen/files/xend-init
/xen/files/xend-conf
/xen/files/xendomains-conf
/xen/files/xendomains-init
Formatted, documented, and ready for Portage.
-- Travis Cross
I have successfully installed, uninstalled, and reinstalled the Xen virtual
machine monitor, the Xend daemon, and the Xen dom0 and domU kernels
(xen-sources) on two separate machines (an IBM e-Server (P4) and a custom Asus /
Athlon-XP box) using the ebuilds provided here.
This bug has gone on for awhile so here is a recap of what needs to go into portage:
app-emulation/xen/xen-2.0.5.ebuild #59355
app-emulation/xen/files/2.0.5/xend-conf #57187
app-emulation/xen/files/2.0.5/xend-init #57234
app-emulation/xen/files/2.0.5/xendomains-conf #57233
app-emulation/xen/files/2.0.5/xendomains-init #57444
sys-kernel/xen-sources/xen-sources-2.6.10.ebuild #53759
portage/app-virtualization would be a good category to add now as there will be
a lot more of this kind of thing coming down the pipe, especially as Intel and
AMD are adding features to their chips to support this. Obviously app-emulation
is a decent second choice.
<evangelism>
Xen has _a lot_ of buzz right now (and major backers). I think it would be wise
to get this into Portage asap. Not only are people looking for this, but lots
of people are writing about it, and are noting what distributions are in the
game. When Gentoo does get mentioned, it is noted that there are 'experimental
ebuilds on Bugzilla' - not the most inviting phrase to the new user, not to
mention that this bug is being linked as the 'go-to point'. This is what
Portage is for.
I'm a busy person, and I know that you (the devs) are as well, so I understand
that we get overloaded. This needs to be a priority, however. We should be
ahead of everyone in this game. Besides, these ebuilds are ready.
</evangelism>
-- Travis Cross
Thank you from the bottom of my (two-chambered) heart! I probably won't have
time to test this over the coming weekend, but a three-day weekend is coming up,
and Xen is high on my list.
Question: as an x86-specific package, isn't this always going to be ~x86?
Congratulations to all of those who've put in the hard work to get this going!!
For the moment, Xen is x86-only, but the core developers are already at work on
amd64, em64t, and ia64 ports, with plans for ports to ppc and arm. The 64-bit
ports are set to appear in version 3.0, which is scheduled to be released in July.
-- Travis
Well, I wanted to try out Xen this week and was both shocked and horrified that
it isn't in portage yet :-O
May I humbly suggest that one of the people who contributed to this ebuild,
knows about Xen, and has a little free time, step up and volunteer to become a
dev and maintain it?
Created an attachment (id=59688) [details]
xen-2.0.6.ebuild
Version bump.
Also fixed make install for the python tools, later versions of Xen require
XEN_PYTHON_NATIVE_INSTALL=1, otherwise they get installed to /usr/lib/python.
xen-sources revision bump. added ebuild for 2.4 kernel.
- inherited kernel-2.eclass instead of kernel.eclass
- set SRC_URI with KERNEL_URI.
- Removed SLOT=0, since kernel sources get slotted from eclass.
- updated XEN_V to 2.0.6
Created an attachment (id=59703) [details]
xen0-sources-2.6.11.10.ebuild
Reworked ebuild to include support for 2.6.11.10, as well as other kernel
patches included with Xen 2.0.6.
(ebuild could do with some checking though)
I had problems compiling xen 2.0.6 with "-fprefetch-loop-arrays". Shall I post
the error message here or open a new bug?
Next problem comes up, when it builds the docs:
################# cut ############################
make: Entering directory `/var/tmp/portage/xen-2.0.6/work/xen-2.0/docs'
latex src/interface.tex >/dev/null
latex src/user.tex >/dev/null
if [ -e interface.toc ] ; then latex src/interface.tex >/dev/null ; fi
if [ -e user.toc ] ; then latex src/user.tex >/dev/null ; fi
make[1]: Entering directory `/var/tmp/portage/xen-2.0.6/work/xen-2.0/docs'
install -d -m0755 html/interface
latex2html -split 0 -show_section_numbers -toc_depth 3 -nonavigation \
-numbered_footnotes -local_icons -noinfo -math -dir html/interface \
src/interface.tex 1>/dev/null 2>/dev/null
install -d -m0755 ps
dvips -Ppdf -G0 -o ps/interface.ps.new interface.dvi
make[1]: *** [html/interface/index.html] Error 126
make[1]: Leaving directory `/var/tmp/portage/xen-2.0.6/work/xen-2.0/docs'
make: *** [html] Error 2
make: *** Waiting for unfinished jobs....
This is dvips(k) 5.92b Copyright 2002 Radical Eye Software (www.radicaleye.com)
' TeX output 2005.05.24:2018' -> ps/interface.ps.new
<tex.pro><alt-rule.pro><texc.pro><8r.enc><aae443f0.enc><bbad153f.enc>
<f7b6d320.enc><texps.pro><special.pro>. <cmmi8.pfb><cmr10.pfb><cmsy10.pfb>
<cmmi10.pfb>[1<figs/xenlogo.eps>] [2] [1] [2] [1] [2] [3] [4] [5] [6] [7] [8]
[9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23]
[24] [25] [26]
mv ps/interface.ps.new ps/interface.ps
make: *** Waiting for unfinished jobs....
rm user.dvi interface.dvi
make: Leaving directory `/var/tmp/portage/xen-2.0.6/work/xen-2.0/docs'
!!! ERROR: app-emulation/xen-2.0.6 failed.
!!! Function src_compile, Line 36, Exitcode 2
!!! compiling docs failed
!!! If you need support, post the topmost build error, NOT this status message.
################# cut ############################
I could reproduce this error with:
/var/tmp/portage/xen-2.0.6/work/xen-2.0 # make -C doc
[...]
same make error
[...]
Without the "doc" use flag everything builds fine :)
Created an attachment (id=59947) [details]
app-emulation/xen/files/2.0.6/xend-init
XenSV has been removed in 2.0.6 by upstream devs. (deactivated in
tools/Makefile)
conf.d/xen is not needed anymore, new init.d/xen attached.
(In reply to comment #91)
> Well, I wanted to try out Xen this week and was both shocked and horrified that
> it isn't in portage yet :-O
>
> May I humbly suggest that one of the people who contributed to this ebuild,
> knows about Xen, and has a little free time, step up and volunteer to become a
> dev and maintain it?
Have you received any reaction on this request? It seems current Gentoo developers are not
interested in / to busy for maintaining Xen. Are the contributors interested in becoming a
dev?
(In reply to comment #102)
> (In reply to comment #91)
> > Well, I wanted to try out Xen this week and was both shocked and horrified that
> > it isn't in portage yet :-O
> >
> > May I humbly suggest that one of the people who contributed to this ebuild,
> > knows about Xen, and has a little free time, step up and volunteer to become a
> > dev and maintain it?
>
> Have you received any reaction on this request? It seems current Gentoo
developers are not
> interested in / to busy for maintaining Xen. Are the contributors interested
in becoming a
> dev?
I believe the process for becoming a Gentoo developer, should someone wish to,
is documented at
http://www.gentoo.org/proj/en/devrel/index.xml
There is a developer recruiting effort as well. The lead for that is Deedra M.
Waters (dmwaters@gentoo.org).
BTW, I just rebuilt the machine that's the target for my Xen testing. I've got
Gentoo running on it and am in the process of getting the memory up from 256 MB
to a full GB. I got it up to 512 MB last night but for some reason the hardware
isn't seeing the second DIMM.
Am I stepping on any dev toes if I work on this?
vmware (not opesource) is in the portage tree and the only opensource
equivalent
xen is not there. comme on.
If you have trouble booting into the kernel and the boot process fails with
"cannot open root device" or similar, make sure that you have
CONFIG_PARTITION_ADVANCED=y
CONFIG_BLK_DEV_IDEDISK=y
CONFIG_BLK_DEV_IDEDISK=y
set in your .config
(I only found this after days of searching the web)
I know this is not really ebuild-related but as we have no central thread on the
forums I thought I appended this here.
OK, lets get this into the tree.
First step, there are a few ebuilds and such that are attached to this bug.
Lets, make sure that there is only one for each component (ie. kernel, utils, etc).
Second, the kernel ebuilds default to a domain0 config. Can we make it so that
domain0 or domainU is chosen via a USE flag?
Third, it would be better if we could derive off of the gentoo-sources kernel
rather than vanilla (lots of fixes, we have a great kernel team!). It should
probably be enough to apply the gentoo-base patches (we may want to avoid
gentoo-extras as it contains things like bootsplash, etc). However, we should
keep xen-sources as close to the gentoo-sources ebuilds as possible (for maint.
reasons).
Fourth, before they go into the tree, some good documentation needs to be
written. I notice a howto has been started (good job so far!). What we really
need is a document which explains the *concepts* as well as a brief howto. For
instance, "what is different between dom0 and domU?" "Do domains > 0 write
directly to disk or to disk image?" and most important "How do you install
gentoo in a domain > 0?"
I'm really appreciative of everyone's work so far. Lets tackle these things and
I think we'll be in great shape!
(In reply to comment #109)
> First step, there are a few ebuilds and such that are attached to this bug.
> Lets, make sure that there is only one for each component (ie. kernel, utils,
etc).
BTW, make sure that the ebuilds are merged, not just old versions removed.
#109
Xen sources are really just a patched in new archetecture to the base linux
kernel. It would be better to have the xen-sources as the xen distributed
sources (like the vanilla kernel sources) and then get a patch into the gentoo
sources to support the xen archetecture (possiably with a xen use flag). If you
base the Xen sources off gentoo you will end up with total confusion about what
is being installed and potentially lots of bug reports to xen mailing list about
problems which are caused by gentoo patches.
This is why I suggested applying the gentoo-base vs gentoo-base +
gentoo-extras.
gentoo-base is really just basic hardware fixes, backports and such. I'm
definately not suggesting that we add a xen use flag to the gentoo-sources.
I'm
saying we should have a separate xen-sources package, but that that package
should apply the basic fixes from gentoo-base.
#109 first point
The kernel sources are the same for both domain0 or domainU. The only
difference is the .config file. As all users are going to want to have both
types of domains a use flag might not be the best way to select the config file.
Perhaps we should just have it set to the domain0 dofault config because
kernels compile with this config should work in both domain0 and domainU.
On the topic of which kernel patches to apply:
The primary modifications made to the kernel by the Xen patches remove
privileged processor instructions and replace them with calls to the Xen Virtual
Machine Monitor. I have not studied the gentoo-base patches in detail, but I
would fear that applying even those might introduce the real possibility of new
and difficult to trap bugs.
At the very least, I doubt anyone on the xen-devel or xen-users lists would take
a concern or bug report seriously from a user without a 'clean' xen kernel.
Maybe the appropriate solution is to add a 'gentoo-base-patches' USE flag to the
xen-sources ebuild. I believe '-gentoo-base-patches' (off) would be the
appropriate default.
Running a kernel under Xen is fundamentally different than running 'on the
metal' because the Xen Virtual Machine Monitor stands between the linux kernel
and the hardware. Therefore kernel interaction with processor(s), memory, and
the pci bus (including agp) must be handled the way the VMM expects, or the
kernel will not get what it expects, and will likely seg fault.
If there is an understanding that processor and system bus related changes stay
out of the gentoo-base patches (or an awareness in the kernel team of the
requirements of Xen), then it probably would not be an issue. But otherwise, I
would personally feel more comfortable with my servers running on the 'clean'
xen sources.
Cheers,
-- Travis
ok, I just spent a few minutes looking through gentoo-base patches. Most of it
is patches against arches that xen doesn't support. The nightmare of weeding
those patches out and keeping the right patches is too much. I agree, lets do
just a vanilla + xen.
How about docs, anyone want to tackle that?
On the topic of documentation:
The Xen Users' Manual is a relatively good introduction to the concepts and the
'howto' of Xen:
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/readmes/user/user.html
The difficult part isn't getting Xen setup, it is getting everything else set up
around it.
When I get a spare moment, I will start on a more complete Gentoo+Xen HOWTO
which will include some guidance on best practices.
The challenge in putting together good documentation at this point is that Xen
is still changing rapidly.
As there are other good resources to learn about Xen, I do not believe that the
creation of gentoo-specific documentation should be an obstacle to getting this
package merged quickly.
I believe that the most important expectation of a gentoo user is that after
she learns about an application and takes an interest in it, that when she opens
a console and types 'cd /usr/portage ; locate xen' that she'll find something.
Cheers,
-- Travis
A couple reasons why I think docs should be a precursor in this case:
1. I won't put anything in the tree until I'm sure it works ok. I can't test
it right now as I don't have a free x86 box. However, I'm moving my webserver
to a ppc box this weekend/week and once that is done, I will test xen. This
gives us time to put some good docs together (and a good time for me to review
the docs when I am actually testing Xen).
2. Because of the complexity of xen (compared to just "emerge gnome"), some
good docs are definately in order.
3. As soon as this is in the tree, people will demand docs. Lets just have
them out of the gate.
4. Docs are in order exactly *because* xen is changing (people need up to
date info).
5. As any good programmer will tell you, docs are more important than code.
They should not be an afterthought.
Remember, if a package doesn't work out of the box with a simple "emerge foo",
then we need clear docs to explain how to get it working.
I'm not looking for something complicated. I think 4 basic sections are our
ultimate goal:
1. Explain Xen concepts
2. HOWTO Gentoo in domain 0
3. HOWTO Gentoo in domain >= 1
4. Tips/Tricks/Thoughts/Hacks/etc
This shouldn't take much time. Be creative and have fun. And, while it'd be
great to have complete docs out of the gate, I'd settle for less than that.
Just so long as we have *something*.
On the topic of getting down to the needed files only:
It seems the following attachments should be marked as obsolete:
#43894
#44963
#51453
#52423
#52424
#53759
#56536
#57231
-- Travis
#119:
Fair enough. I suppose we should take the opportunity to clear a wide trail for
other to follow.
-- Travis
also, it would probably be best to file a separate bug for the hwclock issue
against baselayout (particularly because I don't maintain it).
Created an attachment (id=60552) [details]
sys-kernel/xen-sources/xen-sources-2.6.11.ebuild (gentoo base patches)
I have serious reservations about going this way but, this is a build that will
use only the gentoo base patches. Quite a few of them are not really
applicable because they are for archetectures not supported by xen, and this
has had nothing more then 'a does it compile/boot' test.
#116 & #123
I share your reservations. Let's follow our intuition and mark #60552 as
obsolete, rolling back to #59694 ?
-- Travis
sorry to make you do extra work, we decided not to apply gentoo-base (for the
very reason you mentioned; see comment #116).
Created an attachment (id=60561) [details]
sys-kernel/xen-sources/xen-sources-2.6.11.ebuild
Some of the changes in the gentoo base patch should probably be kept
* moved xen source url to XEN_URI
* removed 0 from EXTRAVERSION because the source is not domain0 specific.
* removed the www.kernel.org url from HOMEPAGE is this link really needed?
* added a dependency for the specific xen version. We don't want to have to
debug problems caused by running old domains with new hypervisor and vis-versa.
* removed reference to domain0 in the DESCRIPTION and added the kernel and xen
version.
* renamed to xen-sources-2.6.11.10.ebuild because not domain specific.
If needed I can review, test and convert any documentation to GuideXML (for
those unfamiliar with that).
I am setting up a system now and will document it as I go. I intend to
document
how to setup a domain0,domain1 using a logical volume.
(In reply to comment #127)
> If needed I can review, test and convert any documentation to GuideXML (for
> those unfamiliar with that).
I'm sure that GDP wannabies (fox2mike, smithj) can do that, too, and with great
pleasure.
Created an attachment (id=60562) [details]
basic rough outline of the required documentation
I have to go now but this is what I have done sofare. This is a very rough
outline of the required documentation as suggested. It is a straight text
file. There is very little comments and a number of parts missing. I will try
and get back to this latter today.
xend-init needs "before dhcp" and xendomains-init needs "after dhcp" for xen0
domains that provide dhcp to its own guests.
I just started playing around with Xen today. All I've done so far is download
the original 2.0.6 source tarball and follow the instructions. Everything built
fine, although I had to put quite a few of the prerequisites in
/etc/portage/package.keywords; Xen wants some things more recent than "stable".
So far only two glitches:
1. Sound didn't come up. I think that the Xen kernel just needs to be rebuilt
with my sound card configured. I normally build a kernel with only the base
sound and emerge "alsa-driver".
2. Everything else, including X/gdm/KDE came up fine. But I couldn't get "xend"
to start. That appears to be caused by something in "twisted", since it's
complaining about Python. I emerged the latest version of "twisted"; Xen may be
more picky than that. I'm going to try using the version of "twisted" that comes
with Xen.
Once I get past these two hurdles, I'll attempt to build another VM on my other
hard drive and put CentOS or Fedora (or maybe Sarge) on it. This box only has
512 MB, so I'm not sure it can handle more than two VMs but it should be able to
deal with two, since it was running single ones in 256 MB a week ago.
Incidentally, if you build the whole works ("make world") from the Xen source,
it downloads a vanilla kernel, patches it, configures it and builds it. Then
when you do "make install", it places everything in "/boot". All you need to do
is edit "grub.conf" and reboot. Would it be simpler to make an ebuild using that
strategy?
Most people will need to configure their xen kernels for their hardware so it
makes sense having a xen-sources package. Try using Twisted version 1.3.0.
(In reply to comment #133)
> Most people will need to configure their xen kernels for their hardware so it
> makes sense having a xen-sources package. Try using Twisted version 1.3.0.
Yeah ... it took me a while to figure out how to configure and rebuild the
kernel in the source distribution. And the build croaked on one of the ISA sound
drivers. I will probably reboot in about an hour; if that works, I'll test the
ebuilds, etc., tonight.
(In reply to comment #134)
> (In reply to comment #133)
> > Most people will need to configure their xen kernels for their hardware so it
> > makes sense having a xen-sources package. Try using Twisted version 1.3.0.
>
> Yeah ... it took me a while to figure out how to configure and rebuild the
> kernel in the source distribution. And the build croaked on one of the ISA sound
> drivers. I will probably reboot in about an hour; if that works, I'll test the
> ebuilds, etc., tonight.
Sound works ... xend works (with Twisted 1.3) ... life is good :). This is
posted from Gentoo running from Xen.
Created an attachment (id=60746) [details]
xen-lvm
I've automated most of the LVM setup for guests. The script is attached, along
with any other (but not all) notes I've collected. It would be very useful to
anyone totally clueless about how to start. It will probably be useful to
anyone writing docs.
Considering this ebuild works can it be put in Portage? Masked of course, so
any
comments can be made...
I know I'm perhaps a bit impatient but this bug exists for almost 7 months with
great interest of the community. If it works documentation will be created by
users.
(In reply to comment #138)
> Created an attachment (id=61091) [edit] [details]
> patch for /etc/init.d/clock to skip hwclock in xenU
>
> I would open a bug for baselayout, but since xen still isn't in portage (!!!)
> there's no point.
You can still open a new bug for baselayout (Add it as a blocker to this bug). I
don't think the maintainers of baselayout would deny the patch soley because
this ebuild isn't in portage yet. They'll *should* understand they need to fix
baselayout before zen works but there's no reason to add it to portage if it
doesn't work yet (Damm chicken! Errr.. egg!)
I am currently working on documentation along the lines of what Nathaniel
McCallum described in #119, sorry for the delay. There should be official
documentation before this is put in the tree. If you are happy to install it
without official documentation then you are free to download the ebuild from
here as many people allready have.
(In reply to comment #141)
> There should be official documentation before this is put in the tree.
I'm quite sure that GDP won't add any documents concerning unofficial ebuilds.
Would anyone mind putting up a .tgz of the required files for portage on the
wild web since the devs apparently aren't going to add this to portage? :)
Or just email me the files and i'll put them up.
Created an attachment (id=61951) [details]
tarball: portage tree for xen and conf-files
The attachment contains the /usr/local/portage tree and the entries you have to
add to /etc/portage/package.keywords and /etc/make.conf.
Created an attachment (id=62232) [details]
xml source file for xen guide
This is the source file for a gentoo xen install guide I have been writting.
It still needs some work but feedback would be good. I will not have internet
access until the 7/6 so don't expect a response before then.
(In reply to comment #145)
> This is the source file for a gentoo xen install guide I have been writting.
> It still needs some work but feedback would be good. I will not have
internet
> access until the 7/6 so don't expect a response before then.
I'd recommend creating another bugreport for Xen documentation as it is closely
related to the GDP.
I'd have several comments about coding style (no tabs, better separation of
<p>s etc, line width and usage of <c> for emphasis, capital letters,...), but
here are the "real" ones:
a) I'd explain what does "hypervisor" mean before first usage of this word
b) don't mention "filesystem" as a part of Xen domain; AFAIK Xen itself (the
hypervisor) doesn't care about guest domain's kernel internals
c) I don't see a reason for domain0 to be compiled with "blockdev/network
frontend support" - is't the backend enough?
d) "Scrub memory before freeing it to Xen" is not required, AFAIK.
e) "generic/default IDE chipset support" is not needed if you select your IDE
chipset driver, AFAIK
f) `cp System.map /boot/System.map-2.6.11.10-xen` - IMHO dsd has something to
say about this point :-)
g) "domain0 kernel" is *not* "booted", it is "loaded" by hypervisor
h) don't duplicate stuff from Handbook (network configuration etc), maintaining
twice the amount of text is PITA
i) "Start the hypervisor access daemon and add it to the default domain." -
hypervisor is already running, you're only starting one of the possible
management interfaces (iirc, it has been soem time since I stopped reading xen
ML)
j) TLS can't be disabled by USE=-tls, that USE flag is called "nptl". But you
should probably ask glibc maintainer...
k) `mv /lib/tls /lib/tls.disable` is ugly hack
I know, this issue has been raised a thousand times in other packages.
Some CFlags need to be filtered, when you use hardened.
make[1]: Entering directory
`/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen/common'
gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix
include -Wall -Werror -pipe
-I/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen/include -Wno-pointer-arith
-Wredundant-decls -O3 -fomit-frame-pointer -msoft-float -m32 -march=i686
-DNDEBUG -c ac_timer.c -o ac_timer.o
gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix
include -Wall -Werror -pipe
-I/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen/include -Wno-pointer-arith
-Wredundant-decls -O3 -fomit-frame-pointer -msoft-float -m32 -march=i686
-DNDEBUG -c dom0_ops.c -o dom0_ops.o
gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix
include -Wall -Werror -pipe
-I/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen/include -Wno-pointer-arith
-Wredundant-decls -O3 -fomit-frame-pointer -msoft-float -m32 -march=i686
-DNDEBUG -c dom_mem_ops.c -o dom_mem_ops.o
dom_mem_ops.c: In function `do_dom_mem_op':
/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen/include/asm/mm.h:160: error: can't
find a register in class `BREG' while reloading `asm'
make[1]: *** [dom_mem_ops.o] Error 1
make[1]: Leaving directory `/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen/common'
make: *** [/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen/xen] Error 2
make: Leaving directory `/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen'
!!! ERROR: app-emulation/xen-2.0.6 failed.
!!! Function src_compile, Line 32, Exitcode 2
!!! compiling xen failed
!!! If you need support, post the topmost build error, NOT this status message.
(In reply to comment #148)
> I know, this issue has been raised a thousand times in other packages.
>
> Some CFlags need to be filtered, when you use hardened.
>
> make[1]: Entering directory
Here's a simple "fix" unless you figured it out yourself.
# emerge xen
When you see "Source unpacked", hit ^Z and edit /var/tmp/portage/xen-2.0.6/work/
xen-2.0/xen/arch/x86/Rules.mk
Change the line
CFLAGS := -nostdinc -fno-builtin -fno-common -fno-strict-aliasing
to
CFLAGS := -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -nopie -
fno-stack-protector -fno-stack-protector-all
..and save.
Let emerge continue by putting the process into the foreground again
# fg
Someone ought to come up with a real patch for those of us running GCC hardened.
Xen 2.0.7 released.
Anyone care to update the tarball?
"Although everyone is working flat out to get Xen 3.0 ready, we thought
it was about time we released a Xen 2.0.7 update as there have been a
bunch of fixes queued up in the testing tree for a while. For example:
* 'noirqbalance' parameter to workaround buggy ES7000 chipsets
* vm relocation tools fixes
* fix fp bug that was triggered by certain user kernel configs
* fix for the console warning when udp_poll() receives fragmented
skbuffs
Upgrading is probably not critical for most people, but is advisable.
Best,
Ian"
Feedback on the current tarball (rename the ebuild works to bump it to 2.07).
1) The message about mv /lib/tls /lib/tls.disabled.. I moved it and still get
the message. Normal?
2) TLS glibc patch. Any plans to add it to gentoo glibc?
3) Docs. Most people don't use LVM - should mention files etc. (could copy
straight from the config section of http://julien.danjou.info/xen.html)
4) Docs. Mentions tls use flag - does this flag actually exist for any packages?
It's not in use.desc. If a rebuild is necessary mention emerge --newuse.
5) /etc/init.d/xend is odd. There's no output. Added to default runlevel doesn't
seem to do anything.. I still have to manually do '/etc/init.d/xend start'
otherwise I get an error about the server not running.
5.5) rc-status always shows xend service not running.
6) Kernel config section should include bridge module (essential), and maybe
netfilter.
7) xend-config file mentioned in the xen docs is missing. Maybe include an
example if it's not necessary.
8) DEPEND on iptables missing. /etc/xen/scripts/vif-bridge calls 'iptables'.
Has there been any progress on getting xen into portage?
(In reply to comment #151)
> 6) Kernel config section should include bridge module (essential), and maybe
> netfilter.
You don't have to do bridging, you can use normal IPv4 routing.
(In reply to comment #151)
> 2) TLS glibc patch. Any plans to add it to gentoo glibc?
That one already works (there was a minor issue [Bug #99896] with flag-o-matic
strip-flags, but that has been resolved):
- install gcc-3.4.4 (or newer), make it the default compiler.
- add "-mno-tls-direct-seg-refs" to your CFLAGS
- recompile glibc (>=3.3.5) with USE=nptl
- recompile all libraries and apps broken by the compiler update (most C++
libs/apps)
glibc Makefiles recognise the -mno-tls-direct-seg-refs flag, set some #defines,
and use modified asm code for tls access then...
works like a charm here.
> 1) The message about mv /lib/tls /lib/tls.disabled.. I moved it and still get
the message. Normal?
Yes, since some apps access memory in a way similar to glibc without patch. xen
doesn't distinguish that... the performance impact for them is quite low, so
nothing to worry about. And recompiling those apps with
-mno-tls-direct-seg-refs should fix the problem.
I've commited ebuilds. I changed them slightly - xen now installs the sparse
source tree which is picked up later by xen-sources. Not exactly ideal but
there's no good way of providing for different versions of xen with the same
kernel version, and sticking to the gentoo ebuild naming scheme. I've added a
pre3 ebuild with kernel 2.6.12.5 as well. Seems to work well here.
I'll leave this bug open for the inevitable bug reports and stuff I missed :)
Thanks to everyone who contributed!
oh and http://gentoo-wiki.com/HOWTO_Xen_and_Gentoo ... though if someone wants
to maintain official docs later that would be cool too.
The ebuild is broken. It is performing the steps for building the sparse tree
in the incorrect order. During the src_install step the following error appears
--------begin----------
mkdir: cannot create directory
`linux-2.6-xen-sparse/include/asm-xen/xen-public': No such file or directory
cp: `linux-2.6-xen-sparse/include/asm-xen/xen-public': specified destination
directory does not exist
Try `cp --help' for more information.
./
---------end----------
This is occurring because the ebuild is incorrectly assuming that the
"linux-2.6-xen-sparse" directory exists when performing a mkdir and copying some
files. I am attaching a patch that should fix the problem.
I apologize, my mozilla seems to be broken. I was unable to use the browse
button trying to attach the patch. The patch follows:
--- xen-2.0.7.ebuild 2005-09-07 12:18:59.000000000 -0600
+++ xen-2.0.7.ebuild-fix 2005-09-08 12:16:01.000000000 -0600
@@ -74,10 +74,10 @@
# we need to do whatever mkbuildtree would've done for each platform
# linux-2.6: copy public include files, and xenstored.h
- mkdir linux-2.6-xen-sparse/include/asm-xen/xen-public
+ mkdir linux-2.6.11-xen-sparse/include/asm-xen/xen-public
# include files for kernel
rm xen/include/public/COPYING
- cp -dpPR xen/include/public/*
linux-2.6-xen-sparse/include/asm-xen/xen-public
+ cp -dpPR xen/include/public/*
linux-2.6.11-xen-sparse/include/asm-xen/xen-public
# fixme: insert code for other sparse trees here
# install xen kernel sparse trees
(In reply to comment #159)
> http://gentoo-wiki.com/HOWTO_Xen_and_Gentoo isn't that good. It just broke my
system (the "Before
> we begin"-Stuff). Someone who knows what this section is supposed to do should
rewrite/repair it.
Yeah, that was a seriously stupid idea to unmerge old gcc-3.3.x version before
rebuilding world with the new (gcc-3.4.x) one. I changed the order now, no idea
who did put that into wiki. Note, that libstdc++-v3 may be still needed by some
binary-only stuff, in that case you should not put it into package.provided.
Ken - bug fixed, thanks for spotting it.
Phil - sorry, my mistake, was typing from memory. :-(
/etc/xen/scripts/network-bridge relies on ifup and ifdown scripts from redhat,
therefore bridging isn't going to work on gentoo.
It works for me. I haven't looked into why exactly why it works despite
ifup/ifdown failing, maybe the interface is already up by default, or something
else brings it up.
re:
http://lxr.xensource.com/lxr-unstable/source/examples/network-bridge?v=tools-xen-unstable
In op_start() there is a fallback for ifdown:
180 if ! ifdown ${netdev} ; then
181 # if ifup didn't work, see if we have an ip= on cmd line
182 if egrep 'ip=[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+:' /proc/cmdline ;
183 then
184 kip=`sed -e
's!.*ip=\([0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+\):.*!\1!' /proc/cmdline`
185 kmask=`sed -e
's!.*ip=[^:]*:[^:]*:[^:]*:\([^:]*\):.*!\1!' /proc/cmdline`
186 kgate=`sed -e 's!.*ip=[^:]*:[^:]*:\([^:]*\):.*!\1!'
/proc/cmdline`
187 ifconfig ${netdev} 0.0.0.0 down
188 fi
189 fi
and ifup:
200 if ! ifup ${netdev} ; then
201 if [ ${kip} ] ; then
202 # use the addresses we grocked from /proc/cmdline
203 ifconfig ${netdev} ${kip}
204 [ ${kmask} ] && ifconfig ${netdev} netmask ${kmask}
205 ifconfig ${netdev} up
206 [ ${kgate} ] && ip route add default via ${kgate}
207 fi
208 fi
So maybe these are sufficient, however the ifup in op_stop() should mean that
you lose eth0 when stopping a xen machine because there is no fallback to
"ifconfig ${netdev} up"?
241 ifup eth0
xen wont compile:
>>> md5 files ;-) xen-3.0.0_pre20050929.ebuild
>>> md5 files ;-) xen-2.0.7.ebuild
>>> md5 files ;-) files/digest-xen-3.0.0_pre20050929
>>> md5 files ;-) files/xendomains-conf
>>> md5 files ;-) files/xendomains-init
>>> md5 files ;-) files/xend-conf
>>> md5 files ;-) files/xend-init
>>> md5 files ;-) files/digest-xen-2.0.7
>>> md5 src_uri ;-) xen-unstable-20050929.tar.bz2
[...]
gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include
-Wall -Werror -Wno-pointer-arith -pipe
-I/var/tmp/portage/xen-3.0.0_pre20050929/work/xen-unstable-20050929/xen/include
-I/var/tmp/portage/xen-3.0.0_pre20050929/work/xen-unstable-20050929/xen/include/asm-x86/mach-generic
-I/var/tmp/portage/xen-3.0.0_pre20050929/work/xen-unstable-20050929/xen/include/asm-x86/mach-default
-O3 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -DVERBOSE -c
grant_table.c -o grant_table.o
grant_table.c: In function `gnttab_transfer':
grant_table.c:760: error: can't find a register in class `BREG' while reloading
`asm'
make[1]: *** [grant_table.o] Error 1
make[1]: Leaving directory
`/var/tmp/portage/xen-3.0.0_pre20050929/work/xen-unstable-20050929/xen/common'
make: ***
[/var/tmp/portage/xen-3.0.0_pre20050929/work/xen-unstable-20050929/xen/xen] Error 2
make: Leaving directory
`/var/tmp/portage/xen-3.0.0_pre20050929/work/xen-unstable-20050929/xen'
Reopen this bug to re-assign to maintainer.
Additionally, I'd really recommend the normal course of solving bugs - i.e.,
open a *new* bug for *new* issues, dumping all the problems here is just a
terrible mess and is causing tons of bugspam.
Closing. Please re-file any existing bugs. The only things I am aware of is
xen3
requiring udev rules as it seems to break with udevsend, and the ifup/ifdown
thing; I'll fix these in the ebuild soon. The compile problem looks like a gcc
bug.
