|
Description
Stefan de Konink
2004-11-05 06:28:49 UTC
'We' should be read as 'They'. Are you requesting an ebuild, or do you already have one to share? Best regards, Stu Requesting, to download the latest sources bitkeeper tools are needed. Plus, XEN start downloading a kernel which is allready found on most systems in distfiles. I've got some almost-working ebuilds to build and install Xen (tentatively named sys-apps/xen and sys-kernels/xen-sources); if nobody else is already working on this, I should (hopefully) be able to finish them to a generally-working state in the next several days. I've uploaded my current attempts to http://people.pwf.cam.ac.uk/pjt47/xentoo/ - I wouldn't be surprised if I've done various things wrongly, but it appears to work for me (as far as booting Gentoo using a XenLinux kernel and running xend; I haven't tried actually running a virtual Gentoo machine). Current problems include being limited to the 2.6 kernel, not handling unprivileged domains (unless you manually use a different default kernel config file), confusing etc-update by putting binary files in /etc (since I didn't want to automatically mount /boot, and wasn't sure where else to put them), and possibly other things that I've not noticed or remembered. To build the docs, tgif (see bug 36185) is required. inherit mount-boot should sort out the auto mounting of /boot for the ebuild. Aha, thanks - I've updated it to use mount-boot and copy the files directly into /boot. (Updated ebuilds at http://people.pwf.cam.ac.uk/pjt47/xentoo/ again.) I'm not sure what the most convenient way of installing an unprivileged kernel is; I currently provide manual instructions to cp /usr/src/linux-2.6.9-xen0 to linux-2.6.9-xenU and then to switch the default config file. It seems a little wasteful to devote half a gigabyte to two identical copies of the kernel sources, but I don't know what else to do. Use flag (xen0only)? Seperate package? I'd say unpack to two different directories in this ebuild. A user who is low on space could always unmerge the sources once they've built the two kernels they require? Created attachment 43893 [details]
xen-2.0.ebuild
clean up of the app-emulation/xen/xen-2.0.ebuild by Philip Taylor
Created attachment 43894 [details]
xend.rc
Gentoo-ified version of the /etc/init.d/xen script
if someone can create a Gentoo-ified version of the xendomains init.d script, i'll add this package to portage ... Created attachment 43934 [details, diff]
`rm -f` instead of `rm`
this is not good :-)
rm: remove write-protected regular file
`/var/tmp/portage/xen-sources-2.6.9/work/linux-2.6.9-xen0/mkbuildtree'?
xen looks very promising, btw the developers have released version 2.0.1, is anyone able to create a Gentoo-ified version of the xendomains init.d script so that this package can be added to the tree? Created attachment 44961 [details]
xen-2.0.1.ebuild
Have added the cam.ac.uk mirror as the sourceforge mirrors do not carry this
file yet.
Comment on attachment 44961 [details]
xen-2.0.1.ebuild
Version bump to 2.0.1.
Created attachment 44962 [details]
xen-2.0.1.ebuild
Tested version :)
Added the cam.ac.uk mirror as sourceforge does not seem to have it on the
mirrors at present, and one of the mirrors hung for me. Added the cam.ac.uk
webpage also.
Had to use some 'trickery' to get it to compile, as the package is zipped in a
xen-2.0 directory, so I have a mv command. Hopefully this is a fair way to do
it, and should also mean that it is easy to update just by copying and
remnaming the ebuild as the version is bumped.
Will look at the init.d script to see if I can Gentoo-ify it so it can be
merged mainstream!
Created attachment 44963 [details]
xendomains
Gentoo-ified xendomains script.
There is plenty more that could be done to this script - restart and reload are
commented out from the original script.
Haven't played with Xen too much yet, but wondering if it might be worth
setting individual scripts for each domain, cf net.* scripts (eg xen.dom0,
xen.dom1 etc)
Created attachment 44965 [details]
xend
/etc/init.d/xend script, modified to represent that it is starting dom0 as
opposed to Xen in general. Should help remove confusion during boot with the
xenddomains script.
> /etc/init.d/xend script, modified to represent that it is starting dom0 as
opposed to Xen in general
you can't start dom0 as it is started by xen when it boots. stopping dom0 means halting the machine. xend is afaik xen control daemon for performing management tasks like starting/stopping other domains, resource usage control,...
and you'll need another daemon for live migration support.
Good point. Now that I look closer at the script (Why didn't I read it earlier) it is for starting the Control Daemon for the domains. Is it worth modifying the xend script for that? you'll have to compile glibc without nptl in USE flags, otherwise you'll get this warning: *************************************************************** *************************************************************** ** WARNING: Currently emulating unsupported memory accesses ** ** in /lib/tls libraries. The emulation is very ** ** slow, and may not work correctly with all ** ** programs (e.g., some may 'Segmentation fault'). ** ** TO ENSURE FULL PERFORMANCE AND CORRECT FUNCTION, ** ** YOU MUST EXECUTE THE FOLLOWING AS ROOT: ** ** mv /lib/tls /lib/tls.disabled ** *************************************************************** *************************************************************** despite what it says, /lib/tls doesn't exist on my system, but see this message from Ian Pratt (Rik is probably Rik van Riel from Red Hat): NPTL is supported, but we're forced to take traps for almost every attempt to access thread local storage, which is slow (I think Rik said 12-15% on a kernel compile). The only way to improve on this would be to have our own very slightly modified version of libc that could be installed to restore full performance. We're hoping to look into this at some point. Disabling NPTL works for most people. Ian ok, then if i understand this correctly, the xen init.d scripts are really only meant to be used from inside of xen ? if so, i'll have the ebuild stick them in /usr/share/xen/ instead of /etc/init.d/ ... It depends what you think "inside of xen" is. You need two types of a kernel; a host kernel (Xen calls this domain 0) and guest kernel (you can assign any number/name to this domain). When you boot your system with a Xen host kernel you are already inside Xen (domain 0), however to start a guest kernel you need to have xend running. I thinks /etc/init.d is the correct location for both scripts. ok, but the only files that should go into /etc/init.d/ are scripts that have been 'gentoo-ified' yep, only two scripts need to go to /etc/init.d; xend (to start/stop the xen-interface) and xendomains (to start/stop the quest systems during boot/shutdown). if you use xen, you need at least two things - xen kernel itself and patched linux kernel (aka "domain0" in xen terminology). these two thing will provide (almost) the same services as normal vanilla linux kernel, plus some xen stuff, so you can run any distribution (including gentoo :-)) on the top of such system. but this setup is useless if you don't run any virtual domains (you don't have to bother with xen just to run "one instance" of OS), so you'll probably want to create another virtual machines. in order to do it, you'll need xend running inside of domain0 (`/etc/init.d/xend start` is a good way, it is only shell script to actually start xend daemon) and have prepared special kernel (linux, netbsd, freebsd, winxp (well, not available due to licensing issues)) and root fs. then you can start any number of other "machines", or, in Xen terminoology, domains. as you probably want to start all (or some of) these virtual machines at bootup, xen has provided the "xendomains" init script, it looks into /etc/xen/auto/ and starts domains specified there. Michiel is right. Hi All, Thanks for the ebuild. Testing it now. I got the following error: cc1: warning: -fprefetch-loop-arrays not supported for this target (try -march switches) I remember that it's possible to disable certain useflags and cflags for ebuilds (but forgot how). Might be wise to disable this one ? Grtz Ramon Just wanted to let you know that version 2.0.2 has been released. Just wanted to let you know that version 2.0.3 has been released. Question for the devs: I don't want to be the impatient user but isn't it time to put these ebuilds in Portage? There are a lot of people interested if you look at the CC list but almost nothing happens here. I fear that people willing to try Xen might jump over to RHEL or SuSE. Just my 2 euro Question for the devs: I don't want to be the impatient user but isn't it time to put these ebuilds in Portage? There are a lot of people interested if you look at the CC list but almost nothing happens here. I fear that people willing to try Xen might jump over to RHEL or SuSE. Just my 2 euroçents of course, let's not start a flamewar on Gentoo VS ... please. I don't know. I would think not. I have started to use those ebuilds, but I'm ending up righting an ebuild of my own because these ones had hickups. A link to this bug from the forums or elsewhere would help much more in getting this tested and find a proper implementation for it. Then release it to Portage, unless it worked out of the box for everybody elses?!? Ok, what is everybody's general status about this? I have succesfuly builded and implemented Xen on a 2.4 kernel box here. There's still issues to get this done properly thrue a ebuild, but I think I have a sufficient basic grasp of this to generate an ebuild. The biggest issue I had was how to make it download a 2.4 or a 2.6 kernel. As such, I think patching the kernels upstream would be a better approach (I'd even hope it would be directly included into the kernels at www.kernel.org). Because of that, i had problems integrating my .config inside xen0. But I think I can manage it on my next iteration. My other issue is the fact the the vanilla Xen script will install the xenU kernel into /boot. This is somewhet inappropriate because then it requires you to mount it to access the kernel. As a simplification, I simply made the configuration file point directly at the bzImage inside the kernel's source hierarchy (arch/xen/boot/bzImage). Another issue I encounter is the fact that xenU cannot absolutely not work properly if started with devfs. So this caused problems when using Gentoo as a guest, which will complain if devfs is not enabled. Insights about this would be appreciated. I guess I should try to guest something else like Knoppix. What about you guys? Where are you at with this? >Another issue I encounter is the fact that xenU cannot absolutely not work properly if started with devfs. So this caused problems when using Gentoo as a guest, which will complain if devfs is not enabled. Insights about this would be appreciated.
emerge udev
devfs at boot is a requirement of Gentoo. In a Xen point of view, udev would probably cause the same issue. A Xen booted guest seems to need static device nodes in /dev. On monday, I will extract the build steps from the Makefiles to incoporate them directly in the ebuild instead. This will give the flexibility to use a distinct xen patched kernel, instead of having to use "make kernel-bla" from xen. So, I guess that by the end of next week, I'll have a ebuild to submit here. I mainly wanted to know if others did work on this and avoid duplicating stuffs. udev is supported as well, as a mather of fact it will be the default very soon for architectures that support it, same goes for the 2.6 kernel which you need when using udev, we use it on all our Xen-servers, I suggest you take a look at Philip's webpage, he already made an ebuild for the Xen-sources. We need to put this on 2.4 systems. So the limitation is either because of Gentoo's inhability to run with a static /dev, or xen's inhability to run with a dynamic one. I'm doing my second iteration install to check if the ebuilds works well, and I always seem to have issues with the xenU kernel. I'm unsure if xen is what we really want. For now, I'll admit I way prefer the vmware paradigm of virtualization. I mean, shouldnt it be the guest kernel who should get xenified, instead of having a bunch of xenU kernels that are completely unrelated to the guest kernel (that could have various patches applied). I don't get it :( Created attachment 50718 [details]
xen-2.0.4.ebuild
I wasn't able to get the ebuild in #37 to go. First it would not find the file to download. I manually downloaded it from http://www.cl.cam.ac.uk and created the digest. When I went to install it I received the following error. >>> emerge (1 of 1) sys-kernel/xen-2.0.4 to / >>> md5 src_uri ;-) xen-2.0.4-src.tgz >>> Unpacking source... >>> Unpacking xen-2.0.4-src.tgz to /var/tmp/portage/xen-2.0.4/work /usr/lib/portage/bin/ebuild.sh: line 33: cd: /var/tmp/portage/xen-2.0.4/work/xen-2.0.4: No such file or directory sed: can't read tools/libxc/Makefile: No such file or directory sed: can't read tools/libxutil/Makefile: No such file or directory sed: can't read tools/misc/miniterm/Makefile: No such file or directory sed: can't read tools/misc/nsplitd/Makefile: No such file or directory sed: can't read tools/misc/Makefile: No such file or directory sed: can't read tools/xentrace/Makefile: No such file or directory sed: can't read xen/arch/x86/Rules.mk: No such file or directory !!! ERROR: sys-kernel/xen-2.0.4 failed. !!! Function src_unpack, Line 39, Exitcode 2 !!! sed cflags Apparently, ${S} and the contents of the tgz don't agree. Comment on attachment 50718 [details]
xen-2.0.4.ebuild
the ebuild looks dos encoded btw
Created attachment 51453 [details]
another ebuild
That isn't really a nice solution but at least build and install
when trying the latest ebuild, I get the following error. as I posted on the xen mailing list about an earlier version of this ebuild, I suspect the problem is due to makefiles using $(DESTDIR} which is the getting wrong value from the ebuild environment: ACCESS DENIED mkdir: /usr/portage/distfiles/install install: cannot create directory `/usr/portage/distfiles/install': Permission denied unset DISTDIR is present just to address that problem tried the suggestion and the build process got a little further:
[ -d /usr/lib ] || install -d -m0755 -p /usr/lib
install -m0755 libxutil.so.2.0.0 /usr/lib
ACCESS DENIED open_wr: /usr/lib/libxutil.so.2.0.0
install: cannot create regular file `/usr/lib/libxutil.so.2.0.0': Permission denied
make[1]: *** [install] Error 1
make[1]: Leaving directory `/var/tmp/portage/xen-2.0.4/work/xen-2.0/tools/libxutil'
looked inside xen-2.0/tools/libxutil/Makefile and saw that it used $(DESTDIR) on installation. I set DESTDIR to ${WORKDIR} but I got the same failure. any suggestions for what I should try next?
Would it help if somebody (I) provide a rsync mirror for a portage overlay including xen (and tgif)? It seems that no dev wants that in the official tree at the moment... Created attachment 52423 [details]
xen-2.0.4-r1.ebuild
Hi, here is another ebuild with patch to solve DISTDIR/DESTDIR problem.
Created attachment 52424 [details, diff]
xen-2.0.4.patch
Just wanted to let you know that version 2.0.5 has been released. *** Bug 85211 has been marked as a duplicate of this bug. *** The build fails for me for some reason. I get these 'asm' errors.. make[2]: Entering directory `/mnt/data/Gentoo-Tmp/portage/xen-2.0.4-r1/work/xen-2.0/xen/common' gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include -Wall -Werror -pipe -I/mnt/data/Gentoo-Tmp/portage/xen-2.0.4-r1/work/xen-2.0/xen/include -Wno-pointer-arith -Wredundant-decls -O2 -march=pentium3 -fomit-frame-pointer -pipe -falign-functions=64 -mmmx -msse -msse2 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -c ac_timer.c -o ac_timer.o gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include -Wall -Werror -pipe -I/mnt/data/Gentoo-Tmp/portage/xen-2.0.4-r1/work/xen-2.0/xen/include -Wno-pointer-arith -Wredundant-decls -O2 -march=pentium3 -fomit-frame-pointer -pipe -falign-functions=64 -mmmx -msse -msse2 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -c dom0_ops.c -o dom0_ops.o gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include -Wall -Werror -pipe -I/mnt/data/Gentoo-Tmp/portage/xen-2.0.4-r1/work/xen-2.0/xen/include -Wno-pointer-arith -Wredundant-decls -O2 -march=pentium3 -fomit-frame-pointer -pipe -falign-functions=64 -mmmx -msse -msse2 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -c dom_mem_ops.c -o dom_mem_ops.o gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include -Wall -Werror -pipe -I/mnt/data/Gentoo-Tmp/portage/xen-2.0.4-r1/work/xen-2.0/xen/include -Wno-pointer-arith -Wredundant-decls -O2 -march=pentium3 -fomit-frame-pointer -pipe -falign-functions=64 -mmmx -msse -msse2 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -c domain.c -o domain.o gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include -Wall -Werror -pipe -I/mnt/data/Gentoo-Tmp/portage/xen-2.0.4-r1/work/xen-2.0/xen/include -Wno-pointer-arith -Wredundant-decls -O2 -march=pentium3 -fomit-frame-pointer -pipe -falign-functions=64 -mmmx -msse -msse2 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -c elf.c -o elf.o dom_mem_ops.c: In function `free_dom_mem': /mnt/data/Gentoo-Tmp/portage/xen-2.0.4-r1/work/xen-2.0/xen/include/asm/mm.h:160: error: can't find a register in class `BREG' while reloading `asm' Created attachment 53755 [details]
2.0.5 ebuild for hypervisor and dom0 daemons
Created attachment 53756 [details]
xend init.d file
Created attachment 53757 [details]
/etc/conf.d/xendomains file
Created attachment 53758 [details]
init.d file for domU unprivilaged domains
Created attachment 53759 [details]
sys-kernel/xen-sources/xen-sources-2.6.10-r5.ebuild
I am rather suprised that it is taking so long to get xen ebuilds into portage. I guess the lack of any gentoo specific document explaining what is involved in setting it up would be part of the problem. These ebuilds are based on Philip Taylor original ebuilds with some modifications. They have been updated since version 2.0 and are currently running on internal production servers. They will not work with hardened patches (i.e. -pie). probably obvious but. the xen ebuild goes into apps-sys/xen all associated files go into apps-sys/xen/files the xen-sources ebuild goes into sys-kernel/xen-sources Xen will not just work out of the box. You need to create the environment for the operating system instances (i.e. filesystems). In that respect it is very much like UML. To install xen you will need to read the manual http://www.cl.cam.ac.uk/Research/SRG/netos/xen/readmes/user/user.html Question: does Xen run on any architectures other than x86 and x86-64? Does it work with 2.4.25? > Question: does Xen run on any architectures other than x86 and x86-64? No. Even the ia64 support is not very mature - see the xen-devel mailing list for details. > Does it work with 2.4.25? IIRC, dom0 (the domain which can controll another "virtual" domains) must be 2.6. Other domains could run 2.4 kernels or OpenBSD (maybe other BSD derivates, Kip Macy is developing FreeBSD port - consult the mailing list). BTW, why does this bug depend on bug 36185? AFAIK the tgif is required only for building of docs. I've just tried removing the tgif DEPEND line in the ebuild, then compile it with USE=doc : it compiles just fine, and the docs are generated. Is this dependency _really_ needed ? Is there any status update available to this bug? Other distros already use Xen, a lot of users want Xen but nothing seems to happen here. I hope this is the right "noise" to make this bug active again... I'm interested in this too, do we need testers? It seems to me we have too much commotion and are lacking some verified working ebuilds, maybe if someone already au fait with this bug could write a short how-to, with links to what file(s) are needed, and we could go about testing them to help iron out the wrinkles. I think if it was a little clearer and we had sane ebuilds it would be a good candidate for portage. Created attachment 56536 [details]
sys-apps/xen-2.0.5
Added a few dies, since I had a few missing files. Did not change anything
else.
Created attachment 57185 [details]
init.d/xend
Comment on attachment 57185 [details]
init.d/xend
#!/sbin/runscript
# Copyright 1999-2005 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
opts="start stop status restart"
depend() {
before xendomain
}
await_daemons_up() {
for ((i=0; i<5; i++)); do
sleep 1
xend status && return 0
done
return 1
}
stop_all_xendomains(){
einfo "Shutting down all Xen domains"
xm list | cut -d' ' -f1 | grep -Ev "^(Name|Domain-0)$" |
while read dom; do
# Stop all running domains.
ebegin " Stopping domain $dom"
xm shutdown --halt --wait $dom >/dev/null
eend $?
done
}
start() {
ebegin "Starting Xen control daemon"
xend start
xend status || await_daemons_up
eend $?
if [ "$XENSV" = YES ] && xend status; then
ebegin "Starting Xensv"
xensv start
eend $?
fi
}
stop() {
if [ "$(xm list | wc -l)" -gt 2 ]; then
stop_all_xendomains
fi
if [ "$XENSV" = YES ]; then
ebegin "Stopping Xensv"
xensv stop
eend $?
fi
ebegin "Stopping Xen control daemon"
xend stop
eend $?
}
status() {
xend status
}
Comment on attachment 57185 [details]
init.d/xend
#!/sbin/runscript
# Copyright 1999-2005 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
opts="start stop status restart"
depend() {
before xendomain
}
await_daemons_up() {
for ((i=0; i<5; i++)); do
sleep 1
xend status && return 0
done
return 1
}
stop_all_xendomains(){
einfo "Shutting down all Xen domains"
xm list | cut -d' ' -f1 | grep -Ev "^(Name|Domain-0)$" |
while read dom; do
# Stop all running domains.
ebegin " Stopping domain $dom"
xm shutdown --halt --wait $dom >/dev/null
eend $?
done
}
start() {
ebegin "Starting Xen control daemon"
xend start
xend status || await_daemons_up
eend $?
if [ "$XENSV" = YES ] && xend status; then
ebegin "Starting Xensv"
xensv start
eend $?
fi
}
stop() {
if [ "$(xm list | wc -l)" -gt 2 ]; then
stop_all_xendomains
fi
if [ "$XENSV" = YES ]; then
ebegin "Stopping Xensv"
xensv stop
eend $?
fi
ebegin "Stopping Xen control daemon"
xend stop
eend $?
}
status() {
xend status
}
Comment on attachment 57185 [details]
init.d/xend
Sorry about the last two comments. I'm about to learn how to submit files to
bugzilla :-(
Created attachment 57186 [details]
init.d/xend
Created attachment 57187 [details]
sys-apps/xen/files/xend-conf
Created attachment 57188 [details]
init.d/xendomains
Created attachment 57189 [details]
sys-apps/xen/xen-2.0.5.ebuild
new Attachments: xend-init: - support for starting xensv (via xend-conf) - now shuts down all running domains before stopping xend - removed lots of " new Attachments: xend-init: - support for starting xensv (via xend-conf) - now shuts down all running domains before stopping xend - removed lots of "¦¦ die" statements (makes no sense in init scripts) xend-conf: - new file, used to enable xensv xendomains-init: - simplified a lot - xendomains-conf removed, as it wasn't really neccessary xen-2.0.5.ebuild: - use emake instead of make - newinitd & newconfd functions used instead of cp all: - gentoo conforming headers added I would really like to see this in portage ASAP. Could someone please test this new ebuild and init-files? Created attachment 57231 [details]
sys-apps/xen/xen-2.0.5.ebuild
sys-apps/xen-2.0.5.ebuild
* removed tgif which does not seem to be necessary to compile the documentation
* added debug flag (needed for verbose debugging messages).
* re-added xendomains-conf which is needed, as explained below.
Created attachment 57232 [details]
sys-apps/xen/files/xendomains-init
/files/xendomains-init
* moved AUTODIR back into the config script. This is needed for running groups
of domains, or domains with specific options based on the runlevel.
* added ${} around variables
* changed starting from auto -> ${AUTODIR}
Created attachment 57233 [details]
sys-apps/xen/files/xendomains-conf
/files/xendomains-conf -> /etc/conf.d/xendomains
* removed INITD and LOCKFILE variables as they are not used
Created attachment 57234 [details]
sys-apps/xen/files/xend-init
/files/xend-init -> /etc/init.d/xend
* added net dependency because xend needs network setup before starting. The
default setup specifically require eth0 but as this is a setting it seemed like
a bad idea putting it in the init script.
* removed stop_all_xendomains function. It is not necessary to step though all
domains when shutting down in xend. This is just a safety/sanity check incase
a domain has been created outside the init scripts.
these are modifications to Christoph Gysin files above. You will need the conf.d/xend file above to use them. The AUTODIR has been moved into a configuration file because this allows the use of multiple groups of domains. This works in a similar way to /etc/init.d/net.eth0. Old attachments made obsolete. conf.d/xend renamed to sys-apps/xen/files/xend-conf I've just tested Edward Middleton's modifications. Everything works as expected. Can anybody else confirm this, so we can put it into portage? i've used the ebuild xen-sources-2.6.10.ebuild (id=53759) and did this steps: make ARCH=xen menuconfig make ARCH=xen make ARCH=xen modules_install make ARCH=xen install but no /boot/xen.gz something missing in the ebuild? there's only xen.gz in the xen-2.0-install.tgz * /boot/xen.gz is the hypervisor (mediates operating system access to the hardware) installed with sys-apps/xen * /boot/vmlinuz is a domain0-N kernel created from the sys-kernel/xen-sources * these are not in the same ebuild because there are a number of operating systems other then linux (various bsd's, plan9 etc) that will run under the hypervisor so it does not make sense packaging them all in one ebuild. Created attachment 57444 [details]
sys-apps/xen/files/xendomains-init
* If you try and start/stop a domain that is allready running an error occurs.
I have added a check is_running to see weather the domain is running before
starting or stopping.
spanky, seems to be gentooified, etc -- you want this? no, i dont use it ;) i found qemu to be an easier solution ... Apples and oranges, SpanKY. :) The files seem to be good enough for getting it into portage! When I run /etc/init.d/xend start I get the following error: * Starting Xen control daemon... Traceback (most recent call last): File "/usr/sbin/xend", line 27, in ? from xen.xend.server import SrvDaemon File "/usr/lib/python/xen/xend/server/SrvDaemon.py", line 35, in ? from xen.xend.server import SrvServer File "/usr/lib/python/xen/xend/server/SrvServer.py", line 28, in ? from twisted.web import server, static ImportError: No module named web And shouldn't there be an ebuild to create the xen-sources? This problem is caused by the twisted 2.0.0 packages being unmasked. In twisted 2.0.0 the package has been split into a number of smaller packages. I have not tested with 2.0.0 at the moment. The simple solution is to change the sys-apps/xen/xen-2.0.5.ebuild dependency from >=dev-python/twisted-1.3.0 to =dev-python/twisted-1.3.0 so that only the 1.3.0 package meets the dependency. Created attachment 59355 [details]
/app-emulation/xen/xen-2.0.5.ebuild
Based on Edward Middleton's latest ebuild (57231).
Changes:
* Added pkg_postinst() instructions describing where to look for help and what
to do next to get Xen up and running.
* Specified a specific version of dev-python/twisted (1.3.0) since 2.0.0 seems
to break Xen.
* Put needed script files in /xen/files/${PV}/ directory so that version bumps
will be easy (recommended method from Gentoo Ebuild HOWTO).
* Suggesting it be put into portage in /app-emulation/ (actually, I'm torn
about this as it doesn't feel quite right; maybe we should create an
/app-virtualization/ :) )
Required ./files:
/xen/files/xend-init
/xen/files/xend-conf
/xen/files/xendomains-conf
/xen/files/xendomains-init
Formatted, documented, and ready for Portage.
-- Travis Cross
I have successfully installed, uninstalled, and reinstalled the Xen virtual machine monitor, the Xend daemon, and the Xen dom0 and domU kernels (xen-sources) on two separate machines (an IBM e-Server (P4) and a custom Asus / Athlon-XP box) using the ebuilds provided here. This bug has gone on for awhile so here is a recap of what needs to go into portage: app-emulation/xen/xen-2.0.5.ebuild #59355 app-emulation/xen/files/2.0.5/xend-conf #57187 app-emulation/xen/files/2.0.5/xend-init #57234 app-emulation/xen/files/2.0.5/xendomains-conf #57233 app-emulation/xen/files/2.0.5/xendomains-init #57444 sys-kernel/xen-sources/xen-sources-2.6.10.ebuild #53759 portage/app-virtualization would be a good category to add now as there will be a lot more of this kind of thing coming down the pipe, especially as Intel and AMD are adding features to their chips to support this. Obviously app-emulation is a decent second choice. <evangelism> Xen has _a lot_ of buzz right now (and major backers). I think it would be wise to get this into Portage asap. Not only are people looking for this, but lots of people are writing about it, and are noting what distributions are in the game. When Gentoo does get mentioned, it is noted that there are 'experimental ebuilds on Bugzilla' - not the most inviting phrase to the new user, not to mention that this bug is being linked as the 'go-to point'. This is what Portage is for. I'm a busy person, and I know that you (the devs) are as well, so I understand that we get overloaded. This needs to be a priority, however. We should be ahead of everyone in this game. Besides, these ebuilds are ready. </evangelism> -- Travis Cross Thank you from the bottom of my (two-chambered) heart! I probably won't have time to test this over the coming weekend, but a three-day weekend is coming up, and Xen is high on my list. Question: as an x86-specific package, isn't this always going to be ~x86? Congratulations to all of those who've put in the hard work to get this going!! For the moment, Xen is x86-only, but the core developers are already at work on amd64, em64t, and ia64 ports, with plans for ports to ppc and arm. The 64-bit ports are set to appear in version 3.0, which is scheduled to be released in July. -- Travis xen-2.0.6 is out Well, I wanted to try out Xen this week and was both shocked and horrified that it isn't in portage yet :-O May I humbly suggest that one of the people who contributed to this ebuild, knows about Xen, and has a little free time, step up and volunteer to become a dev and maintain it? Created attachment 59688 [details]
xen-2.0.6.ebuild
Version bump.
Also fixed make install for the python tools, later versions of Xen require
XEN_PYTHON_NATIVE_INSTALL=1, otherwise they get installed to /usr/lib/python.
Created attachment 59694 [details]
sys-kernel/xen-sources/xen-sources-2.6.11.ebuild
revision bump, now slotted
Created attachment 59695 [details]
sys-kernel/xen-sources/xen-sources-2.4.30.ebuild
ebuild for 2.4 kernel, no changes
xen-sources revision bump. added ebuild for 2.4 kernel. - inherited kernel-2.eclass instead of kernel.eclass - set SRC_URI with KERNEL_URI. - Removed SLOT=0, since kernel sources get slotted from eclass. - updated XEN_V to 2.0.6 Comment on attachment 59695 [details]
sys-kernel/xen-sources/xen-sources-2.4.30.ebuild
This one doesn't work. I'll try again later. Sorry.
Created attachment 59703 [details]
xen0-sources-2.6.11.10.ebuild
Reworked ebuild to include support for 2.6.11.10, as well as other kernel
patches included with Xen 2.0.6.
(ebuild could do with some checking though)
Created attachment 59710 [details]
xen0-sources-2.6.11.10.ebuild
Minor EXTRAVERSION fix (forgotten "." resulted in kernel names of
vmlinuz-2.6.1110-xen0).
I had problems compiling xen 2.0.6 with "-fprefetch-loop-arrays". Shall I post the error message here or open a new bug? Next problem comes up, when it builds the docs: ################# cut ############################ make: Entering directory `/var/tmp/portage/xen-2.0.6/work/xen-2.0/docs' latex src/interface.tex >/dev/null latex src/user.tex >/dev/null if [ -e interface.toc ] ; then latex src/interface.tex >/dev/null ; fi if [ -e user.toc ] ; then latex src/user.tex >/dev/null ; fi make[1]: Entering directory `/var/tmp/portage/xen-2.0.6/work/xen-2.0/docs' install -d -m0755 html/interface latex2html -split 0 -show_section_numbers -toc_depth 3 -nonavigation \ -numbered_footnotes -local_icons -noinfo -math -dir html/interface \ src/interface.tex 1>/dev/null 2>/dev/null install -d -m0755 ps dvips -Ppdf -G0 -o ps/interface.ps.new interface.dvi make[1]: *** [html/interface/index.html] Error 126 make[1]: Leaving directory `/var/tmp/portage/xen-2.0.6/work/xen-2.0/docs' make: *** [html] Error 2 make: *** Waiting for unfinished jobs.... This is dvips(k) 5.92b Copyright 2002 Radical Eye Software (www.radicaleye.com) ' TeX output 2005.05.24:2018' -> ps/interface.ps.new <tex.pro><alt-rule.pro><texc.pro><8r.enc><aae443f0.enc><bbad153f.enc> <f7b6d320.enc><texps.pro><special.pro>. <cmmi8.pfb><cmr10.pfb><cmsy10.pfb> <cmmi10.pfb>[1<figs/xenlogo.eps>] [2] [1] [2] [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21] [22] [23] [24] [25] [26] mv ps/interface.ps.new ps/interface.ps make: *** Waiting for unfinished jobs.... rm user.dvi interface.dvi make: Leaving directory `/var/tmp/portage/xen-2.0.6/work/xen-2.0/docs' !!! ERROR: app-emulation/xen-2.0.6 failed. !!! Function src_compile, Line 36, Exitcode 2 !!! compiling docs failed !!! If you need support, post the topmost build error, NOT this status message. ################# cut ############################ I could reproduce this error with: /var/tmp/portage/xen-2.0.6/work/xen-2.0 # make -C doc [...] same make error [...] Without the "doc" use flag everything builds fine :) Created attachment 59947 [details]
app-emulation/xen/files/2.0.6/xend-init
XenSV has been removed in 2.0.6 by upstream devs. (deactivated in
tools/Makefile)
conf.d/xen is not needed anymore, new init.d/xen attached.
Comment on attachment 59694 [details]
sys-kernel/xen-sources/xen-sources-2.6.11.ebuild
obsolete, since we now have an ebuild for 2.6.11.10
(In reply to comment #91) > Well, I wanted to try out Xen this week and was both shocked and horrified that > it isn't in portage yet :-O > > May I humbly suggest that one of the people who contributed to this ebuild, > knows about Xen, and has a little free time, step up and volunteer to become a > dev and maintain it? Have you received any reaction on this request? It seems current Gentoo developers are not interested in / to busy for maintaining Xen. Are the contributors interested in becoming a dev? (In reply to comment #102) > (In reply to comment #91) > > Well, I wanted to try out Xen this week and was both shocked and horrified that > > it isn't in portage yet :-O > > > > May I humbly suggest that one of the people who contributed to this ebuild, > > knows about Xen, and has a little free time, step up and volunteer to become a > > dev and maintain it? > > Have you received any reaction on this request? It seems current Gentoo developers are not > interested in / to busy for maintaining Xen. Are the contributors interested in becoming a > dev? I believe the process for becoming a Gentoo developer, should someone wish to, is documented at http://www.gentoo.org/proj/en/devrel/index.xml There is a developer recruiting effort as well. The lead for that is Deedra M. Waters (dmwaters@gentoo.org). BTW, I just rebuilt the machine that's the target for my Xen testing. I've got Gentoo running on it and am in the process of getting the memory up from 256 MB to a full GB. I got it up to 512 MB last night but for some reason the hardware isn't seeing the second DIMM. Am I stepping on any dev toes if I work on this? vmware (not opesource) is in the portage tree and the only opensource equivalent xen is not there. comme on. If you have trouble booting into the kernel and the boot process fails with "cannot open root device" or similar, make sure that you have CONFIG_PARTITION_ADVANCED=y CONFIG_BLK_DEV_IDEDISK=y CONFIG_BLK_DEV_IDEDISK=y set in your .config (I only found this after days of searching the web) I know this is not really ebuild-related but as we have no central thread on the forums I thought I appended this here. Created attachment 60478 [details, diff]
patch for /etc/init.d/clock to skip hwclock in xenU
I've worked with all your ebuilds - 4 installs so far. Thanks. First (and very rough) pass at a HOWTO. I'll be out of town for the next 4 days, but please comment and I'll clean and update soon. http://forums.gentoo.org/viewtopic-p-2462628.html#2462628 OK, lets get this into the tree. First step, there are a few ebuilds and such that are attached to this bug. Lets, make sure that there is only one for each component (ie. kernel, utils, etc). Second, the kernel ebuilds default to a domain0 config. Can we make it so that domain0 or domainU is chosen via a USE flag? Third, it would be better if we could derive off of the gentoo-sources kernel rather than vanilla (lots of fixes, we have a great kernel team!). It should probably be enough to apply the gentoo-base patches (we may want to avoid gentoo-extras as it contains things like bootsplash, etc). However, we should keep xen-sources as close to the gentoo-sources ebuilds as possible (for maint. reasons). Fourth, before they go into the tree, some good documentation needs to be written. I notice a howto has been started (good job so far!). What we really need is a document which explains the *concepts* as well as a brief howto. For instance, "what is different between dom0 and domU?" "Do domains > 0 write directly to disk or to disk image?" and most important "How do you install gentoo in a domain > 0?" I'm really appreciative of everyone's work so far. Lets tackle these things and I think we'll be in great shape! (In reply to comment #109) > First step, there are a few ebuilds and such that are attached to this bug. > Lets, make sure that there is only one for each component (ie. kernel, utils, etc). BTW, make sure that the ebuilds are merged, not just old versions removed. #109 Xen sources are really just a patched in new archetecture to the base linux kernel. It would be better to have the xen-sources as the xen distributed sources (like the vanilla kernel sources) and then get a patch into the gentoo sources to support the xen archetecture (possiably with a xen use flag). If you base the Xen sources off gentoo you will end up with total confusion about what is being installed and potentially lots of bug reports to xen mailing list about problems which are caused by gentoo patches. This is why I suggested applying the gentoo-base vs gentoo-base + gentoo-extras. gentoo-base is really just basic hardware fixes, backports and such. I'm definately not suggesting that we add a xen use flag to the gentoo-sources. I'm saying we should have a separate xen-sources package, but that that package should apply the basic fixes from gentoo-base. #109 first point The kernel sources are the same for both domain0 or domainU. The only difference is the .config file. As all users are going to want to have both types of domains a use flag might not be the best way to select the config file. Perhaps we should just have it set to the domain0 dofault config because kernels compile with this config should work in both domain0 and domainU. Sounds good to me. On the topic of which kernel patches to apply: The primary modifications made to the kernel by the Xen patches remove privileged processor instructions and replace them with calls to the Xen Virtual Machine Monitor. I have not studied the gentoo-base patches in detail, but I would fear that applying even those might introduce the real possibility of new and difficult to trap bugs. At the very least, I doubt anyone on the xen-devel or xen-users lists would take a concern or bug report seriously from a user without a 'clean' xen kernel. Maybe the appropriate solution is to add a 'gentoo-base-patches' USE flag to the xen-sources ebuild. I believe '-gentoo-base-patches' (off) would be the appropriate default. Running a kernel under Xen is fundamentally different than running 'on the metal' because the Xen Virtual Machine Monitor stands between the linux kernel and the hardware. Therefore kernel interaction with processor(s), memory, and the pci bus (including agp) must be handled the way the VMM expects, or the kernel will not get what it expects, and will likely seg fault. If there is an understanding that processor and system bus related changes stay out of the gentoo-base patches (or an awareness in the kernel team of the requirements of Xen), then it probably would not be an issue. But otherwise, I would personally feel more comfortable with my servers running on the 'clean' xen sources. Cheers, -- Travis ok, I just spent a few minutes looking through gentoo-base patches. Most of it is patches against arches that xen doesn't support. The nightmare of weeding those patches out and keeping the right patches is too much. I agree, lets do just a vanilla + xen. How about docs, anyone want to tackle that? On the topic of documentation: The Xen Users' Manual is a relatively good introduction to the concepts and the 'howto' of Xen: http://www.cl.cam.ac.uk/Research/SRG/netos/xen/readmes/user/user.html The difficult part isn't getting Xen setup, it is getting everything else set up around it. When I get a spare moment, I will start on a more complete Gentoo+Xen HOWTO which will include some guidance on best practices. The challenge in putting together good documentation at this point is that Xen is still changing rapidly. As there are other good resources to learn about Xen, I do not believe that the creation of gentoo-specific documentation should be an obstacle to getting this package merged quickly. I believe that the most important expectation of a gentoo user is that after she learns about an application and takes an interest in it, that when she opens a console and types 'cd /usr/portage ; locate xen' that she'll find something. Cheers, -- Travis Comment on attachment 59355 [details]
/app-emulation/xen/xen-2.0.5.ebuild
Marking xen-2.0.5.ebuild as obsolete since we've now moved to xen-2.0.6.
A couple reasons why I think docs should be a precursor in this case: 1. I won't put anything in the tree until I'm sure it works ok. I can't test it right now as I don't have a free x86 box. However, I'm moving my webserver to a ppc box this weekend/week and once that is done, I will test xen. This gives us time to put some good docs together (and a good time for me to review the docs when I am actually testing Xen). 2. Because of the complexity of xen (compared to just "emerge gnome"), some good docs are definately in order. 3. As soon as this is in the tree, people will demand docs. Lets just have them out of the gate. 4. Docs are in order exactly *because* xen is changing (people need up to date info). 5. As any good programmer will tell you, docs are more important than code. They should not be an afterthought. Remember, if a package doesn't work out of the box with a simple "emerge foo", then we need clear docs to explain how to get it working. I'm not looking for something complicated. I think 4 basic sections are our ultimate goal: 1. Explain Xen concepts 2. HOWTO Gentoo in domain 0 3. HOWTO Gentoo in domain >= 1 4. Tips/Tricks/Thoughts/Hacks/etc This shouldn't take much time. Be creative and have fun. And, while it'd be great to have complete docs out of the gate, I'd settle for less than that. Just so long as we have *something*. On the topic of getting down to the needed files only: It seems the following attachments should be marked as obsolete: #43894 #44963 #51453 #52423 #52424 #53759 #56536 #57231 -- Travis #119: Fair enough. I suppose we should take the opportunity to clear a wide trail for other to follow. -- Travis also, it would probably be best to file a separate bug for the hwclock issue against baselayout (particularly because I don't maintain it). Created attachment 60552 [details]
sys-kernel/xen-sources/xen-sources-2.6.11.ebuild (gentoo base patches)
I have serious reservations about going this way but, this is a build that will
use only the gentoo base patches. Quite a few of them are not really
applicable because they are for archetectures not supported by xen, and this
has had nothing more then 'a does it compile/boot' test.
#116 & #123 I share your reservations. Let's follow our intuition and mark #60552 as obsolete, rolling back to #59694 ? -- Travis sorry to make you do extra work, we decided not to apply gentoo-base (for the very reason you mentioned; see comment #116). Created attachment 60561 [details]
sys-kernel/xen-sources/xen-sources-2.6.11.ebuild
Some of the changes in the gentoo base patch should probably be kept
* moved xen source url to XEN_URI
* removed 0 from EXTRAVERSION because the source is not domain0 specific.
* removed the www.kernel.org url from HOMEPAGE is this link really needed?
* added a dependency for the specific xen version. We don't want to have to
debug problems caused by running old domains with new hypervisor and vis-versa.
* removed reference to domain0 in the DESCRIPTION and added the kernel and xen
version.
* renamed to xen-sources-2.6.11.10.ebuild because not domain specific.
If needed I can review, test and convert any documentation to GuideXML (for those unfamiliar with that). I am setting up a system now and will document it as I go. I intend to document how to setup a domain0,domain1 using a logical volume. (In reply to comment #127) > If needed I can review, test and convert any documentation to GuideXML (for > those unfamiliar with that). I'm sure that GDP wannabies (fox2mike, smithj) can do that, too, and with great pleasure. Created attachment 60562 [details]
basic rough outline of the required documentation
I have to go now but this is what I have done sofare. This is a very rough
outline of the required documentation as suggested. It is a straight text
file. There is very little comments and a number of parts missing. I will try
and get back to this latter today.
xend-init needs "before dhcp" and xendomains-init needs "after dhcp" for xen0 domains that provide dhcp to its own guests. I just started playing around with Xen today. All I've done so far is download
the original 2.0.6 source tarball and follow the instructions. Everything built
fine, although I had to put quite a few of the prerequisites in
/etc/portage/package.keywords; Xen wants some things more recent than "stable".
So far only two glitches:
1. Sound didn't come up. I think that the Xen kernel just needs to be rebuilt
with my sound card configured. I normally build a kernel with only the base
sound and emerge "alsa-driver".
2. Everything else, including X/gdm/KDE came up fine. But I couldn't get "xend"
to start. That appears to be caused by something in "twisted", since it's
complaining about Python. I emerged the latest version of "twisted"; Xen may be
more picky than that. I'm going to try using the version of "twisted" that comes
with Xen.
Once I get past these two hurdles, I'll attempt to build another VM on my other
hard drive and put CentOS or Fedora (or maybe Sarge) on it. This box only has
512 MB, so I'm not sure it can handle more than two VMs but it should be able to
deal with two, since it was running single ones in 256 MB a week ago.
Incidentally, if you build the whole works ("make world") from the Xen source,
it downloads a vanilla kernel, patches it, configures it and builds it. Then
when you do "make install", it places everything in "/boot". All you need to do
is edit "grub.conf" and reboot. Would it be simpler to make an ebuild using that
strategy?
Most people will need to configure their xen kernels for their hardware so it makes sense having a xen-sources package. Try using Twisted version 1.3.0. (In reply to comment #133) > Most people will need to configure their xen kernels for their hardware so it > makes sense having a xen-sources package. Try using Twisted version 1.3.0. Yeah ... it took me a while to figure out how to configure and rebuild the kernel in the source distribution. And the build croaked on one of the ISA sound drivers. I will probably reboot in about an hour; if that works, I'll test the ebuilds, etc., tonight. (In reply to comment #134) > (In reply to comment #133) > > Most people will need to configure their xen kernels for their hardware so it > > makes sense having a xen-sources package. Try using Twisted version 1.3.0. > > Yeah ... it took me a while to figure out how to configure and rebuild the > kernel in the source distribution. And the build croaked on one of the ISA sound > drivers. I will probably reboot in about an hour; if that works, I'll test the > ebuilds, etc., tonight. Sound works ... xend works (with Twisted 1.3) ... life is good :). This is posted from Gentoo running from Xen. Created attachment 60746 [details]
xen-lvm
I've automated most of the LVM setup for guests. The script is attached, along
with any other (but not all) notes I've collected. It would be very useful to
anyone totally clueless about how to start. It will probably be useful to
anyone writing docs.
Considering this ebuild works can it be put in Portage? Masked of course, so any comments can be made... I know I'm perhaps a bit impatient but this bug exists for almost 7 months with great interest of the community. If it works documentation will be created by users. Created attachment 61091 [details, diff]
patch for /etc/init.d/clock to skip hwclock in xenU
I would open a bug for baselayout, but since xen still isn't in portage (!!!)
there's no point.
(In reply to comment #138) > Created an attachment (id=61091) [edit] > patch for /etc/init.d/clock to skip hwclock in xenU > > I would open a bug for baselayout, but since xen still isn't in portage (!!!) > there's no point. You can still open a new bug for baselayout (Add it as a blocker to this bug). I don't think the maintainers of baselayout would deny the patch soley because this ebuild isn't in portage yet. They'll *should* understand they need to fix baselayout before zen works but there's no reason to add it to portage if it doesn't work yet (Damm chicken! Errr.. egg!) I think we have documentation: http://forums.gentoo.org/viewtopic-t-344571.html I am currently working on documentation along the lines of what Nathaniel McCallum described in #119, sorry for the delay. There should be official documentation before this is put in the tree. If you are happy to install it without official documentation then you are free to download the ebuild from here as many people allready have. (In reply to comment #141) > There should be official documentation before this is put in the tree. I'm quite sure that GDP won't add any documents concerning unofficial ebuilds. Would anyone mind putting up a .tgz of the required files for portage on the wild web since the devs apparently aren't going to add this to portage? :) Or just email me the files and i'll put them up. Created attachment 61951 [details]
tarball: portage tree for xen and conf-files
The attachment contains the /usr/local/portage tree and the entries you have to
add to /etc/portage/package.keywords and /etc/make.conf.
Created attachment 62232 [details]
xml source file for xen guide
This is the source file for a gentoo xen install guide I have been writting.
It still needs some work but feedback would be good. I will not have internet
access until the 7/6 so don't expect a response before then.
Created attachment 62233 [details]
html version of guide
(In reply to comment #145) > This is the source file for a gentoo xen install guide I have been writting. > It still needs some work but feedback would be good. I will not have internet > access until the 7/6 so don't expect a response before then. I'd recommend creating another bugreport for Xen documentation as it is closely related to the GDP. I'd have several comments about coding style (no tabs, better separation of <p>s etc, line width and usage of <c> for emphasis, capital letters,...), but here are the "real" ones: a) I'd explain what does "hypervisor" mean before first usage of this word b) don't mention "filesystem" as a part of Xen domain; AFAIK Xen itself (the hypervisor) doesn't care about guest domain's kernel internals c) I don't see a reason for domain0 to be compiled with "blockdev/network frontend support" - is't the backend enough? d) "Scrub memory before freeing it to Xen" is not required, AFAIK. e) "generic/default IDE chipset support" is not needed if you select your IDE chipset driver, AFAIK f) `cp System.map /boot/System.map-2.6.11.10-xen` - IMHO dsd has something to say about this point :-) g) "domain0 kernel" is *not* "booted", it is "loaded" by hypervisor h) don't duplicate stuff from Handbook (network configuration etc), maintaining twice the amount of text is PITA i) "Start the hypervisor access daemon and add it to the default domain." - hypervisor is already running, you're only starting one of the possible management interfaces (iirc, it has been soem time since I stopped reading xen ML) j) TLS can't be disabled by USE=-tls, that USE flag is called "nptl". But you should probably ask glibc maintainer... k) `mv /lib/tls /lib/tls.disable` is ugly hack I know, this issue has been raised a thousand times in other packages. Some CFlags need to be filtered, when you use hardened. make[1]: Entering directory `/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen/common' gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include -Wall -Werror -pipe -I/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen/include -Wno-pointer-arith -Wredundant-decls -O3 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -c ac_timer.c -o ac_timer.o gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include -Wall -Werror -pipe -I/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen/include -Wno-pointer-arith -Wredundant-decls -O3 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -c dom0_ops.c -o dom0_ops.o gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include -Wall -Werror -pipe -I/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen/include -Wno-pointer-arith -Wredundant-decls -O3 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -c dom_mem_ops.c -o dom_mem_ops.o dom_mem_ops.c: In function `do_dom_mem_op': /var/tmp/portage/xen-2.0.6/work/xen-2.0/xen/include/asm/mm.h:160: error: can't find a register in class `BREG' while reloading `asm' make[1]: *** [dom_mem_ops.o] Error 1 make[1]: Leaving directory `/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen/common' make: *** [/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen/xen] Error 2 make: Leaving directory `/var/tmp/portage/xen-2.0.6/work/xen-2.0/xen' !!! ERROR: app-emulation/xen-2.0.6 failed. !!! Function src_compile, Line 32, Exitcode 2 !!! compiling xen failed !!! If you need support, post the topmost build error, NOT this status message. (In reply to comment #148) > I know, this issue has been raised a thousand times in other packages. > > Some CFlags need to be filtered, when you use hardened. > > make[1]: Entering directory Here's a simple "fix" unless you figured it out yourself. # emerge xen When you see "Source unpacked", hit ^Z and edit /var/tmp/portage/xen-2.0.6/work/ xen-2.0/xen/arch/x86/Rules.mk Change the line CFLAGS := -nostdinc -fno-builtin -fno-common -fno-strict-aliasing to CFLAGS := -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -nopie - fno-stack-protector -fno-stack-protector-all ..and save. Let emerge continue by putting the process into the foreground again # fg Someone ought to come up with a real patch for those of us running GCC hardened. Xen 2.0.7 released. Anyone care to update the tarball? "Although everyone is working flat out to get Xen 3.0 ready, we thought it was about time we released a Xen 2.0.7 update as there have been a bunch of fixes queued up in the testing tree for a while. For example: * 'noirqbalance' parameter to workaround buggy ES7000 chipsets * vm relocation tools fixes * fix fp bug that was triggered by certain user kernel configs * fix for the console warning when udp_poll() receives fragmented skbuffs Upgrading is probably not critical for most people, but is advisable. Best, Ian" Feedback on the current tarball (rename the ebuild works to bump it to 2.07). 1) The message about mv /lib/tls /lib/tls.disabled.. I moved it and still get the message. Normal? 2) TLS glibc patch. Any plans to add it to gentoo glibc? 3) Docs. Most people don't use LVM - should mention files etc. (could copy straight from the config section of http://julien.danjou.info/xen.html) 4) Docs. Mentions tls use flag - does this flag actually exist for any packages? It's not in use.desc. If a rebuild is necessary mention emerge --newuse. 5) /etc/init.d/xend is odd. There's no output. Added to default runlevel doesn't seem to do anything.. I still have to manually do '/etc/init.d/xend start' otherwise I get an error about the server not running. 5.5) rc-status always shows xend service not running. 6) Kernel config section should include bridge module (essential), and maybe netfilter. 7) xend-config file mentioned in the xen docs is missing. Maybe include an example if it's not necessary. 8) DEPEND on iptables missing. /etc/xen/scripts/vif-bridge calls 'iptables'. Has there been any progress on getting xen into portage? (In reply to comment #151) > 6) Kernel config section should include bridge module (essential), and maybe > netfilter. You don't have to do bridging, you can use normal IPv4 routing. Created attachment 66585 [details]
emerge xen-2.0.6 on AMD64 failes due too compile error
specified app-emulation/xen ~* as keyword.
(In reply to comment #151) > 2) TLS glibc patch. Any plans to add it to gentoo glibc? That one already works (there was a minor issue [Bug #99896] with flag-o-matic strip-flags, but that has been resolved): - install gcc-3.4.4 (or newer), make it the default compiler. - add "-mno-tls-direct-seg-refs" to your CFLAGS - recompile glibc (>=3.3.5) with USE=nptl - recompile all libraries and apps broken by the compiler update (most C++ libs/apps) glibc Makefiles recognise the -mno-tls-direct-seg-refs flag, set some #defines, and use modified asm code for tls access then... works like a charm here. > 1) The message about mv /lib/tls /lib/tls.disabled.. I moved it and still get the message. Normal? Yes, since some apps access memory in a way similar to glibc without patch. xen doesn't distinguish that... the performance impact for them is quite low, so nothing to worry about. And recompiling those apps with -mno-tls-direct-seg-refs should fix the problem. There is a Problem in XEN that was fixed some days ago... http://bugzilla.xensource.com/bugzilla/show_bug.cgi?id=76,http://bugzilla.xensource.com/bugzilla/attachment.cgi?id=48 could someone patch the xen-sources? cheers, daniel I've commited ebuilds. I changed them slightly - xen now installs the sparse source tree which is picked up later by xen-sources. Not exactly ideal but there's no good way of providing for different versions of xen with the same kernel version, and sticking to the gentoo ebuild naming scheme. I've added a pre3 ebuild with kernel 2.6.12.5 as well. Seems to work well here. I'll leave this bug open for the inevitable bug reports and stuff I missed :) Thanks to everyone who contributed! oh and http://gentoo-wiki.com/HOWTO_Xen_and_Gentoo ... though if someone wants to maintain official docs later that would be cool too. The ebuild is broken. It is performing the steps for building the sparse tree in the incorrect order. During the src_install step the following error appears --------begin---------- mkdir: cannot create directory `linux-2.6-xen-sparse/include/asm-xen/xen-public': No such file or directory cp: `linux-2.6-xen-sparse/include/asm-xen/xen-public': specified destination directory does not exist Try `cp --help' for more information. ./ ---------end---------- This is occurring because the ebuild is incorrectly assuming that the "linux-2.6-xen-sparse" directory exists when performing a mkdir and copying some files. I am attaching a patch that should fix the problem. I apologize, my mozilla seems to be broken. I was unable to use the browse
button trying to attach the patch. The patch follows:
--- xen-2.0.7.ebuild 2005-09-07 12:18:59.000000000 -0600
+++ xen-2.0.7.ebuild-fix 2005-09-08 12:16:01.000000000 -0600
@@ -74,10 +74,10 @@
# we need to do whatever mkbuildtree would've done for each platform
# linux-2.6: copy public include files, and xenstored.h
- mkdir linux-2.6-xen-sparse/include/asm-xen/xen-public
+ mkdir linux-2.6.11-xen-sparse/include/asm-xen/xen-public
# include files for kernel
rm xen/include/public/COPYING
- cp -dpPR xen/include/public/*
linux-2.6-xen-sparse/include/asm-xen/xen-public
+ cp -dpPR xen/include/public/*
linux-2.6.11-xen-sparse/include/asm-xen/xen-public
# fixme: insert code for other sparse trees here
# install xen kernel sparse trees
http://gentoo-wiki.com/HOWTO_Xen_and_Gentoo isn't that good. It just broke my system (the "Before we begin"-Stuff). Someone who knows what this section is supposed to do should rewrite/repair it. (In reply to comment #159) > http://gentoo-wiki.com/HOWTO_Xen_and_Gentoo isn't that good. It just broke my system (the "Before > we begin"-Stuff). Someone who knows what this section is supposed to do should rewrite/repair it. Yeah, that was a seriously stupid idea to unmerge old gcc-3.3.x version before rebuilding world with the new (gcc-3.4.x) one. I changed the order now, no idea who did put that into wiki. Note, that libstdc++-v3 may be still needed by some binary-only stuff, in that case you should not put it into package.provided. Ken - bug fixed, thanks for spotting it. Phil - sorry, my mistake, was typing from memory. :-( /etc/xen/scripts/network-bridge relies on ifup and ifdown scripts from redhat, therefore bridging isn't going to work on gentoo. It works for me. I haven't looked into why exactly why it works despite ifup/ifdown failing, maybe the interface is already up by default, or something else brings it up. re: http://lxr.xensource.com/lxr-unstable/source/examples/network-bridge?v=tools-xen-unstable In op_start() there is a fallback for ifdown: 180 if ! ifdown ${netdev} ; then 181 # if ifup didn't work, see if we have an ip= on cmd line 182 if egrep 'ip=[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+:' /proc/cmdline ; 183 then 184 kip=`sed -e 's!.*ip=\([0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+\):.*!\1!' /proc/cmdline` 185 kmask=`sed -e 's!.*ip=[^:]*:[^:]*:[^:]*:\([^:]*\):.*!\1!' /proc/cmdline` 186 kgate=`sed -e 's!.*ip=[^:]*:[^:]*:\([^:]*\):.*!\1!' /proc/cmdline` 187 ifconfig ${netdev} 0.0.0.0 down 188 fi 189 fi and ifup: 200 if ! ifup ${netdev} ; then 201 if [ ${kip} ] ; then 202 # use the addresses we grocked from /proc/cmdline 203 ifconfig ${netdev} ${kip} 204 [ ${kmask} ] && ifconfig ${netdev} netmask ${kmask} 205 ifconfig ${netdev} up 206 [ ${kgate} ] && ip route add default via ${kgate} 207 fi 208 fi So maybe these are sufficient, however the ifup in op_stop() should mean that you lose eth0 when stopping a xen machine because there is no fallback to "ifconfig ${netdev} up"? 241 ifup eth0 xen wont compile:
>>> md5 files ;-) xen-3.0.0_pre20050929.ebuild
>>> md5 files ;-) xen-2.0.7.ebuild
>>> md5 files ;-) files/digest-xen-3.0.0_pre20050929
>>> md5 files ;-) files/xendomains-conf
>>> md5 files ;-) files/xendomains-init
>>> md5 files ;-) files/xend-conf
>>> md5 files ;-) files/xend-init
>>> md5 files ;-) files/digest-xen-2.0.7
>>> md5 src_uri ;-) xen-unstable-20050929.tar.bz2
[...]
gcc -nostdinc -fno-builtin -fno-common -fno-strict-aliasing -iwithprefix include
-Wall -Werror -Wno-pointer-arith -pipe
-I/var/tmp/portage/xen-3.0.0_pre20050929/work/xen-unstable-20050929/xen/include
-I/var/tmp/portage/xen-3.0.0_pre20050929/work/xen-unstable-20050929/xen/include/asm-x86/mach-generic
-I/var/tmp/portage/xen-3.0.0_pre20050929/work/xen-unstable-20050929/xen/include/asm-x86/mach-default
-O3 -fomit-frame-pointer -msoft-float -m32 -march=i686 -DNDEBUG -DVERBOSE -c
grant_table.c -o grant_table.o
grant_table.c: In function `gnttab_transfer':
grant_table.c:760: error: can't find a register in class `BREG' while reloading
`asm'
make[1]: *** [grant_table.o] Error 1
make[1]: Leaving directory
`/var/tmp/portage/xen-3.0.0_pre20050929/work/xen-unstable-20050929/xen/common'
make: ***
[/var/tmp/portage/xen-3.0.0_pre20050929/work/xen-unstable-20050929/xen/xen] Error 2
make: Leaving directory
`/var/tmp/portage/xen-3.0.0_pre20050929/work/xen-unstable-20050929/xen'
Reopen this bug to re-assign to maintainer. Additionally, I'd really recommend the normal course of solving bugs - i.e., open a *new* bug for *new* issues, dumping all the problems here is just a terrible mess and is causing tons of bugspam. Close again. Closing. Please re-file any existing bugs. The only things I am aware of is xen3 requiring udev rules as it seems to break with udevsend, and the ifup/ifdown thing; I'll fix these in the ebuild soon. The compile problem looks like a gcc bug. |
