* The current ebuilds fail on hardened due to the outdated ipxe-nopie patch -> use the ipxe-nopie-4.2.0.patch from the virtualization overlay * The current ebuilds list dependencies not strictly required to build/run xen-tools, including python and ocaml (both _very_large_ codebases) -> introduce use flags for 'ocaml' and 'python' (I build and use xen-tools without these, but I'm not sure exactly which tools might require one or the other; so, I cannot really contribute a meaningful patch [I just rip out the unneeded parts]) -> probably also need use constraints, such as 'xend? ( python )' and 'pygrub? ( python )'
Some of the "CDEPEND" do not seem to be required at run-time (e.g. bin86 and dev86), while some other packages that are, aren't listed as RDEPEND (OK, gcc is part of the system profile, but I don't think gnutls is, but qemu won't run without the latter)
(In reply to comment #1) > Some of the "CDEPEND" do not seem to be required at run-time > (e.g. bin86 and dev86), while some other packages that are, > aren't listed as RDEPEND (OK, gcc is part of the system profile, sys-devel/gcc should definitely not be in any *DEPEND. > but I don't think gnutls is, but qemu won't run without the latter) RDEPEND="${CDEPEND} ... So that's all right. CDEPEND normally stands for "common depend". There is no mention of gnutls in the ebuild, however, so you are probably right about that.
> RDEPEND="${CDEPEND} ... Right. That's the problem exactly. > So that's all right. CDEPEND normally stands for "common depend". I think you misunderstand. Some packages in CDEPEND should only be in DEPEND and *not* in RDEPEND, since they are *not* needed at run-time.
Just to put this in perspective, I'm building a minimalistic virtualization host using the --root and --root-deps options for emerge and pulling in dependencies that are not required at run-time is not what makes me happy… (-; Of course, I _can_ just massage the ebuild to my liking, which is what I'm doing anyway; but, I thought, others might not want unnecessary cruft on their systems either (e.g. ocaml, which does not appear to be necessary neither at build nor at run-time for the toolstack I'm using [xl], but then again -- I could be missing something).
I'll make a few inquiries.
all right save re-inventing the wheel, what do you do to cut out optional ocaml from the many Makefile entries?
Created attachment 336728 [details, diff] patch to latest ebuild (4.2.1)
(In reply to comment #6) > all right save re-inventing the wheel, what do you do to cut out optional > ocaml from the many Makefile entries? Nothing whatsoever. (-; To quote the README: --- In addition to the above there are a number of optional build prerequisites. Omitting these will cause the related features to be disabled at compile time: * Development install of Ocaml (e.g. ocaml-nox and ocaml-findlib). Required to build ocaml components which includes the alternative ocaml xenstored. --- I've attached the patch I'm using. The ocaml bits should be fine, the rest may be too specific (gcc as a dependency for instance ;0) and definitely *not* widely tested... While you're at it, could it make sense to remove the boot-loader RDEPEND for xen proper?
a.m * The current ebuilds fail on hardened due to the outdated ipxe-nopie patch -> use the ipxe-nopie-4.2.0.patch from the virtualization overlay oh groan, you are right here. I somehow lost a line in copying it over. I have corrected it, but the commit will come in one collective hit. Not quite there yet. ok let's go thru this (last) patch with numerous changes; adding a global python flag and making the various changes conditional on its setting is drawing a looong bow. +REQUIRED_USE="hvm? ( qemu ) sdl? ( hvm ) xend? ( python ) pygrub? ( python ) pam? ( python )" That would need some further in-house discussion & scrutiny, and the intent of the xend flag was to isolate and minimise the python aspect already. While noted, it's overly ambitious. + ${PAM_DEPS} + ${XEND_DEPS} looks ok, I'll run it past mgorny. + sys-devel/gcc this to my understanding has already been categorically refuted. hvm? ( x11-proto/xproto + sys-devel/bin86 sys-devel/dev86 looks ok, I shall test it out. While you're at it, could it make sense to remove the boot-loader RDEPEND for xen proper? I'll check but I think not
spoke too soon. -> use the ipxe-nopie-4.2.0.patch from the virtualization overlay Point 1. If it was ever named ipxe-nopie-4.2.0.patch, it's not now, it's ipxe-nopie.patch Point 2. If it ever worked out o' the box, it's time has well passed. boot_prompt_option.patch gpxe-git-0edf2405b457 gpxe-git-a803ef3dfeac NOT ONE of the patches are present in the current at at least my xen-4.2.0 Point 3. ~/cvsPortage/gentoo-x86/app-emulation/xen-tools $ find /mnt/gen2/TmpDir/portage/app-emulation/xen-tools-4.2.0-r2/work -name ipxe-nopie.patch yields a blank. Again, a BLANK. The virtualization overlay's patch is supposedly making and adding a new patch, by a method that might work in upstream's systems but dismally does NOTHING in ours. I have had to both re-make the ipxe-nopie.patch, note without the PV to make it generic, and make a 2nd. patch to patch tools/firmware/etherboot/patches/series which then adds the intended ipxe-nopie.patch to it. Point 4. Since I have a hardened likely so old it may not apply, the hardened team will need to test this combo, something not previously yet done. Oh the joys of xen
In reply to Comment#9, -- the ipxe-nopie stuff started working again right after I filed the bug; so, I'm not sure really what is needed or isn't needed anymore... (Sorry...) -- as for the python USE flag: since there is such a "global" USE flag already, it seemed prudent to re-use it; unfortunately, mostly due to 'pygrub', 'xend' is not sufficient to cover all python dependencies (on the other hand, 'pam' _may_, in fact, fit in under 'xend' -- I don't know) _To me_, it would certainly make sense to disable *all* python-dependent stuff if/when the user has '-python' in their USE flags... -- as for sys-devel/gcc, do as you please; since it's installed already on most systems, including it can be likened to a NOP, but in my specific setup I do actually need it (but that shouldn't burden others; so, it's OK) -- finally, for 'grub' being listed in xen-4.2.1.ebuild's RDEPEND, I'll just say that this makes zero sense to me. No kernels RDEPEND on 'grub' (or any other bootloader, but some do RDEPEND on 'gcc'!!! ;-)). 'grub' isn't even part of the 'system' set. Why should xen be different? Try 'grep -rl grub /usr/portage/sys-kernel'! In response to Comment 10, The original ipxe-nopie.patch on the 4.1.x series was called just that (i.e. without any numbers attached). Once that broke (with 4.1.3, I believe, which never showed up in the main Portage-tree AFAICT), numbered versions started to appear. I'll attach the one I was using on 4.2.0..
Created attachment 337304 [details, diff] ipxe-nopie-4.2.0.patch
Created attachment 337306 [details, diff] Patch to fix building on 32-bit hardened systems I thought I'd share this as well... (-; Just in case anyone follows Citrix in running a 32-bit dom0.
-- the ipxe-nopie stuff , attachment 337304 [details, diff] The attached patch looks like it addresses the current state of the xensource package, however its opening lines --- a/tools/firmware/etherboot/patches/ipxe-nopie.patch 1970-01-01 +++ b/tools/firmware/etherboot/patches/ipxe-nopie.patch 2011-03-27 simply don't work out o' the box and as such its basically useless. It's patching a patch. The patch isn't present on unpack. I have fixed it, so it's not an issue. -- as for the python USE flag: since there is such a "global" USE flag already, it seemed prudent to re-use it; _To me_, it would certainly make sense to disable *all* python-dependent stuff if/when the user has '-python' in their USE flags... Well, no derogatory tone intended, but in this you're a lone voice baying at the moon. The use of the python flag in global scope sent shudders down the spines of those few devs who perused it, and that's the polite summary. Your suggestion beckons the questions, Why a python free build?, What have you against python? What are you doing trying to fix something that isn't broken?, and so forth. Python is integral in gentoo portage and the devs at xensource designed the build to incorporate python. While I (we) welcome the submission of ideas and suggestions, in this case, I for one am in no way inclined to tease apart what was carefully crafted and joined into a complex build by the devs of xensource. In summary, what those informed and educated devs have joined in holy Makefiles and setup.pys, let no gentoo devs put asunder. -- finally, for 'grub' being listed in xen-4.2.1.ebuild's RDEPEND you have struck a chord here. There is some support for this which fundamentally second guesses the devs who set it in the 1st. place, but stranger things have happened. I shall make an inquiry into it and remove it pending the reply and see if the sky caves in. --Comment13. good one, thanks
(In reply to comment #14) > -- the ipxe-nopie stuff , attachment 337304 [details, diff] [details, diff] > > The attached patch looks like it addresses the current state of the > xensource package, however its opening lines > --- a/tools/firmware/etherboot/patches/ipxe-nopie.patch 1970-01-01 > +++ b/tools/firmware/etherboot/patches/ipxe-nopie.patch 2011-03-27 > > simply don't work out o' the box and as such its basically useless. It's > patching a patch. The patch isn't present on unpack. I have fixed it, so > it's not an issue. Well, well... I'm not sure how to say this without any feather ruffling, but you do realize that patch can (and does) _create_ new files, right? The ipxe-nopie.patch worked that way and worked just fine (with relevant modifications as the versions progressed) ever since Ralf submitted it... Your "fix" on the other hand fails. Practically, because it's all too easy to mess up -patch/diff- syntax when hand-editing (otherwise working) files. It's also a whole lot less elegant to "manually" copy a file that patch can (and did already) create for you... --- make[5]: Entering directory `/var/tmp/portage/app-emulation/xen-tools-4.2.1-r1/work/xen-4.2.1/tools/firmware/etherboot' rm -rf ipxe gzip -dc ipxe.tar.gz | tar xf - for i in $(cat patches/series) ; do \ patch -d ipxe -p1 --quiet <patches/$i || exit 1 ; \ done patch: **** Only garbage was found in the patch input. make[5]: *** [ipxe/src/arch/i386/Makefile] Error 1 make[5]: Leaving directory `/var/tmp/portage/app-emulation/xen-tools-4.2.1-r1/work/xen-4.2.1/tools/firmware/etherboot' --- On the other hand, I have some good news for you: 4.2.1 seems to build fine *without* patching ipxe at all... (Possibly, because we get your tarball [which may already be patched] instead of going to xensource; don't know...) > -- as for the python USE flag: since there is such a "global" USE flag > already, > it seemed prudent to re-use it; > _To me_, it would certainly make sense to disable *all* python-dependent > stuff if/when the user has '-python' in their USE flags... > > Well, no derogatory tone intended, but in this you're a lone voice baying at > the moon. The use of the python flag in global scope sent shudders down the > spines of those few devs who perused it, and that's the polite summary. > Your suggestion beckons the questions, Why a python free build?, What have > you against python? What are you doing trying to fix something that isn't > broken?, and so forth. Python is integral in gentoo portage and the devs at > xensource designed the build to incorporate python. While I (we) welcome the > submission of ideas and suggestions, in this case, I for one am in no way > inclined to tease apart what was carefully crafted and joined into a complex > build by the devs of xensource. In summary, what those informed and > educated devs have joined in holy Makefiles and setup.pys, let no gentoo > devs put asunder. LOL. I have absolutely nothing against python and I do realize that my target environment is somewhat special (I alluded to this fact in Comment #8 and gave a bit of a background in Comment #4), as it only includes the bare necessities to keep Xen with xl happy. (No ocaml, no python, no editor, nothing...) Of course, my *build* environment does have python, since I'm using portage's finesse to get a minimal target together... (-; > -- finally, for 'grub' being listed in xen-4.2.1.ebuild's RDEPEND > > you have struck a chord here. There is some support for this which > fundamentally second guesses the devs who set it in the 1st. place, but > stranger things have happened. I shall make an inquiry into it and remove it > pending the reply and see if the sky caves in. Thanks!
(In reply to comment #15) > (In reply to comment #14) > > -- the ipxe-nopie stuff , attachment 337304 [details, diff] [details, diff] [details, diff] > > > > The attached patch looks like it addresses the current state of the > > xensource package, however its opening lines > > --- a/tools/firmware/etherboot/patches/ipxe-nopie.patch 1970-01-01 > > +++ b/tools/firmware/etherboot/patches/ipxe-nopie.patch 2011-03-27 > > > > simply don't work out o' the box and as such its basically useless. It's > > patching a patch. The patch isn't present on unpack. I have fixed it, so > > it's not an issue. > > Well, well... I'm not sure how to say this without any feather ruffling, > but you do realize that patch can (and does) _create_ new files, right? > The ipxe-nopie.patch worked that way and worked just fine (with relevant > modifications as the versions progressed) ever since Ralf submitted it... > > Your "fix" on the other hand fails. Practically, because it's all too easy > to mess up -patch/diff- syntax when hand-editing (otherwise working) files. > It's also a whole lot less elegant to "manually" copy a file that patch can > (and did already) create for you... > > --- > make[5]: Entering directory > `/var/tmp/portage/app-emulation/xen-tools-4.2.1-r1/work/xen-4.2.1/tools/ > firmware/etherboot' > rm -rf ipxe > gzip -dc ipxe.tar.gz | tar xf - > for i in $(cat patches/series) ; do \ > patch -d ipxe -p1 --quiet <patches/$i || exit 1 ; \ > done > patch: **** Only garbage was found in the patch input. > make[5]: *** [ipxe/src/arch/i386/Makefile] Error 1 > make[5]: Leaving directory > `/var/tmp/portage/app-emulation/xen-tools-4.2.1-r1/work/xen-4.2.1/tools/ > firmware/etherboot' > --- Thanks to lines 7 and 8 from app-emulation/xen-tools/files/ipxe-nopie.patch: #--- tools/firmware/etherboot/patches/ipxe-nopie.patch 1970-01-01 01:00:00.000000000 +0100 #+++ tools/firmware/etherboot/patches/ipxe-nopie.patch 2011-03-27 17:45:13.929697782 +0200 Patch doesn't like the comment characters.
hmm ok good, well in part. So you say it does in fact create and add the patch all in 1, and the patch in your attachment 337304 [details, diff] is correct and works? I have finished preparing a number of patches and this seemed to not work in another, however, do a re-test and re-confirm for me that the state of that patch is good and I shall re-set it and take out the add-nopie.patch since it will be redundant. Being specific to hardened I needed a tester anyways. Also add whether it should be conditional as in the ebuild or unconditional as alexxy had it originally. I shall next take grub out of xen.
well, cancel the test run. I have run and tested it, it's fine. The initial content was wrong and this method of making the patch by a/ b/ I wasn't familiar with, only with the use of /dev/null so I have picked up an important snippet. Still would like your take on whether it should be conditional.
(In reply to comment #18) > well, cancel the test run. I have run and tested it, it's fine. The > initial content was wrong and this method of making the patch by a/ b/ I > wasn't familiar with, only with the use of /dev/null so I have picked up an > important snippet. Still would like your take on whether it should be > conditional. To quote myself from Comment#15 (I know that was a long one, and this bit was hiding right after the included error dump): " On the other hand, I have some good news for you: 4.2.1 seems to build fine " *without* patching ipxe at all... (Possibly, because we get your tarball " [which may already be patched] instead of going to xensource; don't know...) in other words, you don't need this patch anymore! In the meantime, I checked your tarball and the Makefile in there has empty CFLAGS (i.e. without -nopie) and it still builds fine with hardened gcc-4.6.3. I have no idea what changed or how other compilers might react; so, I cannot say for sure that omitting the patch altogether is safe on all systems, but it does work on mine. (If I had the time to investigate I'd look at the original error report and see if the ipxe sources changed to fix the error internally before getting rid of this patch, but seeing how portage "testing" is often used to validate such ideas, you could just go ahead and remove the patch and wait for complaints. ;0) As for your original question: conditional or not, I don't think -nopie hurts anyone, but if you keep this patch having it conditional should work fine. (I have both hardened and plain systems with Xen and they plain ones never coughed about this _at_all_)
" On the other hand, I have some good news for you: 4.2.1 seems to build fine. ok fine. This implies 4.2.1 is fine without it, 4.2.0 still needs it. You ought realise the ebuild simply acquires the xensource tarball. SRC_URI="http://bits.xensource.com/oss-xen/release/${PV}/xen-${PV}.tar.gz Peruse and you will see 4.2.0 is up for the rigors of testing to go stable. Anything distinguishing their state is noteworthy. "but if you keep this patch having it conditional should work fine." agreed. done. "but seeing how portage "testing" is often used to validate such ideas" Quite so. Thanks for your input. Re-open as you see fit.
