ok, firstly a quick description of the problem: a large number of packages either utilize xargs in their ebuilds, or inside their own build systems. with the ever-growing environment, we are starting to hit the arbitrary 20kb limit that GNU xargs imposes. The problem is coming up because _every_ function and variable that is defined in the shell takes up space in the environment, and when you have large eclasses and ebuilds, things start to go beyond the 20kb limit. The OpenOffice ebuild has this problem presently (bug 31378). I was working on some major changes to my php eclass today, and afterwards, the xargs invocation I have in mod_php was causing weird compile failures. 124 ebuilds currently use xargs directly. Baselayout was previously effected with this problem, and the invocation: ( find /var/lock -type f -print0 | xargs -0 rm -f -- 1>&2 ) got changed to: ( find /var/lock -type f -exec rm -f {} \; ) While similiar variations on this can be made in all our ebuilds, we would still get the problems from the package's build systems. I don't believe this is the answer then, also as my mod_php case required a long sed expression to replace what was being done very simply before. either ebuild.sh needs to reduce the size of the enviroment it's passing in (dangerous i think, and won't nessicarily solve the problem), or we need to adjust the 20kb limit in xargs. see lines 303-304 of findutils-4.1.20/xargs/xargs.c for their limit.
I have made a patch and a new ebuild and am ready to commit it. I want to do this soon as I get tons of bugs related to this. As I now know the cause I have no problem with fixing xargs for this.
no objections to it here. i've moved the limit to 64kb for xargs on my personal machines, but i'm wondering where we should put the proper limit.
I committed 4.1.7-r5 (stable) and 4.1.20-r1 (unstable) with limits of 50k. This indeed is quite arbitrary, but it solves all current problems. However we might want to consider to look at removing fully the broken behaviour from xargs, and make it allocate the buffer it needs.
does debian fix this ? or perhaps redhat ?
I don't know. I needed a temporary fix rather fast as the stable openoffice-1.1.0 was failing. (When I marked it stable it wasn't an issue yet)
I checked Debian, Redhat, Mandrake, and none of them presently include a patch for it. Both Debian and Redhat included patches for it previously, as they seem to have had some other tool that had problems just like portage for us, but they since deprecated those tools and removed the patches (more than a year ago).
An big problem was functions.sh from baslayout which due to misbehaving bash did a 'set -a'. I removed this in CVS again - not sure though if my issues will be fixed, but should help here (and one or two other bugs).
Not much that portage can do about this... and I'd imagine it's handled. Someone want to determine if it is, and close this?
paul, we all set on this now ?
There seem to be no problems anymore. I don't know however if that will stay this way. I guess it depends on the environment size increases of portage. I'll mark it later so that we can look into actual fundamental fixes when problems arise.
