By wrapping etc-update's three portageq calls in a single python command, etc-update gets a huge speed boost. # time echo -1 | ./etc-update >/dev/null real 0m8.575s user 0m6.675s sys 0m1.899s # time echo -1 | /usr/sbin/etc-update >/dev/null real 0m21.541s user 0m18.341s sys 0m2.892s Patch coming up...
Created attachment 61928 [details, diff] the patch It's safe to split by newlines here since CONFIG_PROTECT* and USERLAND can't possibly contain any newlines themselves (CONFIG_PROTECT* are incremental, so any whitespace will be changed to a single space, and USERLAND is set by portage itself), but if you want to be safe in case etc-update will need more variables later on, I can try to modify this to use '\0' as the separator instead.
this would be even better i think: eval $(python -c 'import portage print "USERLAND=\""+portage.settings["USERLAND"]+"\";" print "CONFIG_PROTECT=\""+portage.settings["CONFIG_PROTECT"]+"\";" print "CONFIG_PROTECT_MASK=\""+portage.settings["CONFIG_PROTECT_MASK"]+"\";" ') that way you dont have to worry about newlines or anything
That won't work correctly if any string contains a $, a \, a ", a `, a newline or tab (they'll be replaced with spaces in your version, which is not always correct); it's just as likely to work now, but more likely to break in the future :) But, it would be good, I think, if $(...) became "$(...)", if the assignments became VAR='value', and if the values got a s/'/'\\''/g.
Putting a hold on feature requests for portage as they are drowning out the bugs. Most of these features should be available in the next major version of portage. But for the time being, they are just drowning out the major bugs and delaying the next version's progress. Any bugs that contain patches and any bugs for etc-update or dispatch-conf can be reopened. Sorry, I'm just not good enough with bugzilla. ;)
> Any bugs that contain patches and any bugs for etc-update or dispatch-conf can be reopened. Alrighty.
Created attachment 74448 [details, diff] faster-etc-update Updated patch for portage-2.0.53.
actually i just added support to portageq to handle multiple vars so you could do `portageq envvar A B C D` and get all four back, each on a line
and actually, i just added support to trunk so you can do: eval $(portageq envvar -v USERLAND CONFIG_PROTECT CONFIG_PROTECT_MASK) $ portageq envvar -v ARCH PORTDIR USERLAND ARCH='amd64' PORTDIR='/usr/portage' USERLAND='GNU'
> actually i just added support to portageq to handle multiple vars > > so you could do `portageq envvar A B C D` and get all four back, each on a line Is that a good idea? It has the same potential problem with newlines. > and actually, i just added support to trunk so you can do: > eval $(portageq envvar -v USERLAND CONFIG_PROTECT CONFIG_PROTECT_MASK) > > $ portageq envvar -v ARCH PORTDIR USERLAND > ARCH='amd64' > PORTDIR='/usr/portage' > USERLAND='GNU' Great! Does this correctly change single quotes in variables too?
i dont see a problem ... portage strips out newlines from variables > > $ portageq envvar -v ARCH PORTDIR USERLAND > > ARCH='amd64' > > PORTDIR='/usr/portage' > > USERLAND='GNU' > > Great! Does this correctly change single quotes in variables too? no ... maybe it should print out " instead of '
(In reply to comment #10) > > Great! Does this correctly change single quotes in variables too? > > no ... maybe it should print out " instead of ' In that case, it should change double quotes, dollar signs, backticks and backslashes in variables to be sure to get correct results. Probably easier to stick with ' and change that.
> > no ... maybe it should print out " instead of ' > > In that case, it should change double quotes, dollar signs, backticks and > backslashes in variables to be sure to get correct results. right, which is why i picked ' instead of " all in all, the quoting issue doesnt affect this bug as the variables we are after should not have quotes in them
> i dont see a problem ... portage strips out newlines from variables Sorry, overlooked this. portage only strips out newlines from incremental variables. Try $ mkdir $'/tmp/port\nage' $ PORTAGE_TMPDIR=$'/tmp/port\nage' portageq envvar PORTAGE_TMPDIR for example. > right, which is why i picked ' instead of " > > all in all, the quoting issue doesnt affect this bug as the variables we are > after should not have quotes in them PORTAGE_TMPDIR can legitimately contain any character (think "/var/tmp/portage's \`domain'" - sure, I wouldn't use it, but it's valid, isn't it?), and it's used by etc-update.
Bleh, I just noticed etc-update already doesn't properly quote PORTAGE_TMPDIR... > TMP="${PORTAGE_TMPDIR}/$$" > rm -rf ${TMP} 2> /dev/null > mkdir ${TMP} || die "failed mkdir command!" 1 So while I do personally consider it a bug, I guess it's not too big a deal if it already doesn't work now.
> PORTAGE_TMPDIR can legitimately contain any character yes, but atm, a hell of a lot more would break than just etc-update if there were such things that required quoting, such as a significant number of ebuilds which never quote $D or $S or $WORKDIR or ...
in r3934
This has been released in 2.1.1_pre3-r2.