Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 219066 - eix's PORTDIR_CACHE_METHOD=parse|ebuild* spews warnings
Summary: eix's PORTDIR_CACHE_METHOD=parse|ebuild* spews warnings
Status: RESOLVED UPSTREAM
Alias: None
Product: Gentoo/Alt
Classification: Unclassified
Component: Prefix Support (show other bugs)
Hardware: All Linux
: High minor (vote)
Assignee: Gentoo non-Linux Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-04-23 20:35 UTC by Jeremy Olexa (darkside) (RETIRED)
Modified: 2008-05-12 17:07 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2008-04-23 20:35:11 UTC
(This is probably directed at the eix devs)

2 part question, a) why does the ebuild explicitly contain a defaulted configure call (--with-portdir-cache-method)? let alone a slower one. b) why does embassy.eclass spew warnings?

% time update-eix
Reading Portage settings ..
Building database (/home/jolexa/portage/linux-32/var/cache/eix) ..
[0] "gentoo_prefix" /home/jolexa/portage/global/portage_tree/ (exec: /home/jolexa/portage/linux-32) (cache: parse|ebuild*)
     Reading  72%/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found
/home/jolexa/portage/global/portage_tree//eclass/embassy.eclass: line 20: tr: command not found                                                             100%
[1] "" /home/jolexa/portage/global/overlays (exec: /home/jolexa/portage/linux-32) (cache: parse|ebuild*)
     Reading 100%
Applying masks ..
Calculating hash tables ..
Writing database file /home/jolexa/portage/linux-32/var/cache/eix ..
Database contains 1760 packages in 151 categories.

real    2m30.098s
user    1m20.207s
sys     0m53.216s

% time PORTDIR_CACHE_METHOD=parse update-eix
Reading Portage settings ..
Building database (/home/jolexa/portage/linux-32/var/cache/eix) ..
[0] "gentoo_prefix" /home/jolexa/portage/global/portage_tree/ (cache: parse)
     Reading 100%
[1] "" /home/jolexa/portage/global/overlays (exec: /home/jolexa/portage/linux-32) (cache: parse|ebuild*)
     Reading 100%
Applying masks ..
Calculating hash tables ..
Writing database file /home/jolexa/portage/linux-32/var/cache/eix ..
Database contains 1760 packages in 151 categories.

real    0m8.267s
user    0m2.661s
sys     0m2.522s

Furthermore, you may want to fix the output of --dump:

# STRING
# Portage cache-backend that should be used for PORTDIR
# (metadata/sqlite/cdb/portage-2.0/portage-2.1/none[*]/ebuild[*]/eix[*][:*])
PORTDIR_CACHE_METHOD='parse|ebuild*'

"none" doesn't exist anymore according to the man page and trials.
Comment 1 Emil Beinroth 2008-05-06 22:51:26 UTC
Because it's probably the best one for ALT/prefix users, since they cannot use the default "metadata" -> see http://bugs.gentoo.org/show_bug.cgi?id=209239#c19

The "parse" method will parse ebuilds with our internal sh-parser, which can only handle variables and basic brace-substitutions like bar="$foo". So this will yield wrong results for ebuilds like sys-kernel/gentoo-sources, because eclasses and function calls are involved in setting DESCRIPTION.

parse|ebuild* OTOH employs the "ebuild*" method as a fallback, for when "parse" yields questionable results (for example, missing variables like IUSE, KEYWORDS and so forth). "ebuild*" will let ebuild(1) or ebuild.sh (undocumented) generate a flat cache file, and later read that file itself to get the correct data for the ebuild in question.

So, in conclusion: parse|ebuild* may be slower, but it should yield the correct results.


I have no idea what's up with the tr-issue. Can you please put a echo "$PATH" before the offending line in embassy.eclass, any report back? Thanks in advance.
Comment 2 Martin Väth 2008-05-07 09:32:32 UTC
I had posted a reply yesterday afternoon, but it seems it got lost...
Anyway, I had posted more or less the same as Emil; the only difference is
that I had suggested to put the line
   echo "PATH=$PATH" >/tmp/something
into embassy.eclass (i.e. with redirection to a file, since probably the output
will get lost). Then please post the content of /tmp/something after running
update-eix and the paths where "tr" can be found on your system,
e.g. the output of
  bash -c "type -a tr"

BTW: Thanks for remarking about the false --dump documentation.
This is already fixed in eix svn trunk.
Comment 3 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2008-05-07 17:22:31 UTC
First, thanks for the explanation of the different caching types! prefix just needs a rsync mirror..problem solved ;)

(In reply to comment #2)
>    echo "PATH=$PATH" >/tmp/something
> into embassy.eclass (i.e. with redirection to a file, since probably the 

Well, I definitely found a problem there. It is unclear to me whether or not portage sanitizes its PATH or not but 'echo $PATH' in embassy.eclass returns:

:/home/jolexa/portage/linux-32/usr/lib/portage/bin:

How can I get to the bottom of this? Any ideas?
Comment 4 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2008-05-07 17:29:30 UTC
Actually, maybe "tr" shouldn't even BE in the eclass..
Comment 5 Fabian Groffen gentoo-dev 2008-05-07 18:45:41 UTC
that looks like a somehow messed up PATH.  Is the eclass messing with PATH?  (Or the previous eclass?)  Portage uses the PATH as defined in DEFAULT_PATH (etc/make.globals)
Comment 6 Martin Väth 2008-05-07 20:52:02 UTC
(In reply to comment #5)
> Portage uses the PATH as defined in DEFAULT_PATH (etc/make.globals)

Well, with PORTDIR_CACHE_METHOD='parse|ebuild*', update-eix does not call
the full "portage" ebuild but only the undocumented
/usr/lib/portage/bin/ebuild.sh
In "standard" portage, this script calculates a PATH on its own.
Maybe in prefix this script is rather different and relies on PATH being set?
(currently, update-eix does not set PATH at all when calling the script).

Is the result the same when you use PORTDIR_CACHE_METHOD='parse|ebuild'
(without '*')? (In this case, "full" portage is used).
If yes, then the reason probably is that some eclass (or an ebuild which
imports embassy) messes up PATH.
Comment 7 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2008-05-07 23:04:43 UTC
(In reply to comment #6)
> Is the result the same when you use PORTDIR_CACHE_METHOD='parse|ebuild'
> (without '*')? (In this case, "full" portage is used).
> If yes, then the reason probably is that some eclass (or an ebuild which
> imports embassy) messes up PATH.

Yes, PORTDIR_CACHE_METHOD='parse|ebuild' works without issues.

Comment 8 Martin Väth 2008-05-08 07:03:11 UTC
Then I guess that update-eix should export the current PATH to ebuild.sh:
Since DEFAULT_PATH does not exist in "standard" portage, this is probably
the only reliable information on PATH (and if ebuild.sh sets up a full path
as in standard portage, it does not matter anyway).

PATH is now simply passed in current eix svn trunk. Please test.
Comment 9 Fabian Groffen gentoo-dev 2008-05-08 07:19:35 UTC
in prefix the ebuild.sh does this:

export PATH="${DEFAULT_PATH}:${PORTAGE_BIN_PATH}:${ROOTPATH}"

So I guess either DEFAULT_PATH needs to be set, or (perhaps better?) make.globals need to be sourced, where DEFAULT_PATH and some other (sometimes necessary) stuff is defined.
Comment 10 Martin Väth 2008-05-08 15:53:19 UTC
(In reply to comment #9)
> export PATH="${DEFAULT_PATH}:${PORTAGE_BIN_PATH}:${ROOTPATH}"

Thanks. This explains of course the strange PATH
(And it explains why my previous "fix" would probably not have worked).
Anyway, I think update-eix will continue to export PATH (if it is set in
ebuild.sh, it does not harm).

> make.globals need to be sourced

Yes, it seems that all portage settings variables should be exported,
i.e. anything which is set in make.globals/make.conf/profile...

Please test new eix svn trunk.
Comment 11 Jeremy Olexa (darkside) (RETIRED) archtester gentoo-dev Security 2008-05-12 17:07:54 UTC
(In reply to comment #10)
> Please test new eix svn trunk.

% ./update-eix
Reading Portage settings ..
Building database (/home/jolexa/portage/linux-32/var/cache/eix) ..
[0] "gentoo_prefix" /home/jolexa/portage/global/portage_tree/ (exec: /home/jolexa/portage/linux-32) (cache: parse|ebuild*)
     Reading 100%
[1] "" /home/jolexa/portage/global/overlays (exec: /home/jolexa/portage/linux-32) (cache: parse|ebuild*)
     Reading 100%
Applying masks ..
Calculating hash tables ..
Writing database file /home/jolexa/portage/linux-32/var/cache/eix ..
Database contains 1768 packages in 151 categories.

% ./eix --version
eix 0.12.5 (i686-pc-linux-gnu)

Confirmed. I'll close this bug as Resolved, UPSTREAM. Please re-open if needed.