For testing, I moved my portage tree to /tmp/portage/tree eix-update still expects /usr/portage, this is wrong because the tree moved set PORTDIR=/tmp/portage/tree in make.conf doesn't work set PORTDIR=/tmp/portage/tree in /etc/eixrc/00-eixrc doesn't work PORTDIR=/tmp/portage/tree eix-update works Reproducible: Always Steps to Reproduce: For testing, I moved my portage tree to /tmp/portage/tree eix-update still expects /usr/portage, this is wrong because the tree moved set PORTDIR=/tmp/portage/tree in make.conf doesn't work set PORTDIR=/tmp/portage/tree in /etc/eixrc/00-eixrc doesn't work PORTDIR=/tmp/portage/tree eix-update works Actual Results: Eix doesn't recognize the correct portage tree When eix is recognizing the wrong result: For testing, I moved my portage tree to /tmp/portage/tree eix-update still expects /usr/portage, this is wrong because the tree moved set PORTDIR=/tmp/portage/tree in make.conf doesn't work set PORTDIR=/tmp/portage/tree in /etc/eixrc/00-eixrc doesn't work PORTDIR=/tmp/portage/tree eix-update works When eix is recognizing the right results it updates the tree as usual Expected Results: eix recognizes the correct portage tree and lets me search "i.e eix gcc" emerge --info Portage 2.2.20 (python 2.7.10-final-0, default/linux/amd64/13.0, gcc-4.9.3, glibc-2.20-r2, 4.1.2-gentoo x86_64) ================================================================= System uname: Linux-4.1.2-gentoo-x86_64-Intel-R-_Core-TM-_i5-5300U_CPU_@_2.30GHz-with-gentoo-2.2 KiB Mem: 7840032 total, 3953560 free KiB Swap: 0 total, 0 free Timestamp of repository gentoo: Thu, 16 Jul 2015 00:45:01 +0000 sh bash 4.3_p39 ld GNU ld (Gentoo 2.25 p1.2) 2.25 app-shells/bash: 4.3_p39::gentoo dev-lang/perl: 5.22.0::gentoo dev-lang/python: 2.7.10::gentoo, 3.4.3::gentoo dev-util/cmake: 3.2.3::gentoo dev-util/pkgconfig: 0.28-r3::gentoo sys-apps/baselayout: 2.2::gentoo sys-apps/openrc: 0.17::gentoo sys-apps/sandbox: 2.6-r1::gentoo sys-devel/autoconf: 2.69-r1::gentoo sys-devel/automake: 1.14.1::gentoo, 1.15::gentoo sys-devel/binutils: 2.25-r1::gentoo sys-devel/gcc: 4.9.3::gentoo sys-devel/gcc-config: 1.8::gentoo sys-devel/libtool: 2.4.6-r1::gentoo sys-devel/make: 4.1-r1::gentoo sys-kernel/linux-headers: 4.1::gentoo (virtual/os-headers) sys-libs/glibc: 2.20-r2::gentoo Repositories: gentoo location: /tmp/portage/tree sync-type: webrsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000 qutebrowser location: /var/lib/layman/qutebrowser masters: gentoo priority: 50 ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo" CXXFLAGS="-O2 -pipe" DISTDIR="/tmp/portage/dist" EMERGE_DEFAULT_OPTS="--quiet --jobs=1 " FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="" LDFLAGS="-Wl,-O1 -Wl,--as-needed" MAKEOPTS="-j4" PKGDIR="/tmp/portage/pkg" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_EXTRA_OPTS="--exclude-from=/etc/portage/package.excludes" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/tmp/portage/tmpdir" USE="amd64 bash-completion readline systemd threads" ABI_X86="32 64" CURL_SSL="openssl" ELIBC="glibc" INPUT_DEVICES="evdev" KERNEL="linux" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby22" USERLAND="GNU" VIDEO_CARDS="intel i965" USE_PYTHON="3.4 2.7" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS
what is your eix version?
Installed versions: 0.30.11(17:38:11 07/16/15)(strong-optimization -debug -dep -doc -linguas_de -linguas_ru -nls -optimization -security -sqlite -strong-security -swap-remote -tools)
The new way to specify repositories is by using repos.conf. Current versions of portage contain /usr/share/portage/config/repos.conf which specifies the location if no /etc/portage/repos.conf exists. It would be wrong (and incompatible with future portage versions) to let PORTDIR override repos.conf (if at least one repos.conf exists). The *environment* variable PORTDIR is honoured anyway to allow quick overrides without specifying a full repos.conf, but this is an eix hack and might be removed in the future (since it is incompatible with portage). Any change in current eix behaviour would trigger incompatibility with portage (if not with current version, then with future version and declared plans of the portage team to deprecate PORTDIR completely.)
So, what do you suggest i do to have my expanded portage tree in /tmp/ and the compressed tree reside on my hardrive. I unpack when needed.
(In reply to cryptopsy from comment #4) > So, what do you suggest i do to have my expanded portage tree in /tmp/ and > the compressed tree reside on my hardrive. I unpack when needed. Again, as PORTDIR is deprecated, the solution here would be to modify or create /etc/portage/repos.conf that specifies the location as /tmp/portage/tree.
(In reply to Ian Stakenvicius from comment #5) > (In reply to cryptopsy from comment #4) > > So, what do you suggest i do to have my expanded portage tree in /tmp/ and > > the compressed tree reside on my hardrive. I unpack when needed. > > Again, as PORTDIR is deprecated, the solution here would be to modify or > create /etc/portage/repos.conf that specifies the location as > /tmp/portage/tree. There are also of course system based options too -- for instance, 'mount --bind /tmp/portage/tree /usr/portage' after unpacking would work for your particular use case.
I do not know what is your intention with the unpacking. A few general remarks: 1. Using a predictable name in /tmp is a security risk (since /tmp is world-writable). If you are using a predictable name, why not put it in /var/cache with appropriate permissions and set repos.conf appropriately? (You can also temporarily override the content of repos.conf by putting the content into "PORTAGE_REPOSITORIES", in case you do not want to touch a system file from a shell script.) 2. If you are trying to mount a squashed version of the portage tree, you might want to have a look at the squashmount package from the mv overlay.
my /etc/portage/repos.conf/gentoo.conf: [DEFAULT] main-repo = gentoo [gentoo] location = /tmp/portage/tree sync-type = webrsync sync-uri = rsync://rsync.gentoo.org/gentoo-portage auto-sync = yes # for daily squashfs snapshots #sync-type = squashdelta #sync-uri = mirror://gentoo/../snapshots/squashfs I unpack the portage tree to /tmp/portage/tree when I need it, /tmp is tmpfs. Otherwise it resides compressed on the drive.
Created attachment 410372 [details] make.conf I used to use PORTDIR up until I discovered this bug, turns out I don't set it in make.conf. This is the make.conf that I used when I generated the bug.
I originally referred to it as PORTDIR in my oriignal report, because that's what its called in emerge --info when i double checked the parameters (since i had many xterm open, i didn't want to confuse it)
(In reply to cryptopsy from comment #10) > I originally referred to it as PORTDIR in my oriignal report, because that's > what its called in emerge --info when i double checked the parameters (since > i had many xterm open, i didn't want to confuse it) So then the issue is actually that eix doesn't work wiht repos.conf the way it is? You're running eix-update, and only otherwise running eix queries, when /tmp/portage/tree exists anr your tree archive has been unpacked into it, right?
correct, i am using this as a temporary fix PORTIDR="/tmp/portage/tree" eix-sync && eix-update , everytime i want to use eix after unpacking
(In reply to cryptopsy from comment #12) > correct, i am using this as a temporary fix > PORTIDR="/tmp/portage/tree" eix-sync && eix-update , everytime i want to use > eix after unpacking Oh so the issue you are seeing is that you *need* to set PORTDIR= to make it work??
(In reply to Ian Stakenvicius from comment #13) > (In reply to cryptopsy from comment #12) > > correct, i am using this as a temporary fix > > PORTIDR="/tmp/portage/tree" eix-sync && eix-update , everytime i want to use > > eix after unpacking > > Oh so the issue you are seeing is that you *need* to set PORTDIR= to make it > work?? eix doesn't recognize repos.conf's location of the portage tree if i'm not mistaken, but since PORTDIR is obsolete i shouldn't have to be using that.
So your repos.conf is correct, portage recognizes it, but eix does not? Finally, it becames clear why you consider this an eix bug... First, let us exclude the "usual suspects": 1. Are you sure that /etc/portage/repos.conf/gentoo.conf and its parent directories are readable/accessible by user:group portage:portage? 2. What is the output of eix-update --print PORTDIR 3. What is the output of (this should be one line; bugtilla will break it): PORTDIR= PORTAGE_REPOSITORIES=$(cat /etc/portage/repos.conf/*) eix-update --print PORTDIR 4. Does it change something if you replace /*) by /gentoo.conf) in 3?
1. yes 2. /tmp/portage/tree/ 3. /tmp/portage/tree/ 4. stays the same
So, by 2, eix-update actually has calculated the correct PORTDIR from repos.conf. It is completely mysterious to me how in this case the behaviour of eix-update and of PORTDIR=/tmp/portage/tree eix-update might differ. Can you still reproduce the problem (with your current repos.conf)?
Since /tmp/ is tmpfs, i have to eix-update after unpacking the tree with that PORTDIR parameter before it. I can reproduce it that way. I cleaned /tmp/portage and extract the compressed tree to produce /tmp/portage/tree/* . Then I esync. Now the eix stuff With eix-sync && eix-update , does not work. with PORTIDR="/tmp/portage/tree" eix-sync && eix-update , works. To verify this once more, i would have to reboot, but I cannot because i have some jobs running. I am not sure where the eix database is to remove it so I can try again without rebooting.
1. The eix database is in /var/cache/eix/, although I do not know what this has to do with your problem (the database is recreated with eix-update anyway). 2. In your command line PORTIDR="/tmp/portage/tree" eix-sync && eix-update you do *not* pass PORTDIR to eix-update. So, in fact, you are calling eix-update in both cases with the same environment. Perhaps for some reason you have generally the problem that syncing fails for the first time. Perhaps there is an issue with timestamps on your tmpfs? So far, you actually did not tell us what fails. ("eix expects the portage tree in /usr/portage" is a speculation about the cause of something but not a description of the problem.)
It seems to work now, without PORTDIR defined in a bash function, or exported. I don't understand what's going on. I have been rebooting trying new things. For example, 'eix openra' shows the use flags: {cg tools}. But 'equery uses' shows !!! No USE flags found for games-strategy/openra-20141029-r1
Are you sure that you have a correct (and up-to-date) metadata/md5-cache directory in your portage tree? You have to update it manually if you use e.g. git for syncing. If you do not have, portage might generate a (partial) substitute in /var/cache/edb, but this is then stored under the portdir path when it is generated and might confuse portage (and depending on your cache method also eix) if you move the portage tree.
ls -ls metadata/ total 92K drwxr-xr-x 7 root root 260 Sep 3 23:18 . drwxr-xr-x 136 root root 2.8K Aug 26 14:45 .. -rw-r--r-- 1 root root 148 Aug 25 00:31 .gitignore drwxr-xr-x 2 root root 400 Sep 3 23:18 dtd drwxr-xr-x 2 root root 42K Sep 3 23:18 glsa -rw-r--r-- 1 root root 72K Aug 28 10:18 herds.xml drwxr-xr-x 2 root root 60 Aug 16 17:18 install-qa-check.d -rw-r--r-- 1 root root 1.2K Sep 3 00:37 layout.conf drwxr-xr-x 131 root root 2.6K Jul 15 17:52 md5-cache drwxr-xr-x 81 root root 1.7K Sep 3 23:18 news -rw-r--r-- 1 root root 29 Sep 3 00:42 timestamp -rw-r--r-- 1 root root 32 Sep 3 00:45 timestamp.chk -rw-r--r-- 1 root root 43 Sep 3 00:40 timestamp.x just unpacked my tree, 'eix gcc' says "no matches found". The command used to unpack was has the following elements: esync; eix-sync && eix-update; # the one that produce the result esync; export PORTDIR="/tmp/portage/tree"; eix-sync && eix-update # previously it was notel; with the export, 'eix gcc' shows correctly
Both of your commands are somewhat redundant since eix-sync should already call eix-update, unless you have a special configuration. However, this should not play a role for the problem. Please post the output of the last "eix-update" in both cases. In case they are identical: Please check whether the content of the metadata/md5-cache subdirectory is identical in both cases. Does something change (in one of the two cases) if you call rm -rf /var/cache/edb/dep before the last call to eix-update?
eix-update: eix is hashed (/usr/bin/eix) eix-update # doesn't work Reading Portage settings .. Building database (/var/cache/eix/portage.eix) .. cannot open /usr/portage/profiles/categories: No such file or directory [0] '' /usr/portage/ (cache: metadata-md5-or-flat) Reading category 0|0 (100%) EMPTY! Applying masks .. Calculating hash tables .. Writing database file /var/cache/eix/portage.eix .. Database contains 0 packages in 0 categories. export PORTDIR="/tmp/portage/tree"; eix-update #works Reading Portage settings .. Building database (/var/cache/eix/portage.eix) .. [0] 'gentoo' /tmp/portage/tree/ (cache: metadata-md5-or-flat) Reading category 161|161 (100%) Finished [1] '' /usr/portage (cache: parse|ebuild*#metadata-md5#metadata-assign#assign) Reading category 161|161 (100%) EMPTY! Applying masks .. Calculating hash tables .. Writing database file /var/cache/eix/portage.eix .. Database contains 15554 packages in 161 categories.
It seems that the problem is hardly reproducable, especially since eix uses the PORTDIR environment variable only in exactly one place, namely to calculate the internal portdir which is output with --print PORTDIR (and above you got the same result in both cases...) Above you had said that exporting PORTDIR or not does not make a difference at all, now it does again? What has changed? If you can now really reproduce that PORTDIR=/tmp/portage/tree eix-update and eix-update give different output (please check again!). Does then (hopefully) also PORTIDIR=/tmp/portage/tree eix-update --print PORTDIR and eix-update --print PORTDIR give different output? If yes, my first conjecture is that there is now again a problem with reading your repos.conf (e.g. a permission problem); check perhaps with strace eix-update --print PORTDIR whether repos.conf is successfully opened. Also PORTAGE_REPOSITORIES=$(cat /etc/portage/repos.conf/*) eix-update --print PORTDIR and the same with prefixed PORTDIR="" or PORTDIR=/tmp/portage/tree, respectively.
comment #24 is valid, but PORTIDIR=/tmp/portage/tree eix-update --print PORTDIR and eix-update --print PORTDIR produce the same results. BUT, this is only after my tree has already been deployed. If i reboot and start from scratch, I don't know if it will. Perhaps this is where the confusion arises. I will reboot after some jobs finish running.
with eix-update () { export PORTDIR=/tmp/portage/tree; command eix-update "$@"; } overlays are not recognized by eix-0.30.11 when doing eix-update For example, the gamerlay overlay adds rtcw-9999, i can emerge it but i can't see it. I know the overlays were added properly with layman -l
(In reply to cryptopsy from comment #27) > overlays are not recognized by eix-0.30.11 when doing eix-update This can be a rather different story: Are the _overlays_ or their _packages_ not recognized? More precisely, does OVERLAYS_LIST=all eix --not list the overlays (alternatively: Does the output of eix-update contain the overlays)? If not: Where does layman produce its repos.conf? Post this file. Might it be that this file or - more likely - one of its parent directories is sometimes(?) not readable (in case of directories not +rx) by portage:portage? (Somehow it seems that this is the case with your /etc/portage/repos.conf directory). And how about the portage tree/overlay directories themselves? (If they are not readable/accessible, eix-update also ignores them silently). So you perhaps start eix-update in your script with a different user than root? Check with strace whether eix-update tries to open that file and whether this is successful.
OVERLAYS_LIST=all eix --not lists the overlays: [0] "gentoo" /tmp/portage/tree/ [1] /usr/portage No matches found I suppose there should be others listed as well, since layman -l shows * qutebrowser [Git ] eix-update doesn't list that overlay, neither does PORTDIR=/tmp/portage/tree eix-update drwxr-xr-x 139 root root 2.8K Jan 2 13:36 tree repos.conf is rx too strace of eix STDOUT and STDERR shows open("/etc/portage/repos.conf/", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = -1 EACCES (Permission denied) open("/etc/portage/repos.conf", O_RDONLY) = -1 EACCES (Permission denied) but it is: drwx------ 1 root root 44 Sep 14 16:17 repos.conf and i am running eix-update
eix[-update] drops permissions to portage:portage as soon as possible. Set EIX_USER, EIX_GROUP, EIX_UID, EIX_GID if you want to modify this.
is that something i should be doing?
I cannot prescribe your security concepts. If you make the portage-related data (in particular /etc/portage and its content) readable (and /var/cache/eix writable) for user (or group) portage you do not have to fiddle with these permissions. If you think that even portage has too high privileges for eix, you can introduce another group. Or if you want to raise the privileges of eix you can also keep the original user/group root. The latter is of course the most dangerous approach, since malevolently crafted data in /etc/portage (or in some portage tree/overlay or in the environment or in /etc/eixrc or in ...) might be possibly be used to let eix do bad things (it is very likely that eix - like any complex program - has bugs which might be exploited by specially crafted input data).
