Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 661006 - sys-apps/portage-2.3.40-r1: crash when performing postinstall actions on cross compiled packages
Summary: sys-apps/portage-2.3.40-r1: crash when performing postinstall actions on cros...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal major (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks: 659322
  Show dependency tree
 
Reported: 2018-07-12 15:08 UTC by Gordon Bos
Modified: 2018-10-12 19:32 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Gordon Bos 2018-07-12 15:08:03 UTC
When unmerging (using depclean) the still active older binutils emerge makes the switch to the newer version. Apparently this is internal code, because I cannot replicate the crash when running eselect. The crash also does not occur when I have the newer version marked active before doing the unmerge.

Reproducible: Always

Actual Results:  
Relevant portion of output:

<<<          dir /usr/armv5tel-softfloat-linux-gnueabi/binutils-bin/2.29.1
--- !empty   dir /usr/armv5tel-softfloat-linux-gnueabi/binutils-bin
--- !empty   dir /usr/armv5tel-softfloat-linux-gnueabi
--- !empty   dir /usr
--- !empty   dir /etc/env.d/binutils
--- !empty   dir /etc/env.d
--- !empty   dir /etc
 * Switching to armv5tel-softfloat-linux-gnueabi-2.30 ...                                                         [ ok ]
!!! Error: SYSROOT (currently /usr/armv5tel-softfloat-linux-gnueabi/) must equal / or ROOT (currently /).
Traceback (most recent call last):
  File "/usr/lib/python-exec/python2.7/env-update", line 35, in <module>
    portage.env_update(makelinks)
  File "/usr/lib/python2.7/site-packages/portage/proxy/objectproxy.py", line 31, in __call__
    return result(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/portage/util/env_update.py", line 51, in env_update
    eprefix = portage.settings["EPREFIX"]
  File "/usr/lib/python2.7/site-packages/portage/proxy/objectproxy.py", line 44, in __getitem__
    return object.__getattribute__(self, '_get_target')()[key]
  File "/usr/lib/python2.7/site-packages/portage/__init__.py", line 651, in _get_target
    return _get_legacy_global(name)
  File "/usr/lib/python2.7/site-packages/portage/_legacy_globals.py", line 36, in _get_legacy_global
    portage.db = portage.create_trees(**kwargs)
  File "/usr/lib/python2.7/site-packages/portage/__init__.py", line 541, in create_trees
    env=env, sysroot=sysroot, eprefix=eprefix)
  File "/usr/lib/python2.7/site-packages/portage/package/ebuild/config.py", line 378, in __init__
    locations_manager.set_root_override(make_conf.get("ROOT"))
  File "/usr/lib/python2.7/site-packages/portage/package/ebuild/_config/LocationsManager.py", line 318, in set_root_override
    raise InvalidLocation(self.sysroot)
portage.exception.InvalidLocation: /usr/armv5tel-softfloat-linux-gnueabi/

 * Please remember to run:

 *   # . /etc/profile
Comment 1 Gordon Bos 2018-07-13 09:13:51 UTC
Just had another binary package reference the SYSROOT location of the crossdev machine. Unsure what changed, but I've not had this problem before during the more than five years that this system has been running.

Raising importancy of this bug from minor to major.
Comment 2 Jonas Stein gentoo-dev 2018-07-21 15:12:05 UTC
It is sad to read that you have problems with the software. The situation seems to be a bit more complicate and requires some analysis.
We can not help you efficiently via bug tracker. The bug tracker aims rather on specific problems in .ebuilds and less on individual systems. 

I have had very good experience on the gentoo IRC [1] with questions like this. Of course there are also forums and mailing lists [2,3].
I hope you understand, that I will close the bug here therefore and wish you good luck on one of the mentioned channels [4].
Please reopen the ticket in order to provide an indication for an specific error in an ebuild or any gentoo related product.

[1] https://www.gentoo.org/get-involved/irc-channels/
[2] https://forums.gentoo.org/
[3] https://www.gentoo.org/get-involved/mailing-lists/all-lists.html
[4] https://www.gentoo.org/support/
Comment 3 Gordon Bos 2018-07-23 17:34:48 UTC
I fear you did not read the submitted bug correctly.

The problem is not with a specific system. The problem is that the buildpkg option in portage adds local environment settings from a crossdev system (where systemroot can't be / as this is already used by the the crossdev host system) to the package so that it becomes invalid for the target system if that tries to use those.

In the 5 years that I've used this setup I've never had this issue. This just popped up in the last month or so. I'm unsure whether the issue is with these env vars being stored now where they were not before, or portage now using then where it didn't before - although it does still install to the correct folders, so apparently it's only an issue for some portage procedures.
Comment 4 Gordon Bos 2018-08-04 14:39:06 UTC
Reopening as this is becoming more and more of an issue:

 * Switching native-compiler to armv5tel-softfloat-linux-gnueabi-6.4.0 ...!!! Error: SYSROOT (currently /usr/armv5tel-softfloat-linux-gnueabi/) must equal / or ROOT (currently /).
Traceback (most recent call last):
  File "/usr/lib/python-exec/python2.7/env-update", line 35, in <module>
    portage.env_update(makelinks)
  File "/usr/lib/python2.7/site-packages/portage/proxy/objectproxy.py", line 31, in __call__
    return result(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/portage/util/env_update.py", line 51, in env_update
    eprefix = portage.settings["EPREFIX"]
  File "/usr/lib/python2.7/site-packages/portage/proxy/objectproxy.py", line 44, in __getitem__
    return object.__getattribute__(self, '_get_target')()[key]
  File "/usr/lib/python2.7/site-packages/portage/__init__.py", line 651, in _get_target
    return _get_legacy_global(name)
  File "/usr/lib/python2.7/site-packages/portage/_legacy_globals.py", line 36, in _get_legacy_global
    portage.db = portage.create_trees(**kwargs)
  File "/usr/lib/python2.7/site-packages/portage/__init__.py", line 541, in create_trees
    env=env, sysroot=sysroot, eprefix=eprefix)
  File "/usr/lib/python2.7/site-packages/portage/package/ebuild/config.py", line 378, in __init__
    locations_manager.set_root_override(make_conf.get("ROOT"))
  File "/usr/lib/python2.7/site-packages/portage/package/ebuild/_config/LocationsManager.py", line 318, in set_root_override
    raise InvalidLocation(self.sysroot)
portage.exception.InvalidLocation: /usr/armv5tel-softfloat-linux-gnueabi/


From what I can see this version of portage is wrongfully referencing the SYSROOT environment variable defined in the package where it should use the local defined SYSROOT variable.

In effect this means that with this version of portage we can no longer use crossdev machines to build packages for slower or embedded machines.
Comment 5 James Le Cuirot gentoo-dev 2018-08-06 09:59:53 UTC
Thanks for the CC. I'm just back from holiday so I have some catching up to do but I agree that this is important so I'll try to look tonight. I have a tentative patch that makes some changes to the relevant code but I doubt it will help this particular case. Here it is anyway.

https://github.com/chewi/portage/commit/89f262263caa8b480b1dbd3bd49fd67c8a9945e3
Comment 6 James Le Cuirot gentoo-dev 2018-08-06 21:10:50 UTC
I've tried to reproduce this but I haven't been able to. To clue you in a bit, the change happened in Portage 2.3.32 in order to support new features in EAPI 7. The above patch loosens the check but it will probably still trip up here. I believe this could happen with any package. I tried cross-building some random package (killproc) on my amd64 system, installed it as a binary package on my arm system, and then uninstalled it but it all went fine.

2.3.40 isn't the latest so please could you try 2.3.44 and report back?
Comment 7 Gordon Bos 2018-08-07 11:48:27 UTC
Looking at that patch, that seems to want to accomplish the exact opposite? The problem is not with the crossdev machine but with the target machine. The trigger here seems to be that inside the binary package - which I'm currently unsure how to split in it's individual components - there is an `environment.bz2` that declares SYSROOT as /usr/${CHOST}

Additional info:
For some reason I can no longer replicate the issue by re-installing gcc. I can still replicate it by removing the active binutils with the following test sequence though.

    emerge -1avkO =binutils-2.29.1-r1
    eselect binutils 1
    . /etc/profile
    emerge -C =binutils-2.29.1-r1

I have also tried to override the value in environment.bz2 by running it like this:

    SYSROOT="/" emerge -C =binutils-2.29.1-r1

That made no difference.

Note:
The bewildering part here is that this environment setting appears to be ignored by (as far as I can tell) every other part of portage. i.e. the package is installed to / and not to /usr/${CHOST} and it also uninstalls from /. I am in fact puzzled why this variable is even made part of the binary package as to me it seems this is a local setting that should be set when calling portage - like is done in /usr/bin/cross-emerge
Comment 8 James Le Cuirot gentoo-dev 2018-08-07 11:57:08 UTC
(In reply to Gordon Bos from comment #7)
> Looking at that patch, that seems to want to accomplish the exact opposite?

Before, it checked whether SYSROOT != ROOT but only when SYSROOT != /. Now it only does this when ROOT = /. As I said, it doesn't help your case. I need it to bootstrap new systems into an empty ROOT.

> I
> can still replicate it by removing the active binutils with the following
> test sequence though.
> 
>     emerge -1avkO =binutils-2.29.1-r1
>     eselect binutils 1
>     . /etc/profile
>     emerge -C =binutils-2.29.1-r1

I will try this later. Maybe there is something special about binutils.

> Note:
> The bewildering part here is that this environment setting appears to be
> ignored by (as far as I can tell) every other part of portage. i.e. the
> package is installed to / and not to /usr/${CHOST} and it also uninstalls
> from /. I am in fact puzzled why this variable is even made part of the
> binary package as to me it seems this is a local setting that should be set
> when calling portage - like is done in /usr/bin/cross-emerge

I agree, we probably forgot to add SYSROOT to a blacklist somewhere. I just want to be able to reproduce it first so I can be sure that I've fixed it. :)
Comment 9 James Le Cuirot gentoo-dev 2018-08-07 22:11:19 UTC
Good news, I've managed to reproduce the issue using your steps. Brain is tired right now though so I'll try to work out what this means tomorrow.
Comment 10 James Le Cuirot gentoo-dev 2018-08-07 22:59:53 UTC
Managed to muster some brainpower before bed. It actually triggers within binutils-config when it calls portageq. You can reproduce it very simply like this.

SYSROOT=/foo portageq envvar CHOST

This is doing the right thing though. As Gordon says, the variable should not have been carried from the cross environment in the first place. I'm going to try adding it to the unset list in bin/save-ebuild-env.sh. However, I notice that EROOT isn't in this list either and yet that doesn't appear in the environment file so perhaps this is the wrong place.
Comment 11 Zac Medico gentoo-dev 2018-08-08 18:31:26 UTC
We've discussed in #gentoo-portage that __filter_readonly_variables filters EROOT in bin/phase-functions.sh, an we may be able to make it filter SYSROOT unconditionally since SYSROOT propagates from the python side between phases (it's in the environ_whitelist for the config.environ() method).
Comment 12 Gordon Bos 2018-08-08 21:05:13 UTC
Unsure how to interpret that. Does that mean that future portage will handle existing binary packages correctly, or that it will build new binary packages without the conflicting env var?
Comment 13 Zac Medico gentoo-dev 2018-08-08 21:17:50 UTC
It will handle existing binary packages correctly, since __filter_readonly_variables will discard the SYSROOT setting from the binary package's environment.
Comment 14 James Le Cuirot gentoo-dev 2018-08-08 21:25:22 UTC
Tried it out and it's looking good. Here's the PR.

https://github.com/gentoo/portage/pull/356
Comment 15 Larry the Git Cow gentoo-dev 2018-08-08 21:45:48 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=24d4a557cba289327b67b0d339e04995cc293020

commit 24d4a557cba289327b67b0d339e04995cc293020
Author:     James Le Cuirot <chewi@gentoo.org>
AuthorDate: 2018-08-08 21:22:50 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2018-08-08 21:26:31 +0000

    bin/phase-functions.sh: Filter SYSROOT unconditionally
    
    It is propagated in every EAPI because it was used unofficially before
    EAPI 7.
    
    Bug: https://bugs.gentoo.org/661006
    Closes: https://github.com/gentoo/portage/pull/356

 bin/phase-functions.sh | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
Comment 16 Gordon Bos 2018-08-09 07:52:35 UTC
Confirmed. That solves the crash.