Bug 715974 – boostrap-prefix.sh: stage1: python install fails because pip needs pyexpat

• Status: RESOLVED FIXED
• Product: Gentoo/Alt

[Description Alexei Colin 2020-04-03 03:01:01 UTC]

In stage1 a (non-fatal) failure occurs during install of python, during install of pip. Pip needs pyexpat, but pyexpat is disabled in build config. Since libexpat is a library with no dependencies, the attached patch proposes to bootstrap it and enable pyexpat module, so that the python install is not doomed to never succeed.

Tested on CentOS7. If there is interest, I might be able to help test on OS X and Windows.

Reproducible: Always

Steps to Reproduce:
1. ./bootstrap-prefix.sh myprefix stage1

Actual Results:  
This (non-fatal) error in the output log:

    ModuleNotFoundError: No module named 'pyexpat'
    make: *** [Makefile:1102: install] Error 1
    ??? Python failed to install *sigh* continuing anyway
    * Python bootstrapped

Expected Results:  
After applying the attached patch:

    Collecting setuptools
    Collecting pip
    Installing collected packages: setuptools, pip
    Successfully installed pip-18.1 setuptools-40.6.2
    * Python bootstrapped

[Comment 1 Alexei Colin 2020-04-03 03:03:59 UTC]

Created attachment 629252 [details, diff]
Patch to bootstrap libexpat and enable pyexpat for pip

[Comment 2 CoelacanthusHex 2020-05-24 09:59:05 UTC]

Same on my Arch, and the patch is useful but out-of-date

[Comment 3 Benda Xu 2020-05-24 13:21:09 UTC]

Thanks for bringing this up.  However, we do not want to expand the things to compile at stage1 if not absolutely necessary.  Pip is of no use to us.

Please use the full featured Python from stage 3.

[Comment 4 Alexei Colin 2020-05-24 17:34:45 UTC]

Then, this message should be removed:

    ??? Python failed to install *sigh* continuing anyway

The failure path is then the normal path, and the tool should not print an error.

By the way, it takes negligible time to compile expat and it has no dependencies. Not sure why you are worried about including it as a dependency of a component that is required (Python).

[Comment 5 Fabian Groffen 2020-05-24 18:05:34 UTC]

We should disable expat from the bootstrap python I think.  The reason the thing continues whilst complaining is to indicate that we're in a sub-optimal state, but it we get enough to run portage, then it's going to be resolved eventually.

[Comment 6 Alexei Colin 2020-05-24 18:59:07 UTC]

@Fabian, agreed that the error should stay non-fatal and prefix build should continue, also agreed that disabling expat would be ideal, but that's the issue -- expat is disabled already, but the build of Python (specifically PIP) does not succeed without expat. Maybe the correct fix is to disable pip, isn't it?

[Comment 7 Fabian Groffen 2020-05-24 19:00:36 UTC]

ah, yes, pip is not required at this stage indeed.

[Comment 8 Benda Xu 2020-05-25 00:45:12 UTC]

(In reply to Alexei Colin from comment #6)
> @Fabian, agreed that the error should stay non-fatal and prefix build should
> continue, also agreed that disabling expat would be ideal, but that's the
> issue -- expat is disabled already, but the build of Python (specifically
> PIP) does not succeed without expat. Maybe the correct fix is to disable
> pip, isn't it?

That makes more sense here.  As an absolute minimal python is needed here, if your patch were disabling pip, I would be happy to accept it.

[Comment 9 Benda Xu 2020-05-25 00:47:53 UTC]

(In reply to Alexei Colin from comment #4)

> By the way, it takes negligible time to compile expat and it has no
> dependencies. Not sure why you are worried about including it as a
> dependency of a component that is required (Python).

Be careful introducing new dependencies in stage1.  It is the most fragile part of bootstrap, what works rock solid now might introduce failures in the future.  That's why I am super conservative on stage1.

[Comment 10 Alexei Colin 2020-05-28 21:03:02 UTC]

Created attachment 642382 [details, diff]
Patch to disable Pip from Python build.

Attached a tiny patch that disables Pip. Tested build through all stages with patch on top of latest master (c9c2e1e) on centos7: same error without patch, no error with patch. @Benda, how does this one look?

[Comment 11 Larry the Git Cow 2020-05-29 06:22:47 UTC]

The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/proj/prefix.git/commit/?id=91cc96f4ef5489dcf33724b5c3c615b59b7f5047

commit 91cc96f4ef5489dcf33724b5c3c615b59b7f5047
Author:     Alexei Colin <[e-mail]>
AuthorDate: 2020-05-28 18:23:49 +0000
Commit:     Fabian Groffen <[e-mail]>
CommitDate: 2020-05-29 06:22:10 +0000

    bootstrap-prefix.sh: exclude pip from Python build #715974
    
    Pip is not needed but it fails to build due to lack of pyexpat,
    which generates a non-fatal error during boostrap of stage1:
    
        ModuleNotFoundError: No module named 'pyexpat'
        make: *** [Makefile:1102: install] Error 1
        ??? Python failed to install *sigh* continuing anyway
        * Python bootstrapped
    
    Closes: https://bugs.gentoo.org/715974
    Signed-off-by: Alexei Colin <[e-mail]>
    Signed-off-by: Fabian Groffen <[e-mail]>

 scripts/bootstrap-prefix.sh | 1 +
 1 file changed, 1 insertion(+)

[Comment 12 Fabian Groffen 2020-05-29 06:23:54 UTC]

Thanks for taking the time to figure it out, patch applied!