Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 938764 - distutils-r1.eclass inappropriately calls cargo_env when CARGO_OPTIONAL=1 and not using Cargo
Summary: distutils-r1.eclass inappropriately calls cargo_env when CARGO_OPTIONAL=1 and...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Eclasses (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords: PullRequest
Depends on:
Blocks:
 
Reported: 2024-08-31 04:27 UTC by Matt Whitlock
Modified: 2024-09-08 18:21 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Whitlock 2024-08-31 04:27:01 UTC
The following commit broke distutils-r1.eclass:

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9b2def86a9c7fd8821a5b83df7d250fc4c73540f

When the ebuild sets CARGO_OPTIONAL=1 and chooses not to invoke cargo_gen_config, then distutils_pep517_install fails with the following error:

cargo_env python3.12 -m gpep517 build-wheel --prefix=/usr --backend poetry.core.masonry.api --output-fd 3 --wheel-dir /var/tmp/portage/net-p2p/core-lightning-24.05/work/lightning-24.05-python3_12/wheel
 * ERROR: net-p2p/core-lightning-24.05::local failed (compile phase):
 *   FATAL: please call cargo_gen_config before using cargo_env
 * 
 * Call stack:
 *     ebuild.sh, line  136:  Called src_compile
 *   environment, line 4937:  Called distutils-r1_src_compile
 *   environment, line 2442:  Called _distutils-r1_run_foreach_impl 'python_compile'
 *   environment, line  993:  Called python_foreach_impl 'distutils-r1_run_phase' 'python_compile'
 *   environment, line 4372:  Called multibuild_foreach_variant '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_compile'
 *   environment, line 3678:  Called _multibuild_run '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_compile'
 *   environment, line 3676:  Called _python_multibuild_wrapper 'distutils-r1_run_phase' 'python_compile'
 *   environment, line 1443:  Called distutils-r1_run_phase 'python_compile'
 *   environment, line 2426:  Called python_compile
 *   environment, line 4161:  Called python_foreach_subdir 'distutils-r1_python_compile'
 *   environment, line 4380:  Called distutils-r1_python_compile
 *   environment, line 2248:  Called distutils_pep517_install '/var/tmp/portage/net-p2p/core-lightning-24.05/work/lightning-24.05-python3_12/install'
 *   environment, line 2771:  Called cargo_env 'python3.12' '-m' 'gpep517' 'build-wheel' '--prefix=/usr' '--backend' 'poetry.core.masonry.api' '--output-fd' '3' '--wheel-dir' '/var/tmp/portage/net-p2p/core-lightning-24.05/work/lightning-24.05-python3_12/wheel'
 *   environment, line 2003:  Called die
 * The specific snippet of code:
 *       [[ -n ${_CARGO_GEN_CONFIG_HAS_RUN} ]] || die "FATAL: please call cargo_gen_config before using ${FUNCNAME}";


Please use a more specific condition in distutils-r1.eclass:

-	has cargo ${INHERITED} && cmd+=( cargo_env )
+	[[ ${_CARGO_GEN_CONFIG_HAS_RUN} ]] && cmd+=( cargo_env )
Comment 1 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2024-08-31 05:43:10 UTC
@chewi, could you add a fix to https://github.com/gentoo/gentoo/pull/38041, please?
Comment 2 James Le Cuirot gentoo-dev 2024-08-31 13:10:36 UTC
I had previously said that ebuilds should call cargo_src_unpack unconditionally, but I suppose this changes makes sense.
Comment 3 Matt Whitlock 2024-09-03 21:18:34 UTC
(In reply to James Le Cuirot from comment #2)
> I had previously said that ebuilds should call cargo_src_unpack
> unconditionally, but I suppose this changes makes sense.

Why should we unpack potentially many crates if we're not going to use them? Waste of time and disk I/O.

cargo.eclass says this about CARGO_OPTIONAL:

# If you enable CARGO_OPTIONAL, you have to set BDEPEND on virtual/rust
# for your package and call at least cargo_gen_config manually before using
# other src_functions or cargo_env of this eclass.
# Note that cargo_gen_config is automatically called by cargo_src_unpack.

That's all well and good, but the requirement to call cargo_gen_config prior to calling distutils-r1_src_compile is not intuitive. Python/Distutils has nothing to do with Rust/Cargo.
Comment 4 James Le Cuirot gentoo-dev 2024-09-03 22:05:29 UTC
Other packages like dev-python/blake3 put the crates behind a USE flag. You're probably already doing this since it would download them all unconditionally otherwise. Then cargo_src_unpack effectively does nothing. I've made the change anyway though, it's in the queue to be merged.
Comment 5 Matt Whitlock 2024-09-03 22:14:42 UTC
(In reply to James Le Cuirot from comment #4)
> Other packages like dev-python/blake3 put the crates behind a USE flag.
> You're probably already doing this since it would download them all
> unconditionally otherwise. Then cargo_src_unpack effectively does nothing.

Ah, okay. I was thinking that cargo_src_unpack would try to unpack everything listed in ${CRATES} regardless of whether it had actually been conditionally selected in ${SRC_URI}. I didn't realize it's as smart as it is.

For my own edification: where does cargo.eclass mention that cargo_src_unpack should be called unconditionally? I am trying to understand how it was okay for distutils-r1.eclass to assume cargo_gen_config would always have been called whenever cargo.eclass has been inherited.

> I've made the change anyway though, it's in the queue to be merged.

Thank you. :)
Comment 6 James Le Cuirot gentoo-dev 2024-09-04 07:48:59 UTC
Admittedly the eclass doesn't say that, but I found that's what almost every ebuild was already doing when someone else raised this issue with me. I didn't catch yours since it's not in the main repo.
Comment 7 Larry the Git Cow gentoo-dev 2024-09-08 18:21:45 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=08cb1492fe62e8f15aa0d546db59ba27ec7a5ab3

commit 08cb1492fe62e8f15aa0d546db59ba27ec7a5ab3
Author:     James Le Cuirot <chewi@gentoo.org>
AuthorDate: 2024-08-31 13:18:39 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2024-09-08 18:21:29 +0000

    distutils-r1.eclass: Don't run cargo_env unless cargo_gen_config has run
    
    Normally, failing to run cargo_gen_config results in an error, but that
    is unhelpful for ebuilds with optional Cargo support.
    
    Closes: https://bugs.gentoo.org/938764
    Signed-off-by: James Le Cuirot <chewi@gentoo.org>
    Signed-off-by: Michał Górny <mgorny@gentoo.org>
    Closes: https://github.com/gentoo/gentoo/pull/38041
    Signed-off-by: Sam James <sam@gentoo.org>

 eclass/distutils-r1.eclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)