Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 892041 - dev-python/pyfuse3-3.2.2: emerge fails
Summary: dev-python/pyfuse3-3.2.2: emerge fails
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: AMD64 Linux
: Normal normal
Assignee: Piotr Karbowski (RETIRED)
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-01-26 03:59 UTC by Stopi
Modified: 2023-05-02 14:07 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info (emerge--info.txt,6.19 KB, text/plain)
2023-01-26 04:00 UTC, Stopi
Details
emerge log (build.log,3.05 KB, application/octet-stream)
2023-01-26 04:01 UTC, Stopi
Details
the patch that worked (test.patch,154 bytes, patch)
2023-01-27 05:32 UTC, Stopi
Details | Diff
the final portage log (emerge.log,14.34 KB, text/plain)
2023-01-27 05:33 UTC, Stopi
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Stopi 2023-01-26 03:59:32 UTC
It just says "Permission denied: 'cython3'" which didn't lead me to a any solution.

The stable tree is up to date, I ran both "emerge @preserved-rebuild" and "emerge --depclean".
So python is currently 3.10.9

No issue with any other package.

Reproducible: Always

Steps to Reproduce:
emerge -1 dev-python/pyfuse3
Actual Results:  
emerge fails

Expected Results:  
I expect this package to build, or at least give me more information about what's wrong.

The closest (and only) thing I found on the web was https://github.com/cschwan/sage-on-gentoo/issues/354
Comment 1 Stopi 2023-01-26 04:00:38 UTC
Created attachment 849211 [details]
emerge --info
Comment 2 Stopi 2023-01-26 04:01:02 UTC
Created attachment 849213 [details]
emerge log
Comment 3 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-01-26 04:03:44 UTC
Do you have some odd permissions on /usr/bin or similar? Do you have a 'cython3' installed? (I don't.)
Comment 4 Stopi 2023-01-26 05:09:50 UTC
Hi, there's no 'cython3' in the current tree.
I have dev-python/cython-0.29.32 installed with PYTHON_TARGETS="python3_10".

# eix cython
[I] dev-python/cython
     Available versions:  0.29.32^t ~0.29.33^t {doc emacs test PYTHON_TARGETS="pypy3 python3_9 python3_10 python3_11"}
     Installed versions:  0.29.32^t(12:02:44 01/26/23)(-doc -emacs -test PYTHON_TARGETS="python3_10 -pypy3 -python3_9 -python3_11")
     Homepage:            https://cython.org/ https://github.com/cython/cython/ https://pypi.org/project/Cython/
     Description:         A Python to C compiler

rebuilding this package doesn't fix the issue.
Comment 5 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-01-26 05:13:18 UTC
(In reply to Stopi from comment #4)
> Hi, there's no 'cython3' in the current tree.

Right, but I'm wondering if you even have a 'cython3' binary on your system. I assume no.

What about the /usr/bin question?
Comment 6 Stopi 2023-01-26 05:20:53 UTC
# locate cython3
<empty result>

# ls -lahd /usr/bin
drwxr-xr-x 3 root root 90K 25 janv. 10:22 /usr/bin
Comment 7 Stopi 2023-01-26 05:29:30 UTC
Thanks Sam, with your help I fixed this issue by doing:

# cd /usr/bin
# ln -s cython cython3

Now pyfuse3 is installed.
Comment 8 Piotr Karbowski (RETIRED) gentoo-dev 2023-01-26 15:57:03 UTC
I do not think this is fix, I do do not have cython3 in $PATH and it works just fine. I think you should remove this symlink and focus on fixing the root cause for this oddities.
Comment 9 Piotr Karbowski (RETIRED) gentoo-dev 2023-01-26 16:04:49 UTC
Given your hardened string in emerge info, you can try to check what is going on there, like:

    # su - portage --shell=/bin/bash

and then try to run cython3. Perhaps part of your $PATH lacks +r on some directories, or whatever RBAC you use (tomoyo, selinux, apparmor?) result in this error.

Looks like this is the code that you hit the wall on

    def run(self):
        cython = None
        for c in ('cython3', 'cython'):
            try:
                version = subprocess.check_output([c, '--version'],
                                              universal_newlines=True, stderr=subprocess.STDOUT)
                cython = c
            except FileNotFoundError:
                pass
        if cython is None:
            raise SystemExit('Cython needs to be installed for this command') from None


Maybe as it goes though $PATH, it goes to a directory it has no +r on and then it crash,
Comment 10 Piotr Karbowski (RETIRED) gentoo-dev 2023-01-26 16:10:28 UTC
Sorry for the follow up comments. I somewhat reproduced your thingy.

    piotr@megumin tmp % cd ~/tmp
    piotr@megumin tmp % mkdir baz
    mkdir: created directory 'baz'
    piotr@megumin tmp % export PATH="$PATH:$(pwd)/baz"
    piotr@megumin tmp % chmod 000 baz
    mode of 'baz' changed from 0700 (rwx------) to 0000 (---------)
    piotr@megumin tmp % python
    Python 3.10.9 (main, Dec  9 2022, 17:26:00) [GCC 12.2.1 20221203] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import subprocess
    >>> subprocess.run(["cython3"])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "/usr/lib/python3.10/subprocess.py", line 503, in run
        with Popen(*popenargs, **kwargs) as process:
      File "/usr/lib/python3.10/subprocess.py", line 971, in __init__
        self._execute_child(args, executable, preexec_fn, close_fds,
      File "/usr/lib/python3.10/subprocess.py", line 1847, in _execute_child
        raise child_exception_type(errno_num, err_msg, err_filename)
    PermissionError: [Errno 13] Permission denied: 'cython3'
    >>> 

Looks like my theory with walking over $PATH makes sense. Fix your $PATH.
Comment 11 Piotr Karbowski (RETIRED) gentoo-dev 2023-01-26 17:29:43 UTC
I will go ahead and close it as INVALID as there's nothing here that would indicate that there's fault in pyfuse3 itself.

You most likely can run the following:

    for i in ${PATH//:/ }; do namei --long "$i"; echo; done

And see which of them lacks the required permissions.

In case however that you see that it is pyfuse3 issue after all, feel free to reopen.
Comment 12 Stopi 2023-01-27 05:08:43 UTC
Hi Piotr. Thanks for trying to help me on that one.

This system was installed "quite recently" (last year) and have been well maintained, I didn't deviate from the stable tree.

I did not tamper with the filesystem, nor did I install any RBAC solution (yet).

When it comes to the $PATH, it looks good :

$ for i in ${PATH//:/ }; do namei --long "$i"; echo; done
f: /usr/local/sbin
drwxr-xr-x root root /
drwxr-xr-x root root usr
drwxr-xr-x root root local
drwxr-xr-x root root sbin

f: /usr/local/bin
drwxr-xr-x root root /
drwxr-xr-x root root usr
drwxr-xr-x root root local
drwxr-xr-x root root bin

f: /usr/sbin
drwxr-xr-x root root /
drwxr-xr-x root root usr
drwxr-xr-x root root sbin

f: /usr/bin
drwxr-xr-x root root /
drwxr-xr-x root root usr
drwxr-xr-x root root bin

f: /sbin
drwxr-xr-x root root /
drwxr-xr-x root root sbin

f: /bin
drwxr-xr-x root root /
drwxr-xr-x root root bin

f: /opt/bin
drwxr-xr-x root root /
drwxr-xr-x root root opt
drwxr-xr-x root root bin

f: /my/scripts
drwxr-xr-x root root /
drwxr-xr-x root root my
drwx------ root root scripts

But moreover, your explaination doesn't make sense to me.

First, if you don't have cython3 on your system, and neither do I (remember the locate command came empty), then how this can be resulting on portage not accessing a path?

Secondly, while I'm not a python developper, the snippet of code you shared seemed to look for both 'cython3' and 'cython' then issue a 'Cython needs to be installed for this command' error message on failure (I may be wrong on that one).
But 'cython' is in the path and portage have access to it.
Then the error I get is different, it's about a 'permission denied'.

Finally, I can't say for sure if it's a pyfuse3 bug, it might be something more on the cython side, which would still be a bug to me.
Clearly the error message produced while emerging pyfuse3 is not accurate enough  for us to find the root cause.

You're right, my symbolic link doesn't solve that root cause, but it did solve the bug I had and pyfuse3 could emerge after that; so I reported it here in case that would give a clue on what was going on.
Comment 13 Stopi 2023-01-27 05:31:38 UTC
Hey, I think there's an issue with pyfuse3 after all, so I re-open this ticket, as you mentioned.

While not being a python developper, I wanted to try something quick and it seemed to have worked.

I notice the setup code was just checking for 'FileNotFoundError', so I decided to handle a more broad type of exception.
I've search on the web about python syntax and came up with a patch of my own (in /etc/portage/patches/dev-python/pyfuse3/), just for the sake of trying.

Please, have a look at my patch as it allowed the pyfuse3 to compile on my device (yes I previously deleted the symlink I made from cython to cython3).
I'll also attach the final log produced by portage.

I'm still puzzled on why it originally failed in my environment and not in yours tho.
Comment 14 Stopi 2023-01-27 05:32:50 UTC
Created attachment 849257 [details, diff]
the patch that worked
Comment 15 Stopi 2023-01-27 05:33:24 UTC
Created attachment 849259 [details]
the final portage log
Comment 16 Stopi 2023-01-27 05:53:00 UTC
Oh damn, I feel so silly!
I just realized there was indeed an issue with my $PATH:

f: /my/scripts
drwxr-xr-x root root /
drwxr-xr-x root root my
drwx------ root root scripts

This directory isn't accessible by portage.
Changing the rights here also leads to a successful emerge, without the need of my patch.

However, I'd like to keep the rights on this directory that way.
The content of '/my/scripts' should not be accessible to anyone but root.
...and that should not prevent portage to compile.

Please consider my patch anyway (or a custom version of it), because I still think the setup should not just fail because it cannot read the content of one path.

Again, many thanks for your help.
Comment 17 Stopi 2023-01-27 06:24:16 UTC
Considering the issue isn't on Gentoo's side but rather on upstream's, I just opened an issue on github.
https://github.com/libfuse/pyfuse3/issues/63
Comment 18 Piotr Karbowski (RETIRED) gentoo-dev 2023-01-27 09:56:09 UTC
I would prefer not to merge this patch as this is something that shouldn't really happen in first place, with part of $PATH been inaccessible. If we would merge it, we open ourselves to any other corner cases that shouldn't ever happen in first place.

I would consider it an misconfiguration. I understand that you want /my/scripts to be accessible when running as root, so I would suggest you change they way you modify your PATH. Portage does sanitize $PATH by reloading it to drop whatever user might modified there, sounds like you used /etc/env.d/ to add it to your PATH, which would then sneak its way into Portage execution. Instead use /root/.bashrc or /etc/profile, those are sourced at login, and Portage wouldn't take it in, that would let you retain your shell's $PATH without affecting Portage.
Comment 19 Stopi 2023-01-27 10:50:10 UTC
Thanks for the info, I will consider doing so and use ~/.bashrc; I thought adding a custom file in /etc/env.d was the way to go when it comes to change my $PATH. At least that was what I understood when reading the Gentoo doc.

In my head using .bashrc for that sounded like an ugly hack in the first place. After years of experiment, I agree it will solve several problems I had while using /etc/env.d

I also tried to play with ROOT_PATH in /etc/env.d but that one left me confused sometimes.
Comment 20 Piotr Karbowski (RETIRED) gentoo-dev 2023-01-27 11:00:52 UTC
Rather than hack I'd call it scoping. If you want it to be system wide, you use env.d, if you want it to be user-wide, you use user configuration files. Since your directory is root accessible only it makes little sense to have it system wide, with only exception if you'd want to call it via init scripts, but for that I'd go with /usr/local/sbin/ instead.

Since always I had something like it in my .zshrc

if ! [[ "${PATH}" =~ "^${HOME}/bin" ]]; then
    export PATH="${HOME}/bin:${PATH}"
fi

and never looked back.
Comment 21 Sergey 'L29Ah' Alirzaev 2023-05-02 13:33:28 UTC
Got hit by the same problem; thank you for your patch!
Comment 22 Piotr Karbowski (RETIRED) gentoo-dev 2023-05-02 14:07:19 UTC
You might want consider fixing the root cause which is inaccessible $PATH entry. I do not mind patching pyfuse3 but I still consider this to be user error and a condition that should never occur. If you can get me some valid usecases where $PATH contain locations that exist but are not traversalable due to permissions, I can add patch there.