Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 892133 - >=net-misc/streamlink-5: Doesn't work from unprivileged user
Summary: >=net-misc/streamlink-5: Doesn't work from unprivileged user
Status: RESOLVED WORKSFORME
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Sam James
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-01-27 03:27 UTC by Michael
Modified: 2023-11-09 00:58 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge-info (info,7.84 KB, text/plain)
2023-01-27 03:27 UTC, Michael
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michael 2023-01-27 03:27:26 UTC
Created attachment 849253 [details]
emerge-info

$ streamlink -v
Traceback (most recent call last):
  File "/usr/bin/streamlink", line 5, in <module>
    from streamlink_cli.main import main
  File "/usr/lib/python3.10/site-packages/streamlink_cli/main.py", line 19, in <module>
    from streamlink.stream.stream import Stream, StreamIO
  File "/usr/lib/python3.10/site-packages/streamlink/stream/__init__.py", line 3, in <module>
    from streamlink.stream.hls import HLSStream, MuxedHLSStream
  File "/usr/lib/python3.10/site-packages/streamlink/stream/hls.py", line 11, in <module>
    from Crypto.Util.Padding import unpad
ModuleNotFoundError: No module named 'Crypto.Util.Padding'

However, works with elevated privileges:
$ sudo streamlink -v
[cli][info] streamlink is running as root! Be careful!
usage: streamlink [OPTIONS] <URL> [STREAM]

Use -h/--help to see the available options or read the manual at https://streamlink.github.io

Pycryptodome appears to be installed just fine:
emerge -pv dev-python/pycryptodome
[ebuild   R    ] dev-python/pycryptodome-3.16.0::gentoo  PYTHON_TARGETS="python3_10 (-pypy3) -python3_9 -python3_11" 0 KiB

Upstream thinks one of us is doing something stupid, and I admit, I'd think that too. 
https://github.com/streamlink/streamlink/issues/5083
Comment 1 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-01-27 03:29:33 UTC
Can't reproduce:
```
$ streamlink  -v
usage: streamlink [OPTIONS] <URL> [STREAM]

Use -h/--help to see the available options or read the manual at https://streamlink.github.io

$ streamlink --version
streamlink 5.2.1
```

Do you have PYTHONPATH or something set in your environment?
Comment 2 Michael 2023-01-27 03:33:13 UTC
No, it's unset
Comment 3 Michael 2023-01-27 03:34:08 UTC
sudo wouldn't change it anyway
Comment 4 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-01-27 03:45:03 UTC
(In reply to Michael from comment #3)
> sudo wouldn't change it anyway

There's a limited number of things which can break this, so you'll have to forgive me for asking questions about each of them.

And yes, it would, because sudo only perserves certain environment variables?

If you write a small Python program with just 'from Crypto.Util.Padding import unpad' inside, what happens when you execute that?
Comment 5 Michael 2023-01-27 03:51:36 UTC
Sure, I will be happy to help troubleshoot. It is as weird to me as it is to you:

```
cat test.py 
from Crypto.Util.Padding import unpad
```

python ./test.py 
Traceback (most recent call last):
  File "/home/corvus/soft/streamlink-debug/./test.py", line 1, in <module>
    from Crypto.Util.Padding import unpad
ModuleNotFoundError: No module named 'Crypto.Util.Padding'

We get our minimal reproducer. :-)
Comment 6 Michael 2023-01-27 04:11:30 UTC
$ yt-dlp --version
Traceback (most recent call last):
  File "/usr/lib/python3.10/site-packages/yt_dlp/dependencies.py", line 27, in <module>
    from Cryptodome.Cipher import AES as Cryptodome_AES
ModuleNotFoundError: No module named 'Cryptodome'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.10/yt-dlp", line 5, in <module>
    from yt_dlp import main
  File "/usr/lib/python3.10/site-packages/yt_dlp/__init__.py", line 18, in <module>
    from .cookies import SUPPORTED_BROWSERS, SUPPORTED_KEYRINGS
  File "/usr/lib/python3.10/site-packages/yt_dlp/cookies.py", line 18, in <module>
    from .aes import (
  File "/usr/lib/python3.10/site-packages/yt_dlp/aes.py", line 5, in <module>
    from .dependencies import Cryptodome_AES
  File "/usr/lib/python3.10/site-packages/yt_dlp/dependencies.py", line 37, in <module>
    Cryptodome_AES.new(b'abcdefghijklmnop')
  File "/home/corvus/.local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 95, in new
    return AESCipher(key, *args, **kwargs)
  File "/home/corvus/.local/lib/python3.10/site-packages/Crypto/Cipher/AES.py", line 59, in __init__
    blockalgo.BlockAlgo.__init__(self, _AES, key, *args, **kwargs)
  File "/home/corvus/.local/lib/python3.10/site-packages/Crypto/Cipher/blockalgo.py", line 141, in __init__
    self._cipher = factory.new(key, *args, **kwargs)
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats

$ sudo yt-dlp --version
2023.01.06

The other revdep of cryptodome on my system behaves the same. :-(
Comment 7 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-01-27 04:18:02 UTC
(In reply to Michael from comment #5)
> Sure, I will be happy to help troubleshoot. It is as weird to me as it is to
> you:
> 
> ```
> cat test.py 
> from Crypto.Util.Padding import unpad
> ```
> 
> python ./test.py 
> Traceback (most recent call last):
>   File "/home/corvus/soft/streamlink-debug/./test.py", line 1, in <module>
>     from Crypto.Util.Padding import unpad
> ModuleNotFoundError: No module named 'Crypto.Util.Padding'
> 
> We get our minimal reproducer. :-)

heh :)

- Okay, any better with 'python -E ./test.py'?
- If you temporarily uninstall pycryptodome forcefully (quickpkg it first), does streamlink still work as root? (Put pycryptodome back afterwards).
- I assume re-emerging pycryptodome makes no difference?
- What does 'find /usr/lib/python* -exec qfile -o {} +' give? (Hopefully no output at all.)
Comment 8 Michael 2023-01-27 04:28:00 UTC
(In reply to Sam James from comment #7)
> (In reply to Michael from comment #5)
> > Sure, I will be happy to help troubleshoot. It is as weird to me as it is to
> > you:
> > 
> > ```
> > cat test.py 
> > from Crypto.Util.Padding import unpad
> > ```
> > 
> > python ./test.py 
> > Traceback (most recent call last):
> >   File "/home/corvus/soft/streamlink-debug/./test.py", line 1, in <module>
> >     from Crypto.Util.Padding import unpad
> > ModuleNotFoundError: No module named 'Crypto.Util.Padding'
> > 
> > We get our minimal reproducer. :-)
> 
> heh :)
> 
> - Okay, any better with 'python -E ./test.py'?

No, same.

> - If you temporarily uninstall pycryptodome forcefully (quickpkg it first),
> does streamlink still work as root? (Put pycryptodome back afterwards).

No, it says `ModuleNotFoundError: No module named 'Crypto'`

> - I assume re-emerging pycryptodome makes no difference?

None.

> - What does 'find /usr/lib/python* -exec qfile -o {} +' give? (Hopefully no
> output at all.)

``` 
# find /usr/lib/python* -exec qfile -o {} +
/usr/lib/python3.10/site-packages/Brotli-1.0.9-py3.10.egg-info.backup.0000
/usr/lib/python3.9
/usr/lib/python3.9/site-packages
/usr/lib/python3.9/site-packages/Brotli-1.0.9-py3.9.egg-info.backup.0000
```

Finds a little trash.
Comment 9 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-01-27 04:52:55 UTC
You should remove those orphaned files it found (the egg, and all of 3.9) as an aside.

- If you run 'env -i python ./test.py', does that work as your user?
- I just realised I assumed that 'test.py' works as root. It does, right?
- Failing that, we can strace the root invocation and the user invocation and compare what's different.
Comment 10 Michael 2023-01-27 09:14:58 UTC
(In reply to Sam James from comment #9)
> You should remove those orphaned files it found (the egg, and all of 3.9) as
> an aside.

Yes, I removed the trash. The system is 13 years old, cobwebs, dustbunnies everywhere. I swipe them away as I encounter them. I even used portage-orphaned-files or some such, when it started feeling like grandfather's attic.  I should show you emerge.log. It's 50MB long, starts in 2010.


> - If you run 'env -i python ./test.py', does that work as your user?

Yes, same:

$ env -i python ./test.py 
Traceback (most recent call last):
  File "/home/corvus/soft/streamlink-debug/./test.py", line 1, in <module>
    from Crypto.Util.Padding import unpad
ModuleNotFoundError: No module named 'Crypto.Util.Padding'

> - I just realised I assumed that 'test.py' works as root. It does, right?

Yes, returns no errors. 

> - Failing that, we can strace the root invocation and the user invocation
> and compare what's different.

I guess so.
Comment 11 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-11-09 00:58:02 UTC
I don't think I can really do anything about this. This isn't specific to streamlink and it's somehow specific to your system.

There's something off with site-packages or one of the import paths of Python on your machine, I guess.