Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 916182 - sys-apps/portage: attempting to compare str.startswith(b".") in files for sets
Summary: sys-apps/portage: attempting to compare str.startswith(b".") in files for sets
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal major (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on: 916031
Blocks:
  Show dependency tree
 
Reported: 2023-10-23 16:49 UTC by Mazunki Hoksaas
Modified: 2023-10-25 05:49 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info after src change (emerge.info,44.19 KB, text/plain)
2023-10-23 16:57 UTC, Mazunki Hoksaas
Details
emerge --info after src change (emerge.info,23.15 KB, text/plain)
2023-10-23 17:05 UTC, Mazunki Hoksaas
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Mazunki Hoksaas 2023-10-23 16:49:41 UTC
I'm unable to run emerge since the last update. I've tracked down the issue to the recent utf8-change to paths.

```
File "/usr/lib/python3.12/site-packages/portage/_sets/__init__.py", line 243, in _parse
    newsets = setclass.multiBuilder(
              ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/portage/_sets/files.py", line 187, in multiBuilder
    if d in vcs_dirs or d.startswith(b".") or d.endswith(b"~"):
```

Replacing the binary string into a standard string fixes this on my machine, but I'm not sure how that change would affect non-utf8 systems. This error only appears if I have a subdirectory under `/etc/portage/sets/`. Would using `pathlib` be a good way to avoid these problems in the future?
Comment 1 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-10-23 16:51:30 UTC
Please don't truncate backtraces. Please also include emerge --info.
Comment 2 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-10-23 16:53:20 UTC
Yeah, ultimately we want to switch to pathlib. There's a large Unicode translation layer which is being bypassed for UTF locales now, thanks to a lot of work from Zac. Ultimately we'll start removing it as well.

Incremental porting to pathlib is welcome.
Comment 3 Mazunki Hoksaas 2023-10-23 16:57:13 UTC
This is the full trace log. I can't run `emerge --info` without modifying `/usr/lib/python3.12/site-packages/portage/_sets/files.py:186`. 

I'm uploading the `emerge --info` I get after modifying it.

I believe `mkdir /etc/portage/sets/whatever` is enough to replicate the issue.

```Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.12/emerge", line 59, in <module>
    retval = emerge_main()
             ^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/_emerge/main.py", line 1265, in emerge_main
    emerge_config = load_emerge_config(action=myaction, args=myfiles, opts=myopts)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/portage/proxy/objectproxy.py", line 31, in __call__
    return result(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/_emerge/actions.py", line 2915, in load_emerge_config
    root_config = RootConfig(settings, root_trees, setconfig)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/_emerge/RootConfig.py", line 25, 
in __init__
    self.sets = self.setconfig.getSets()
                ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/portage/_sets/__init__.py", line 311, in getSets
    self._parse()
  File "/usr/lib/python3.12/site-packages/portage/_sets/__init__.py", line 243, in _parse
    newsets = setclass.multiBuilder(
              ^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/site-packages/portage/_sets/files.py", line 186, in multiBuilder
    if d in vcs_dirs or d.startswith(b".") or d.endswith(b"~"):
                        ^^^^^^^^^^^^^^^^^^
TypeError: startswith first arg must be str or a tuple of str, not bytes
```
Comment 4 Mazunki Hoksaas 2023-10-23 16:57:42 UTC
Created attachment 873298 [details]
emerge --info after src change
Comment 5 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-10-23 17:00:40 UTC
..portage 3.0.41? Am I missing something..?
Comment 6 Mazunki Hoksaas 2023-10-23 17:05:58 UTC
Created attachment 873315 [details]
emerge --info after src change
Comment 7 Mazunki Hoksaas 2023-10-23 17:06:32 UTC
Whoops. I accidentally ran `>>` instead of `>` to save the info file. Sorry for that.
Comment 8 Sam James archtester Gentoo Infrastructure gentoo-dev Security 2023-10-23 17:16:43 UTC
thanks - makes much more sense now!
Comment 9 Larry the Git Cow gentoo-dev 2023-10-24 01:49:23 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=92f34ea2f00ea68b2b5c8374ba6b38287adb1628

commit 92f34ea2f00ea68b2b5c8374ba6b38287adb1628
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2023-10-23 17:14:21 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2023-10-24 01:48:56 +0000

    StaticFileSet: Fix os.walk for utf8_mode
    
    Bug: https://bugs.gentoo.org/916182
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/_sets/files.py | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)
Comment 10 Larry the Git Cow gentoo-dev 2023-10-25 05:49:19 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2b9128b401def05cac46f9e6d66048e5a0d888b8

commit 2b9128b401def05cac46f9e6d66048e5a0d888b8
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2023-10-25 05:48:31 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2023-10-25 05:48:40 +0000

    sys-apps/portage: add 3.0.54
    
    Closes: https://bugs.gentoo.org/905869
    Closes: https://bugs.gentoo.org/915551
    Closes: https://bugs.gentoo.org/915896
    Closes: https://bugs.gentoo.org/916106
    Closes: https://bugs.gentoo.org/916108
    Closes: https://bugs.gentoo.org/916112
    Closes: https://bugs.gentoo.org/916116
    Closes: https://bugs.gentoo.org/916139
    Closes: https://bugs.gentoo.org/916141
    Closes: https://bugs.gentoo.org/916142
    Closes: https://bugs.gentoo.org/916149
    Closes: https://bugs.gentoo.org/916182
    Closes: https://bugs.gentoo.org/916231
    Closes: https://bugs.gentoo.org/916235
    Closes: https://bugs.gentoo.org/916240
    Closes: https://bugs.gentoo.org/916242
    Closes: https://bugs.gentoo.org/916245
    Closes: https://bugs.gentoo.org/916248
    Closes: https://bugs.gentoo.org/899898
    Closes: https://bugs.gentoo.org/906129
    Closes: https://bugs.gentoo.org/906156
    Closes: https://bugs.gentoo.org/916031
    Signed-off-by: Sam James <sam@gentoo.org>

 sys-apps/portage/Manifest              |   1 +
 sys-apps/portage/portage-3.0.54.ebuild | 238 +++++++++++++++++++++++++++++++++
 2 files changed, 239 insertions(+)