Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 676074 - app-portage/layman-2.4.2-r2 with EPYTHON=python3.7 - Adding an overlay fails - Exception caught installing repository "musl": unterminated character set at position 1
Summary: app-portage/layman-2.4.2-r2 with EPYTHON=python3.7 - Adding an overlay fails ...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 3 votes (vote)
Assignee: Layman Overlay Manager project
URL:
Whiteboard:
Keywords: PullRequest
Depends on:
Blocks: 710650
  Show dependency tree
 
Reported: 2019-01-22 21:34 UTC by eroen
Modified: 2020-04-24 21:45 UTC (History)
16 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description eroen 2019-01-22 21:34:13 UTC
occam / # layman -a musl

 * Adding overlay...

 * CLI: Errors occurred processing action add
 * Exception caught installing repository "musl":
 * unterminated character set at position 1


This seems to be caused by the configuration parsing not working correctly under python 3.7. self.config['protocol_filter'] returns ['[]'] instead of an empty list when it is not explicitly set.


occam / # EPYTHON=python3.7 layman --add musl

 * Adding overlay...
> /usr/lib/python3.7/site-packages/layman/overlays/overlay.py(100)add()
-> self.sources = self.filter_protocols(self.sources)
(Pdb) step
--Call--
> /usr/lib/python3.7/site-packages/layman/overlays/overlay.py(130)filter_protocols()
-> def filter_protocols(self, sources):
(Pdb) next
> /usr/lib/python3.7/site-packages/layman/overlays/overlay.py(135)filter_protocols()
-> _sources = []
(Pdb) 
> /usr/lib/python3.7/site-packages/layman/overlays/overlay.py(136)filter_protocols()
-> if not self.config['protocol_filter']:
(Pdb) 
> /usr/lib/python3.7/site-packages/layman/overlays/overlay.py(139)filter_protocols()
-> for source in sources:
(Pdb) 
> /usr/lib/python3.7/site-packages/layman/overlays/overlay.py(140)filter_protocols()
-> for protocol in self.config['protocol_filter']:
(Pdb) 
> /usr/lib/python3.7/site-packages/layman/overlays/overlay.py(141)filter_protocols()
-> protocol = protocol.lower()
(Pdb) 
> /usr/lib/python3.7/site-packages/layman/overlays/overlay.py(143)filter_protocols()
-> if protocol == 'git+ssh':
(Pdb) 
> /usr/lib/python3.7/site-packages/layman/overlays/overlay.py(145)filter_protocols()
-> protocol += '://'
(Pdb) 
> /usr/lib/python3.7/site-packages/layman/overlays/overlay.py(146)filter_protocols()
-> if re.search('^' + protocol, source.src):
(Pdb) 
re.error: unterminated character set at position 1
> /usr/lib/python3.7/site-packages/layman/overlays/overlay.py(146)filter_protocols()
-> if re.search('^' + protocol, source.src):
(Pdb) p protocol
'[]://'
(Pdb) p self.config['protocol_filter']
['[]']
(Pdb)
Comment 1 Helmut Jarausch 2019-03-09 14:34:47 UTC
A patch / workaround is the following:

--- /tmp/argsparser.py.ORIG	2019-03-06 15:08:03.000000000 +0100
+++ /usr/lib/python3.7/site-packages/layman/argsparser.py	2019-03-09 15:31:58.296182966 +0100
@@ -364,6 +364,8 @@
                 protocol_filter = self.options[key]
             if self.config.has_option('MAIN', 'protocol_filter'):
                 protocol_filter = self.config.get('MAIN', 'protocol_filter')
+                if  isinstance(protocol_filter, str) and protocol_filter=='[]' :
+                  protocol_filter= []
             if protocol_filter:
                 if not isinstance(protocol_filter, list):
                     protocol_filter = [e.strip() for e in protocol_filter.split(',')]
Comment 2 David Heidelberg (okias) 2019-03-29 20:03:44 UTC
I can confirm
Comment 3 Søren Dalby Larsen 2019-05-29 09:19:37 UTC
I can confirm as well.
Comment 4 rx80 2019-06-25 13:53:25 UTC
This bug has been here for half a year now, making layman unusable.... I can confirm that the patch in Comment 1 fixes it.
Comment 5 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2019-07-04 03:27:27 UTC
I can comfirm the general error, but the fix doesn't work for me.

Additionally, there's complaints on gentoo-user about this:

https://archives.gentoo.org/gentoo-user/message/3c2bdb834d16ba1b5aa818f6d06a9ee9

For me, the only workaround that works is setting EPYTHON to a different version other than python3.7, where the different version is in :


tr " " "\n" < /var/db/pkg/app-portage/layman-2.4.2-r2/USE  | grep python_targets
python_targets_python3_6
python_targets_python3_7
Comment 6 Jakub Hrbek 2019-07-23 18:57:56 UTC
Seems to be issue with compatibility on python3.7 -> recommends hotfix by blacklisting python3_7 for layman untill fix is issued.

===

rogisek / # em -va layman

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] app-portage/layman-2.4.2-r2::gentoo  USE="git -bazaar -cvs* -darcs -g-sorcery -gpg* -mercurial -sqlite -squashfs -subversion -sync-plugin-portage -test" PYTHON_TARGETS="python3_7 -pypy -python2_7 -python3_5 -python3_6" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

Would you like to merge these packages? [Yes/No] n

Quitting.

rogisek / # EPYTHON=python3.7 layman -a bobwya

 * Adding overlay...
 * Overlay "bobwya" is not official. Continue installing? [y/n]: y

 * CLI: Errors occurred processing action add
 * Exception caught installing repository "bobwya":
 * unterminated character set at position 1

rogisek / # EPYTHON=python3.6 layman -a bobwya

 * Adding overlay...
 * Overlay "bobwya" is not official. Continue installing? [y/n]: y

 * CLI: Errors occurred processing action add
 * Exception caught installing repository "bobwya":
 * unterminated character set at position 1

rogisek / # em -va layman

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] app-portage/layman-2.4.2-r2::gentoo  USE="git -bazaar -cvs* -darcs -g-sorcery -gpg* -mercurial -sqlite -squashfs -subversion -sync-plugin-portage -test" PYTHON_TARGETS="python3_6* python3_7 -pypy -python2_7 -python3_5" 0 KiB

Total: 1 package (1 reinstall), Size of downloads: 0 KiB

Would you like to merge these packages? [Yes/No] y
>>> Running pre-merge checks for app-portage/layman-2.4.2-r2
>>> Emerging (1 of 1) app-portage/layman-2.4.2-r2::gentoo
>>> Installing (1 of 1) app-portage/layman-2.4.2-r2::gentoo
>>> Jobs: 1 of 1 complete                           Load avg: 1.86, 1.58, 1.90
>>> Auto-cleaning packages...

>>> No outdated packages were found on your system.

 * GNU info directory index is up-to-date.
rogisek / # EPYTHON=python3.6 layman -a bobwya

 * Adding overlay...
 * Overlay "bobwya" is not official. Continue installing? [y/n]: y
 * Running Git... # ( cd /var/lib/layman  && /usr/bin/git clone https://github.com/bobwya/miscellaneous_ebuilds.git /var/lib/layman/bobwya )
Cloning into '/var/lib/layman/bobwya'...
fatal: unable to find remote helper for 'https'
 * Failure result returned from Git
 * Running Git... # ( cd /var/lib/layman/bobwya  && /usr/bin/git config user.name "layman" )
 * [Errno 2] No such file or directory: '/var/lib/layman/bobwya': '/var/lib/layman/bobwya'
 * 
 * Trying next source of listed sources...
 * Running Git... # ( cd /var/lib/layman  && /usr/bin/git clone git://github.com/bobwya/miscellaneous_ebuilds.git /var/lib/layman/bobwya )
Cloning into '/var/lib/layman/bobwya'...
remote: Enumerating objects: 98, done.
remote: Counting objects: 100% (98/98), done.
remote: Compressing objects: 100% (68/68), done.
remote: Total 14061 (delta 54), reused 56 (delta 25), pack-reused 13963
Receiving objects: 100% (14061/14061), 37.98 MiB | 20.55 MiB/s, done.
Resolving deltas: 100% (9924/9924), done.
 * Running Git... # ( cd /var/lib/layman/bobwya  && /usr/bin/git config user.name "layman" )
 * Running Git... # ( cd /var/lib/layman/bobwya  && /usr/bin/git config user.email "layman@localhost" )
 * Successfully added overlay(s) bobwya.
Comment 7 Mads 2019-08-06 09:36:58 UTC
Added 2.4.2-r3 with the argsparse patch from Helmut[1] to my overlay[2] if anyone needs it.

[1] https://bugs.gentoo.org/676074#c1
[2] https://gitlab.com/madsl/fol4
Comment 8 Michael Rohleder (mroh) 2019-08-17 11:39:49 UTC
I can confirm that Helmuts patch fixes this.
Comment 9 vltg0903 2019-12-19 00:00:50 UTC
Problem persists, observed on two machines (both up to date) with

PYTHON_SINGLE_TARGET="python3_7"
PYTHON_TARGETS="python3_7"

set in /etc/portage/make.conf. The patch helped.
Comment 10 Emil Medve 2020-01-19 19:00:46 UTC
A simple workaround is to un-comment the (empty) 'protocol_filter :' line in layman.cfg
Comment 11 Zentoo 2020-02-01 11:16:08 UTC
(In reply to Emil Medve from comment #10)
> A simple workaround is to un-comment the (empty) 'protocol_filter :' line in
> layman.cfg

I confirm that simple workaround is fixing the problem here too.
(~amd64 and full python 3.7 system)
Comment 12 Davide Palma 2020-02-13 12:59:39 UTC
Your comment is so similar to what i was going to write that i'm going to copy-paste it lol

I confirm that simple workaround is fixing the problem here too.
(~amd64 and full python 3.7 system)
Comment 13 Arnaud Vallette d'Osia 2020-03-25 08:56:26 UTC
Emil Medve's solution does the trick here! Thanks :)
Comment 14 Martin Mokrejš 2020-03-26 13:45:02 UTC
I can confirm that runing =app-portage/layman-2.4.2-r2 under =dev-lang/python-3.7.7 gives me still the error.

A quick workaround for me was 

EPYTHON=python3.6 layman -a foo

thanks to @kentnl

https://archives.gentoo.org/gentoo-user/message/15f0601314a0c3c69f99f486bff37413
Comment 15 Brian Dolbec (RETIRED) gentoo-dev 2020-04-24 19:14:37 UTC
will be fixed in layman-2.4.3 to be released soon
Comment 16 Brian Dolbec (RETIRED) gentoo-dev 2020-04-24 21:45:55 UTC
Fixed in layman-2.4.3