Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 530256

Summary: sys-apps/portage-2.2.14 - fails to read repos.conf with leading spaces (error occurs with python-2.7.7 but not with python-3.4.1)
Product: Portage Development Reporter: Sam Jorna (wraeth) <wraeth>
Component: Core - ConfigurationAssignee: Portage team <dev-portage>
Status: RESOLVED WONTFIX    
Severity: normal CC: alexander
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---
Bug Depends on:    
Bug Blocks: 658182    
Attachments: emerge --info

Description Sam Jorna (wraeth) gentoo-dev 2014-11-24 00:55:51 UTC
After having just updated system with the roll to python-3.4 and rebuilding several packages (portage included), portage now fails to read /etc/portage/repos.conf when the file contains leading spaces, for example:

[myrepo]
    location = /path/to/repo
    etc.

This means that any overlays managed by repos.conf cease to exist as far as portage is concerned, meaning packages have been "removed from tree" and bumped versions in overlays no longer exist causing downgrades.

Reproducible: Always

Steps to Reproduce:
1. Update python (now on 2.7.7, 3.3.5-r1, 3.4.1)
2. Ensure repos.conf contains leading spaces
3. Attempt to emerge package or update a set
Actual Results:  
!!! Error while reading repo config file: File contains parsing errors: 
/etc/portage/repos.conf
	[line  2]: u'    location = /usr/local/portage/devel\n'
	[line  3]: u'    sync-type = git\n'
	[line  4]: u'    sync-uri = file:///home/sjorna/dev/portage/devel\n'
	[line  7]: u'    location = /usr/local/portage/wraeth\n'
	[line  8]: u'    sync-type = git\n'
	[line  9]: u'    sync-uri = https://github.com/wraeth/GentooOver

Expected Results:  
Normal portage operation

I'm not entirely certain what is causing this - I know it's a change to python, but unsure how to track this down reasonably.

Also, version according to `emerge --version`:
Portage 2.2.14 (python 2.7.7-final-0, default/linux/amd64/13.0/desktop/kde/systemd, gcc-4.8.3, glibc-2.19-r1, 3.17.1-gentoo-r1 x86_64)
Comment 1 Sam Jorna (wraeth) gentoo-dev 2014-11-24 00:56:30 UTC
Created attachment 390170 [details]
emerge --info
Comment 2 Arfrever Frehtes Taifersar Arahesis 2014-11-24 16:18:38 UTC
Such syntax was never officially supported.
Comment 3 Zac Medico gentoo-dev 2014-11-24 18:42:47 UTC
(In reply to wraeth from comment #0)
> !!! Error while reading repo config file: File contains parsing errors: 
> /etc/portage/repos.conf

I can reproduce this with python-2.7.7 (but not python-3.4.1) if I indent the [DEFAULT] section. It seems to handle indentation of other sections without error.

There's an example in the python docs which seems to suggest that it is valid to indent sections and values:

https://docs.python.org/3/library/configparser.html#supported-ini-file-structure
Comment 4 Zac Medico gentoo-dev 2014-11-24 18:48:25 UTC
Note that (In reply to Zac Medico from comment #3)
> There's an example in the python docs which seems to suggest that it is
> valid to indent sections and values:
> 
> https://docs.python.org/3/library/configparser.html#supported-ini-file-
> structure

The python-2.7.x docs do not have the above mentioned documentation section:

	https://docs.python.org/2/library/configparser.html

I guess maybe you'll have to use python-3.4 if you want the indentation support.

(In reply to wraeth from comment #0)
> Also, version according to `emerge --version`:
> Portage 2.2.14 (python 2.7.7-final-0,
> default/linux/amd64/13.0/desktop/kde/systemd, gcc-4.8.3, glibc-2.19-r1,
> 3.17.1-gentoo-r1 x86_64)

This shows your default python interpreter is python-2.7. You can use eselect python to change it to python-3.4.
Comment 5 Sam Jorna (wraeth) gentoo-dev 2014-11-25 00:36:53 UTC
(In reply to Zac Medico from comment #4)
> (In reply to wraeth from comment #0)
> > Also, version according to `emerge --version`:
> > Portage 2.2.14 (python 2.7.7-final-0,
> > default/linux/amd64/13.0/desktop/kde/systemd, gcc-4.8.3, glibc-2.19-r1,
> > 3.17.1-gentoo-r1 x86_64)
> 
> This shows your default python interpreter is python-2.7. You can use
> eselect python to change it to python-3.4.

  root@cerberus :1676: ~ # eselect python list
  Available Python interpreters:
    [1]   python2.7
    [2]   python3.3 *
    [3]   python3.4

I've been on python3 system-wide for a while now. That being said, with the rebuilds from python-3.4 -> python-3.3 in PYTHON_TARGETS, portage was rebuilt and is now reporting:

  Portage 2.2.14 (python 3.3.5-final-0 ... )

As a test, I changed system python to 3.4 and rebuilt portage, and got this:

  * Building package for python3.3 only while python3.4 is active.
  * Please consider switching the active Python 3 interpreter:

And emerge is now reporting its version as:

  Portage 2.2.14 (python 2.7.7-final-0 ... )
Comment 6 Zac Medico gentoo-dev 2014-11-25 00:50:26 UTC
You shouldn't have to rebuild portage to change the python interpreter it uses, unless you need to change PYTHON_TARGETS so that portage installs modules for the desired python version(s).

Locally, I can use eselect python to change the default interpreter, and 'emerge --version' immediately reflects the selected python version. If that doesn't work for you, then it sounds like we've got a problem with dev-lang/python-exec. What version of dev-lang/python-exec do you have, and what PYTHON_TARGETS is portage built for? This should show us:

   emerge -pv --nodeps dev-lang/python-exec sys-apps/portage
Comment 7 Sam Jorna (wraeth) gentoo-dev 2014-11-25 01:21:28 UTC
(In reply to Zac Medico from comment #6)
> You shouldn't have to rebuild portage to change the python interpreter it
> uses, unless you need to change PYTHON_TARGETS so that portage installs
> modules for the desired python version(s).

My system is largely default, as this is my stable work system. This issue presented when PYTHON_TARGETS was changed to default to "python2_7 python3_4", causing portage and a number of other packages to rebuild.

> Locally, I can use eselect python to change the default interpreter, and
> 'emerge --version' immediately reflects the selected python version.

This does appear to work to a point - changing system python /without/ rebuilding portage causes emerge to report:

  root@cerberus :1702: ~ # eselect python set 3                                                                                                               
  root@cerberus :1703: ~ # eselect python show                                                                                                                
  python3.4
  root@cerberus :1704: ~ # emerge --version                                                                                                                   
  Portage 2.2.14 (python 2.7.7-final-0 ... )

I then rebuilt portage with

  PYTHON_TARGETS="python2_7 python3_4" emerge -1O sys-apps/portage

(the --nodeps was because of a conflict with webapp-config that i'll look at later). After the rebuild, however, portage is reporting the correct python version.

> If that
> doesn't work for you, then it sounds like we've got a problem with
> dev-lang/python-exec. What version of dev-lang/python-exec do you have, and
> what PYTHON_TARGETS is portage built for? This should show us:
> 
>    emerge -pv --nodeps dev-lang/python-exec sys-apps/portage

Note that this is *before* the above rebuild of portage and changes to system python:

  emerge -pvO dev-lang/python-exec sys-apps/portage
  These are the packages that would be merged, in order:
  [binary   R    ] dev-lang/python-exec-2.0.1-r1:2  PYTHON_TARGETS="(jython2_5) (jython2_7) (pypy) (pypy3) (python2_7) (python3_3) (python3_4)" 0 KiB
  [binary   R   ~] sys-apps/portage-2.2.14  USE="(ipc) -build -doc -epydoc (-selinux) -xattr" LINGUAS="-ru" PYTHON_TARGETS="python2_7 python3_3 -pypy -python3_4" 0 KiB

*After* the change, with PYTHON_TARGETS="python2_7 python3_4" in make.conf shows what you'd expect as well:

  emerge -pvO dev-lang/python-exec sys-apps/portage                                                                                  
  These are the packages that would be merged, in order:
  [binary   R    ] dev-lang/python-exec-2.0.1-r1:2  PYTHON_TARGETS="(jython2_5) (jython2_7) (pypy) (pypy3) (python2_7) (python3_3) (python3_4)" 0 KiB
  [binary   R   ~] sys-apps/portage-2.2.14  USE="(ipc) -build -doc -epydoc (-selinux) -xattr" LINGUAS="-ru" PYTHON_TARGETS="python2_7 python3_4 -pypy -python3_3" 0 KiB

In short, it doesn't look like there's a bug in dev-lang/python-exec (since #c5 had a system python of 3.4 but portage built with targets 2.7 and 3.3); however at the time of the bug report, PYTHON_TARGETS="python3_4" had been made default and packages rebuilt.

It could be, though, that because PYTHON_TARGETS="python3_4" was made default while the system python (and the python3 preference) remained as-is (as in, on python3.3) this caused portage to revert to the only matching set - PYTHON_TARGETS="python2_7" and python2 preference of python2.7.
Comment 8 Zac Medico gentoo-dev 2014-11-25 02:02:34 UTC
If the sys-apps/portage ebuild doesn't die when one of the configured PYTHON_TARGETS is not available, then that's a bug.
Comment 9 Sam Jorna (wraeth) gentoo-dev 2014-11-25 02:17:03 UTC
When the change to PYTHON_TARGETS defaults was made, it was set to PYTHON_TARGETS="python2_7 python3_4". Installed python was 2.7, 3.3, 3.4; and I didn't make any change to system python - it remained on the default 3.3. The selected python2 interpreter was python2.7.

Python-3.4 was available on the system, but was not selected as the default interpreter (either as the system default or the selected python3 implementation).

Python-2.7 was selected as the default python2 implementation, but not the default system python.

So, it seems to me that this may be an unexpected result of the change to PYTHON_TARGETS - the change to default PYTHON_TARGETS removed the version that was selected as system interpreter, causing portage to revert to the only matched "preferred" interpreter for which it was built - python2.7. This, in turn, caused the errors with repos.conf since python2 doesn't support leading spaces in ini files.
Comment 10 Zac Medico gentoo-dev 2014-11-25 19:54:04 UTC
(In reply to Zac Medico from comment #8)
> If the sys-apps/portage ebuild doesn't die when one of the configured
> PYTHON_TARGETS is not available, then that's a bug.

I've tested this locally, and it appears to die correctly.

(In reply to wraeth from comment #9)
> So, it seems to me that this may be an unexpected result of the change to
> PYTHON_TARGETS - the change to default PYTHON_TARGETS removed the version
> that was selected as system interpreter, causing portage to revert to the
> only matched "preferred" interpreter for which it was built - python2.7.
> This, in turn, caused the errors with repos.conf since python2 doesn't
> support leading spaces in ini files.

I'm not familiar with how python-exec handles this kind of situation, but it sounds like a plausible explanation to me.
Comment 11 Zac Medico gentoo-dev 2020-04-01 22:50:57 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=0a2c520e735f285a11e2486cb7b4fe1c8e4d0dac

commit 0a2c520e735f285a11e2486cb7b4fe1c8e4d0dac
Author:     Michał Górny <mgorny@gentoo.org>
AuthorDate: 2020-03-19 09:52:09 +0100
Commit:     Michał Górny <mgorny@gentoo.org>
CommitDate: 2020-03-19 10:41:00 +0100

    sys-apps/portage: Drop py2
    
    Signed-off-by: Michał Górny <mgorny@gentoo.org>