Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 543814 - app-portage/mirrorselect ignores continuation lines (backslash) in make.conf
Summary: app-portage/mirrorselect ignores continuation lines (backslash) in make.conf
Status: IN_PROGRESS
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Tools (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage Tools Team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks:
 
Reported: 2015-03-19 18:24 UTC by Jack
Modified: 2020-05-20 21:29 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
Use shlex to support multi-line GENTOO_MIRRORS assignments (write_make_conf-support-multi-line-GENTOO_MIRRORS-bu.patch,5.69 KB, patch)
2019-05-27 07:42 UTC, Zac Medico
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jack 2015-03-19 18:24:10 UTC
If GENTOO_MIRRORS in make.conf is split over more than one line, using a backslash to indicate continuation, mirrorselect only replaces the first line, putting the new GENTOO_MIRRORS at the end of the file, but leaving all the previous continuation lines intact, leaving make.conf in an invalid state.  (I'm using 2.2.2)

Reproducible: Always

Steps to Reproduce:
1. use backslash and continuation lines for GENTOO_MIRRORS in make.conf
2. run mirrorselect (-s5 in my example)
3.
Actual Results:  
The new GENTOO_MIRRORS is placed at the end of the file, all on one line.  The original GENTOO_MIRRORS is removed, but only the first line, not the continuation lines.

Expected Results:  
All lines making up GENTOO_MIRRORS are removed.  Optionally, I would like to see the new value also use continuation lines with one server per line, but I'm willing to let that be a separate enhancement request.

I'm calling this a normal bug, but it does leave make.conf in an invalid state.  Since I caught the problem before running emerge, I don't know what problems it would cause - hopefully only warning messages.
Comment 1 Jack 2017-08-18 18:15:59 UTC
I just confirmed the problem is still present with versions 2.2.2-r2 and 2.2.3.
Comment 2 Jack 2019-05-26 23:38:19 UTC
Problem still present in mirrorselect-2.2.4-r1
Comment 3 Zac Medico gentoo-dev 2019-05-27 03:34:25 UTC
The faulty code is located at https://gitweb.gentoo.org/proj/mirrorselect.git/tree/mirrorselect/configs.py?h=2.2.4#n78:

>	regex = re.compile('^%s=.*' % var)
>	for line in lines:
>		if regex.match(line):
>			lines.remove(line)
Comment 4 Zac Medico gentoo-dev 2019-05-27 05:01:11 UTC
We can use a shlex instance to detect the line number where the GENTOO_MIRRORS setting ends. This is the same approach that portage uses to read make.conf, and mirrorselect already uses this approach in the get_filesystem_mirrors function here:

https://gitweb.gentoo.org/proj/mirrorselect.git/tree/mirrorselect/configs.py?h=2.2.4#n153
Comment 5 Zac Medico gentoo-dev 2019-05-27 07:42:10 UTC
Created attachment 577868 [details, diff]
Use shlex to support multi-line GENTOO_MIRRORS assignments
Comment 6 Larry the Git Cow gentoo-dev 2019-05-27 16:42:45 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/mirrorselect.git/commit/?id=42238f4ff13ae6c75bc1925f366429a35c26317d

commit 42238f4ff13ae6c75bc1925f366429a35c26317d
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2019-05-27 06:11:28 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2019-05-27 07:39:39 +0000

    write_make_conf: support multi-line GENTOO_MIRRORS (bug 543814)
    
    Use shlex to support multi-line GENTOO_MIRRORS assignments.
    
    Bug: https://bugs.gentoo.org/543814
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 .gitignore                    |  1 -
 mirrorselect/configs.py       | 33 +++++++++++++++++++++++++++------
 setup.py                      | 37 ++++++++++++++++++-------------------
 tests/__init__.py             |  0
 tests/test_write_make_conf.py | 41 +++++++++++++++++++++++++++++++++++++++++
 5 files changed, 86 insertions(+), 26 deletions(-)
Comment 7 Larry the Git Cow gentoo-dev 2019-05-27 18:11:06 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=82bbe253084208ac9216f8a5a822e8e15361d7f4

commit 82bbe253084208ac9216f8a5a822e8e15361d7f4
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2019-05-27 17:47:08 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2019-05-27 18:10:57 +0000

    app-portage/mirrorselect: version bump to 2.2.5
    
    Bug: https://bugs.gentoo.org/543814
    Bug: https://bugs.gentoo.org/582508
    Package-Manager: Portage-2.3.67, Repoman-2.3.13
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 app-portage/mirrorselect/Manifest                  |  1 +
 app-portage/mirrorselect/mirrorselect-2.2.5.ebuild | 39 ++++++++++++++++++++++
 2 files changed, 40 insertions(+)
Comment 8 Jack 2020-05-20 21:29:35 UTC
I confirm 2.2.6-r1 fixes the problem.  Is there any reasonable way to get it to use backslashes and one mirror per line, especially if the previous version had it that way?  (I agree it may not be worth the effort it would take, but ask just in case.)