Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 434672 - app-portage/ufed should look for make.* in new location: /etc/portage
Summary: app-portage/ufed should look for make.* in new location: /etc/portage
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Tools (show other bugs)
Hardware: All Linux
: Normal normal with 1 vote (vote)
Assignee: Portage Tools Team
URL:
Whiteboard:
Keywords:
: 435314 (view as bug list)
Depends on:
Blocks:
 
Reported: 2012-09-11 02:28 UTC by Jan Psota
Modified: 2012-12-27 16:33 UTC (History)
14 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Patch to let Portage.pm look into /etc/portage/make.conf first (ufed-0.40.1-add-portage-subdir-to-pm.patch,500 bytes, patch)
2012-09-18 19:53 UTC, Sven Eden
Details | Diff
Patch to let ufed look into /etc/portage/make.conf first (ufed-0.40.1-add-portage-subdir-to-ufed.patch,1.57 KB, patch)
2012-09-18 19:54 UTC, Sven Eden
Details | Diff
Add a check for both paths (unless symlinking each other) to Portage.pm (ufed-0.40.1-add_new_path_to_pm.patch,2.25 KB, patch)
2012-09-20 17:39 UTC, Sven Eden
Details | Diff
Patch to make ufed use the findings on the proper make.conf or error out if none is found (ufed-0.40.1-add_new_path_to_ufed.patch,1.74 KB, patch)
2012-09-20 17:42 UTC, Sven Eden
Details | Diff
Fix warnings/errors of perl/ufed if one of the make.conf is empty or blankless (ufed-0.40.1-fix-one-line-make-conf.patch,555 bytes, patch)
2012-09-20 17:53 UTC, Sven Eden
Details | Diff
Add virtual/perl-File-Spec to RDEPEND in the most current ufed ebuild (add_perl-file-Spec-rdepend_to_ebuild.patch,325 bytes, patch)
2012-09-20 17:55 UTC, Sven Eden
Details | Diff
make.conf placement autodetection on prepare stage (ufed-0.40.1-r1.ebuild.patch,599 bytes, patch)
2012-09-23 22:24 UTC, Jan Psota
Details | Diff
Patch to let Portage.pm look into /etc and /etc/portage for make.conf/profile (ufed-0.40.1-add_new_path_to_pm.patch,3.90 KB, patch)
2012-10-04 18:14 UTC, Sven Eden
Details | Diff
Merge both make.conf's in a nonintrusive way (ufed-0.40.1-make.conf-read.patch,371 bytes, patch)
2012-10-15 17:41 UTC, Martin Väth
Details | Diff
Write into correct make.conf (ufed-0.40.1-make.conf-write.patch,1.16 KB, patch)
2012-10-15 17:47 UTC, Martin Väth
Details | Diff
Patch for /etc/{,portage}/make.profile (ufed-0.40.1-make.profile-path.patch,442 bytes, patch)
2012-10-15 17:49 UTC, Martin Väth
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Psota 2012-09-11 02:28:23 UTC
...given in summary.

Resolution - sed will do the job:
        sed -i 's:/etc/make\.\(conf\|profile\):/etc/portage/make.\1:g' \
                `grep -rl 'etc/make\.' .`
in src_prepare() after first epatch.

It of course won't work with old file locations,
but people using ufed know what they do.
Comment 1 Brian Dolbec gentoo-dev 2012-09-11 03:49:33 UTC
It is actually more complicated than that.

The old locations are still valid, as well, you can use both locations.  The new location will override anything from the old in a stacking way.

If you are able to help, and would like to see ufed continue.  It could use some maintainer help.  It has been without an active developer for some time.

Volunteers welcome.
Comment 2 Jan Psota 2012-09-12 06:54:46 UTC
> The old locations are still valid, as well, you can use both locations.  The
> new location will override anything from the old in a stacking way.
> 
Both locations!? We are steel suffering of 8086 backward compatibility...

> If you are able to help, and would like to see ufed continue.  It could use
> some maintainer help.  It has been without an active developer for some time.
> 
> Volunteers welcome.
>
I could do it, but I think it's stupid to let users use 2 locations.
There are many users, who will symlink /etc/make.conf to /etc/portage/
if you won't forbid it.

I can make ufed test in which place is make.conf and read both
.conf and .profile from that location.
Comment 3 Paul Varner (RETIRED) gentoo-dev 2012-09-18 05:32:59 UTC
*** Bug 435314 has been marked as a duplicate of this bug. ***
Comment 4 Sven Eden 2012-09-18 19:53:31 UTC
Created attachment 324238 [details, diff]
Patch to let Portage.pm look into /etc/portage/make.conf first

This patch lets ufed look into /etc/portage/make.conf first before trying /etc/make.conf.

Reason: I do not want a symlink /etc/make.conf -> /etc/portage/make.conf. ;)
Comment 5 Sven Eden 2012-09-18 19:54:22 UTC
Created attachment 324240 [details, diff]
Patch to let ufed look into /etc/portage/make.conf first

This patch lets ufed look into /etc/portage/make.conf first before trying /etc/make.conf.

Reason: I do not want a symlink /etc/make.conf -> /etc/portage/make.conf. ;)

Note: There are two patches.
Comment 6 Brian Dolbec gentoo-dev 2012-09-19 01:37:11 UTC
Sven,  Thank you for the patches, but they are incorrect.

The correct order for make.conf reads are as follows.

1) read in /etc/make.conf if it exists and is not a symlink to /etc/portage/make.conf

2) read /etc/portage/make.conf (if it exists) and add it to whatever was read in from step1.  Any settings read in from this file may override previous settings read in step 1 or add to them.

3) make sure at least one make.conf was read, error if neither were found.

4) if /etc/portage/make.conf exists, then changes should be saved there rather than in /etc/make.conf


Both locations are still valid and using both on the same system is also valid. At least for now and the foreseeable future (subject to change).
Comment 7 Sven Eden 2012-09-19 08:54:48 UTC
Hi Brian,

thank you very much for the fast feedback.

(In reply to comment #6)
> Sven,  Thank you for the patches, but they are incorrect.
> 
> The correct order for make.conf reads are as follows.
> 
> 1) read in /etc/make.conf if it exists and is not a symlink to
> /etc/portage/make.conf
> 
> 2) read /etc/portage/make.conf (if it exists) and add it to whatever was
> read in from step1.  Any settings read in from this file may override
> previous settings read in step 1 or add to them.

Does that mean that portage is doing it this way? And if so, why? As make.conf and make.profile are permanently moved, it should be the new primary location, with the old one being a fallback if the new isn't there.

And this would mean someone could have _two_ make.conf files. How could a user know which setting is valid and not overridden and how could ufed know? Of course there could be a note somewhere. Or a message. Or a news entry, or whatever.

But what would happen once users start to actually have two *different* versions of the file. They do make changes manually or via ufed and will naturally change /etc/make.conf (by mistake or because ufed doesn't know yet) - but without any effect because the stuff they change is reverted by /etc/portage/make.conf ? The forums will explode. Probably.

Let's see what portage is doing:

--------
sed-notebook ~ # cp /etc/portage/make.conf /etc/make.conf
sed-notebook ~ # strace emerge --pretent --update --newuse @system 2>&1 | grep "make.conf"
open("/etc/make.conf", O_RDONLY)        = 3
open("/var/lib/layman/make.conf", O_RDONLY) = 3
open("/etc/portage/make.conf", O_RDONLY) = 3
open("/var/lib/layman/make.conf", O_RDONLY) = 3
open("/etc/make.conf", O_RDONLY)        = 3
open("/var/lib/layman/make.conf", O_RDONLY) = 3
open("/etc/portage/make.conf", O_RDONLY) = 3
open("/var/lib/layman/make.conf", O_RDONLY) = 3
read(4, "K setting in make.conf\nEAPI=4\nHO"..., 46) = 46
^C
--------

This looks to me like portage doing exactly what you suggested for ufed to mirror. (At least portage-2.2.0_alpha129 that I am using.)

Well, this can be patched in. But I am in doubt whether it is wise to do so or rather dangerous...

> 
> 3) make sure at least one make.conf was read, error if neither were found.

I did not change the error behavior. The patches just make ufed to look into /etc/portage/make.conf and fall back to /etc/make.conf if the first is not present. Yes, there is no symlink check, yet. If this is necessary I could add one.
Ok, there is no try to read /etc/make.conf if /etc/portage/make.conf is present but could not be read. Should such a behavior be added? Wouldn't it wreak havoc? If /etc/portage/make.conf is present but can not be read or written to by root, there is a serious problem with the user system (no space left on device maybe) that has to be fixed asap instead of falling back to an obsolete file that would fail in this very scenario, too, and would not help configuring the USE-Flags.

Okay, I have no idea whether the latter assumption is correct. What would be portage behavior if /etc/portage/make.conf is present but not readable?

However, according to points 1) and 2) the reading pattern has to be changed nevertheless, and this would change the error behavior, too.

> 
> 4) if /etc/portage/make.conf exists, then changes should be saved there
> rather than in /etc/make.conf

Yes, that is what is happening:

--------
sed-notebook ~ # touch /etc/make.conf
sed-notebook ~ # l /etc/make.conf /etc/portage/make.conf
-rw-r--r-- 1 root root    0 19. Sep 10:25 /etc/make.conf
-rw-r--r-- 1 root root 4,8K 19. Sep 10:11 /etc/portage/make.conf
sed-notebook ~ # ufed
Warning: source command found in /etc/portage/make.conf. Flags may
be saved incorrectly if the sourced file modifies them.
sed-notebook ~ # l /etc/make.conf /etc/portage/make.conf
-rw-r--r-- 1 root root    0 19. Sep 10:25 /etc/make.conf
-rw-r--r-- 1 root root 4,8K 19. Sep 10:25 /etc/portage/make.conf
--------

You see, the changes are saved in /etc/portage/make.conf and /etc/make.conf is not touched any more.

But obeying points 1 to 3 will lead to extra steps to be necassary to have point 4 fulfilled as well.

> 
> Both locations are still valid and using both on the same system is also
> valid. At least for now and the foreseeable future (subject to change).

Okay, I did know both where valid. Must be or all user systems would get shattered without a manual move. But I did not know that it is in order to have two (different?) make.conf files.

So it seems that it is necessary to mirror what portage is doing. No matter how dangerous it might be. But I can't see a way to make sure the changes made with ufed are saved in a way they find their way into the next emerge process.

Thanks again for the feedback!
Comment 8 Brian Dolbec gentoo-dev 2012-09-19 14:30:23 UTC
Sven, yes, portage is doing it the way I explained.  For the vast majority of users, they will use only the one make.conf.  So all saving should prefer the new location if both exist.  It does after all have the final say.  If a user uses both, they will need to track their changes properly.

It was zmedico that was suggesting a complex setup using both for someone that was trying to export a central common /etc/make.conf for a group of computers. the individual machines would import /etc/make.conf via nfs, etc., then use /etc/portage/make.conf to make any individual changes needed for that system. This makes it easier for a sysadmin to manage groups of machines. So then having ufed default to saving in the new location if the new location is used, would always do the right thing for at least 99% of users.
Comment 9 Sven Eden 2012-09-20 08:41:36 UTC
(In reply to comment #8)
> (snip) the individual machines would import /etc/make.conf via nfs,
> etc., then use /etc/portage/make.conf to make any individual changes needed
> for that system. (snip)

Ah! Okay, I never thought of this. I'll add the proper steps to ufed asap (maybe even today) and add patches here for inspection after I have their result to work properly on my system. Hopefully I can get it right this time.

And for my last post, I meant "But I can't see a*nother* way to make sure...", of course. Amazing how six missing letters can change the meaning of a whole sentence this complete...

Sorry for the long posts here mailed around, but I wanted to be damn sure that I understand the whole issue properly before I make another push on the code.
Comment 10 Brian Dolbec gentoo-dev 2012-09-20 09:11:07 UTC
Sven, no problem.  Just glad I was able to explain it correctly.  I'm sure everyone cc'd here will appreciate your work :)

I don't know perl and don't really use ufed, so have been avoiding delving into it myself...
Comment 11 Sven Eden 2012-09-20 17:39:38 UTC
Created attachment 324410 [details, diff]
Add a check for both paths (unless symlinking each other) to Portage.pm

This patch makes the Modules to load /etc/make.conf first, if it is present and not a symlink to /etc/portage/make.conf.
The Module then loads /etc/portage/make.conf if it is present and not a symlink to /etc/make.conf ; possibly overwriting settings read from /etc/make.conf now.
The Module saves the full path of the last loaded make.conf. Therefore it is either /etc/make.conf if no /etc/portage/make.conf is present or a symlink to the prior, or the latter if present.

This behavior is tested, but please see notes two patches below!
Comment 12 Sven Eden 2012-09-20 17:42:03 UTC
Created attachment 324412 [details, diff]
Patch to make ufed use the findings on the proper make.conf or error out if none is found

This patch makes ufed to use the findings of Portage.pm in the following ways:
- Use the path the Module found and verified for text output.
- Write changes only to the path specified by the Module.
- Error out if the Module couldn't find a valid make.conf in either path.

Please see notes one patch down!
Comment 13 Sven Eden 2012-09-20 17:53:39 UTC
Created attachment 324422 [details, diff]
Fix warnings/errors of perl/ufed if one of the make.conf is empty or blankless

I have tested the above patches with a normal make.conf in both locations, with an /etc/make.conf consosting of just one line USE="-kde gnome" (the opposite of what my regular make.conf states) and an empty /etc/make.conf.

In all three cases ufed does what it should do, but the latter two scenarios show a tiny little bug. In the first case (the one-liner) perl moans about an uninitialized value and in the second case it dies with a parse error.

This patch fixes both issues, but needs the above patches to be applied first.

Note: This patchset should take care of all four points described in comment #6 by Brian Dolbec. But I have one problem myself, maybe it is just a thinking error.
In the above scenario with the one-liner, ufed states a [-] for both kde (from /etc/make.conf) and gnome (from /etc/portage/make.conf overriding the former setting correctly) as expected. But if I take the negation out (resulting in ( ) ) and save the changes the following happens:
- /etc/make.conf is the set one-liner ; This is the expected and wanted behavior
- /etc/portage/make.conf neither has a flag for kde, nor for gnome now ; This is correct, too.
But: If I open ufed again, gnome now has a [+] (from /etc/make.conf) and the [-] for kde is back.

The problem I have with this is: How should ufed change (in any way) to avoid utter confusion in such scenarios? An update @world will not do what an unaware user would expect now.

Thank you for your thoughts - and your patience!
Comment 14 Sven Eden 2012-09-20 17:55:39 UTC
Created attachment 324424 [details, diff]
Add virtual/perl-File-Spec to RDEPEND in the most current ufed ebuild

To let Portage.pm check /etc/make.conf and /etc/portage/make.conf against symlinking each other, the module has to use File::Spec - which is now a runtime dependency.
Comment 15 Brian Dolbec gentoo-dev 2012-09-21 03:28:08 UTC
Thank you Sven, they look good for me, but then I don't know perl ;)

I'm waiting for some confirmation review ... :)  and we'll get them applied.
Comment 16 Kamen Dokov 2012-09-23 14:56:30 UTC
I'm trying those patches but i'm getting an error with:
add_perl-file-Spec-rdepend_to_ebuild.patch

patch.out file says:

***** add_perl-file-Spec-rdepend_to_ebuild.patch *****
PWD: /var/tmp/portage/app-portage/ufed-0.40.1-r1/work/ufed-0.40.1

======================================================

PATCH COMMAND:  patch -p0 -g0 -E --no-backup-if-mismatch  < '/usr/local/portage/app-portage/ufed/files/add_perl-file-Spec-rdepend_to_ebuild.patch'

======================================================
can't find file to patch at input line 3
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|--- ufed-0.40.1-r1.ebuild.ori	2012-09-20 19:33:57.579520094 +0200
|+++ ufed-0.40.1-r1.ebuild	2012-09-20 19:34:46.716189325 +0200
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored

patch program exited with status 1
======================================================

PATCH COMMAND:  patch -p1 -g0 -E --no-backup-if-mismatch  < '/usr/local/portage/app-portage/ufed/files/add_perl-file-Spec-rdepend_to_ebuild.patch'

======================================================
can't find file to patch at input line 3
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|--- ufed-0.40.1-r1.ebuild.ori	2012-09-20 19:33:57.579520094 +0200
|+++ ufed-0.40.1-r1.ebuild	2012-09-20 19:34:46.716189325 +0200
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored

patch program exited with status 1
======================================================

PATCH COMMAND:  patch -p2 -g0 -E --no-backup-if-mismatch  < '/usr/local/portage/app-portage/ufed/files/add_perl-file-Spec-rdepend_to_ebuild.patch'

======================================================
can't find file to patch at input line 3
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|--- ufed-0.40.1-r1.ebuild.ori	2012-09-20 19:33:57.579520094 +0200
|+++ ufed-0.40.1-r1.ebuild	2012-09-20 19:34:46.716189325 +0200
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored

patch program exited with status 1
======================================================

PATCH COMMAND:  patch -p3 -g0 -E --no-backup-if-mismatch  < '/usr/local/portage/app-portage/ufed/files/add_perl-file-Spec-rdepend_to_ebuild.patch'

======================================================
can't find file to patch at input line 3
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|--- ufed-0.40.1-r1.ebuild.ori	2012-09-20 19:33:57.579520094 +0200
|+++ ufed-0.40.1-r1.ebuild	2012-09-20 19:34:46.716189325 +0200
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored

patch program exited with status 1
======================================================

PATCH COMMAND:  patch -p4 -g0 -E --no-backup-if-mismatch  < '/usr/local/portage/app-portage/ufed/files/add_perl-file-Spec-rdepend_to_ebuild.patch'

======================================================
can't find file to patch at input line 3
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|--- ufed-0.40.1-r1.ebuild.ori	2012-09-20 19:33:57.579520094 +0200
|+++ ufed-0.40.1-r1.ebuild	2012-09-20 19:34:46.716189325 +0200
--------------------------
No file to patch.  Skipping patch.
1 out of 1 hunk ignored

patch program exited with status 1



And this is my ebuild:

# Copyright 1999-2012 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: /var/cvsroot/gentoo-x86/app-portage/ufed/ufed-0.40.1-r1.ebuild,v 1.1 2012/08/12 11:44:53 ssuominen Exp $

EAPI=4
inherit eutils multilib

DESCRIPTION="Gentoo Linux USE flags editor"
HOMEPAGE="http://www.gentoo.org/"
SRC_URI="mirror://gentoo/${P}.tar.bz2
	http://dev.gentoo.org/~truedfx/${P}.tar.bz2"

LICENSE="GPL-2"
SLOT="0"
KEYWORDS="~alpha ~amd64 ~arm ~hppa ~ia64 ~m68k ~mips ~ppc ~ppc64 ~s390 ~sh ~sparc ~x86 ~x86-fbsd"
IUSE=""

DEPEND="sys-libs/ncurses"
RDEPEND="${DEPEND}
	dev-lang/perl"

src_prepare() {
	epatch "${FILESDIR}"/${P}-make.globals-path.patch
	epatch "${FILESDIR}"/${P}-add_new_path_to_pm.patch
	epatch "${FILESDIR}"/${P}-add_new_path_to_ufed.patch
	epatch "${FILESDIR}"/${P}-fix-one-line-make-conf.patch
	epatch "${FILESDIR}"/add_perl-file-Spec-rdepend_to_ebuild.patch
}

src_configure() {
	econf --libexecdir=/usr/$(get_libdir)/ufed
}
Comment 17 teidakankan 2012-09-23 20:55:34 UTC
(In reply to comment #14)
> Created attachment 324424 [details, diff] [details, diff]
> Add virtual/perl-File-Spec to RDEPEND in the most current ufed ebuild
> 
> To let Portage.pm check /etc/make.conf and /etc/portage/make.conf against
> symlinking each other, the module has to use File::Spec - which is now a
> runtime dependency.

I applied your patches and get the following error message when running ufed:

 # ufed
/etc/make.profile is not a symlink
Compilation failed in require at /usr/sbin/ufed line 9.
BEGIN failed--compilation aborted at /usr/sbin/ufed line 9.


Note: I have moved my /etc/make.profile to /etc/portage/make.profile.
Comment 18 Jan Psota 2012-09-23 22:22:59 UTC
> I applied your patches and get the following error message when running ufed:
> /etc/make.profile is not a symlink
> [...]
> 
> Note: I have moved my /etc/make.profile to /etc/portage/make.profile.

Why didn't you go first, simplest, shortest way proposed in bug description?

Better way - checking and correcting on emerge stage - ebuild patch follows.
Added to bleeding-edge, too.
Comment 19 Jan Psota 2012-09-23 22:24:20 UTC
Created attachment 324748 [details, diff]
make.conf placement autodetection on prepare stage
Comment 20 Sven Eden 2012-09-24 11:42:08 UTC
@Kamen: comment #16:
Okay, adding a patch to the ebuild without further description was maybe a bit rash. This patch is only needed to see where I put the required dependency. If you have virtual/perl-File-Spec installed, there is no need to patch the ebuild. (You couldn't anyway)

@teidakankan@gmail.com: comment 17:
My patches have nothing to do with make.profile. I have a symlink from /etc/make.profile to /etc/portage/make.profile, and ufed isn't correctly reading the profile at all.
So make.profile has to be addressed as well, but I do not have the time to do it today, maybe tomorrow. (Can't promise, sorry)

@Jan: comment #19:
Because, as Brian already stated, would be a bad idea to do.

First: You deny users to change their systems.
Second: In the case where a global /etc/make.conf from several systems is used and individual /etc/portage/make.conf files adapt global settings, your approach would wreak havoc.

See comment #8

---

Nevertheless, the handling of make.profile is still to be changed.
Comment 21 Sven Eden 2012-10-04 18:14:29 UTC
Created attachment 325674 [details, diff]
Patch to let Portage.pm look into /etc and /etc/portage for make.conf/profile

This patch drops the previous introduced dependency to File::Spec and uses a much more convenient way to solve symlinks from and to make.conf and make.profile.

make.profile is now correctly read if /etc/make.profile symlinks to /etc/portage/make.profile and vice versa.
Comment 22 Martin Väth 2012-10-15 17:41:21 UTC
Created attachment 326628 [details, diff]
Merge both make.conf's in a nonintrusive way

To read both make.conf's it is not necessary to resolve any symlinks.
(If one is a symlink to the other then it is sourced twice
which does not hurt either).

The corresponding change is very unintrusive and adds just one
one line of code.

For writing, see the next patch which is also rather short.
Comment 23 Martin Väth 2012-10-15 17:47:37 UTC
Created attachment 326630 [details, diff]
Write into correct make.conf

The attached patch makes a backup and modifies /etc/make.conf
if it exists, otherwise if makes a backup and modifies /etc/portage/make.conf.

I think it is saner to "default" to the "global" /etc/make.conf
since ufed does not have a mode to handle individual modifications
(which is what might need to be done in /etc/portage/make.conf).
Comment 24 Martin Väth 2012-10-15 17:49:40 UTC
Created attachment 326632 [details, diff]
Patch for /etc/{,portage}/make.profile

Since also make.profile may exist in /etc or in /etc/portage,
another patch is necessary; this is completely independent of the
above patches.

If you want to test all previous 3 patches easily,
you can install ufed-0.40.1-r2 of the mv overlay (available over layman).
Comment 25 Brian Dolbec gentoo-dev 2012-10-16 01:56:30 UTC
(In reply to comment #23)
> Created attachment 326630 [details, diff] [details, diff]
> Write into correct make.conf
> 
> The attached patch makes a backup and modifies /etc/make.conf
> if it exists, otherwise if makes a backup and modifies
> /etc/portage/make.conf.
> 
> I think it is saner to "default" to the "global" /etc/make.conf
> since ufed does not have a mode to handle individual modifications
> (which is what might need to be done in /etc/portage/make.conf).


I disagree.  If both /etc/make.conf and /etc/portage/make.conf exist, then changes should be saved in /etc/portage/make.conf.  The new location takes priority over the old location and will override settings from it.  Also there are installs that are exporting /etc/make.conf for global settings in a group of machines.  Individual machines make their changes in /etc/portage/make.conf.

So saving to /etc/make.conf may error due to permissions or make changes globally when it was intended to be local only.
Comment 26 Martin Väth 2012-10-16 08:51:02 UTC
(In reply to comment #25)
> If both /etc/make.conf and /etc/portage/make.conf exist, then
> changes should be saved in /etc/portage/make.conf

To change it, would be trivial in my (2nd) patch (just the paths in
the first two patched lines would need to be exchanged).

However, with your suggestion my setup which I consider as natural
would not be possible:

My hand-edited config is all in /etc/portage/make.conf
while the "automatic" /etc/make.conf contains only the USE-flags:
This way, I can be absolutely sure that ufed will not touch anything
of my "valuable hand-edited" config, and to make a backup of my config,
I essentially only need to copy /etc/portage.

Maybe it would make sense to let the user somehow customize which
file should be used (or preferred) for writing: This would also
solve the problem ufed has with "source": Then the user could choose a
completely different file if he wants.

(Essentially, only the first line of the 2nd patch would need to be
modified correspondingly to arrange this)
Comment 27 Kamen Dokov 2012-10-16 09:26:27 UTC
May be it's good idea to have 2 ufed ebuilds:
ufed_legacy for /etc/make.conf

and

ufed for /etc/portage/make.conf

or respectively a USE flag called "legacy" ?
Comment 28 Sven Eden 2012-10-16 14:18:33 UTC
One note beforehand: I just noticed that I have uploaded the wrong patch above. Sorry, sorry, sorry, I'll upload the correct one shortly.


(In reply to comment #22)
> Created attachment 326628 [details, diff] [details, diff]
> Merge both make.conf's in a nonintrusive way
The problem with this is, that it simply adds another hardcoded path. What happens when /etc/portage/make.conf does not exist?


(In reply to comment #23)
> Created attachment 326630 [details, diff] [details, diff]
> Write into correct make.conf
With this setup ufed will destroy the usage of two make.conf files with /etc/make.conf being the global and /etc/portage/make.conf being the machine dependent file.
Further the overwriting of the settings form /etc/make.conf by the settings in /etc/portage/make.conf is no invention of ufed but imposed by portage. ufed really should mirror portage's behavior.


(In reply to comment #24)
> Created attachment 326632 [details, diff] [details, diff]
> Patch for /etc/{,portage}/make.profile

Could you please test your patch with /etc/make.profile being a link to /etc/portage/make.profile? Yes. It does not work. Because line 210 sets "/etc/etc/portage/make.profile" as the profile to be read.
Comment 29 Sven Eden 2012-10-16 14:20:54 UTC
(In reply to comment #28)
> One note beforehand: I just noticed that I have uploaded the wrong patch
> above. Sorry, sorry, sorry, I'll upload the correct one shortly.

Which was wrong. I should not look at large diffs in a hurry. Sorry for the double post, my fault!
Comment 30 Sven Eden 2012-10-16 14:23:33 UTC
Comment on attachment 324424 [details, diff]
Add virtual/perl-File-Spec to RDEPEND in the most current ufed ebuild

The new patch does not need this dependency any more.
Comment 31 Brian Dolbec gentoo-dev 2012-10-16 14:42:42 UTC
Martin, perhaps we can make it configurable.  Since that would be the only thing to save and ufed is already sourcing make.conf, we put it there.  Perhaps UFED_WRITE="/etc/make.conf"  for your setup.  If UFED_WRITE is not found then prefer /etc/portage/make.conf if both make.conf's are found.  Perhaps if both make.conf's exist and the UFED_WRITE is not found, print a message suggesting it be set in one of the make.conf's.

That should cover all possible use cases.

P.S.  Thank you, I am not a perl programmer.  NOW if I could get some feedback from some of the others in the tools-portage herd... I'll get some patches applied and released.
Comment 32 Martin Väth 2012-10-16 19:59:52 UTC
(In reply to comment #28)
> > Merge both make.conf's in a nonintrusive way
> The problem with this is, that it simply adds another hardcoded path.

Which in this case is correct, since these two paths are also hardcoded
into portage. (Maybe EPREFIX should also be honoured, but I guess
that ufed fails with EPREFIX anyway?).

> What happens when /etc/portage/make.conf does not exist?

The right thing: The open fails, so it is not read (without any error).

> Further the overwriting of the settings form /etc/make.conf by the settings
> in /etc/portage/make.conf is no invention of ufed but imposed by portage.
> ufed really should mirror portage's behavior.

I completely agree, and this is exactly what the patch for reading does.

For writing, it is not clear what is the right thing.
I like the idea with UFED_WRITE in make.conf.
However, there is a better idea, but I am not sure when I will find
time to implement it:
One should remember in which file USE is actually modified.
The last read file which modifies USE is logically the only candidate
where the data should be written. This should cover all reasonable setups
without requiring the user to configure something.

> Could you please test your patch with /etc/make.profile being a link to
> /etc/portage/make.profile? Yes. It does not work.

You are right, strange setups like symlinks to symlinks fail.
However, already with the original ufed you could produce such setups
which fail with ufed since the whole norm_path function is broken,
since it does not honour further symlinks or mounted directories.

Maybe the simplest fix would be to remove norm_path and symlink
resolving completely; this should do the right thing, because
e.g. /etc/make.profile/.. should access the correct path.

The only thing which cannot be treated in this simple way is if
PARENTS contain the magic paths starting with ":" or "REPO:"
(which are currently not handled correctly by ufed anyway).

However, as noted above, I have currently no time to implement
anything at all.
Comment 33 Sven Eden 2012-10-17 08:31:52 UTC
(In reply to comment #32)
> One should remember in which file USE is actually modified.
> The last read file which modifies USE is logically the only candidate
> where the data should be written. This should cover all reasonable setups
> without requiring the user to configure something.

I do like both ideas and have a problem with both ideas. Maybe I am just misunderstanding them. Anyway, this approach will fail if /etc/make.conf is a link to a central make.conf in a network, and this central make.conf gets updated. In this case ufed would then try to modify the network version.

> You are right, strange setups like symlinks to symlinks fail.
Why strange? After moving both files to /etc/portage it is quite natural to symlink them, or do it the other way round.

> However, already with the original ufed you could produce such setups
> which fail with ufed since the whole norm_path function is broken,
> since it does not honour further symlinks or mounted directories.

That's why I added sub follow_link, which fixes this.

> Maybe the simplest fix would be to remove norm_path and symlink
> resolving completely; this should do the right thing, because
> e.g. /etc/make.profile/.. should access the correct path.

Yes, this would be another idea. But I have absolutely no idea whether this would drive various read_* functions nuts or not. That's why I added "follow_link", so call me a coward. ;)

At least the above idea about determining the last modified version would be harder to implement, because you would have to make sure to stat the symlink and not the target. Otherwise both would be last to be modified. (No idea whether I described this good enough)

> The only thing which cannot be treated in this simple way is if
> PARENTS contain the magic paths starting with ":" or "REPO:"
>(which are currently not handled correctly by ufed anyway).

Then it should be added. But before I can make a suggestion (or even provide a patch), I need to find out what these "magic paths" are, and what they are supposed to do. :-( Never heard of them before.

However, I think we are on a good way here. :)
Comment 34 Martin Väth 2012-10-17 16:25:39 UTC
(In reply to comment #33)
> I do like both ideas and have a problem with both ideas. Maybe I am just
> misunderstanding them. Anyway, this approach will fail if /etc/make.conf
> is a link to a central make.conf in a network, and this central make.conf
> gets updated. In this case ufed would then try to modify the network
> version.

Which IMHO is what the user would desire: If this central make.conf is
the user's only place where he has the line USE="..." then probably
the user wants that this central make.conf is modified by ufed.
If the user does not want this, he just can write USE="" into a
local make.conf (e.g. in /etc/portage/make.conf or into a file sourced
by that) and then this local file will get updated.

> > You are right, strange setups like symlinks to symlinks fail.
> Why strange? After moving both files to /etc/portage it is quite natural
> to symlink them, or do it the other way round.

Since ufed is probably the last tool which does not support the
new location, there is no reason for such a symlink. Moreover,
why shouldn't the user just keep the previous symlink but turn it
into a symlink to a symlink?
Anyway, it makes no sense to discuss this, since I agree that it
is better if it works independent of symlinks.

> That's why I added sub follow_link, which fixes this.

Wouldn't it be easier to just use Cwd::realpath instead of norm_path
which should handle all cases correctly (including setups like
overmounted directories if it is correctly implemented)?

> Yes, this would be another idea. But I have absolutely no idea whether
> this would drive various read_* functions nuts or not. That's why I
> added "follow_link", so call me a coward. ;)

An advantage of using the real path is that it is possible to detect
implicit loops. However, if no such tests are made (e.g. by hashing
the results), I think that realpath is not necessary.
However, realpath is necessary if you want to support ":":

> > The only thing which cannot be treated in this simple way is if
> > PARENTS contain the magic paths starting with ":" or "REPO:"
> >(which are currently not handled correctly by ufed anyway).
>
> Then it should be added.

Unfortunately, it is not easy to add, since it requires to emulate
a rather large part of portage (adding it to eix required a remarkable
change to the cascading profile's handling)
Officially, details are specified in app-doc/pms, but IMHO this part
was not clearly formulated, so I try to explain it here:

If a "parent" file contains "gentoo:a/b" then the directory actually
referred to is $GENTOO_ROOT/profiles/a/b where $GENTOO_ROOT is
the root directory of the "gentoo" repository, i.e. that directory
specified in PORTDIR or PORTDIR_OVERLAY such that
$GENTOO_ROOT/profiles/repo_name consists of the text "gentoo".

If a "parent" file conains only ":a/b" then the directory actually
referred to is $CURRENT_ROOT/profiles/a/b where $CURRENT_ROOT is
really ugly to find out: You first must find out to which repository
the current "parent" file belongs. Since repositories can be contained
inside each other this means that you must first calculate the realpath
of the "parent" file and then find from the realpath of all filenames
in PORTDIR and PORTDIR_OVERLAY the one with the *longest* matching
part in the beginning.
Doing this without realpath() is impossible.
Comment 35 Paul Varner (RETIRED) gentoo-dev 2012-10-23 17:23:15 UTC
 have committed the patches from Martin Väth to the ufed repository (http://git.overlays.gentoo.org/gitweb/?p=proj/ufed.git;a=summary)

For the patch that writes the make.conf file, I switched the order to prefer /etc/portage/make.conf.

For testing, I have a live ebuild available at http://dev.gentoo.org/~fuzzyray/overlay/ that can be used to install the version in the git repository.

I want to thank everyone that provided patches and the reason I went with Martin's patches is they were the least intrusive changes to the software that worked on my system.  We still need to work on the implementation the following:

Implement a variable (UFED_WRITE) to select which make.conf file is used to write changes
Update ufed to support profiles that use the "repoistory:path" syntax

Please note that ufed is not really being maintained at the moment, so any and all user provided patches are highly appreciated.
Comment 36 Paul Varner (RETIRED) gentoo-dev 2012-11-02 21:11:01 UTC
ufed-0.40.2 is now in the tree. I will close this bug once we get a version with fixes in stable.
Comment 37 Paul Varner (RETIRED) gentoo-dev 2012-11-06 17:22:22 UTC
Released in gentoolkit-0.3.0.7
Comment 38 Paul Varner (RETIRED) gentoo-dev 2012-11-06 17:55:16 UTC
Re-opening since gentoolkit-0.3.0.7 has nothing to do with ufed.
Comment 39 Paul Varner (RETIRED) gentoo-dev 2012-12-27 16:33:49 UTC
app-portage/ufed-0.40.2-r1 is marked stable on all architectures.