Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 367215 - sys-apps/portage: package moves should not cause a package to block itself (installed app-crypt/gnupg blocks itself)
Summary: sys-apps/portage: package moves should not cause a package to block itself (i...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: Normal normal
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks: 193766 445274
  Show dependency tree
 
Reported: 2011-05-14 09:29 UTC by Christian Burger
Modified: 2012-11-29 23:52 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
emerge --info as requested (emerge-info.text,6.36 KB, text/plain)
2011-05-17 17:17 UTC, Christian Burger
Details
cave info gnupg (gnupg-info,14.55 KB, application/octet-stream)
2011-08-10 03:26 UTC, Daniel Salas
Details
shorter patch (update.py.1.patch,672 bytes, patch)
2012-11-07 21:18 UTC, Christian Burger
Details | Diff
longer patch (but more elegant) (update.py.2.patch,1.20 KB, patch)
2012-11-07 21:21 UTC, Christian Burger
Details | Diff
patch to commit 228a860476d7543608b469c569ec1d4e70aa7f59 (avoid-self-blockers.patch,1.71 KB, patch)
2012-11-09 13:00 UTC, Christian Burger
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Christian Burger 2011-05-14 09:29:40 UTC
After reinstalling app-crypt/gnupg because of a required USE flag change on one of the packages it depends on (dev-libs/libgpg-error), the package blocked itself.

$ cat /var/db/pkg/app-crypt/gnupg-2.0.17/RDEPEND 

!static? ( >=dev-libs/libassuan-2 >=dev-libs/libgcrypt-1.4 >=dev-libs/libgpg-error-1.7 >=dev-libs/libksba-1.0.7 >=dev-libs/pth-1.3.7 >=net-misc/curl-7.10 adns? ( >=net-libs/adns-1.4 ) bzip2? ( app-arch/bzip2 ) pcsc-lite? ( >=sys-apps/pcsc-lite-1.3.0 ) openct? ( >=dev-libs/openct-0.5.0 ) smartcard? ( =virtual/libusb-0* ) ldap? ( net-nds/openldap ) ) || ( app-crypt/pinentry app-crypt/pinentry-qt ) virtual/mta !app-crypt/gnupg !<=app-crypt/gnupg-2.0.1 selinux? ( sec-policy/selinux-gnupg ) nls? ( virtual/libintl )

Paludis' cave (the package manager I use) modified the block of the old package's name app-crypt/gpg-agent to the new one -- the package was moved in 2Q-2006.

According to this

http://www.mail-archive.com/gentoo-dev@lists.gentoo.org/msg42102.html

blocking the old package name is unnecessary and in my case resulted in an unresolvable dependency tree. Could someone please remove the block?

I know how to fix the problem: Throwing out the self-block from DEPEND or RDEPEND in /var/db/pkg. But still: The problem might reappear next time I update app-crypt/gnupg.

Thank a lot.
Comment 1 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2011-05-17 01:25:10 UTC
Where did you get that copy of the ebuild? The RDEPEND you list there is NOT the same as the ebuild.

Specifically, you have "!app-crypt/gnupg !<=app-crypt/gnupg-2.0.1", while the ebuild only has "!<=app-crypt/gnupg-2.0.1".

Also, where is your emerge --info output?

I cannot reproduce your problem.
Comment 2 Christian Burger 2011-05-17 17:16:41 UTC
Thanks for your reply.

(In reply to comment #1)
> Where did you get that copy of the ebuild? The RDEPEND you list there is NOT
> the same as the ebuild.

No, it is not the same, you are right, it is the one from the /var/db/pkg after I installed the package via 'cave' the Paludis package manager. I am sorry if my explanation was too short.

> Specifically, you have "!app-crypt/gnupg !<=app-crypt/gnupg-2.0.1", while the
> ebuild only has "!<=app-crypt/gnupg-2.0.1".

In my gnupg-2.0.17.ebuild (directly from the Portage tree) the depend is like you would expect it to be:

RDEPEND="!static? ( ${COMMON_DEPEND_LIBS} )
        ${COMMON_DEPEND_BINS}
        virtual/mta
        !app-crypt/gpg-agent
        !<=app-crypt/gnupg-2.0.1
        selinux? ( sec-policy/selinux-gnupg )
        nls? ( virtual/libintl )"

It is just that cave adjusted the block on the old package name to reflect the new one -- according to a package move/rename done in Q2-2006:

"!app-crypt/gpg-agent !<=app-crypt/gnupg-2.0.1"

becomes

"!app-crypt/gnupg !<=app-crypt/gnupg-2.0.1"

> Also, where is your emerge --info output?

I thought because of the nature of the problem, it would be irrelevant. But as I stated before I use Paludis' cave, so an emerge --info will only be mildly helpful, but I will attach it.

> I cannot reproduce your problem.

I understand that people using emerge do not have that problem. But according to
http://www.gentoo.org/proj/en/devrel/handbook/handbook.xml?part=2&chap=5#doc_chap6
a block to the old package name is not required nor encouraged if a package is renamed or moved, simply let the package manager (pkgcore, emerge or paludis) do its magic -- at least that is my understanding for the moment.
Comment 3 Christian Burger 2011-05-17 17:17:48 UTC
Created attachment 273687 [details]
emerge --info as requested
Comment 4 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2011-05-17 22:10:23 UTC
Portage: not sure if this is strictly a Paludis bug, or a case where Portage isn't picking up the rename in the same way (in the case that the rename does need to be removed).
Comment 5 Zac Medico gentoo-dev 2011-05-17 22:33:04 UTC
(In reply to comment #4)
> Portage: not sure if this is strictly a Paludis bug, or a case where Portage
> isn't picking up the rename in the same way (in the case that the rename does
> need to be removed).

If we fix portage so that the it doesn't create circular blockers when applying package moves, won't that solve the problem?
Comment 6 Zac Medico gentoo-dev 2011-05-17 22:40:34 UTC
(In reply to comment #5)
> It is just that cave adjusted the block on the old package name to reflect the
> new one -- according to a package move/rename done in Q2-2006:
> 
> "!app-crypt/gpg-agent !<=app-crypt/gnupg-2.0.1"
> 
> becomes
> 
> "!app-crypt/gnupg !<=app-crypt/gnupg-2.0.1"

This is a known bug in portage, since it's illogical to create a circular blocker when applying a package move. I'd like to fix portage's update_dbentry function so that it takes a parent argument and avoids creating circular blockers.
Comment 7 Robin Johnson archtester Gentoo Infrastructure gentoo-dev Security 2011-05-17 22:49:09 UTC
Ok, so the gpg-agent block should be removed from the ebuilds?
Comment 8 Zac Medico gentoo-dev 2011-05-18 02:13:26 UTC
(In reply to comment #7)
> Ok, so the gpg-agent block should be removed from the ebuilds?

Well, I guess that's your decision. It wouldn't be a problem if portage's update_dbentry() function handled it better. The code that moves binary/installed packages should also be fixed to drop blockers that become circular as a result of the move.
Comment 9 Daniel Salas 2011-08-10 03:25:53 UTC
I have the same problem. I'm attaching 'cave info gnupg' as it seems is more relevant.
Comment 10 Daniel Salas 2011-08-10 03:26:40 UTC
Created attachment 282777 [details]
cave info gnupg
Comment 11 Zac Medico gentoo-dev 2011-08-11 00:28:22 UTC
(In reply to comment #9)
> I have the same problem. I'm attaching 'cave info gnupg' as it seems is more
> relevant.

We don't need any more info since the problem is well understood. The problem is that portage (and maybe paludis too) creates nonsensical circular blockers when doing package moves. The package move code (in portage, located in the update_dbentry() function) can be fixed to prevent creation of these kinds of nonsensical circular blockers.
Comment 12 Dane Smith (RETIRED) gentoo-dev 2012-11-05 13:47:46 UTC
(In reply to comment #11)
> (In reply to comment #9)
> > I have the same problem. I'm attaching 'cave info gnupg' as it seems is more
> > relevant.
> 
> We don't need any more info since the problem is well understood. The
> problem is that portage (and maybe paludis too) creates nonsensical circular
> blockers when doing package moves. The package move code (in portage,
> located in the update_dbentry() function) can be fixed to prevent creation
> of these kinds of nonsensical circular blockers.

Should this maybe be a Portage bug then? Or would you rather I closed it.
Comment 13 Zac Medico gentoo-dev 2012-11-05 17:39:35 UTC
Yeah, it's a Portage bug.
Comment 14 Christian Burger 2012-11-07 21:17:38 UTC
Just another two cents from me:

1) I have to correct my initial description in retrospect: I later realized that (at least in my case) Paludis was not the cause of the problem. I did use the 'emerge' command shortly before the problem occurred, which caught up with a lot of package moves and so on from the last quarters -- Paludis did those already.

Mixing Paludis and 'emerge' might sometimes not be the best idea.

2) Since version 0.74 Paludis ignores those self-blockers.
http://ciaranm.wordpress.com/2012/04/15/paludis-0-74-0-released/
Though this helps the issue, it does not really solve the problem.

3) As far as I understand it, this is not a bug in Portage, but bad behavior by package maintainers. But I am not here to start a lengthy philosophical debate. :)

4) I will attach two patch proposals; either one should do the job. I did not test them, since I do not use 'emerge' nor do I have an environment set up where I could test intensively.

Hope it helps.
Comment 15 Christian Burger 2012-11-07 21:18:52 UTC
Created attachment 328744 [details, diff]
shorter patch

By regex search and replace, this patch resets obvious self-blockers after the update.
Comment 16 Christian Burger 2012-11-07 21:21:49 UTC
Created attachment 328746 [details, diff]
longer patch (but more elegant)

This patch prevents the offending self-block in the first place: "!cat/pkg" is not replaced.
Comment 17 Zac Medico gentoo-dev 2012-11-09 01:49:04 UTC
Your patches don't apply to the latest (master branch):

  http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git
Comment 18 Christian Burger 2012-11-09 12:58:17 UTC
(In reply to comment #17)
> Your patches don't apply to the latest (master branch):
> 
>   http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git

Ah, cool, I see. And there is a test environment. Nice. :)

Wrote another patch and extended the test case for db-entry-updates. Attaching the output of "git log -u -1"

A related question: Those db-updates only change the db-entries of the packages that are moved? It's not like another already installed package cat/A blocking a certain package cat/B will be updated if cat/B is moved to cat/C. Because then my patch would be harmful.

I checked the source, but it appears to me that fixdbentries() is only used on the package to be moved. But then you (Zac) mentioned a additional "parent" argument:

comment #6:
> blocker when applying a package move. I'd like to fix portage's
> update_dbentry function so that it takes a parent argument and avoids
> creating circular blockers.

That fits into my train of thought. Well, what do I know, first time looking at the Portage source ...

But, I am a little bit confused about that, because if only db-entries of moved packages are updated, there might be dangling references to the old package name in the package database.
Comment 19 Christian Burger 2012-11-09 13:00:21 UTC
Created attachment 329010 [details, diff]
patch to commit 228a860476d7543608b469c569ec1d4e70aa7f59
Comment 20 Zac Medico gentoo-dev 2012-11-14 07:11:31 UTC
(In reply to comment #18)
> A related question: Those db-updates only change the db-entries of the
> packages that are moved? It's not like another already installed package
> cat/A blocking a certain package cat/B will be updated if cat/B is moved to
> cat/C. Because then my patch would be harmful.
> 
> But, I am a little bit confused about that, because if only db-entries of
> moved packages are updated, there might be dangling references to the old
> package name in the package database.

update_dbentry is called for *all* packages, in order to fix those dangling references. So, your patch needs to account for the parent. I've added a parent argument for this:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=0b4e6c27e87e57fac09fa28d3ed1320cb6a83ad9
Comment 22 Christian Burger 2012-11-14 23:19:20 UTC
(In reply to comment #21)
> Account for parent in update_dbentry:
> 
> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;
> h=cae023ad64e403dd4246eab558bbffa9f611b724

Maybe I am wrong, but it seems to me that in

pym/portage/dbapi/vartree.py:

        def move_ent(self, mylist, repo_match=None):
                [...]
                origmatches = self.match(origcp, use_cache=0)
                [...]
                for mycpv in origmatches:
                        [...]
                        fixdbentries([mylist], newpath, parent=mycpv)

the old/original package name is given, but in

pym/portage/update.py:

        def update_dbentry(update_cmd, mycontent, eapi=None, parent=None):
                [...]
                new_atom = Atom(token.replace(old_value, new_value, 1),
                        eapi=eapi)
                [...]
                if new_atom.blocker and parent is not None and \
                        parent.cp == new_atom.cp and \
                        match_from_list(new_atom, [parent]):
                        continue

it is compared to the new package name.
Comment 23 Zac Medico gentoo-dev 2012-11-15 00:05:20 UTC
Thanks, you're right. This should fix it:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=c974a023882485b8eeae35bac35c1f00d1a0725b
Comment 24 Christian Burger 2012-11-15 15:53:25 UTC
(In reply to comment #23)
> Thanks, you're right. This should fix it:
> 
> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;
> h=c974a023882485b8eeae35bac35c1f00d1a0725b

In that case, you might want to remove the method alltogether:


$ grep fixdbentries portage/* -R

pym/portage/update.py:def fixdbentries(update_iter, dbdir, eapi=None, parent=None):
pym/portage/__init__.py:                'portage.update:dep_transform,fixdbentries,grab_updates,' + \


It looks to me like it isn't referenced anywhere anymore.
Comment 25 Zac Medico gentoo-dev 2012-11-15 16:10:19 UTC
Okay, I've added a deprecation warning to fixdbentries:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=a6a9b6dd6342a6009097944b38163ccc66908f0e
Comment 26 Zac Medico gentoo-dev 2012-11-29 23:52:47 UTC
This is fixed in 2.1.11.32 and 2.2.0_alpha143.