Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 286895 - >=dev-python/setuptools-0.6.3-r2 blocks older versions of dev-python/setuptools
Summary: >=dev-python/setuptools-0.6.3-r2 blocks older versions of dev-python/setuptools
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Development (show other bugs)
Hardware: All All
: High normal (vote)
Assignee: Python Gentoo Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-09-29 07:44 UTC by Dennis Schridde
Modified: 2018-08-31 19:42 UTC (History)
6 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dennis Schridde 2009-09-29 07:44:16 UTC
Calculating dependencies ... done!
[ebuild     U ] dev-python/setuptools-0.6.3-r1 [0.6.3] 0 kB
[blocks B     ] <=dev-python/setuptools-0.6.3 ("<=dev-python/setuptools-0.6.3" is blocking dev-python/setuptools-0.6.3-r1)

Total: 1 package (1 upgrade), Size of downloads: 0 kB
Conflict: 1 block (1 unsatisfied)

 * Error: The above package list contains packages which cannot be
 * installed at the same time on the same system.

  ('ebuild', '/', 'dev-python/setuptools-0.6.3-r1', 'merge') pulled in by
    setuptools


Portage 2.2_rc42 (default/linux/amd64/10.0/desktop, gcc-4.4.1, glibc-2.10.1-r0, 2.6.31-gentoo x86_64)
=================================================================                                    
System uname: Linux-2.6.31-gentoo-x86_64-AMD_Athlon-tm-_64_X2_Dual_Core_Processor_5000+-with-gentoo-2.0.1
Timestamp of tree: Tue, 29 Sep 2009 06:45:01 +0000                                                       
app-shells/bash:     4.0_p33                                                                             
dev-java/java-config: 2.1.9-r1                                                                           
dev-lang/python:     2.6.2-r2, 3.1.1-r1                                                                  
dev-python/pycrypto: 2.0.1-r8                                                                            
dev-util/cmake:      2.6.4-r2                                                                            
sys-apps/baselayout: 2.0.1                                                                               
sys-apps/openrc:     0.4.3-r3                                                                            
sys-apps/sandbox:    2.1                                                                                 
sys-devel/autoconf:  2.13, 2.63-r1                                                                       
sys-devel/automake:  1.8.5-r3, 1.9.6-r2, 1.10.2, 1.11                                                    
sys-devel/binutils:  2.19.1-r1                                                                           
sys-devel/gcc-config: 1.4.1                                                                              
sys-devel/libtool:   2.2.6a                                                                              
virtual/os-headers:  2.6.30-r1                                                                           
ACCEPT_KEYWORDS="amd64 ~amd64"                                                                           
CBUILD="x86_64-pc-linux-gnu"                                                                             
CFLAGS="-pipe -O2 -march=athlon64-sse3 -fstack-protector -ftree-vectorize"                               
CHOST="x86_64-pc-linux-gnu"                                                                              
CONFIG_PROTECT="/etc /usr/kde/3.5/env /usr/kde/3.5/share/config /usr/kde/3.5/shutdown /usr/share/config"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d"
CXXFLAGS="-pipe -O2 -march=athlon64-sse3 -fstack-protector -ftree-vectorize"
DISTDIR="/var/cache/portage/distfiles"
EMERGE_DEFAULT_OPTS="--with-bdeps y"
FEATURES="assume-digests distlocks fixpackages metadata-transfer parallel-fetch preserve-libs protect-owned sandbox sfperms strict unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync"
GENTOO_MIRRORS="ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/ http://distfiles.gentoo.org http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LANG="en_GB.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--hash-style=gnu -Wl,--as-needed"
LINGUAS="de"
MAKEOPTS="-j3"
PKGDIR="/var/cache/portage/packages"
PORTAGE_COMPRESS="xz"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/var/cache/portage/gentoo"
PORTDIR_OVERLAY="/var/cache/portage/layman/pcsx2 /var/cache/portage/layman/kde-testing /var/cache/portage/layman/oss-overlay /var/cache/portage/layman/sunrise /var/cache/portage/layman/java-overlay /var/cache/portage/layman/perl-experimental /var/cache/portage/local"
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage"
[...]
Unset:  CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS


Reproducible: Always
Comment 1 Leonard Khoo 2009-09-29 11:09:22 UTC
same problem.
Comment 2 Sebastian Luther (few) 2009-09-29 12:06:20 UTC
You need to manually uninstall the old version.

from the ebuild:
# Avoid silent errors during upgrade from older versions.
DEPEND="!!<=dev-python/setuptools-0.6.3"

Portage doesn't automatically uninstall packages in case of strong blocks (!!).

I'll leave it open for now to prevent duplicates.
Comment 3 Dennis Schridde 2009-09-29 16:59:20 UTC
"Solving" the issue like this seems suboptimal. It is not clear as to why the block appears and what the user should do to fix it.
Comment 4 Sebastian Luther (few) 2009-09-29 17:50:35 UTC
(In reply to comment #3)
> "Solving" the issue like this seems suboptimal. 

Portage devs know about it, not sure if a bug exists.

> It is not clear as to why the block appears 

I don't see a mechanism to provide this information automatically. 

> and what the user should do to fix it.
 
Portage points you to [1], which says:

"To fix a blockage, you can choose to not install the package or unmerge the conflicting package first."

[1] http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?full=1#blocked


I'm assigning it maintainers now because there is another problem. Only >=portage 2.2_rc34 has support for blockers in the same slot. Older portage version ignore it (see bug 270953).
Comment 5 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2009-09-29 18:49:37 UTC
We will leave this bug open to some time after stabilization of 0.6.3-r2.

(In reply to comment #4)
> I'm assigning it maintainers now because there is another problem. Only
> >=portage 2.2_rc34 has support for blockers in the same slot. Older portage
> version ignore it (see bug 270953).

Fixed in dev-python/setuptools-0.6.3-r2.
Comment 6 Leonard Khoo 2009-09-29 21:13:34 UTC
Portage 2.2_rc42 (default/linux/x86/2008.0, gcc-4.4.1, glibc-2.10.1-r0, 2.6.31-gentoo-r1 i686)


[ebuild     U ] dev-python/setuptools-0.6.3-r2 [0.6.3] 0 kB                                                      
[blocks B     ] <dev-python/setuptools-0.6.3-r2 ("<dev-python/setuptools 0.6.3-r2" is blocking dev-python/setuptools-0.6.3-r2)   

Total: 10 packages (8 upgrades, 1 downgrade, 1 reinstall), Size of downloads: 9,092 kB
Conflict: 1 block (1 unsatisfied)                                                     

 * Error: The above package list contains packages which cannot be
 * installed at the same time on the same system.                 

  ('ebuild', '/', 'dev-python/setuptools-0.6.3-r2', 'merge') pulled in by
    dev-python/setuptools required by ('installed', '/', 'net-analyzer/net-snmp-5.4.2.1-r1', 'nomerge')
    dev-python/setuptools required by ('installed', '/', 'dev-python/numpy-1.3.0-r1', 'nomerge')       

Comment 7 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2009-09-29 21:22:09 UTC
All users must manually uninstall older versions of dev-python/setuptools before installation of >=dev-python/setuptools-0.6.3-r2. It is intentional and won't be changed. We don't yet close this bug as INVALID to prevent duplicates.
Comment 8 Davide Pesavento gentoo-dev 2009-09-30 10:38:42 UTC
Arfrever, could you explain the reason for the blocker please?
Comment 9 Richard Cox 2009-09-30 13:39:22 UTC
(In reply to comment #7)
> All users must manually uninstall older versions of dev-python/setuptools
> before installation of >=dev-python/setuptools-0.6.3-r2. It is intentional and
> won't be changed. We don't yet close this bug as INVALID to prevent duplicates.
> 

This morning, setuptools is now blocking the same version of itself.
Comment 10 Sebastian Luther (few) 2009-09-30 14:23:56 UTC
(In reply to comment #9) 
> This morning, setuptools is now blocking the same version of itself.
> 

Please post emerge's output.
Comment 11 Sebastian Luther (few) 2009-09-30 14:25:41 UTC
(In reply to comment #10)

> This morning, setuptools is now blocking the same version of itself.
> 

Please post emerge's output and emerge --version.
Comment 12 Richard Cox 2009-09-30 14:36:49 UTC
(In reply to comment #11)
> (In reply to comment #10)
> 
> > This morning, setuptools is now blocking the same version of itself.
> > 
> 
> Please post emerge's output and emerge --version.
> 

(In reply to comment #11)
> (In reply to comment #10)
> 
> > This morning, setuptools is now blocking the same version of itself.
> > 
> 
> Please post emerge's output and emerge --version.
> 

Sorry, I already fixed it by unmerging and re-emerging setuptools.  It is the same output as seen in comment #6 and in this comment in the forums:

http://forums.gentoo.org/viewtopic-t-794850-highlight-.html

Here is my emerge --version:

Portage 2.2_rc42 (default/linux/amd64/10.0/desktop, gcc-4.3.4, glibc-2.9_p20081201-r2, 2.6.31-gentoo-r1 x86_64)
Comment 13 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2009-09-30 18:43:55 UTC
(In reply to comment #8)
> Arfrever, could you explain the reason for the blocker please?

Upgrade from some older versions tries to replace some directories with regular files, but in this case Portage rather silently renames these regular files and doesn't remove these directories.

These errors will be more verbose in the next versions of Portage:
http://sources.gentoo.org/viewcvs.py/portage?rev=14468&view=rev
Comment 14 Daniel Robbins 2009-09-30 21:57:46 UTC
I've added a fix to Funtoo to eliminate the blocker:

http://github.com/funtoo/portage/commit/ea9ca414b1801cbff883a1d647b293210cb50ff4
Comment 15 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2009-10-02 16:58:59 UTC
Blockers are a feature, not a bug.
Comment 16 Daniel Robbins 2009-10-02 17:05:37 UTC
Passenger airbags in cars are a safety feature -- having your air bags go off all the time for no good reason is a bug. Likewise, unnecessary use of blockers is something I consider to be a bug. Especially a blocker for the previous versions of the same package. That's like having your airbags deploy when you start your car. The driver would not consider that to be a "feature".
Comment 17 Chris Gianelloni 2009-10-02 17:22:00 UTC
(In reply to comment #16)
> Passenger airbags in cars are a safety feature -- having your air bags go off
> all the time for no good reason is a bug. Likewise, unnecessary use of blockers
> is something I consider to be a bug. Especially a blocker for the previous
> versions of the same package. That's like having your airbags deploy when you
> start your car. The driver would not consider that to be a "feature".

No, but the driver would definitely be quite pissed off if the airbags were disabled when pressing the brakes.  Quite simply, your analogy sucks.

The blocker is there to protect the users from a bug in portage, which causes some pretty hefty breakage.  You have removed this protection for your users, while making the astounding claim that you are somehow helping them.  Using something like a vehicle airbag for an analogy evokes a certain reaction from the reader, which I am sure you are aware.  So, are you really trying to fear-monger, or was your choice of example simply poor?  Wouldn't it be much easier to *not* speak in riddles to get your point across?  Simple, plain English seems to work just fine, without trying to evoke fear, add confusion, or otherwise undermine the hard work of the Gentoo team, which is simply trying to protect their users from unintended consequences of their software.
Comment 18 Daniel Robbins 2009-10-02 17:24:38 UTC
Chris, did you actually look at my commit? It actually deals with the problem.
Comment 19 hackablecrap 2009-10-02 17:27:43 UTC
Isn't this package marked ~ anyway? if someone wanted their system to be completely stable they'd probably use stable right?
Comment 20 Daniel Robbins 2009-10-02 17:35:27 UTC
Hackablecrap, we should still try to ensure that unstable is as good as possible. Ask users, they don't like blockers. Also, today's unstable is tomorrow's stable.
Comment 21 Chris Gianelloni 2009-10-02 17:54:29 UTC
(In reply to comment #18)
> Chris, did you actually look at my commit? It actually deals with the problem.

I did not look at it before, but my main point, which was in response to your comment, still stands.  It is simply far too prevalent for people, especially intelligent people, to use analogies which bring about strong emotions to get their point across, rather than using simple to understand technical reasoning.

Had you simply responded that your fix resolved the issue in another way, versus using a poor emotion-based analogy, I wouldn't have said a thing.  Remember, it's not always what we say, but how we say it, that influences people the most.  I think we all can agree that the emotional path never seems to work out right in a technical discussion.  Feelings get hurt, people get upset, and it tends to allow for the louder or more self-assured amongst us to walk all over more meek individuals, even when they have a valid point.  Sticking to the facts keeps things simple and logical for anyone involved.  All of that being said, I think your fix seems to be the better one.  I only have one concern, is there any way we could be wiping out files which do not belong to this package with the "rm -rf"?  It seems like there needs to be a bit more logic/sanity-checking there, unless we're positive that no other package could have files there.  Performing a "rm -rf" anywhere on the live file-system seems like a *very* bad idea.
Comment 22 Daniel Robbins 2009-10-02 18:00:31 UTC
Chris, there are comments in the patch which explains how it works, but essentially what it does is "for every file I am installing that matches a certain pattern, check if it already exists and is a directory. If so, delete the directory."

So based on the original logic of the blocker (ie. if the blocker is correct and complete,) then this code should also be correct and complete.

There is also a safety check to ensure that the path of each directory contains the string "site-packages", to catch potential errors in the string handling that could result in destructive "rm -rf" actions.
Comment 23 Chris Gianelloni 2009-10-02 18:07:40 UTC
(In reply to comment #22)
> Chris, there are comments in the patch which explains how it works, but
> essentially what it does is "for every file I am installing that matches a
> certain pattern, check if it already exists and is a directory. If so, delete
> the directory."

Yes, I saw that.  It doesn't answer my question.  You're using data from the to-be-installed package as a basis for what should be on the live file-system.  While there *shouldn't* be other files on the live file-system, it doesn't mean that there cannot be.  Performing a "rm -rf" anywhere is dangerous and, in my opinion, irresponsible to the users.

> So based on the original logic of the blocker (ie. if the blocker is correct
> and complete,) then this code should also be correct and complete.

Unless the user, for any reason, has any files not belonging to the package in said directories that are being removed.  The blocker doesn't avoid this situation, either, though.

> There is also a safety check to ensure that the path of each directory contains
> the string "site-packages", to catch potential errors in the string handling
> that could result in destructive "rm -rf" actions.

Yes, you error check for problems in the string handling, but not for extraneous files in the already-existing directories.  Like I said, your patch is the better of the two solutions, but still not quite correct to completely resolve the issue.  It also introduces the potential for stomping over files on the user's live system, which seems to me to be a worse issue than forcing the user to unmerge a package by hand.
Comment 24 Daniel Robbins 2009-10-02 18:17:31 UTC
Chris, since it seems like the tone in this thread has reached friendlier levels, let me address the original point I made that seemed to rub you the wrong way.

I do think the point I made is a valid point. Over-use of blockers impacts the automated nature of emerge updates, so they should be avoided if possible. The patch I provided is an example of how this is often possible with a little bit of effort.

When blockers were added to Portage, I viewed them as something that would not be used often, and they were intended to be used for cases where two packages, *due to their nature* (ie. incompatibility, shared files) could not exist. In other words, they were intended to allow for Portage to present "forks in the road," where Portage would inform the user that an either/or choice would need to be made.

Even in these cases, we always tried to look for ways to avoid blockers if possible, by using unique directory paths, etc. In other words, *even for their intended use*, we considered blockers to be a sub-optimal solution.

The use of a blocker in setuptools as it was done here, I would definitely consider to be an interim-type fix and a sub-optimal one. This is not meant to be an insult to any developer, it's just a technical fact and this is a technical forum. 

Blockers are to be avoided and solved in other ways if possible. I would consider a good long-term fix for this particular technical problem (if it happens frequently) to be an upgrade to Portage's merge strategy, so that a package can overwrite its own directories with files, say if they are at least 3 levels deep in the directory tree. So I would escalate this even to a change in Portage functionality if required in order to address this long-term, if necessary.

I hope that gives you insight into my views on blockers -- we should be looking for ways to avoid them and ways to keep Portage working without requiring manual intervention.
Comment 25 Daniel Robbins 2009-10-02 18:24:02 UTC
Chris, extraneous files (files not related to setuptools) will be ignored since the patch uses the to-be-installed files as an initial working set of paths to scan. In my opinion the patch is pretty robust, if that blocker is accurate.

If this kind of problem (a single package needing to replace directories with files) happens periodically, I'd recommend considering an upgrade to Portage's merge strategy to allow these types of upgrades to happen, possibly in conjunction with the elimination of .keep files. (So, something you might call "more sophisticated Portage handling of directory management and ownership".)
Comment 26 Chris Gianelloni 2009-10-02 18:38:25 UTC
(In reply to comment #25)
> Chris, extraneous files (files not related to setuptools) will be ignored since
> the patch uses the to-be-installed files as an initial working set of paths to
> scan. In my opinion the patch is pretty robust, if that blocker is accurate.

OK.  I already explained how the blocker is not technically robust.  Your patch assumes that the blocker *is* technically robust.  I understand what your patch does, you do not need to reiterate it, yet again.  It is, after all, only a few lines of shell.  While it take the to-be-installed files as an initial working set, it doesn't take into account if *any* other process/package/etc. has put files in those same directories.

Rather than (nearly) blindly doing a "rm -rf" on the user's live file-system, a non-blocker solution should verify the contents of the directory versus the known contents of the older version of the package.  If and *only* if there are no files other than those belonging to the package, perform the "rm -rf" on the directory.  Otherwise, it should bail out *immediately* and inform the user, rather than potentially trashing their files.

To make this clearer, the current in-tree blocker solution does *not* cover this case, either, so let's cease the propagation of the assumption that the blocker is even a correct fix.  Using your own example of .keep files, toss a .keep into one of the directories and unmerge setuptools and merge the newer version.  You end up with the same results as if no blocker existed.  Now, create a second package which puts files into any one of these directories, then merge setuptools using your patch, and it wipes out the files belonging to another package!
Comment 27 Daniel Robbins 2009-10-02 19:54:02 UTC
My understanding is that no other ebuilds install files into those directories. If you have any data indicating otherwise, please let me know.
Comment 28 Arfrever Frehtes Taifersar Arahesis (RETIRED) gentoo-dev 2009-10-06 19:37:03 UTC
I deleted blocker for some other reasons.