| Summary: | sys-apps/portage-2.2.14_rc1 trouble emerging on case insensitive volumes | ||
|---|---|---|---|
| Product: | Gentoo/Alt | Reporter: | iloveosxzfs |
| Component: | Prefix Support | Assignee: | Gentoo Prefix <prefix> |
| Status: | RESOLVED FIXED | ||
| Severity: | major | CC: | dev-portage, iloveosxzfs, pacho |
| Priority: | Normal | ||
| Version: | unspecified | ||
| Hardware: | All | ||
| OS: | OS X | ||
| Whiteboard: | |||
| Package list: | Runtime testing required: | --- | |
| Attachments: |
Initial detected file collisions.
Portage broken Implement FEATURES=case-insensitive-fs Implement FEATURES=case-insensitive-fs Implement FEATURES=case-insensitive-fs |
||
Created attachment 385910 [details]
Portage broken
Can reproduce this. Feels like the unmerge code doesn't take the collisions into account. For reference, this is easily fixed by running: ebuild portage-2.2.14_rc1.ebuild merge from $EPREFIX/usr/src/portage/sys-apps/portage. @portage: the respective files are not on the filesystem after merge (FEATURES=-collision-protect), any pointers on where to look are appreciated. (In reply to iloveosxzfs from comment #1) > Created attachment 385910 [details] > Portage broken The log clearly shows the UserQuery.py* files being merged with camelcase names, and then unmerged with lowercase names. In order to avoid this behavior, we would need case-insensitive support in the dblink.isowner method, since that's what's used to check if an unmerge candidate belongs to the new instance that was just merged. (In reply to Zac Medico from comment #4) > (In reply to iloveosxzfs from comment #1) > > Created attachment 385910 [details] > > Portage broken > > The log clearly shows the UserQuery.py* files being merged with camelcase > names, and then unmerged with lowercase names. > > In order to avoid this behavior, we would need case-insensitive support in > the dblink.isowner method, since that's what's used to check if an unmerge > candidate belongs to the new instance that was just merged. First, we need to agree that we won't allow installing two files with the same case-insensitive names. A QA check could suffice for a start but don't know if people really want that limitation. Not sure if we want to restrict case changes. Collision protection already identifies there is a problem (and aborts). The only real problem (IMO) at the moment is that unmerging removes the just installed file. I'll see if I can do something with dblink.isowner as Zac suggests. Case changes can still be accomplished on a case-insensitive file system by doing mv foo bar mv bar Foo though I don't know whether that approach is feasible in this context. (In reply to iloveosxzfs from comment #7) > Case changes can still be accomplished on a case-insensitive file system Not necessarily. For some filesystems there are options to report always in upper or lower case. Right. That is the distinction between the meaning of being case sensitive and the mean of being case preserving. You are right that I was assuming it would be case preserving, which is because I'm particularly thinking of HFS+J which is both case insensitive and case preserving. Created attachment 385946 [details, diff] Implement FEATURES=case-insensitive-fs When case-insensitive-fs is enabled in FEATURES, the dblink.isowner method and the _owners_db class will be case-insensitive. This causes the collision-protect and unmerge code to behave correctly for a case-insensitive file system. If the file system is case-insensitive but case-preserving, then case is preserved in the CONTENTS data of installed packages. I'll be maintaining this patch in the following branch: https://github.com/zmedico/portage/tree/bug_524236 Created attachment 385950 [details, diff]
Implement FEATURES=case-insensitive-fs
This fixes the previous patch so that dblink._match_contents correctly preserves the case of the CONTENTS key that it returns.
I'm against adding anything more to FEATURES. This variable is a horrible mistake, and features should be controlled via separate configuration variables. (In reply to Michał Górny from comment #12) > I'm against adding anything more to FEATURES. This variable is a horrible > mistake, and features should be controlled via separate configuration > variables. Are you sure that you want to add more environment variables instead? Maybe we should also have a way to configure things that doesn't involve environment variables. For example, when we added repos.conf, we replaced PORTDIR and PORTDIR_OVERLAY variables with an INI-style configuration file. Yes, I would like to have proper configuration too. However, I don't have time myself to work on all that. (In reply to Michał Górny from comment #14) > Yes, I would like to have proper configuration too. However, I don't have > time myself to work on all that. In this context, is adding a new environment variable really better than re-using the existing FEATURES variable? What's so bad about the FEATURES variable? (In reply to Zac Medico from comment #15) > (In reply to Michał Górny from comment #14) > > Yes, I would like to have proper configuration too. However, I don't have > > time myself to work on all that. > > In this context, is adding a new environment variable really better than > re-using the existing FEATURES variable? What's so bad about the FEATURES > variable? Yes, anything is better than adding up to that crap pile. What's so bad is that it is a collection of more than dozen totally unrelated options that sometimes collide one with another or force others. It's completely impossible to properly template it for CONFIG_PROTECT, and one basically has to diff manpage from time to time to know when you push yet another magical keywords into it. (In reply to Michał Górny from comment #16) > Yes, anything is better than adding up to that crap pile. What's so bad is > that it is a collection of more than dozen totally unrelated options that > sometimes collide one with another or force others. You seem biased against FEATURES. Anyway, we can discuss its pros/cons on a mailing list, since it's only tangentially related to this bug. > It's completely impossible to properly template it for CONFIG_PROTECT, I'm not sure what you mean here. Are you talking about some interaction between case-insensive support and CONFIG_PROTECT? Anyway, it reminds me that the attached patch doesn't yet include case-insensitive support for CONFIG_PROTECT (though it should not be difficult to implement). Created attachment 386404 [details, diff]
Implement FEATURES=case-insensitive-fs
This fixes the previous patch to include case-insensitive CONFIG_PROTECT support (including etc-update and dispatch-conf).
I've included this in the Prefix branch where it seems to work wonders 2.2.10.1-r1 and 2.2.14 include the case-insensitive-fs patch and enable it through make.globals. People must upgrade through 2.2.10.1-r1 because the patch must be active to emerge 2.2.14. I couldn't find a way to express this in dependencies (Portage complained about circular deps and/or just ignored it) therefore I masked 2.2.14 for the moment. To be unmasked after a week or so from now. For the master branch, I've posted this for review: http://thread.gmane.org/gmane.linux.gentoo.portage.devel/4779 This has been merged into the master branch as these two commits: https://github.com/gentoo/portage/commit/317629c7e7acbae32b56d1dec86d11bc038fc976 https://github.com/gentoo/portage/commit/5ace188b449926dae09aca0a09aea2c2fe216bd7 I will backout the earlier one, and merge from master for the next prefix release (In reply to Fabian Groffen from comment #20) > I masked 2.2.14 for the moment. To be unmasked after a week or so from now. Unmasked. |
Created attachment 385908 [details] Initial detected file collisions. At first we get file collisions. * Detected file collision(s): * * /Users/joe/gentoo/usr/lib/portage/pym/_emerge/UserQuery.py * /Users/joe/gentoo/usr/lib/python2.7/site-packages/_emerge/UserQuery.py * /Users/joe/gentoo/usr/lib/python3.3/site-packages/_emerge/UserQuery.py Moving those files out of the way allows for a "successful" emerge, but when we actually then go try to use it: joe@Josephs-MacBook-Pro ~ $ emerge Traceback (most recent call last): File "/Users/joe/gentoo/usr/bin/emerge", line 50, in <module> retval = emerge_main() File "/Users/joe/gentoo/usr/lib/python2.7/site-packages/_emerge/main.py", line 1043, in emerge_main action=myaction, args=myfiles, opts=myopts) File "/Users/joe/gentoo/usr/lib/python2.7/site-packages/portage/proxy/objectproxy.py", line 30, in __call__ result = object.__getattribute__(self, '_get_target')() File "/Users/joe/gentoo/usr/lib/python2.7/site-packages/portage/proxy/lazyimport.py", line 130, in _get_target __import__(name) File "/Users/joe/gentoo/usr/lib/python2.7/site-packages/_emerge/actions.py", line 71, in <module> from _emerge.depgraph import backtrack_depgraph, depgraph, resume_depgraph File "/Users/joe/gentoo/usr/lib/python2.7/site-packages/_emerge/depgraph.py", line 76, in <module> from _emerge.UserQuery import UserQuery ImportError: No module named UserQuery joe@Josephs-MacBook-Pro ~ $ So I reverted to the prior version of sys-apps/portage using the ebuild command since the portage command itself was now broken. Later, based on the diagnosis of mgorny from #gentoo-portage, I created a case-sensitive ram disk, copied my entire prefix there, and tried again. The emerge succeeded and the new emerge command worked. So I rsync'd the changes from the ram disk back to my real prefix, ejected ram disk, and restored the real prefix to its original location. And the new emerge command is now installed fine. amadio from #gentoo-prefix reports that copying only the UserQuery.py files from Linux into OS X prefix also was a successful workaround.