> 2.1.2 Package Names > A package name may contain any of the characters [A-Za-z0-9+ -]. > It must not begin with a hyphen, and must not end in a hyphen > followed by one or more digits. The "must not end in a hyphen followed by one or more digits" restriction is not enough: "foo-1a" or "foo-2_alpha" should not be allowed as package names neither, for the same reasons "-[0-9]+$" is forbidden. I suggest something like this: It must not begin with a hyphen, and must not end in a hyphen followed by anything matching the package versions syntax described in section 2.2.
Hrm, your two examples don't appear to be ambiguous in any context. Consider: =bar/foo-1a-1 =bar/foo-2_alpha-1.2.3 In both cases, by selecting the last place where there's a hyphen followed immediately by a digit as the split point is enough to parse the spec correctly.
(In reply to comment #1) > Hrm, your two examples don't appear to be ambiguous in any context. Consider: > > =bar/foo-1a-1 > =bar/foo-2_alpha-1.2.3 Sure, there is no ambiguity when you already know whether the string you're facing is versionned or not. Consider: =bar/foo-123-4 ...here too it is easy to cut between the name and the version, but that's not the point. If you were sure to have this additional information (the "is it versionned?" bit) in 100% of the real world cases, then you could as well just drop the restriction about "-[0-9]+$", since it is hardly more useful than what i am suggesting. But it happens that there are some contexts where you just don't exactly know what your string is (no dep operator hint), and you guess from its syntax. It's a trick used by many utilities around (portage-utils and others), which accept both "cat/name" and "cat/name-ver" as arguments (but don't care about real dep atoms, and thus don't expect a leading "=" for the later). I may agree it's not really clean, but it is convenient, and i think it would suck to forbid that just to allow a few hypothetical package names that no one will ever use.
There shouldn't be any cases where both cat/pkg and cat/pkg-ver are legal. The version format wasn't designed to allow this.
(In reply to comment #3) > There shouldn't be any cases where both cat/pkg and cat/pkg-ver are legal. Imo it's part of the many things which shouldn't be, but which are and thus should be taken into account by PMS. > The version format wasn't designed to allow this. How do you explain the restriction on "-[0-9]+$" then?
(In reply to comment #4) > (In reply to comment #3) > > There shouldn't be any cases where both cat/pkg and cat/pkg-ver are legal. > > Imo it's part of the many things which shouldn't be, but which are and thus > should be taken into account by PMS. This predates PMS by a long time. PMS isn't going to change long-standing behaviour on this one. > > The version format wasn't designed to allow this. > > How do you explain the restriction on "-[0-9]+$" then? There has to be a way of splitting a pkg-ver when it is known that a string is a pkg-ver. There does not have to be a way of telling whether a string is a pkg or a pkg-ver.
I'd say we should not care about programs that accept obviously incomplete DEP specs.
(In reply to comment #5) > (In reply to comment #4) > > How do you explain the restriction on "-[0-9]+$" then? > > There has to be a way of splitting a pkg-ver when it is known that a string is > a pkg-ver. Which doesn't answer my question. As you've already pointed, splitting on "the last place where there's a hyphen followed immediately by a digit" is enough when you know you're in front of a pkg-ver, like for instance "=cat/foo-123-4.5". So why does PMS forbids it? Either there's a reason and it should be done in a complete and coherent way (not forbidding only integers, but also integers with a one letter suffix, etc.), or there is no reason and this restriction on "-[0-9]+$" should simply be removed from PMS. (In reply to comment #6) > I'd say we should not care about programs that accept obviously incomplete DEP > specs. When said programs have no reason to care are dep atoms (simply because they don't deal with some package dependencies), i don't see why they should conform to the complete dep atoms spec. Take the recent proposal about automating part of the bug-wrangling work for instance. You can't expect users will put an "=" sign in front of all the "cat/pkg-ver" they talk about. So, when one reports that "foo/bar-3a doesn't compile", should the wrangling script look for maintainers of a pkg named "foo/bar-3a" or "foo/bar"? Forbidding "foo/bar-3a" (and whatever else ends in what may be a package version) as a pkg name would answer the question. It is exactly in the spirit of the already specified "-[0-9]+$" restriction, with the only difference that it is complete and coherent. And sure i have checked it doesn't go against anything used in the current tree, so why not doing it?
(In reply to comment #7) > Which doesn't answer my question. As you've already pointed, splitting on "the > last place where there's a hyphen followed immediately by a digit" is enough > when you know you're in front of a pkg-ver, like for instance > "=cat/foo-123-4.5". So why does PMS forbids it? Either there's a reason and > it should be done in a complete and coherent way (not forbidding only integers, > but also integers with a one letter suffix, etc.), or there is no reason and > this restriction on "-[0-9]+$" should simply be removed from PMS. The restriction is necessary to make the place where a pkg-ver is split well defined. > Take the recent proposal about automating part of the bug-wrangling work for > instance. You can't expect users will put an "=" sign in front of all the > "cat/pkg-ver" they talk about. So, when one reports that "foo/bar-3a doesn't > compile", should the wrangling script look for maintainers of a pkg named > "foo/bar-3a" or "foo/bar"? That's rather easily solved by requesting package and version separately. > Forbidding "foo/bar-3a" (and whatever else ends in what may be a package > version) as a pkg name would answer the question. It is exactly in the spirit > of the already specified "-[0-9]+$" restriction, with the only difference that > it is complete and coherent. And sure i have checked it doesn't go against > anything used in the current tree, so why not doing it? Historical precedent. You're asking for us to introduce a substantially more complicated and more restrictive requirement when there's no need for one. This isn't going to happen.
(In reply to comment #8) > The restriction is necessary to make the place where a pkg-ver is split well > defined. Do you really mean that without this restriction there would be some ambiguities on where to split some pkg-ver? Do you have examples?
(In reply to comment #9) > (In reply to comment #8) > > The restriction is necessary to make the place where a pkg-ver is split well > > defined. > > Do you really mean that without this restriction there would be some > ambiguities on where to split some pkg-ver? Do you have examples? Hrm. Depends upon whether we stick with the ban on . in package names (it's allowed in category names because of CHOST).
In order to prevent ambiguities with some plausible versions in the near future (-scm), it should ban package names which end in a hyphen followed by a valid version-like string. As reported originally, although maybe with a slightly different wording. It looks weird, but 'foo-2b-r1' is not ambiguous. If we want to go with the less restrictive path, we can just ban names ending with a hyphen followed by what could be the first part of a version specification ([0-9]+(\.[0-9]+)* followed by a letter optionally followed by a zero or more suffixes). So, Invalid: * foo-2b * foo-2b_alpha Valid: * foo-2b-bar * foo-2b-r1 * foo-_alpha * foo-scm (if GLEP 54 is accepted)
(In reply to comment #11) > Valid: > * foo-scm (if GLEP 54 is accepted) I mean, it's valid now and would still be valid if GLEP 54 is accepted ;-)
(In reply to comment #7) > As you've already pointed, splitting on "the last place where there's a > hyphen followed immediately by a digit" is enough when you know you're in > front of a pkg-ver, like for instance "=cat/foo-123-4.5". So why does PMS > forbids it? Either there's a reason and it should be done in a complete > and coherent way (not forbidding only integers, but also integers with a > one letter suffix, etc.), or there is no reason and this restriction on > "-[0-9]+$" should simply be removed from PMS. Coming back to this. Is there any technical reason why "-[0-9]+$" in package names is banned? (In reply to comment #1) > Hrm, your two examples don't appear to be ambiguous in any context. > Consider: > > =bar/foo-1a-1 > =bar/foo-2_alpha-1.2.3 > > In both cases, by selecting the last place where there's a hyphen followed > immediately by a digit as the split point is enough to parse the spec > correctly. The same would be true for "=bar/foo-1-1" or "=bar/foo-123-4.5". Also in these cases PN and PV can be split at the last hyphen without any ambiguity. So, couldn't we drop the "must not end in a hyphen followed by one or more digits" restriction?
(In reply to comment #13) > Coming back to this. Is there any technical reason why "-[0-9]+$" in package > names is banned? If memory serves, that was a portage quirk rather than an actual PM issue that led to this- roughly something related to portage's version parsing stripping the operators off first, then parsing the result, basically losing the context of whether or not it was parsing in a versionless or versioned context. I know pkgcore inherited something similar, although that was resolved a few years back. Zac, does the above sound familiar, if so, is it it still in the codebase?
(In reply to comment #13) > Coming back to this. Is there any technical reason why "-[0-9]+$" in package > names is banned? The only technical reason that I can think of is to allow for user interfaces that are able to take a mix of cp and cpv as input, and be able to automatically distinguish between them. For example, emerge --unmerge currently accepts a plain cpv as input, which technically is not a valid atom (without = operator). It also accepts a cp, which is a valid atom, but could also correspond a valid cpv if the "-[0-9]+$" constraint is removed. (In reply to comment #14) > If memory serves, that was a portage quirk rather than an actual PM issue that > led to this- roughly something related to portage's version parsing stripping > the operators off first, then parsing the result, basically losing the context > of whether or not it was parsing in a versionless or versioned context. I'm not sure about that reasoning, but portage.versions module used to be a bit messy. It's since been rewritten with a single regular expression to match each input, which makes it very strict and easy to audit.
Created attachment 300745 [details, diff] Patch for PMS (In reply to comment #15) > The only technical reason that I can think of is to allow for user interfaces > that are able to take a mix of cp and cpv as input, and be able to > automatically distinguish between them. As mentioned earlier, the ambiguity already exists with names like foo-1a and foo-2_alpha. But I think that for an interface that accepts sloppy user input it cannot be expected that results will be accurate in all cases. > For example, emerge --unmerge currently accepts a plain cpv as input, which > technically is not a valid atom (without = operator). It also accepts a cp, > which is a valid atom, but could also correspond a valid cpv if the > "-[0-9]+$" constraint is removed. emerge --unmerge also accepts a package name without category and it doesn't really cause any problems in practice. I think it's time that we remove this arbitrary restriction on package names (and repository names, since commit 87c0398). Patch is attached.
This will break things that do validation, and it's beyond what we can put in an EAPI without GLEP 55.
(In reply to comment #17) > This will break things that do validation, and it's beyond what we can put in > an EAPI without GLEP 55. But we're loosening the spec, not tightening it. So validation tools will accept everything they've accepted so far.
And anything that uses the new rules will break anything that enforces the old ones.
It's clear that we still must not allow packages with such names, at least for some time. But I don't see how lifting the limitation in package managers could break anything.
We don't have a mechanism for saying that "this is still banned, even though the spec says it's not".
(In reply to comment #21) > We don't have a mechanism for saying that "this is still banned, even though > the spec says it's not". We have tree policies. Of course, some custom repositories may ignore them.
Let's put this on the list for EAPI 6.
(In reply to comment #20) > It's clear that we still must not allow packages with such names, at least > for some time. But I don't see how lifting the limitation in package > managers could break anything. It's pretty simple; a proper PM is going to be doing enforcement on the package names, same for any of the bits we deal in; most do regex minimally, and are anchored on that. This isn't something that can be fixed in a later EAPI; package names, like package versions, like categories, like repo_name, like profiles/updates, like <insert every other unversioned format component of our repo>, even if we were to change it in a later EAPI still would be visible/parsed by PM's of this day and age- that hold backwards compatibility thing that EAPI is designed to address. Hell, vdb, binpkgs repo, etc, take your pick- binpkgs in particular since one of the common tricks for restoring a thwacked system is grabbing one of solars tinderbox repos. Point is, PMs of this day and age cannot choke on anything we do in later EAPIs; now if you want to chuck out that requirement, here's the things you're going to have to do to keep me from being a royal pain in y'alls ass: 1) Figure out a way to do this w/out causing breakage. You manage it, it's a bit more viable unless it's completely crack adled. Tracing down exactly which versions of PMs will be broken by it, how far back, etc, is a start. May not be much, but blindly making the change w/out doing that leg work is irresponsible and missing the point of EAPI. 2) Barring a way to pull it off w/out fallout, get the council to sign off on it- this fundamentally is an aesthetic change, bit of perfectionism; if we're going to risk breakage for that, I *will* be that annoying bastard who forces it to their level. :) As for "lets put it on EAPI6's radar", radar sure, but those are my constraints for it becoming changed in EAPI6; pretty sure ciaran is on the same page on that one too.
(In reply to comment #24) > (In reply to comment #20) > > It's clear that we still must not allow packages with such names, at least > > for some time. But I don't see how lifting the limitation in package > > managers could break anything. > > It's pretty simple; a proper PM is going to be doing enforcement on the > package names, same for any of the bits we deal in; most do regex minimally, > and are anchored on that. Phrasing this better; this cannot be fixed unless the plan is to mandate all PMs support the new syntax as part of EAPI6, and then we sit on it for a couple of years. It ain't worth the breakage, especially considering the rules while annoying, don't actually affect anything (the context of a cp vs cpv is known up front; the sole reason to allow this is if we want people to be able to do emerge dev-util/diffball-1.0 rather than emerge dev-util/diffball-1.0).
Portage has changed rules for package names in EAPI="4-python", where it allows dots now. I don't see why this would be different from the problem that we're addressing here.
(In reply to comment #26) > Portage has changed rules for package names in EAPI="4-python", where it > allows dots now. I don't see why this would be different from the problem > that we're addressing here. 4-python is arfrever's (why it's in mainline, I do not know; take that up with zac)- directly committed by arfrever. Only repo that uses it is arfrevers personal python overlay.... which nothing but portage supports, since the rest of us ignored 4-python. In other words, that repo doesn't prove anything when it comes to compatibility; point paludis at it, see how that goes. So... your argument is basically, arfrever did it in his private overlay (accessed strictly by an EAPI he mangles directly), so we should be able to. You're assuming he cared about compatibility in this case; I know he didn't since this, and the same sort of issue for period and USE flags was raised and ignored. That EAPI is unofficial, and his, so he's free to screw shit up as much as he likes- even if it breaks core PMS support (it's stupid, but it's unofficial thus there is jack all we can do when portage goes and enables embrace/extend). Unofficial abuses however don't mean we can do official abuses, so try again. ;)
The only thing I'm saying is that our current rules are nonsensical. Either there is a reason why ${PN} should never look like a valid ${PF}; then we should adopt the original change suggested in comment 0. Or there is no such reason, and then we should loosen the spec. Maybe 4-python was a bad example. However, there were also other changes, like allowing an arbitrary number of digits in a version (which was limited to 8 before), or multiple _alpha, _beta type suffixes.
Right now you're just sitting on your asses and doing nothing -- let someone else find out, then we'll wait another 5 years and see that we can't do that then either. So maybe it's time to stop requiring obedience to that specific rule when reading packages from the tree? We can wait those 5 years to remove the rule, fine. But do something to make it possible in those 5 years instead of talking about it.
That would be the route outlined in comment 20 and following: Ensure that package managers can handle any package names without the limitation, but still require the current rule by tree policy (and enforce it with repoman).
In fact, Portage is more restrictive than the spec and forbids package names like: foo-1a foo-1a_alpha foo-1a-r1 With portage-2.1.11.20: $ repoman manifest Invalid ebuild name: /usr/portage/local/ulm/app-misc/foo-1a/foo-1a-1.ebuild >>> Creating Manifest for /usr/portage/local/ulm/app-misc/foo-1a !!! Invalid package name: 'app-misc/foo-1a-1' Unable to generate manifest. $ ebuild foo-1a-1.ebuild unpack # or any other ebuild command !!! app-misc/foo-1a-1 does not follow correct package syntax.
(In reply to comment #29) > Right now you're just sitting on your asses and doing nothing -- let someone > else find out, then we'll wait another 5 years and see that we can't do that > then either. > > So maybe it's time to stop requiring obedience to that specific rule when > reading packages from the tree? We can wait those 5 years to remove the > rule, fine. But do something to make it possible in those 5 years instead of > talking about it. I told you what needs to be done to deal w/ it properly. Now either you can persist in making noise and not doing the @!#*ing legwork w/out any real contribution /beyond noise or a stupid library to copyfiles/, or you can start on the legwork and stop being noisy I'm sorry that if you want to change a rule like this, you have to do analysis; further, I'm sorry that we're (or at least me) not willing to drop what we're doing and go do what you want at that moment because we view the cost/gain ratio to not be worth it in comparison to other things we could be doing; welcome to FOSS, scratch your own fucking itch rather than being a noisy ass. (In reply to comment #30) > That would be the route outlined in comment 20 and following: Ensure that > package managers can handle any package names without the limitation, but > still require the current rule by tree policy (and enforce it with repoman). Could, but this still is off. See my earlier comments; scope exactly what this will bork; might not actually be anything- the easiest scenario, but someone needs to spend time doing that analysis instead of mouthing off like a 5 year old (see comment #29). If it's not a straight shot, then if you want to do this, trying to block it from being used in gentoo-x86 misses the issue of VDB and binpkg repos; it basically means any old version's going to get busted regardless, meaning that repoman bit is pointless work. As stated, require the parser rules in EAPI blah, then require they not be used for <insert time period.> Or a saner route that also fixes some other issues, fold layout.conf awareness in, and add versioning via that- specifically minimum EAPI version require to look at that repo. Sucks (turns off the whole repo if the PM doesn't support it), but it should be a once in a blue moon scenario; via layout.conf awareness and versioning we can pull in other things, profile format for example, thin manifests, etc. Hadn't thought of the layout.conf route till now; that's basically the sanest approach here imo.
(In reply to comment #27) > (In reply to comment #26) > > Portage has changed rules for package names in EAPI="4-python", where it > > allows dots now. I don't see why this would be different from the problem > > that we're addressing here. > > 4-python ... > Only repo that uses it is arfrevers personal python overlay For clarification, that overlay was renamed to Progress Overlay over 1 year ago (some months after creation of EAPI="4-python"). Funtoo also uses EAPI="4-python" in some packages. The new EAPI, which contains features of EAPI="4-python" and EAPI="5", is called EAPI="5-progress". > So... your argument is basically, arfrever did it in his private overlay > (accessed strictly by an EAPI he mangles directly), so we should be able to. > You're assuming he cared about compatibility in this case I cared about compatibility. Behavior of previous version of Portage had been checked and there were no problems preventing upgrade of Portage. There were only non-fatal errors about malformed update entries in a ${repository_path}/profiles/updates/... file. After upgrade of Portage, update entries were processed correctly.
(In reply to comment #33) > (In reply to comment #27) > > So... your argument is basically, arfrever did it in his private overlay > > (accessed strictly by an EAPI he mangles directly), so we should be able to. > > You're assuming he cared about compatibility in this case > > I cared about compatibility. Not trying to pick a fight, but USE=python2.7 is an example, albeit not directly this one. That's more what I'm riffing there. > Behavior of previous version of Portage had > been checked and there were no problems preventing upgrade of Portage. There > were only non-fatal errors about malformed update entries in a > ${repository_path}/profiles/updates/... file. After upgrade of Portage, > update entries were processed correctly. Suggest identifying the versions involved; also, did you validate existing eapis getting exposed to those chars, or just the new 4-python pathways, and the remaining unversioned pathways (updates for example) ? Will state this isn't the case for the others.
(In reply to comment #34) > USE=python2.7 is an example, albeit not directly this one. Dots in USE flags did not cause more problems than dots in package names. > > Behavior of previous version of Portage had > > been checked and there were no problems preventing upgrade of Portage. There > > were only non-fatal errors about malformed update entries in a > > ${repository_path}/profiles/updates/... file. After upgrade of Portage, > > update entries were processed correctly. > > Suggest identifying the versions involved; also, did you validate existing > eapis getting exposed to those chars, or just the new 4-python pathways Portage does not allow to create Manifest for ebuilds with unsupported PN. When performing other operations, Portage silently ignores package directories with unsupported PN. > and the remaining unversioned pathways (updates for example) ? I described it in comment #33.
Created attachment 324680 [details, diff] Alternative patch that tightens the spec I don't really care if we loosen or tighten the spec. We only shouldn't leave it in the current half-baked state. Portage being stricter than PMS might be a hint in which direction we should go.
(In reply to comment #36) > Created attachment 324680 [details, diff] [details, diff] > Alternative patch that tightens the spec > > I don't really care if we loosen or tighten the spec. We only shouldn't > leave it in the current half-baked state. > > Portage being stricter than PMS might be a hint in which direction we should > go. Please address how existing package managers... all of them... will not go boom if this is changed and they become exposed to those new names. Hate to be a broken record here, but it's getting rediculous no one addresses the breakage, just keeps trying to move forward with an aesthetic wart (there's many, this really is the best use of our time?)
(In reply to comment #36) > Created attachment 324680 [details, diff] [details, diff] > Alternative patch that tightens the spec > > I don't really care if we loosen or tighten the spec. We only shouldn't > leave it in the current half-baked state. > > Portage being stricter than PMS might be a hint in which direction we should > go. To be honest... I don't think we even should tighten it. Beyond the fact it might lead to borkage (admittedly, less likely, and contained to portage)... which rules suck more of the following: 1) Package names must not end in -\d$ 2) Package names must not end in a version. See this long winded/fun chunk of regex that discerns what a version is. Which do you think is more likely to be screwed up by PM authors and ebuild authors alike? As said; obnoxious wart, I strongly think we should ignore it and focus on higher gain propositions.
(In reply to comment #36) > Created attachment 324680 [details, diff] [details, diff] > Alternative patch that tightens the spec By today's council decision, the spec should be tightened. So I'll apply this patch next weekend.
