Created attachment 411800 [details, diff] 0001-Document-necessity-to-revbump-on-RDEPEND-changes.patch Last I heard, the portage team voted out dynamic dependencies, although they are still in place. This will probably change in the near future, so we should make it clear that people have to revbump their ebuilds if they change runtime dependencies, so users get a correct VDB update.
Regarding dynamic dependency removal, from the Council 20140826 summary: > Dynamic dependencies in Portage > =============================== > During discussion, is was remarked that some changes, e.g. to > dependencies in eclasses, could require mass rebuilds of packages. > > Vote: > - "The council asks the Portage team to first outline their long-term > plan regarding removal or replacement of dynamic dependencies, > before they remove this feature. In particular, tree policies and > the handling of eclasses and virtuals need to be clarified." > Accepted unanimously. > > Note added in proof: The Portage team does not intend to remove > dynamic dependencies, but only change their default to "off". Some time ago I enquired about this, but did not receive a definitive answer regarding if this plan was ever completed.
Regarding the patch, I feel it warrants discussion on the mailing list because it bans a behaviour that seems reasonably common.
(In reply to Michael Palimaka (kensington) from comment #1) Well, this is actually irrelevant for ebuild writing policies. Ebuilds are written against PMS and PMS doesn't include dynamic dependencies whatsoever which has been confirmed by the PMS team. This is simply an explicit policy which is already implicit in PMS. The fact that the portage team has decided to remove dynamic-dependencies support just makes it more pressing to document this, but it doesn't depend on it.
(In reply to Michael Palimaka (kensington) from comment #2) It has already been discussed vividly on the mailing list. If you want a different decision, you need to write a PMS patch.
(In reply to Julian Ospald (hasufell) from comment #4) > (In reply to Michael Palimaka (kensington) from comment #2) > > It has already been discussed vividly on the mailing list. If you want a > different decision, you need to write a PMS patch. Could you please provide a link to the discussion where this change was agreed upon?
(In reply to Michael Palimaka (kensington) from comment #5) > (In reply to Julian Ospald (hasufell) from comment #4) > > (In reply to Michael Palimaka (kensington) from comment #2) > > > > It has already been discussed vividly on the mailing list. If you want a > > different decision, you need to write a PMS patch. > > Could you please provide a link to the discussion where this change was > agreed upon? which change do you mean?
(In reply to Julian Ospald (hasufell) from comment #6) > (In reply to Michael Palimaka (kensington) from comment #5) > > (In reply to Julian Ospald (hasufell) from comment #4) > > > (In reply to Michael Palimaka (kensington) from comment #2) > > > > > > It has already been discussed vividly on the mailing list. If you want a > > > different decision, you need to write a PMS patch. > > > > Could you please provide a link to the discussion where this change was > > agreed upon? > > which change do you mean? The one you're proposing.
(In reply to Michael Palimaka (kensington) from comment #7) Please see https://devmanual.gentoo.org/appendices/contributing/index.html
> (In reply to Michael Palimaka (kensington) from comment #7) > > Please see https://devmanual.gentoo.org/appendices/contributing/index.html That's not a link to a mailing list discussion.
(In reply to Michael Palimaka (kensington) from comment #9) > > (In reply to Michael Palimaka (kensington) from comment #7) > > > > Please see https://devmanual.gentoo.org/appendices/contributing/index.html > > That's not a link to a mailing list discussion. That's a link on how to contribute to the devmanual. There is no discussion on the ML necessary unless the editors think so. Comment #4 was referring to the dynamic dependencies discussion on dev-ML, not the devmanual change. Please don't spam this bug report.
(In reply to Julian Ospald (hasufell) from comment #10) > That's a link on how to contribute to the devmanual. There is no discussion > on the ML necessary unless the editors think so. There is a difference between a devmanual change and a policy change. Discussion off the former is indeed at the discretion of the editors, the latter however is not. > Comment #4 was referring to the dynamic dependencies discussion on dev-ML, > not the devmanual change. Thanks for the clarification.
(In reply to Michael Palimaka (kensington) from comment #11) > (In reply to Julian Ospald (hasufell) from comment #10) > > That's a link on how to contribute to the devmanual. There is no discussion > > on the ML necessary unless the editors think so. > > There is a difference between a devmanual change and a policy change. > Discussion off the former is indeed at the discretion of the editors, the > latter however is not. > Exactly and there is no policy change, unless you ignore PMS. The meeting log of the portage team is in $URL. Dynamic deps have been voted off. The only thing they don't know is how to introduce the change. This devmanual patch will help with the transition.
If there is no policy change, then why is dependency changing without revbump a common behaviour? Would a mailing list discussion not help raise awareness of the issue?
(In reply to Michael Palimaka (kensington) from comment #13) > If there is no policy change, then why is dependency changing without > revbump a common behaviour? Would a mailing list discussion not help raise > awareness of the issue? Yes, but that's not within the scope of this bug report. That's what the portage team wanted to do (see the meeting log).
(In reply to Michael Palimaka (kensington) from comment #13) > If there is no policy change, then why is dependency changing without > revbump a common behaviour? Would a mailing list discussion not help raise > awareness of the issue? It's common because it happens to work in portage, where dynamic dependencies (now considered a misfeature and planned for removal) are enabled by default. So most developers will never notice anything wrong.
Created attachment 411926 [details, diff] 0001-general-concepts-ebuild-revisions-rewrite.patch Here's another (much more strongly-worded) rewrite. I think this wording is less likely to lead to mistakes: the existing guidelines only make sense if you already know the rules. Lest I be accused of bikeshedding, I'm also fine with hasufell's version and I think it's important to make progress on this regardless of the details.
(In reply to Michael Orlitzky from comment #16) > Created attachment 411926 [details, diff] [details, diff] > 0001-general-concepts-ebuild-revisions-rewrite.patch This it a very intrusive change which even drops some aspects of existing policy. For example, that "the new ebuild should be based on the previous revision to ensure that fixes aren't dropped accidentally" isn't mentioned any more. Also the wording could be more precise in some places. For example: "A revision number is an -rX suffix" doesn't make sense. (In reply to Julian Ospald (hasufell) from comment #0) > Created attachment 411800 [details, diff] [details, diff] > 0001-Document-necessity-to-revbump-on-RDEPEND-changes.patch Wording is fine with me, with one spelling change ("users" -> "user's").
(In reply to Julian Ospald (hasufell) from comment #14) > (In reply to Michael Palimaka (kensington) from comment #13) > > If there is no policy change, then why is dependency changing without > > revbump a common behaviour? Would a mailing list discussion not help raise > > awareness of the issue? > > Yes, but that's not within the scope of this bug report. That's what the > portage team wanted to do (see the meeting log). Especially new developers might not know what the VDB is. It might help to add an explanation or a reference.
(In reply to Manuel Rüger from comment #18) > (In reply to Julian Ospald (hasufell) from comment #14) > > (In reply to Michael Palimaka (kensington) from comment #13) > > > If there is no policy change, then why is dependency changing without > > > revbump a common behaviour? Would a mailing list discussion not help raise > > > awareness of the issue? > > > > Yes, but that's not within the scope of this bug report. That's what the > > portage team wanted to do (see the meeting log). > > Especially new developers might not know what the VDB is. It might help to > add an explanation or a reference. Sorry, misquoted. Should reference your patch.
(In reply to Ulrich Müller from comment #17) > Wording is fine with me, with one spelling change ("users" -> "user's"). +1.
So, what's the plan to handle tree-wide changes like the introduction of virtual/pkgconfig, or eclass changes - revbump the entire tree?
Created attachment 411950 [details, diff] 0001-Document-necessity-to-revbump-on-RDEPEND-changes.patch I actually like the rewrite patch (in case it fixes the brought up issues), because it assumes a default of revbump and then lists viable exceptions which I think it is a bit easier to follow for newcomers. Anyway, fixed the spelling in my patch and added a very short comment for VDB.
(In reply to Michael Palimaka (kensington) from comment #1) > Regarding dynamic dependency removal, from the Council 20140826 summary: > > > Dynamic dependencies in Portage > > =============================== > > During discussion, is was remarked that some changes, e.g. to > > dependencies in eclasses, could require mass rebuilds of packages. > > > > Vote: > > - "The council asks the Portage team to first outline their long-term > > plan regarding removal or replacement of dynamic dependencies, > > before they remove this feature. In particular, tree policies and > > the handling of eclasses and virtuals need to be clarified." > > Accepted unanimously. > > > > Note added in proof: The Portage team does not intend to remove > > dynamic dependencies, but only change their default to "off". > > Some time ago I enquired about this, but did not receive a definitive answer > regarding if this plan was ever completed. CCing council to see if something changed since this resolution, this looks to me like one more try to force all people to make tons of revbumps and drop all the complaints against this change that were discussed until death in mailing list behind the rug :| As far as I remember, nothing has changed about what to do when we need to do with "tree policies and the handling of eclasses and virtuals need to be clarified." This will likely end up with tons of rebuilds or with people refusing to fix the dependencies until more changes are needed for a bump
(In reply to Michael Palimaka (kensington) from comment #21) > So, what's the plan to handle tree-wide changes like the introduction of > virtual/pkgconfig You revbump. That's like one additional line for your automation script/Makefile: "git mv". > or eclass changes - revbump the entire tree? That's a bit trickier, but I'd go so far to say that dependencies have almost no place in eclasses and that is misuse. To make that more sane, we could allow per-package eclasses (in the package directory), similar to what exherbo already does and lift that restriction for those.
(In reply to Pacho Ramos from comment #23) > CCing council to see if something changed since this resolution, this looks > to me like one more try to force all people to make tons of revbumps and > drop all the complaints against this change that were discussed until death > in mailing list behind the rug :| > You don't seem to understand that it was already voted out and that the council is merely waiting for a strategy to introduce this change. Fixing documentation is one of these steps.
(In reply to Julian Ospald (hasufell) from comment #24) > dependencies have > almost no place in eclasses "almost", because stuff like autotools.eclass just affects build-time, same for cmake-utils.eclass and so on... for those, there's no revbump necessary
(In reply to Julian Ospald (hasufell) from comment #24) > (In reply to Michael Palimaka (kensington) from comment #21) > > So, what's the plan to handle tree-wide changes like the introduction of > > virtual/pkgconfig > > You revbump. That's like one additional line for your automation > script/Makefile: "git mv". Using pkgconfig again as the example, that affects almost 4000 packages. Do we really want to inflict that level of pain on our users?
(In reply to Michael Palimaka (kensington) from comment #27) > (In reply to Julian Ospald (hasufell) from comment #24) > > (In reply to Michael Palimaka (kensington) from comment #21) > > > So, what's the plan to handle tree-wide changes like the introduction of > > > virtual/pkgconfig > > > > You revbump. That's like one additional line for your automation > > script/Makefile: "git mv". > > Using pkgconfig again as the example, that affects almost 4000 packages. Do > we really want to inflict that level of pain on our users? pkgconfig is a buildtime dependency in 99% of the cases, so there is no revbump necessary.
(In reply to Julian Ospald (hasufell) from comment #28) > (In reply to Michael Palimaka (kensington) from comment #27) > > (In reply to Julian Ospald (hasufell) from comment #24) > > > (In reply to Michael Palimaka (kensington) from comment #21) > > > > So, what's the plan to handle tree-wide changes like the introduction of > > > > virtual/pkgconfig > > > > > > You revbump. That's like one additional line for your automation > > > script/Makefile: "git mv". > > > > Using pkgconfig again as the example, that affects almost 4000 packages. Do > > we really want to inflict that level of pain on our users? > > pkgconfig is a buildtime dependency in 99% of the cases, so there is no > revbump necessary. As I said, it is just an example.
So I wonder what we should do if we ever decide to raise the minimum Perl version in the perl-module.eclass dependency string. Roughly 1000 ebuilds.
(In reply to Andreas K. Hüttel from comment #30) > So I wonder what we should do if we ever decide to raise the minimum Perl > version in the perl-module.eclass dependency string. Roughly 1000 ebuilds. The same applies to kde eclasses, which regularly raise deps on kde-plasma and kde-frameworks.
(In reply to Andreas K. Hüttel from comment #30) > So I wonder what we should do if we ever decide to raise the minimum Perl > version in the perl-module.eclass dependency string. Roughly 1000 ebuilds. IMO, it shouldn't even be in the the eclass. Look at haskell packages... they mostly (not consistently though) do it right, so they mostly don't generate any depstrings through eclass functions or variables, but they are generated when the ebuild is created (e.g. via the tool hackport). From that on, all deps are static, as it should be. Fixing such deps can be automated (sed, Makefile, haskell tool, whatever) and is easy to review in git. Laziness is a good thing, but whenever laziness means giving up correctness, then we should not do it. But I feel this warrants a new devmanual patch/bug report, since it's more about eclass writing, not ebuild writing. Even if we only fix this part of the devmanual yet, it is still a huge improvement and will reduce the number of breakages.
(In reply to Julian Ospald (hasufell) from comment #25) > (In reply to Pacho Ramos from comment #23) > > CCing council to see if something changed since this resolution, this looks > > to me like one more try to force all people to make tons of revbumps and > > drop all the complaints against this change that were discussed until death > > in mailing list behind the rug :| > > > > You don't seem to understand that it was already voted out and that the > council is merely waiting for a strategy to introduce this change. There was never a vote. A couple of people who continually disparage Gentoo "decided" that dyndeps would be turned off without any public consultation or consideration of the real-world consequences. > Fixing documentation is one of these steps. How can we fix the documentation if there is no migration strategy? It's been over a year and still nothing.
(In reply to Michael Palimaka (kensington) from comment #33) > > There was never a vote. There was a vote, please refrain from spamming this bug report again. Useful comments are appreciated. I'm really close to involving comrel here, because you are creating more noise than useful comments. > A couple of people who continually disparage Gentoo > "decided" You should stop this offensive behaviour right here. You are talking about the portage team.
(In reply to Julian Ospald (hasufell) from comment #34) > (In reply to Michael Palimaka (kensington) from comment #33) > > > > There was never a vote. > > There was a vote, please refrain from spamming this bug report again. Useful > comments are appreciated. I'm really close to involving comrel here, because > you are creating more noise than useful comments. Could you please link to where this vote took place? I checked the Portage meeting logs but couldn't find it. > > > A couple of people who continually disparage Gentoo > > "decided" > > You should stop this offensive behaviour right here. You are talking about > the portage team. I am not talking about the Portage team.
(In reply to Michael Palimaka (kensington) from comment #35) > > Could you please link to where this vote took place? I checked the Portage > meeting logs but couldn't find it. > It is in the URL field of this bug report. > > I am not talking about the Portage team. Then you are OT and trolling. Please stop.
(In reply to Julian Ospald (hasufell) from comment #36) > (In reply to Michael Palimaka (kensington) from comment #35) > > > > Could you please link to where this vote took place? I checked the Portage > > meeting logs but couldn't find it. > > > > It is in the URL field of this bug report. I read it and didn't see a vote. I'm genuinely asking, because obviously I'm missing something.
@comrel ...can you please help here? kensington is continuously torpedoing this bug report and makes it almost impossible to follow the interesting parts of this bug report/discussion
(In reply to Julian Ospald (hasufell) from comment #25) > (In reply to Pacho Ramos from comment #23) > > CCing council to see if something changed since this resolution, this looks > > to me like one more try to force all people to make tons of revbumps and > > drop all the complaints against this change that were discussed until death > > in mailing list behind the rug :| > > > > You don't seem to understand that it was already voted out and that the > council is merely waiting for a strategy to introduce this change. > > Fixing documentation is one of these steps. I am referring to the "particular" things that are still pending as explained at: https://bugs.gentoo.org/show_bug.cgi?id=516612#c38 This aim to "fix documentation" before handling that looks to me like trying to force that path of needing to always rebuild every time we need to fix a dependency or make a pkg move or use a new virtual. The policy with that is not clear at all and all threads in mailing lists ended with lots of flames and no final decisions.
And, of course, I would like to raise this topic for the next Council meeting to try to clarify the path we need to follow in this topic. Changing the policy now in this way of "fixing devmanual" will only end with some developers not following it (some because they are not even aware of the change, others because they strongly disagree with needing to revbump for this changes), others trying to enforce it via QA or even comrel asking that teams to "punish" the people not following the new devmanual wording.
(In reply to Pacho Ramos from comment #39) The portage team has decided (see the meeting logs) that they want to deprecate dynamic deps. Numerous arguments were brought up on the ML why they are dangerous, cause breakage and don't work consistently or not at all, because they are just a portage quirk (participants of those arguments include PMS, portage and paludis authors who mostly shared a similar opinion). The council just said that we need a proper strategy before _actually_ switching it off. How do you think this can work if we don't fix documentation in the first place? Chicken-and-egg problem? And now guess why no one actively tried to come up with such a strategy... because they knew that this would cause such reactions, so they mostly just procrastinated.
(In reply to Pacho Ramos from comment #40) > And, of course, I would like to raise this topic for the next Council > meeting to try to clarify the path we need to follow in this topic. They will probably decide the same thing... which will drop us back to this very bug report again... and then people will object and require a new council decision... which will drop us back to this bug report err...
(In reply to Julian Ospald (hasufell) from comment #41) > (In reply to Pacho Ramos from comment #39) I remember this topic and I remember this occurred when Zac needed to leave active portage development for a while. I remember I asked him about how could this be handled in a different way that wouldn't involve the useless rebuilds and that lead to him commenting in https://bugs.gentoo.org/show_bug.cgi?id=516612#c38 to clarify still some work is needed before even changing the default (don't think before completely removing the feature) In all that threads you comment you also can see the tons of comments against that change and all the downsides that they will come because of this change... but you simply ignore all of them instead and you keep trying to push the change in multiple ways trying to avoid the conflict... but that will only lead in postponing the conflict to the first time others will need to enforce the updated policy. Then, some people will expect others (QA/Comrel) to impose that changes to other developers.
(In reply to Ulrich Müller from comment #17) > > This it a very intrusive change which even drops some aspects of existing > policy. For example, that "the new ebuild should be based on the previous > revision to ensure that fixes aren't dropped accidentally" isn't mentioned > any more. That was on purpose, although people may disagree. The suggestion is either obvious or wrong but in both cases kind of a waste of a sentence. If you're going to revbump to add a new fix, then obviously you will base it on the previous revision and not some other random ebuild. But what if you're fixing a problem introduced in the last revision? Then the advice is wrong. In either case, I thought the suggestion (as it was written) seemed unhelpful. > Also the wording could be more precise in some places. For example: "A > revision number is an -rX suffix" doesn't make sense. I don't like that either but I didn't know of a better way to fix it. Steal wording from the PMS?
(In reply to Andreas K. Hüttel from comment #30) > So I wonder what we should do if we ever decide to raise the minimum Perl > version in the perl-module.eclass dependency string. Roughly 1000 ebuilds. If someone has 1,000 perl ebuilds installed with wrong runtime dependencies, then yeah, it sucks, but they need to re-emerge those packages to fix the stored dependencies. The only reason to use a distribution is because you're willing to sacrifice a little bit of time to ensure that your dependencies are correct and consistent. And of course nobody will have all 1,000 of those ebuilds installed. And even then, to emerge 1,000 perl ebuilds takes like 5 minutes =) I also agree that dependencies in eclasses are a mistake, but fires should be started one-at-a-time. This change should only affect changing dependencies within an ebuild, at which point the correct course of action is clear according to the PMS.
Just to be clear: 1. there is no policy of relying on runtime dependencies. There never were. It was just developers failing to do stuff properly, and then Portage team inventing workaround for it. Workaround which never worked properly and is causing more harm than good. 2. Not every dependency change needs to apply immediately. Most of eclass changes belong to this category, I think (except for stuff like virtuals switching -- and yes, I know perl team abuses virtuals horribly). If you raise required version of X in the eclass, you usually want to enforce this requirement in ebuilds updated (or built, more realistically) after the eclass change, not before. 3. Revbump on dependency change is not an *unneeded* rebuild. It is a rebuild needed to update metadata and enforce dependency graph integrity. So, I would really appreciate if people stopped arguing about 'unneeded rebuilds' while they're actually causing a lot more unneeded rebuilds. If developers revbumped properly when updating deps, the number of rebuilds would be limited. If developers don't do that, dependency graph becomes randomly broken, and PM can't proceed updating. Now people can do two things. They can either spend hours of time figuring out what became broken and rebuilding random packages until PM can proceed again, or they use stuff like --changed-dep to rebuild everything that ever changed any dependency. The latter implies *a lot of needless rebuilds*, and I'm forced to do them every few weeks.
The URL already contains the meeting log where the portage team decided to disable dynamic deps. But for convenience, here's the meeting summary: https://archives.gentoo.org/gentoo-portage-dev/message/af7b3d3db857f2c9c1ff4d5cfcf827ee and the relevant decision: Dynamic dependencies --------------------- Dynamic dependencies will be turned off. Michał and myself (and I encourage everyone else that has the time to join in) will rebuild world without dynamic dependencies locally on our own work computers, and try to live without them for a while as to stumble upon the potential problems. We will continuously document how to fix whatever mess arises. We might also write some tools to assist migration. After a while of testing, an email will be sent to the developer's mailing list to let them prepare to not being able to rely on dynamic dependencies tricks. Then, pending some reasonable amount of time (likely at least 30 days), we roll out a release with dynamic dependencies disabled. This release will be accompanied by a NEWS item for users and developers alike.
(In reply to Michael Orlitzky from comment #44) > (In reply to Ulrich Müller from comment #17) > > This it a very intrusive change which even drops some aspects of existing > > policy. For example, that "the new ebuild should be based on the previous > > revision to ensure that fixes aren't dropped accidentally" isn't mentioned > > any more. > > That was on purpose, although people may disagree. The suggestion is either > obvious or wrong but in both cases kind of a waste of a sentence. If you're > going to revbump to add a new fix, then obviously you will base it on the > previous revision and not some other random ebuild. But what if you're > fixing a problem introduced in the last revision? Then the advice is wrong. > > In either case, I thought the suggestion (as it was written) seemed > unhelpful. The part containing that sentence was recently migrated from the old devrel handbook and is a policy which is in place since 2004. And I think it's not entirely obvious, as I've already seen devs copying a new revision from their overlay or from a bugzie attachment, thereby accidentally dropping a previous fix. In any case, this is a change separate from the topic of this bug, so it should be in a separate patch (and bug). > > Also the wording could be more precise in some places. For example: "A > > revision number is an -rX suffix" doesn't make sense. > > I don't like that either but I didn't know of a better way to fix it. Steal > wording from the PMS? "Ebuilds may have a revision suffix of the form <c>-rX</c>, where the Gentoo <e>revision number</e> <c>X</c> is a natural number <d/> [...]"
(In reply to Michał Górny from comment #46) [...] > So, I would really appreciate if people stopped arguing about 'unneeded > rebuilds' while they're actually causing a lot more unneeded rebuilds. We refer to them like "unneeded rebuilds" because we don't understand why for regenerating VDB with the values we want to put in the portage tree we need to recompile all the package... even if it will get rebuilt against the same package with a different name. It's similar as the issue with all USE flags forcing to recompile all the packages and, then, we being unable to set runtime only USE flags :S
(In reply to Pacho Ramos from comment #49) > (In reply to Michał Górny from comment #46) > [...] > > So, I would really appreciate if people stopped arguing about 'unneeded > > rebuilds' while they're actually causing a lot more unneeded rebuilds. > > We refer to them like "unneeded rebuilds" because we don't understand why > for regenerating VDB with the values we want to put in the portage tree we > need to recompile all the package... even if it will get rebuilt against the > same package with a different name. It's similar as the issue with all USE > flags forcing to recompile all the packages and, then, we being unable to > set runtime only USE flags :S Yes, we all know that this is not optimal. The point is... no one really knows how to implement dynamic deps, or such non-trivial VDB updates without rebuilding, properly... while not breaking a hundred edge cases. If you know it, please provide patches. And not just PM patches, but also PMS patches, since an assumption like "dynamic deps" really requires a PMS patch. If you want to know more, ask PM developers or re-read the discussion on the ML where a few people really tried hard to explain this. It's not an easy topic indeed, but I'd kindly ask everyone not interested in reviewing the attached patches to move OT discussions elsewhere.
(In reply to Michael Orlitzky from comment #47) > After a while of testing, an email will be sent to the developer's > mailing list to let them prepare to not being able to rely on dynamic > dependencies tricks. Why don't we start with the mail to -dev outlining the plan before we start implementing the change. I have no objection to people preparing patches to the devmanual. That is planning. For a change as contentious as this one I don't think we should start changing the devmanual before there has been a real outline of the plan on the lists. I don't really have a problem with moving away from dynamic dependencies. However, it was the intent of the council (IMO) that this be discussed on the lists before it happens, and updating the devmanual counts as "making it happen" (again, IMO). If nothing else we can use the list discussion to demonstrate the sky isn't falling as those who have been piloting the change share their experiences. No doubt there will not be unanimous consensus behind the change, and we don't need that to proceed. However, it isn't surprising that people want to lodge their objections before somebody starts patching the devmanual, since that does effectively constitute policy. So, I suggest we take discussion to the lists as is normally the case, and hold off on actually implementing changes for the moment. By all means use the bug for things that bugs are good at, like improving the patch (constructively - if we implement the change then there is no reason to not implement it well, and if we don't implement the change it won't be because we were impressed with bugzilla vandalism). FWIW, after coming back from vacation I really enjoyed digging through emerge output to figure out which package(s) needed to be --oneshot-ted just to fix the dependency resolver to complete a modest 100 package update. There may or may not be a sane way to handle dynamic deps, but it is clear that the way we have today isn't it.
If someone wants to start another bikeshed where nobody agrees and nobody is willing to concede that maybe the package manager authors know more than them just this once, go ahead. Maybe we can trick Michał into doing it again. I remain unconvinced at the efficacy of this, but I would sincerely applaud the initiative.
(In reply to Richard Freeman from comment #51) > (In reply to Michael Orlitzky from comment #47) > > After a while of testing, an email will be sent to the developer's > > mailing list to let them prepare to not being able to rely on dynamic > > dependencies tricks. > > Why don't we start with the mail to -dev outlining the plan before we start > implementing the change. > Plan: tell people to stop relying on dynamic dependencies (as in: this bug). Then (at some point) when people actually follow it... turning it off will go almost unnoticed. Not sure what another bikeshed will improve. We are running in circles here.
> > Vote: > > - "The council asks the Portage team to first outline their long-term > > plan regarding removal or replacement of dynamic dependencies, > > before they remove this feature. In particular, tree policies and > > the handling of eclasses and virtuals need to be clarified." > > Accepted unanimously. I've yet to see anything presented regarding the handling of eclasses and virtuals. There hasn't even been a mention on -dev of the intent to remove dynamic dependencies. You're welcome to disagree with the council decision. You're not welcome to ignore it. That's just how we do things. If you'd like to ask the council to change its mind you're welcome to do so. However, I wouldn't really expect much to change. There was a request to clarify policy around virtuals and eclasses, and it doesn't seem unreasonable to do that before we start trying to implement something. This is a big change. The purpose isn't so that we can bikeshed it to death. The purpose is so that everybody can see that there is a reasonable plan before we start implementing it. You're more than welcome to publish the plan, ignore all the replies, and ask the council to give it a thumbs up or down. Then you can pretend that there was no bikeshedding if it makes you happier. Of course, if somebody comes up with a sound objection and you don't address it, then that is on you. But, if your plan is airtight I don't think the council is going to judge the proposal mainly on the basis of counting votes, which seems to be the fear.
(In reply to Richard Freeman from comment #54) > > > Vote: > > > - "The council asks the Portage team to first outline their long-term > > > plan regarding removal or replacement of dynamic dependencies, > > > before they remove this feature. In particular, tree policies and > > > the handling of eclasses and virtuals need to be clarified." > > > Accepted unanimously. > > I've yet to see anything presented regarding the handling of eclasses and > virtuals. There hasn't even been a mention on -dev of the intent to remove > dynamic dependencies. > > You're welcome to disagree with the council decision. You're not welcome to > ignore it. That's just how we do things. > I'm not sure if you have really read all of this bug report. But I can see that it's difficult given the spam that was spread here. Your questions were already answered: * a lot of eclass dependencies are build-time only (e.g. autotools, cmake), so for them there is nothing to do * not all eclass dependency changes need to apply immediately, see #c46 ...if unsure, people should consult dev ML * discourage (but not disallow) the use of runtime-dependencies in eclasses... it should be used sparingly * for virtuals that affect runtime dependencies, you generally have to do revbumps... automate it with scripts and review it in git (which is fairly easy now)... that is basically exactly what I will do once I import libressl into the tree, I will sequentially revbump and test ebuilds * this is not a 100% solution that covers every corner case, but it will cause less breakage than we have today However, not all of that belongs into this bug report. The devmanual is not about listing every possible corner case and eclass writing does not belong into this patch at all. A new bug will have to be created. So, any questions left? Can we move forward now? Or will this be another git migration story?
First, Julian, I applaud your effort to get this needed change to move forward. Second, I applaud Alexander for being so polite in his responses. But, this subject was such a hot potato with one developer in particular (not a portage team member) that we could not even discuss it in our #gentoo-portage dev channel without interference. (I even contemplated banning him from the channel at the time, but that likely would not have helped) So, before we could even formalize a complete solution and plan. The nay-sayer and fear monger-er went shouting to the council "the sky is falling". With the council then interceding telling us we had to do what we were attempting to do in the first place. So, to answer comment #1: The whole thing pretty much KILLED any enthusiasm ANY of the team had of working on portage. We all have better things to do with our life... Have you not noticed, there has been little new work done on portage since? So, if the council can rubber stamp a plan to proceed, the portage team is interested in implementing it, but I doubt any of us will participate until then.
(In reply to Julian Ospald (hasufell) from comment #55) > I'm not sure if you have really read all of this bug report. But I can see > that it's difficult given the spam that was spread here. And that is why a bug is not the right place to have this discussion. > Your questions were > already answered: > And I don't have a big problem with your answers in general, but they still deserve some discussion. > * this is not a 100% solution that covers every corner case, but it will > cause less breakage than we have today > I'm willing to buy into that, but it is still fair to everybody to have the discussion out on the list. > > So, any questions left? Can we move forward now? IMO we cannot move forward now. The council asked for a plan. They didn't ask somebody to file a bug and start making changes, maybe with some elements of a plan buried in the comments but not discussed because discussion would be out of scope. This is a big change. You're basically proposing committing one element of the change directly to production and saying that the fact that other elements of the change are still hazy doesn't matter. The way you normally do big changes is to make sure all the elements of the plan are understood and you have some kind of release plan, and THEN you start changing production. > Or will this be another git migration story? Main limiting factor on git was infra availability to implement all the changes. There really wasn't a lot of decision-making that held things up. The last council decision on the topic was almost a year ago. For the most part when people raised roadblocks the council knocked them down and the git team was allowed to move forward almost exactly as they preferred. I don't really get the complaints about bikeshedding and delay. Sure, it takes a few weeks to go through the council, but it has been a year since the last time this came up. If you put a proposal in front of the council and it takes three months to get a concrete list of decisions/changes/requirements from them, please do complain about bikeshedding. We can't stop people from replying to your proposals. However, in general we don't make decisions by printing out the emails for/against and weighing them. My sense is that the vast majority of the council already agrees with the direction - we just want to be careful to coordinate how we get there. I think you've already made the case that dynamic deps as they exist today don't work. Just cross your t's and we can be through this.
(In reply to Richard Freeman from comment #57) > This is a big change. No. This is what already lies in PMS.
(In reply to Brian Dolbec from comment #56) > > But, this subject was such a hot potato with one developer in particular > (not a portage team member) that we could not even discuss it in our > #gentoo-portage dev channel without interference. (I even contemplated > banning him from the channel at the time, but that likely would not have > helped) So, before we could even formalize a complete solution and plan. > The nay-sayer and fear monger-er went shouting to the council "the sky is > falling". With the council then interceding telling us we had to do what we > were attempting to do in the first place. I get it. I don't think anybody in the council thought the portage team didn't intend to do this in a sensible way, and that is why they simply asked for them to publish a plan first. It was intended to be a reasonable way to address the issue without trying to take the decisions out of their hands. > > So, to answer comment #1: > The whole thing pretty much KILLED any enthusiasm ANY of the team had of > working on portage. We all have better things to do with our life... Again, I get it. Sometimes you need to take a step back. Just keep in mind that in FOSS you can't give up anytime a single person disparages your work. I can't think of anything big in FOSS that didn't have a long line of detractors. ZFS/Btrfs were rampant layering violations. Linux was an obsolete macrokernel. Your favorite license is more/less free than the other one. Let's not mention things that run as PID 1. Generally the last few councils have operated by letting everybody air their grievances and then stepping in to make a decision. Don't feel like you have to have the last reply on every subthread or else it is a concession of defeat. Sure, it is good to speak up somewhere on the discussion, and present clean arguments once. If nothing else it gets you recognition for the good work you're doing and helps everybody else learn how to do similar good work. If you would rather just focus on the code, then perhaps we can try to get others involved in portage documentation/etc who want to take more of an interest in the communication side. We just need to make that need more visible - it sounds like a great way for somebody new but enthusiastic to get involved. And if you have a concern feel free to ping the council on IRC/email/whatever, publicly or privately. I can't think of anybody on the current council who would want to discourage the work you're doing. Don't get caught up on -dev (list or irc) drama and let it drain you. Even if we went full-nazi-mode on the moderation you'd still never stamp out every voice of dissent. You just need to trust that the folks making the decisions will consider your experience and your reasoned arguments, and not worry about the volume of replies. And if the community can't trust us to do that, then we shouldn't be in charge. > > So, if the council can rubber stamp a plan to proceed, the portage team is > interested in implementing it, but I doubt any of us will participate until > then. Well, somebody just needs to write the plan. I think there is already enough interest to do it by those on the list, but if that REALLY is the limiting factor, perhaps we can nudge somebody else to step in and help facilitate.
I'm not interested in going through this bikeshed again, this time on account of the devmanual. Once (as part of Portage) was enough. People don't want us to fix this bug in Portage. So we likely won't for the time being. If someone wants to fix the devmanual part of the problem, that's their battle. It's pretty clear that the Portage team (unanimously) is in favour of deprecating dynamic deps. Removing Portage from the CC list.
(In reply to Julian Ospald (hasufell) from comment #58) > (In reply to Richard Freeman from comment #57) > > This is a big change. > > No. This is what already lies in PMS. It is a change to the reality of how we're doing things. It may or may not be a change to the PMS requirements. Maybe we're doing things wrong today, but making it right is still a big change. If I discovered an accounting mistake at work and our corporation was behind $1B in back taxes, it isn't like I could just tell some clerk to file an amended return to the IRS. That might very well be the likely ultimate outcome, but so much stuff would have to be fixed in light of the discovery it would cause a huge churn in meetings, updated procedures, legal reviews, financial changes to pay the bill, and so on. The legal requirements didn't change - but we had to change to meet them. That is the case here. I think you've made the case that this is a good change to make. We just need to get everybody aligned around how we're going to make the change. Fixing one page in the devmanual is just one part of that. I don't think a lot is needed. Just get it out on the list, let the naysayers have their say, and if necessary the council can stamp it. Maybe a naysayer will actually find something in your plan that could benefit from improvement, and then we'll all be better for the changes you make.
(In reply to Richard Freeman from comment #61) > (In reply to Julian Ospald (hasufell) from comment #58) > > (In reply to Richard Freeman from comment #57) > > > This is a big change. > > > > No. This is what already lies in PMS. > > It is a change to the reality of how we're doing things. It may or may not > be a change to the PMS requirements. > > Maybe we're doing things wrong today, but making it right is still a big > change. > A revision bump is now and has always been required when changing runtime dependencies in an ebuild. What hasufell keeps trying to get through is that we're not changing *anything* as part of this bug, just documenting the existing reality. The fact that documenting the *existing situation* is a prerequisite for the elimination of dynamic deps is a red herring. If a developer is too lazy to revbump, he can just leave the dependencies incorrect in his ebuild. If you think that sounds asinine, congratulations, that's what you're already doing, because dynamic deps aren't standard.
On 16/09/15 16:33, Rich Freeman wrote: > Honestly, I think you really need to have thicker skin here. If you > only want to work on features that NOBODY doesn't like, you're not > going to accomplish much. Thicker skin doesn't enter into it. I removed us from the CC because I don't want to read about this bikeshed. I simply *don't care*. Neither does the rest of the Portage team. > I think it is pretty clear that the council is in favor of this as > well. We just want to see a coordinated plan. > > And you don't have to be the one to author it. I just don't expect > much to happen in terms of getting the tree ready to use the newer > versions of portage in the absence of such a plan. Again, we *don't care*. Nobody in Portage is motivated to work on this. Please don't spam me privately to restate what you have said a thousand times over in public or to effectively tell me to "grow a pair". If someone wants to go through the hassle of authoring this grand unified plan, I genuinely applaud the effort. But I don't think widespread acceptance is going to happen any time soon. That said, I'd be thrilled to be proven wrong, so give us a ping on IRC when you've made enough progress for us to roll out the over a year old patches that fixes the dynamic dependencies bug.
(In reply to Richard Freeman from comment #61) > (In reply to Julian Ospald (hasufell) from comment #58) > > (In reply to Richard Freeman from comment #57) > > > This is a big change. > > > > No. This is what already lies in PMS. > > It is a change to the reality of how we're doing things. No. Everyone who understands that dynamic deps are bad already revbumps according to PMS. The few people who refuse to understand it haven't been able to bring up useful arguments for over a year. I'm not buying into your bikeshed. Was vacation that boring?
(In reply to Alexander Berntsen from comment #63) > Please don't spam me privately to restate what you have said a thousand > times over in public or to effectively tell me to "grow a pair". Apologies, I'll refrain from sending you additional private emails. I'll ask you to refrain from publishing private emails in public forums. That is advice that will probably serve you well elsewhere. > > If someone wants to go through the hassle of authoring this grand unified > plan, I genuinely applaud the effort. But I don't think widespread > acceptance is going to happen any time soon. Again, nobody has to achieve widespread acceptance. The council just asked for an implementation plan. You really only have to convince four of us, and I think most of us are already leaning in your direction. (In reply to Michael Orlitzky from comment #62) > > A revision bump is now and has always been required when changing runtime > dependencies in an ebuild. What hasufell keeps trying to get through is that > we're not changing *anything* as part of this bug, just documenting the > existing reality. The fact that documenting the *existing situation* is a > prerequisite for the elimination of dynamic deps is a red herring. Perhaps. Is it really easier to have a 60+ post running debate in a bug over the change than just posting a summary of the entire plan on -dev? Somebody complained about your change to the council. The council cautiously asked you to do ABC so that they could bless your change. Instead everybody wants to do DEF. Then we can alternatively act shocked that people are raising objections, and cynically express how this just goes to prove that Gentoo is unable to get anything done.
(In reply to Richard Freeman from comment #65) > The council just asked for an implementation plan. It is right in front of your eyes. Are you ignoring it on purpose?
(In reply to Richard Freeman from comment #65) > > Perhaps. Is it really easier to have a 60+ post running debate in a bug > over the change than just posting a summary of the entire plan on -dev? > The goal is to compartmentalize the easy/obvious changes so that some progress can be made. The need to revbump on RDEPEND changes is a fact, independent of the portage team's desire to deprecate dynamic dependencies. If the portage team decides to keep dynamic deps, you'll still need to do a revbump when you change RDEPEND, because not everyone uses portage. That's been true "forever." The council asked for a plan regarding the deprecation of dynamic deps. Forget dynamic deps for now -- consider this fix in isolation. By putting this to the mailing list in the context of dynamic deps, we'll start a shitstorm over a bunch of irrelevant details. And for revbump issue, it's a lot like assembling a committee to legislate the value of pi. Everyone's going to have an opinion, and none of them are going to matter, because the need to revbump on RDEPEND changes is a fact. Pages and pages of arguments about dynamic deps and eclasses and virtuals and personal insults are only going to eclipse the discussion about this one little trivial, correct, and necessary change to the devmanual. My hope was that we could get this necessary and correct change to the devmanual out of the way before even considering a larger plan. That way, the plan contains one less bullet point to argue over, and that bullet point is a change that needs to happen anyway, regardless of what happens with dynamic deps.
(In reply to Richard Freeman from comment #54) > > > Vote: > > > - "The council asks the Portage team to first outline their long-term > > > plan regarding removal or replacement of dynamic dependencies, > > > before they remove this feature. In particular, tree policies and > > > the handling of eclasses and virtuals need to be clarified." > > > Accepted unanimously. > > I've yet to see anything presented regarding the handling of eclasses and > virtuals. There hasn't even been a mention on -dev of the intent to remove > dynamic dependencies. > > You're welcome to disagree with the council decision. You're not welcome to > ignore it. That's just how we do things. > > If you'd like to ask the council to change its mind you're welcome to do so. > However, I wouldn't really expect much to change. There was a request to > clarify policy around virtuals and eclasses, and it doesn't seem > unreasonable to do that before we start trying to implement something. For the record I support this statement 100%. The removal or replacement of dynamic dependencies has been mainly pushed for by the Portage team, who have good reasons for it. Noone of us however knows the entire code in Gentoo equally well, and I can't believe that they have considered every tree-wide detail. (For example, noone from Portage team is also involved in packaging Perl or KDE.) That's why important policy changes go to the lists. The council has done nothing but requested that the current state is not replaced by a half-hearted snap decision.
(In reply to Michael Orlitzky from comment #62) > (In reply to Richard Freeman from comment #61) > > (In reply to Julian Ospald (hasufell) from comment #58) > > > (In reply to Richard Freeman from comment #57) > > > > This is a big change. > > > > > > No. This is what already lies in PMS. > > > > It is a change to the reality of how we're doing things. It may or may not > > be a change to the PMS requirements. > > > > Maybe we're doing things wrong today, but making it right is still a big > > change. > > > > A revision bump is now and has always been required when changing runtime > dependencies in an ebuild. This is incorrect (and exactly this detail has been the subject of one of our quiz questions for ages). Policy has been for many years that a revbump is required only if the installed files change. If Portage changes inadvertently in a way that this policy becomes impossible, that's unfortunate and a bug. If Portage is changed deliberately in a way that this policy becomes impossible, without changing the policy first, that's antisocial behaviour at the least.
Ok, so do it yourself. Have fun.
(In reply to Andreas K. Hüttel from comment #69) > > > > A revision bump is now and has always been required when changing runtime > > dependencies in an ebuild. > > This is incorrect (and exactly this detail has been the subject of one of > our quiz questions for ages). > > Policy has been for many years that a revbump is required only if the > installed files change. Which policy? According to the PMS, you need to do a revision bump. And I should point out that changing RDEPEND does in fact change an installed file: $ cat /var/db/pkg/sys-power/cpupower-3.18/RDEPEND sys-apps/pciutils !<sys-apps/linux-misc-apps-3.6-r2 !sys-power/cpufrequtils So the quiz answer is not inconsistent with performing a revision bump for an RDEPEND change.
(In reply to Michael Orlitzky from comment #71) > Which policy? According to the PMS, you need to do a revision bump. And I > should point out that changing RDEPEND does in fact change an installed file: > > $ cat /var/db/pkg/sys-power/cpupower-3.18/RDEPEND > sys-apps/pciutils !<sys-apps/linux-misc-apps-3.6-r2 !sys-power/cpufrequtils Usually we define "installed files" of a package as the ones listed in the CONTENTS file of the VDB. Files in the VDB itself are not part of that. > So the quiz answer is not inconsistent with performing a revision bump for > an RDEPEND change.
(In reply to Ulrich Müller from comment #72) > (In reply to Michael Orlitzky from comment #71) > > Which policy? According to the PMS, you need to do a revision bump. And I > > should point out that changing RDEPEND does in fact change an installed file: > > > > $ cat /var/db/pkg/sys-power/cpupower-3.18/RDEPEND > > sys-apps/pciutils !<sys-apps/linux-misc-apps-3.6-r2 !sys-power/cpufrequtils > > Usually we define "installed files" of a package as the ones listed in the > CONTENTS file of the VDB. Files in the VDB itself are not part of that. > I'm only trying to point out that reality may actually be consistent with the quiz answer. But for the record, here is a quote from my recruiter: The rule is: “If any file changes on the next emerge, then you need a revbump”. The word "installed" does not appear, and /var/db/pkg/sys-power/cpupower-3.18/RDEPEND is certainly a file.
(In reply to Michael Orlitzky from comment #67) > By putting this to the mailing list in the context of dynamic > deps, we'll start a shitstorm over a bunch of irrelevant details. When you know a change is going to cause a 100-post flamewar, you're not going to avoid it by just quietly making the change anyway. About the only thing worse than politics is pretending that it doesn't exist. It doesn't get you anywhere. Maybe you feel better in the end about not getting anything done, but if your goal was to accomplish something you've lost. So, we still have the flamewar, but now it is in bugzilla where it is all the more painful to manage. The underlying issue here is that there is a desire to make a change that is contentious. If it weren't we wouldn't be having this discussion, and there wouldn't be concern about a mess on the mailing lists. When we want to make contentious changes that impact multiple projects in Gentoo, we have a mechanism for doing so, and it is the Council. My advice is to take a short break, and we can try again when we've all cooled off. Maybe we'll lend a hand, but right now I still owe an action item on the whole qt/gtk debate. And I don't disagree with you that we might consider moving forward with the simple case of changing RDEPENDS in packages apart from eclasses/virtuals/etc. I'll go ahead and propose that on the list now as I don't think it needs a lot of thought to at least consider that, and we obviously have a patch available.
(In reply to Richard Freeman from comment #74) > > And I don't disagree with you that we might consider moving forward with the > simple case of changing RDEPENDS in packages apart from > eclasses/virtuals/etc. I'll go ahead and propose that on the list now as I > don't think it needs a lot of thought to at least consider that, and we > obviously have a patch available. Thank you. I wasn't proposing that we avoid the issue forever, just that we not hedge this little three-line patch on the outcome of a war over dynamic deps.
