It would help --newuse if we could tell which dependencies are purely detected at runtime and which need build time support. This way --newuse could just pull in the dependencies for things that are purely detected at runtime. The syntax could for example be: RDEPEND=" runtime: sys-app/something buildtime: sys-app/else" With buildtime as default for backwards compatibility. I posted an RFC to gentoo-dev about this for more comments.
I'm not sure I understand the suggestion. If you check things only in RDEPEND vs things in both RDEPEND and DEPEND, doesn't that take care of it?
we (gnome herd) discussed about what should be done for gstreamer plugins and something like this, having the package manager know or be smart about runtime pluggable deps would be cool to avoid rebuilding for a simple use flag change (codecs in the gst case).
This is a good idea IMHO. The problem (at least, a small space of improvement exists here) of current separation of dependency is that it is still not granular enough. The current RDEDEND includes dynamic library and standalone program (festival as to stardict-3.0.{0,1}). The former one is also needed at build time,or link time to be specific. While the latter one is purely needed at run time. If we are able to distinguish between the two, not only this will help --newuse, but also helps cross compiling. Be aware that the separation of DEPEND/RDEPEND plays an important role in cross compiling. The dependencies in DEPEND will be installed on host system. while those deps in RDEPEND will be installed on host systems. Currently many ebuild's DEPEND and RDEPEND have overlap between each other. In cross compiling, this will result in some unneeded package installed on the host system. It will be great if we can distinguish between build time RDEPEND and pure runtime RDEPEND. I have discussed this with someone before: http://r0bertz.blogspot.com/2007/03/some-ideas-about-dependrdepend.html
*** Bug 174552 has been marked as a duplicate of this bug. ***
(In reply to comment #1) > I'm not sure I understand the suggestion. If you check things only in RDEPEND > vs things in both RDEPEND and DEPEND, doesn't that take care of it? ditto. If we want better granularity I would vote for bug #174552#c5
(In reply to comment #0) > It would help --newuse if we could tell which dependencies are purely detected > at runtime and which need build time support. This way --newuse could just > pull in the dependencies for things that are purely detected at runtime. I'm also having trouble understanding the idea, could you provide a real example and/or a more detailed explanation?
(In reply to comment #6) > I'm also having trouble understanding the idea, could you provide a real > example and/or a more detailed explanation? > Here is my interpretation for you. In java, you could have a package (BLAH) that requires JDBC support. The JDBC API is included within the Java runtime environment. So from the perspective of building the application, no other dependencies are required. Of coarse if you want to run BLAH then you will need a jdbc driver. ( like jdbc-mysql for example ) therefore the RDEPEND in this situation could be RDEPEND="runtime: mysql? ( dev-java/jdbc-mysql )" Does everyone agree with that?
(In reply to comment #7) > In java, you could have a package (BLAH) that requires JDBC support. The JDBC > API is included within the Java runtime environment. So from the perspective > of building the application, no other dependencies are required. Of coarse if > you want to run BLAH then you will need a jdbc driver. ( like jdbc-mysql for > example ) > > therefore the RDEPEND in this situation could be > > RDEPEND="runtime: mysql? ( dev-java/jdbc-mysql )" > > Does everyone agree with that? Putting other deps aside: DEPEND="" RDEPEND="mysql? ( dev-java/jdbc-mysql )" Where is the difference? - dep only in RDEPEND is runtime only - dep in both DEPEND and RDEPEND is both buildtime and runtime (here RDEPEND="buildtime: dep" ??, would it be also in DEPEND?)
Is the only use case for this where ebuilds are using USE flags purely to pull in additional plugin-type packages?
(In reply to comment #8) > - dep only in RDEPEND is runtime only > - dep in both DEPEND and RDEPEND is both buildtime and runtime (here > RDEPEND="buildtime: dep" ??, would it be also in DEPEND?) Anyway, the point is we should be able to distinguish between deps like autotools/dynamics libs/standalone programs. We have to decide on whether we need this kind of separation. Then we decide on the implementation details. "DEPEND only/both DEDEND&RDEPDEN/RDEPEND only" could be a solution. However our implementation does not work like this, does it?
RECOMMENDS or a similar variable could be a better approach, fully backwards compatible and with more applications (and more intuitive!). Let's put an example of the possible behaviour: RECOMMENDS="myuse? ( cat/foo )" Depending on user settings (never install recomendations automatically, always install them...) RECOMMENDS will be just printed so the users are aware of the recomendations, or handled as the Petteri's RDEPEND="runtime: cat/foo". Actually, from the package manager's view point RECOMMENDS would be something like optional on-demand (un)pluggable PDEPENDs. That way we can also remove a lot of pkg_postinst code printing things like "You may want install foo to get bar feature." Probably RECOMMENDS would need a different syntax and support descriptions for each dep, of maybe this could be done somewhere else?
(In reply to comment #11) > RECOMMENDS or a similar variable could be a better approach, fully backwards > compatible and with more applications (and more intuitive!). Yep, that's what I was getting at with my question in comment#9. CRAN has a Suggests: key which is used for this purpose. Note that there are still plenty of other reasons to switch to the labels syntax and a single DEPENDENCIES variable.
(In reply to comment #12) > > Note that there are still plenty of other reasons to switch to the labels > syntax and a single DEPENDENCIES variable. > Could you point which reasons are? Or maybe any thread on @-dev about it?
(In reply to comment #8) > > Where is the difference? > > - dep only in RDEPEND is runtime only > - dep in both DEPEND and RDEPEND is both buildtime and runtime (here > RDEPEND="buildtime: dep" ??, would it be also in DEPEND?) > The package manager can't know about the code paths in the ebuild. Even if it's runtime detected currently the ebuild might be installing some extra files with the use flag etc. With the new syntax the developer guaranties that reinstalling is not needed.
(In reply to comment #13) > (In reply to comment #12) > > Note that there are still plenty of other reasons to switch to the labels > > syntax and a single DEPENDENCIES variable. > > Could you point which reasons are? Or maybe any thread on @-dev about it? There have been a rather large number of other "we need a new *DEPEND variable to deal with ..." requests in the past -- enough that trying to deal with them all through *DEPEND variables gets silly. I believe spb has a fairly up to date list of all the dependency roles that have been considered necessary.
(In reply to comment #14) > The package manager can't know about the code paths in the ebuild. Even if it's > runtime detected currently the ebuild might be installing some extra files with > the use flag etc. With the new syntax the developer guaranties that > reinstalling is not needed. Ok, so what you want is to mark certain deps (use flags?) that should not trigger a rebuild with --newuse and relatives?
(In reply to comment #16) > > Ok, so what you want is to mark certain deps (use flags?) that should not > trigger a rebuild with --newuse and relatives? > That's a specific part of the technical side but not the only one. We're trying to specify "dependencies" which can be used at runtime but has nothing to do with the build and installation process. That is, recommendations or suggestions, but not strict dependencies which certain app needs to build or run. Currently we have no standard method to handle this. Some people use an USE flag which just pulls a dep, creating a problem with --newuse (for example), and other people print a message on pkg_postinst recommending some packages... IMO both methods are suboptimal.
Ok, so really what people are after is a way of marking dependencies as required, recommended or suggested?
Created attachment 137968 [details] dependency labels doc I wrote this up a while ago. It seems to cover what's required...
(In reply to comment #19) > Created an attachment (id=137968) [edit] > dependency labels doc > > I wrote this up a while ago. It seems to cover what's required... > This looks pretty good. I still don't know how we'd document why each dep is recommended or suggested.
(In reply to comment #20) > I still don't know how we'd document why each dep is recommended or suggested. DEPENDENCIES=" post,suggested: app-misc/my-plugin post,required: app-misc/my-required-plugin " Note that by the labels proposal, there're various equivalent ways of specifying this, some more verbose than others... For example: DEPENDENCIES=" post: ( app-misc/my-required-plugin suggested: app-misc/my-plugin ) " gives the same result.
(In reply to comment #19) > Created an attachment (id=137968) [edit] > dependency labels doc > > I wrote this up a while ago. It seems to cover what's required... So, are we going to implement this? This looks really good!
(In reply to comment #22) > (In reply to comment #19) > > Created an attachment (id=137968) [edit] > > dependency labels doc > > > > I wrote this up a while ago. It seems to cover what's required... > > So, are we going to implement this? > This looks really good! I suggest holding off for a bit. I'm pretty sure there'll be quite a few new revisions of that document before it's even considered appropriate for a formal proposal... In particular, the ABI stuff isn't suitable for Gentoo, and we could probably do with a couple of new type labels (fetch dependencies, test dependencies, probably others). Regarding labels, there's also a suggestion to use them within SRC_URI: SRC_URI=" mirrors-then-listed: http://blah/whatever local-mirrors-only: http://blah/nomirrorable listed-then-mirrors: http://blah/betterthanprimaryuri " I'm not aware of anyone having written this up properly, but there's an experimental implementation available for it in Paludis 0.26.
(In reply to comment #20) > This looks pretty good. > > I still don't know how we'd document why each dep is recommended or suggested. > There is no mechanism inherent for that, which I can see, apart from the Changelog. I agree it would be useful to have this visible to the user in a similar fashion to metadata for USE flags, so I guess it'd be a candidate for metadata.xml in the same way. (In reply to comment #23) > Regarding labels, there's also a suggestion to use them within SRC_URI: > > SRC_URI=" > mirrors-then-listed: http://blah/whatever > local-mirrors-only: http://blah/nomirrorable > listed-then-mirrors: http://blah/betterthanprimaryuri > " > Looks OK; wouldn't it be simpler just to label them first, local and last?
(In reply to comment #24) > There is no mechanism inherent for that, which I can see, apart from the > Changelog. It's a class of labels. post,suggested: or post,recommended: or post,required:. > I agree it would be useful to have this visible to the user in a > similar fashion to metadata for USE flags, so I guess it'd be a candidate for > metadata.xml in the same way. Can't go in metadata.xml. No package manager mandatory information can be in XML. > (In reply to comment #23) > > Regarding labels, there's also a suggestion to use them within SRC_URI: > > > > SRC_URI=" > > mirrors-then-listed: http://blah/whatever > > local-mirrors-only: http://blah/nomirrorable > > listed-then-mirrors: http://blah/betterthanprimaryuri > > " > > > Looks OK; wouldn't it be simpler just to label them first, local and last? There're six labels. The names we're using in the current experiment are mirrors-first, mirrors-only, listed-only, listed-first, local-only, manual. But whilst more concise, they're not as clear for illustrative purposes.
(In reply to comment #19) > Created an attachment (id=137968) [edit] > dependency labels doc > > I wrote this up a while ago. It seems to cover what's required... > This idea of labels is wonderful and will properly categorize (or could categorize) all possible dependency types. There are several examples where this could be useful; gstreamer plugins, runtime-detected plugins like for totem, are just a few applications. Now, I'm new at this and I could be horribly wrong/idiotic, so please bear with me :) As far as I can see, the attachment does not specify how this new system will interact with the user for the dependency graph. What I mean by this is that USE flags are (similar) optional dependencies and are specified in make.conf or package.use for association with a package. But with this dependency categorisation, how does one specify "I want so and so 'suggested' or 'recommended' runtime dependency Y of package X to be associated with package X." ? As I understand it, the user shouldn't just `emerge Y` (it'll soil the world file), and if he does `emerge --oneshot Y`, how do we know which package he wants it for? Do we associate Y with every package that suggests/recommends it? Having a package.labels (sys-apps/package suggested) is an incomplete solution since it doesn't specify exactly which dependencies one wants, and only says "I want all suggested deps to be installed". Thoughts?
(In reply to comment #26) > As far as I can see, the attachment does not specify how this new system will > interact with the user for the dependency graph. Correct. Package manager interfaces and configuration are not specified by EAPIs or PMS, and are beyond the scope of this discussion.
(In reply to comment #27) > Correct. Package manager interfaces and configuration are not specified by > EAPIs or PMS, and are beyond the scope of this discussion. Okay, they are not specified by them, and that means Package manager interfaces are beyond the scope of the spec. But that hardly means that it's beyond the scope of this discussion. After all, if a spec is made without looking at possible consequences, it could easily cause problems during implementation.
(In reply to comment #28) > Okay, they are not specified by them, and that means Package manager interfaces > are beyond the scope of the spec. But that hardly means that it's beyond the > scope of this discussion. After all, if a spec is made without looking at > possible consequences, it could easily cause problems during implementation. And if a spec is made in such a way that it assumes that a package manager is going to work in a particular way, then it's likely going to cause problems for anyone doing things a different way. A discussion about package manager specifics isn't helpful here; ensuring that the data provided to the package manager by ebuilds is accurate and done so in a way which isn't unnecessarily complex is what matters.
(In reply to comment #29) > And if a spec is made in such a way that it assumes that a package manager is > going to work in a particular way, then it's likely going to cause problems for > anyone doing things a different way. Well, I'd say if someone wants to do something in a different way, they have to make sure that that different way doesn't break everything else =) Kind of like how arch teams have to work with herds when marking things stable. The current gnupg-2 stab is a rough example of this; it's a new and unusual problem which requires coordination between several different teams for things to go smoothly. > A discussion about package manager > specifics isn't helpful here; ensuring that the data provided to the package > manager by ebuilds is accurate and done so in a way which isn't unnecessarily > complex is what matters. But what's the use of that data if the package manager or the users cannot use it properly? What you are saying is akin to saying "All we have to do is build a sturdy and simple foundation for this building. How the building's load will be distributed is it's own problem, not ours."
(In reply to comment #30) > (In reply to comment #29) > > And if a spec is made in such a way that it assumes that a package manager is > > going to work in a particular way, then it's likely going to cause problems for > > anyone doing things a different way. > > Well, I'd say if someone wants to do something in a different way, they have to > make sure that that different way doesn't break everything else =) Designing things that way is a large part of why it's so hard to do anything with ebuilds that Portage doesn't do. > But what's the use of that data if the package manager or the users cannot use > it properly? What you are saying is akin to saying "All we have to do is build > a sturdy and simple foundation for this building. How the building's load will > be distributed is it's own problem, not ours." If the data is correct, the package manager can use it properly. Even if the data's there but the package manager doesn't use it at all yet, it leaves room for people to do useful things with it. In particular, what we absolutely don't want is a specification that limits people to either having to use a configuration file to keep track of suggestions or having to have the user specify each time what they want. The way things are done currently that's possible; if you start discussing an implementation that's based around, say, a configuration file, your assumptions implicitly restrict things to being done that way. The discussion should be about the data, not the policy it imposes.
(In reply to comment #31) > If the data is correct, the package manager can use it properly. Even if the > data's there but the package manager doesn't use it at all yet, it leaves room > for people to do useful things with it. > So if the data isn't entirely useful to the package manager, and we want more data, we have another EAPI/PMS to change the data given again? It seems far from optimal. Foresight, anyone? > In particular, what we absolutely don't want is a specification that limits > people to either having to use a configuration file to keep track of > suggestions or having to have the user specify each time what they want. The > way things are done currently that's possible; if you start discussing an > implementation that's based around, say, a configuration file, your assumptions > implicitly restrict things to being done that way. The configuration file was merely an example of the way it could be done. There could be several better ways of doing it. All I'm saying is throwing data at the package manager is all fine and good, but it's hardly the most optimal solution. Ask the package manager what it wants, whether it can use it, or make use of it sometime in the future, and then give the data in a proper format to the package manager or give it more data if it requires. > The discussion should be about the data, not the policy it imposes. > If that's the consensus, very well. I will cease pointing out implementation problems in this discussion. Sorry for the bugspam everyone :)
(In reply to comment #32) > (In reply to comment #31) > > If the data is correct, the package manager can use it properly. Even if the > > data's there but the package manager doesn't use it at all yet, it leaves room > > for people to do useful things with it. > > > > So if the data isn't entirely useful to the package manager, and we want more > data, we have another EAPI/PMS to change the data given again? It seems far > from optimal. Foresight, anyone? If the data is there and correct, the package manager can make use of it as it sees fit. If the data is only designed to be used in one particular way, the package manager can only use it in that way. Foresight is in providing data that doesn't limit how it's used -- a lack of foresight is why we have the current DEPEND/RDEPEND mess. > All I'm saying is throwing data at the package manager is all fine and good, > but it's hardly the most optimal solution. Ask the package manager what it > wants, whether it can use it, or make use of it sometime in the future, and > then give the data in a proper format to the package manager or give it more > data if it requires. If you only give the package manager what it wants, you prevent people from writing package managers that do other things. Making a package manager ignore things it doesn't consider relevant is easy; making it use data that isn't there is often impossible.
(In reply to comment #33) Okay, this debate has turned into an idealogical argument about how things are to be discussed and developed which will lead nowhere. Everyone has his or her own ideas as to how things are to be done, and this is really the worst place to discuss this :) Now, my original concern, rephrased. The concept of "recommended" and "suggested" is very much like the Debian package format to the point where it seems to have been ripped off from there ;) Binary package managers happily take it in, since they track package dependencies differently from portage/paludis/pkgcore. However, I think the portage system (not the package manager) by design, cannot really handle optional dependencies unless you have a compulsory USE flag associated with every optional dependency without *drastic* changes. But if the intention is to add features that are not useful now, but could be used by some package manager sometime in the future, then by all means, go ahead. Just my 2 paise :^)
(In reply to comment #34) > Now, my original concern, rephrased. The concept of "recommended" and > "suggested" is very much like the Debian package format to the point where it > seems to have been ripped off from there ;) Actually, it was ripped off from CRAN, and extended to be Debian-concept-compatible later. > However, I think the portage system (not the package manager) by design, cannot > really handle optional dependencies unless you have a compulsory USE flag > associated with every optional dependency without *drastic* changes. Actually, they can rather easily. The simplest implementation is as follows: * When showing a list of packages to install, post (we'll ignore non-post suggestions for this) suggestions are indicated after the package suggesting them. The package manager does not install these suggestions. * The user can choose to install these suggestions by hand using --oneshot or the equivalent. * When checking for unused packages (depclean in Portage terms), suggestions that are installed are considered part of the dep tree. * When doing deep upgrades, suggestions that are installed are considered part of the dep tree. Now, that isn't the *best* implementation. But it's a very easy one.
I think the discussion about PM handling should be over. Nirbheek there are various models for suggestion handling which could work quite well and each package manager will choose whatever they want. So let's stay on-topic and discuss those implementation details when this proposal is more mature (and probably in another bug, or in gentoo-dev mailing list) ;-)
Could someone explain why these labels are preferable to new variables?
(In reply to comment #37) > Could someone explain why these labels are preferable to new variables? Two reasons. First, you'd need a hundred or more variables to specify all the necessary combinations. Second, people are already using COMMON_DEPEND lots. This removes the need for it by letting you use build+run: as a label.
What about backwards compatibility? This will break any eclass using current *DEPEND. Have you thought any compatibility method? Or will eclasses need to be adapted?
(In reply to comment #39) > What about backwards compatibility? > This will break any eclass using current *DEPEND. Have you thought any > compatibility method? Or will eclasses need to be adapted? It's an EAPI proposal, so the usual EAPI rules apply.
(In reply to comment #25) > (In reply to comment #24) > > There is no mechanism inherent for that, which I can see, apart from the > > Changelog. > > It's a class of labels. post,suggested: or post,recommended: or post,required:. Understood. > > > I agree it would be useful to have this visible to the user in a > > similar fashion to metadata for USE flags, so I guess it'd be a candidate for > > metadata.xml in the same way. > > Can't go in metadata.xml. No package manager mandatory information can be in > XML. > The question was about having documentation as to why the packages are recommended visible to the user, not where the deps are specified. The doc info for the user for this should go in metadata.xml, in the same way as the USE description imo.
*** Bug 317337 has been marked as a duplicate of this bug. ***
If people want to push this forward... glep process comes to mind. Changes to the format of this sort affect devs pretty heavily, so they ought to get a full say in it.
So, if we want dependency labels... what does a dependency label in a || () block mean?
(In reply to comment #44) > So, if we want dependency labels... what does a dependency label in a || () > block mean? In exheres-0 it's forbidden. It's probably not worth trying to figure out what it means...
After 45 comments, it isn't entirely clear to me what this bug is about. If I take the original comment, it looks like it is mostly a special case of bug 424283, and maybe some aspects are also covered by bug 327701. Closing as duplicate of the former bug. Feel free to reopen (or open a fresh bug) if you disagree. *** This bug has been marked as a duplicate of bug 424283 ***
