I have come accross a problem with lablgtk (http://bugs.gentoo.org/show_bug.cgi?id=39181) due to the fact that ocaml is not binary compatible from version to version and hence any libraries needs to be recompiled on every upgrade (standalone apps have no problem because they're statically linked, usually). So i tried to think of a way to avoid problems and there are basically four ways i found: - Rebuild all libraries on ocaml upgrade: it is difficult to automate. Using `qpkg -nc -q -I -n dev-lang/ocaml`, we get libraries and standalone apps, so user intervention is needed if we don't want to make the upgrade time much longer. - Make multiple ocaml versions coexist (i.e: like debian), and arrange things so as to check easily if a particular library is installed for a particular version. (e.g: Knowing the package you're in will build for ocaml-3.07, `opengl? ( dev-ml/lablgl-0.99 for ocaml-3.07 )`. This would be a nice-feature of portage-ng, but it seems a bit... too much. - Rebuild selectively. Somehow when a caml library is installed, we keep (in /var/db/pkg/) the ocaml version it was built with, if it's not the same as the ocaml version used for building our package, we automaticaly reemerge the incriminated library. This would work for the future, but not now that there are already installed lablgl versions for example. This is somewhat similar to the precedent proposition. A hack to emulate this behavior can still be used today, just try to compile a program that links with the installed library version and see if it fails, if yes reemerge, otherwise the library has the good version. - Force reemerging of dependent libraries. I'm not sure if its possible with current portage to force a reemerge, but this would be the simplest way to avoid clashes today. Ideas ? I'll mask lablgtk-2.2.0 until we find a solution. P.S: we should have an ml or dev-ml mail alias, seemant told me to bug you about that george :)
>seemant told me to bug you about that george :) So, why is it assigned to bug-wranglers then? ;) Well, seriously, I only occasionally stumbled upon this bug and I suspect this is more clearly the one you had questions about. Sorry about not getting to them yet, - I got carried away somewhat, but I promise to respond within a few days :). Reassigning this one, now that we have lang-ml alias. George
Sorry for taking so long to spot this bug, but I got here finally :). Actually, considering that portage-ng is in the works, I think the best course of action won't involve much on our part for now. But lets get to the details: 1 and 4 seem to be essentially the same (rebuild dependants, possibly selectively). This shoudn't be done from within the ocaml ebuild, as this will go against the dependency logic and other metadata (and will eventually result in a mess) and is kind of against the "Gentoo philosophy" (to keep user in controll, which includes not doing non-trivial unexpected changes automatically). The way similar issues are dealt with regularly to this time is to do necessary preps on the involved packages to insure that en-masse update goes smoothly and then put a note in the base package (in pkg_postinst) asking user to run revdep-rebuild on this package (its either part of portage or gentoolkit), leaving the user with the choice to rebuild necessary packages selectively. Effectively this is #1. 2. Is a nice alternative, if it does not cause too much pain. This is what SLOT's are for, but this would require quite a bit of changes to caml and the libraries in questions I would imagine (to make different versions of ocaml coexist, which will likely involve changin its installation routine and symlink handling, and adjust all related libraries correspondingly). So, basically, this is a question of acceptable tradeoff between our and user's convenience :). 3. Seems to be a cludge (I mean the version tracking under /var/db/pkg part and similar stuff) and IMHO is best left until portage-ng, as it should cover this and whatever resolution we do now will have to be reworked.. Plus anyway this will be the additional dependency or metadata tracking and such tasks are best left for the package manager, otherwise we are going to face (potentially severe) problems in the future.. George
Ok, i've played around with the different ideas and finally stalled on point 2. I created an ocaml.eclass and an ocaml-config script (from gcc-config) that allows to have multiple versions of ocaml installed and multiple versions of each library, one for each compiler. Basically, it works this way: dev-lang/ocaml is changed to add information about itself to an /etc/env.d/ocaml file (library path, sitelib path, ...) ocaml-config allows one to switch between ocaml versions and get info about the current one. Ebuilds for ocaml libraries inherit ocaml.eclass which sets the slot to $OCAML_VERSION. Then it is fairly easy to use qpkg to rebuild all libraries of the previous compiler version for the selected one. Also, when slot syntax is add for deps, it will be really easy to update the ebuilds to ensure good versions are installed, although for now it will stay the same mess if people don't rebuild on each new ocaml version. I also created a findlib.eclass which exports only findlib_src_{preinst,install} which sets things up nicely for findlib-enabled packages. I attach a tarball containing ocaml-config, modified ocaml and some libraries i updated (not all, some personal), plus the two eclasses. Tell me what you think!
It'll be easier if i just put it on my dev.g.o space, as it's big. See: http://dev.gentoo.org/~mattam/ocaml-overlay
Ok, looking thirough this finally. Sorry about the delay - the notification to this bug cought me while I was mostly on the go, returned only couple of weeks ago.. Anyway, the modifications seem reasonable wrt the discussed implementation of #2 approach. I'll do some more testing, but right now I was looking at whether we can use PDEPEND to sole this at least partially. Unfortunately it does not look like simple addition of PDEPEND solves the issue. At least so far I was unable to make it do anything usefull. I'll have to ask around for the particulars I guess. In any case, these modifications make it possible to have two versions installed in parallel and switch in flight. That's an interesting addition in itself, that is if you are willing to maintain it ;). Ok, that's it for now, I'll get back with more details shortly. Just wanted to drop a line here that this is not forgotten :). George
I am not sure if the idea with slots is very good. If ocaml would change much between the versions, it would be okay, but as it is... I have always had the feeling that slots should be used when versions differ much and there is a good reason for using many of them in parallel (and easier maintainance doesn't qualify as a good reason in my opinion). Of course this solution is better than no solution; this is perfectly acceptable as a temporary solution before portage-ng (OTOH experience taughts us that temporary solutions last for the longest time). But I don't feel much comfortable about having many installed versions of ocaml just because these libraries there need to be recompiled... it isn't as if it breaks my (non-library) binaries or anything. I would much prefer for the outdated libraries to be somehow recompiled automagically (or even better, to notify me about such a need) so I don't have to hunt down every outdated ocaml library myself whenever I upgrade. That revdep-rebuild thingy that George suggests in comment #2 sounds reasonable, although I have no idea what it really is, and have never actually seen it ;).
>(or even better, to notify me about such a need) Ok, *that* can be done trivially: just add a pkg_postinst function and make it spit out a few lines with ewarn "new version of ocaml has been installed" ewarn ", please recompile all your caml libraries!" or some such. It should also be possible to check on whether this is a new version, just a -rX update or a new install (autoclean takes place afterwards). As for automatizing this, PDEPEND does not seem to do the thing unfortunately, so I'll probably try to pester a few people in case I did something wrong :). Otherwise you can really tru revdep-rebuild. I personally hadn't need to use it so far, but from what I know it should do the thing. Try revdep-rebuild -h. It comes either with portage or gentoolkit. George
I meant that it should find the libraries that need reemerging and give me a list... hunting them down manually is some pain. But it should be automated enough to just issue a single command to reemerge them all by itself.
After giving it some thought, I came to the conclusion that these all should be reemerged automatically when a new ocaml is built. ``Gentoo philosophy'' is all good and true, but we don't want any dysfunctional libraries floating around, either. Issuing a warning before compiling ocaml should be sufficient.
First, i'm ready to support multiple ocaml versions on the same machine, and i want to do it so that it's not a nightmare for regular users. I actually have a script that allows to recompile all libraries on ocaml-compiler updates (given the compiler use the ocaml-config scheme and the packages were built using the new scheme). I updated the my overlay and the ocaml-overlay.tar.gz on dev.gentoo.org/~mattam to include documented eclasses and the aforementioned script. To test, just emerge the two different ocaml versions, use ocaml-config to set your prefered one (say 3.07), emerge some packages, then switch versions and run the script like this: ocaml-rebuild.sh 3.07 -pv, ocaml-rebuild.sh 3.07. It should be possible to adapt the script for the transition, i'll do if you're satisfied with the current solution.
Forgot to say, i also added some einfos in dev-lang/ocaml ebuild's post_insts that describe the procedure.
As i see no responses and that ocaml-3.07 is now required by mldonkey (which is the most used ocaml app in portage i think), i will move ocaml-3.07 to stable with my script, just tailored for the 3.06 -> 3.07 transition. Hopefully when 3.08 will come to stable multilib will be implemented in portage (i'm gonna try to work on it at least :) and the transition will be cleaner.
Some good news ! The dynamic slotting mechanism that has been discussed (by me and some others, although not enough to make it a GLEP) on gentoo-dev recently and will be implemented by jstubbs in the foreseable future will solve all problems with ocaml and it's libraries we discussed here. I'm marking the bug LATER, hoping i'll close it next time...
