Bugzilla Bug 182028

Inside portage, the ebuilds for CVS/SUBVERSION version of software are labeled
as *-9999.ebuild wich is someway 'OK'. The matter is when you sync portage's
data, and you do an update deep world, as it checks the ebuild's version, it
will never update all theese *-9999 packages installed.

Exposing my idea: How about implementing this inside the 'deep' parameter of
emerge? :

Have a copy of work directory for each -9999 package in /var/tmp/portage/...
and when you do an emerge --update --deep world, has a cvs checkout (or
equivalent) and somewhat like a monitor to see if this workdir has changed. If
so, reemerge this -9999 version with new version in workdir (something has
changed in CVS) instead of doing it manually.

Of course, do this optionally, because of that I suggested to implement in deep
option of emerge. (Or maybe creating another option like '--update-cvs').

I don't know if it is easy or hard or viable to do, but I wanted to suggest it
as I think suggestions are always good :)

Thanks.

Reproducible: Always

(In reply to comment #0)
> Have a copy of work directory for each -9999 package in /var/tmp/portage/...
> and when you do an emerge --update --deep world, has a cvs checkout (or
> equivalent) and somewhat like a monitor to see if this workdir has changed. If
> so, reemerge this -9999 version with new version in workdir (something has
> changed in CVS) instead of doing it manually.

This would be best implemented as some time of EAPI extension.  We'd like to
add an optional src_fetch() phase for ebuilds like this to do their fetching. 
We can create a mechanism for this phase to notify the package manager about
the latest revision that is currently available.

Another thing that I'd like to see is the ability to have multiple branches in
different slots.  A -9999 suffix implies that there might only be support for a
single slot, but that seems like it would be an artificial limitation.

Hello,

I and zmedico have discusted another approach:

We could add the current VCS revision as metadata of the just emerged
9999-versioned package.

Emerge would need another optional command switch (just as --newuse, maybe
called --devel) that would enable the dependency calculation process of the
9999-versioned ebuilds to connect to the VCS, read the current version, compare
it with the already installed package's metadata, and if not equal it will set
the package as upgradeable.

I guess this would be inexpenssive, because there are relatively few
VCS-packages, and the revision fetch would need just a few bytes.

(In reply to comment #2)
> We could add the current VCS revision as metadata of the just emerged
> 9999-versioned package.

Not all SCMs are 'revision'ed, such as DARCS. Don't get me wrong, all can be
calculated using various methods after updating, but it isn't always as simple
as asking the SCM's server if there's a newer version.

> ..
> it with the already installed package's metadata, and if not equal it will set
> the package as upgradeable.

A couple of interesting notes about this method is people may only want their
SCM packages updated only so often, for example, for high traffic SCMs, such as
for a package for mozilla-firefox which users may not want to update daily, but
maybe once a week, or maybe even two.

I scripted such a device, update-live-ebuilds (see the forums) which detects
these updates and acts accordingly to whatever the user wants to do. It
supports paludis, portage, pkgcore; darcs, cvs, mercurial, svn, and git. It
supports asking the user before installation, deferred installs, and masking
(and actually much more that I won't go into here). 

This is not saying I don't want such a device in portage, quite the opposite,
I'm simply saying it maybe more trouble than first realized.

could we see your script, maybe would be worth inclusion in gentoolkit

(In reply to comment #4)
> could we see your script, maybe would be worth inclusion in gentoolkit

Well, the git repository is at hosted at repo.or.cz, a review is always
welcome.

I don't think it is a good idea to include in gentoolkit, as it's constantly
being updated to be lighter and better and requires configuration files (does
any gentoolkit stuff support that?). OTOH, I wouldn't mind an official ebuild. 
If people want to use update-live-ebuilds, they already use some type of SCM
and don't mind installing GIT to install ULE. This is convenient for me, as
when I'm ready for world testing, I simply merge unstable to master and people
update when using update-live-ebuilds itself :).

There are a few minor fixes in the unstable branch, there is also some broken
stuff which is why it's not merged back yet (and I haven't had time to fix it
up this week). One of the fixes is much better commenting on what's going on
inside, so this is probably review-worthy, but the broken stuff can't be
ignored, obviously. OTOH; the stable branch is super stable but doesn't yet
include the direct-to-SCM talking stuff or the commenting yet that I'm working
on in unstable.

ULE can be installed from the MPD overlay, or the ebuild is also in the git
repository below (under the docs/ directory).

gitweb: http://repo.or.cz/w/ule.git
git: git://repo.or.cz/ule.git

Comments / Flames welcome.

In terms of implementing multiple slots, you could just use the existing
-cvs[0-9] extension, which can be followed by any version spec you want. This
was discussed in a Council meeting wrt -scm suffix; the extension has been part
of portage for ages, and the package manager always knows it's dealing with a
vcs ebuild (it has to for the version ordering.)

(In reply to comment #6)
> it has to for the version ordering.

Funny thing about that (I do hope I'm not misunderstanding the syntax)....

[dleverton@shiny-one ~] $ python
Python 2.4.4 (#1, Oct 27 2007, 11:37:00)
[GCC 4.1.2 (Gentoo 4.1.2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from portage_versions import vercmp
>>> vercmp("cvs.1.5", "2.0")
1
>>>

(In reply to comment #7)
> (In reply to comment #6)
> > it has to for the version ordering.
> 
> Funny thing about that (I do hope I'm not misunderstanding the syntax)....
>
You appear to be misinterpreting the result.

> >>> from portage_versions import vercmp
> >>> vercmp("cvs.1.5", "2.0")
> 1
> 
Yes, a cvs version is always considered newer than any non-cvs version so it's
returned a positive integer. So it has correctly considered the version
information given (no output would indicate that the versions weren't
conformant.)

Changing this to having version 2.0 compare greater than cvs.1.5, is something
that would have to be discussed with a wider audience (and completely flies
against the behaviour of -9999 ebuilds which are well understood.) The
intention of having a vcs build is to keep current with a project; extending
this into branching is a bit beyond the standard use-case. A user tracking a
project in vcs would imo know when to switch back to the standard ebuilds if
that should be needed.

What I meant wrt multiple slots was that you could have cvs.1 and cvs.2 each
with a different SLOT value.

(In reply to comment #8)
> Changing this to having version 2.0 compare greater than cvs.1.5, is something
> that would have to be discussed with a wider audience (and completely flies
> against the behaviour of -9999 ebuilds which are well understood.) The
> intention of having a vcs build is to keep current with a project; extending
> this into branching is a bit beyond the standard use-case. A user tracking a
> project in vcs would imo know when to switch back to the standard ebuilds if
> that should be needed.

media-sound/amarok-1.4.9999-r2, so at least one other person thinks supporting
branches is a good idea.  If this was changed to a cvs.1.4 or the like, it
would completely break anything that had a dependency on amarok 2 or greater.

> What I meant wrt multiple slots was that you could have cvs.1 and cvs.2 each
> with a different SLOT value.

And what would be the point of that, unless the two ebuilds pulled from
different branches?

(In reply to comment #9)
> > What I meant wrt multiple slots was that you could have cvs.1 and cvs.2 each
> > with a different SLOT value.
> 
> And what would be the point of that, unless the two ebuilds pulled from
> different branches?
> 
That is the point; wake up and please stop spamming when you haven't even read
the discussion properly.

(In reply to comment #10)
> That is the point; wake up and please stop spamming when you haven't even read
> the discussion properly.

And with two ebuilds with .cvs. versions pulling from different branches, say
1.4 and 2.0, how does an ebuild depend upon anything 2.0 or newer? That was the
point in his question; wake up and please stop attacking people when you
haven't read their comments properly.

(In reply to comment #10)
> That is the point; wake up and please stop spamming when you haven't even read
> the discussion properly.

If that's the kind of contribution you're going to make, we're no longer
interested in having your contributions on PMS bugs. Please refrain from
posting any further.

(In reply to comment #11)
> And with two ebuilds with .cvs. versions pulling from different branches, say
> 1.4 and 2.0, how does an ebuild depend upon anything 2.0 or newer? That
> was the point in his question.

Thanks for the explanation; it was necessary. I believe SLOT deps would cover
that use case, although perhaps some minor UI modifications would be needed in
terms of display.

It might be useful, in maintenance terms, to make the restriction that SLOTs
within a CP (per-repo) must order asciibetically (if it's not already
specified.) As for feature branches (multiple within same slot) adding a label
would cover that.

(In reply to comment #13)
> (In reply to comment #11)
> > And with two ebuilds with .cvs. versions pulling from different branches,
> > say 1.4 and 2.0, how does an ebuild depend upon anything 2.0 or newer? That
> > was the point in his question.
> 
> Thanks for the explanation; it was necessary. I believe SLOT deps would cover
> that use case, although perhaps some minor UI modifications would be needed in
> terms of display.

Compatibility and SLOTs do not always overlap. 1.4 could (theoretically) be
incompatible with 2.0 yet still both using the same SLOT.

What's wrong with having a version like 1.4-live and 2.0-live?

As far as per-week vs per-day vs per-revision updates, my live.eclass
(armagetron overlay) uses an environment variable to let the user decide when
to update.

> What's wrong with having a version like 1.4-live and 2.0-live?
>
Nothing at all. I don't have any objections to -live or w/e else you want to
call this anymore (fwtw.) I accept that cvs[0-9]+ doesn't give arbitrary
versions, and that people want this capability, which is also more transparent
to upstream.

> As far as per-week vs per-day vs per-revision updates, my live.eclass
> (armagetron overlay) uses an environment variable to let the user decide when
> to update.
> 
Cool; per-pkg config can easily be done already to pass that (or any other)
variable to ebuild.sh, so the user will be able to "set and forget" to a
certain extent.