Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 608168 - <portage>/dev-perl/ & perl-core/ metadata description lacks accuracy.
Summary: <portage>/dev-perl/ & perl-core/ metadata description lacks accuracy.
Status: CONFIRMED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo Perl team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-02-04 02:20 UTC by Teika kazura
Modified: 2017-07-04 03:58 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Teika kazura 2017-02-04 02:20:22 UTC
Both perl-core and dev-perl's metadata.xml give "longdescription", but their contents are the same, instead of being distinguishable; they read:
------------------------------------------------------------------------
The <category name> category contains libraries and utilities relevant to the
Perl programming language.
------------------------------------------------------------------------
As dilfridge explained [1] perl-core modules are the part of the Perl language itself, but distributed separately. I'm not sure about dev-perl, but I guess they come from third-party, or something like that.

[1] In the Jul 2014 post: http://dilfridge.blogspot.jp/2014/07/perl-in-gentoo-dev-langperl-virtuals.html

I submit the following draft for the description of perl-core:
------------------------------------------------------------------------
The perl-core category contains perl modules. Depending on the versions, they are included in the Perl language itself, but are also distributed separately. When a package in perl-core is installed, it takes precedence over the counterpart in dev-lang/perl.

Packages in perl-core should not be depended on directly, but instead via virtual/perl-*.
------------------------------------------------------------------------

Sorry for mixing a separate issue here, but I think virtual/perl-* should be collected under a new category perl-virt/. It's because updating dev-lang/perl is often a mess, and having perl-virt/ makes things more understandable to Gentoo users. (Search for "perl slot" or "conflicts" in Gentoo forum.) Its description can be:
------------------------------------------------------------------------
The perl-virt category contains virtual packages, allowing a choice between the module in perl-core/ and the one included in dev-lang/perl.
------------------------------------------------------------------------

Thanks Gentoo dev. Regards.
Comment 1 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2017-02-04 02:52:22 UTC
(In reply to Teika kazura from comment #0)
> Both perl-core and dev-perl's metadata.xml give "longdescription", but their
> contents are the same, instead of being distinguishable; they read:
> ------------------------------------------------------------------------
> The <category name> category contains libraries and utilities relevant to the
> Perl programming language.
> ------------------------------------------------------------------------

This is a technically correct statement, in fairness. The details of their differences
is kinda lengthy and hard to explain succinctly.

> As dilfridge explained [1] perl-core modules are the part of the Perl
> language itself, but distributed separately. I'm not sure about dev-perl,
> but I guess they come from third-party, or something like that.

(expanding on [1]) 

stuff in perl-core is stuff that is "also shipped in perl itself", and by default
where possible, we prefer to use Perl itself.

perl-core basically exists so we can get these facets of perl in versions newer than
those shipped with perl, without necessitating we patch perl itself, and without
necessitating we create complex problems stemming from the missmatch we're constantly
fighting where upstream uses /module/ based dependency specifications while portage
can only handle /package/ based dependency specifications, and the module<=>package relationship
is many-to-one, and there's no guarantees that the module will even have the same version number as the containing package does.

It is also worth mentioning that sometimes, things in perl-core/ get "deprecated in perl",
so it stops shipping with dev-lang/perl, and we then switch to shipping that component "From CPAN"
by relocating it to dev-perl/ and fixing the necessary deps ( and then nuking the related virtual )

A good example of this was Module::Build, which shipped first with perl 5.9, and was later removed from perl 5.21+

In this interim, we fortunately had a virtual/perl-Module-Build, that packages that needed that depended on. 

Whereas most people who think only in "package" dependencies would have only depended on dev-lang/perl itself, which would have created a really fun problem for us to solve when it got removed from perl 5.21, due to all the things that would have silently been requiring a thing
but not depending on it!

Fortunately, we only had to pkgmove perl-core/Module-Build dev-perl/Module-Build, tweak virtual/perl-Module-Build to point to dev-perl/Module-Build, and then run sed over the package
tree replacing "virtual/perl-Module-Build" with "dev-perl/Module-Build", which was much preferable to "install everything in portage and see what breaks"

So in short really, our use of virtual/ and perl-core/ is a process that gives us leverage
and granularity. 


> I submit the following draft for the description of perl-core:
> ------------------------------------------------------------------------
> The perl-core category contains perl modules. Depending on the versions,
> they are included in the Perl language itself, but are also distributed
> separately. When a package in perl-core is installed, it takes precedence
> over the counterpart in dev-lang/perl.
> 
> Packages in perl-core should not be depended on directly, but instead via
> virtual/perl-*.

I accept this draft is mostly correct, and is an enhancement over what stands.

I'm detail oriented though and I think it could consider borrowing more elements
of my longer description above, but I'm not good at doing so succinctly :)


> Sorry for mixing a separate issue here, but I think virtual/perl-* should be
> collected under a new category perl-virt/. It's because updating
> dev-lang/perl is often a mess, and having perl-virt/ makes things more
> understandable to Gentoo users. (Search for "perl slot" or "conflicts" in
> Gentoo forum.) Its description can be:
> ------------------------------------------------------------------------
> The perl-virt category contains virtual packages, allowing a choice between
> the module in perl-core/ and the one included in dev-lang/perl.
> ------------------------------------------------------------------------

Personally, I would object to this, mostly because of the large scale work it would involve,
and it would prove negligible benefit to users or portage, and would create measurable negatives
due to how pervasively they are used.

And it wouldn't fix the core issue that exists, that portage has problems handling slot conflicts.

I have a partial solution to this problem that gets you past this without needing @world upgrades,
which in part should reduce the number of slot-conflicts you have interacting with each other during an upgrade.

   emerge -va app-admin/gentoo-perl-helpers

Here there's a simple tool you can use to see all the perl virtuals you have installed
without needing to be a wizard:

   gentoo-perl installed-perl-virtuals

And if you're doing a major perl upgrade (eg: 5.20.* to 5.22.*)

   gentoo-perl installed-perl-virtuals 5.20 5.22 

Assuming run as root, Will query some portage wizardry
and generate an upgrade set in /etc/portage/sets 

Which, after waiting some time for it to complete, 
can be installed with

    emerge -va1 -k n @perl-upgrade

Which so far gives a better upgrade plan than

    emerge --uvatDN @world

I know this is also diverging into off-topic stuff though :) Its just a newish tool and I need people to test it for me and tell me what breaks.

Though the core tip here is that

 portageq match / "virtual/perl-*"

Works just as well as a future

 portageq match / "perl-virt/*" 

Would.
Comment 2 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2017-02-04 02:55:45 UTC
> And if you're doing a major perl upgrade (eg: 5.20.* to 5.22.*)
> 
>    gentoo-perl installed-perl-virtuals 5.20 5.22 
> 

*CORRECTION*

gentoo-perl gen-upgrade-sets 5.20 5.22

gentoo-perl list-commands-desc # will list all the other options

gentoo-perl help gen-upgrade-sets # will give you other info that might prove helpful. 

Doc fixes there welcome too ;)
Comment 3 Teika kazura 2017-02-09 02:04:43 UTC
>> I submit the following draft for the description of perl-core:
>> ...
> I accept this draft is mostly correct, and is an enhancement over
> what stands.  I'm detail oriented though and I think it could
> consider borrowing more elements of my longer description above, but
> I'm not good at doing so succinctly :)

I understand you, but at the same time I think my draft's length is moderate for metadata.xml. To reconcile both, I created a page "Perl in Gentoo" in Gentoo Wiki, and put some explanation there, and also a link to your comment. I reckon it's a right direction.

I also launched a forum thread titled "Having trouble upgrading Perl? Use this thread!": https://forums.gentoo.org/viewtopic-p-8028098.html (I think it should be sticky.) In both, I've introduced your shining, brand-new tool and wrote basic usage. (It seems we both came in right time. :)

I wrote:
> I think virtual/perl-* should be collected under a new category perl-virt/

Sorry for my terrible English. I meant it "in an ideal world, it'd be better".


And yes, I tried gentoo-perl-helpers (0.1.0. Wow 0.2.0 is there), and it worked like a charm to upgrade 5.22 to 5.24. (In fact "emerge -p". First it was necessary to unmerge @perl-cleanup.)

Offtopic, about upgrading to unstable (~arch) Perl:
1. If gentoo-perl-helpers fails upgrading to ~arch Perl, is it a bug of gentoo-perl-helpers? Otherwise, it's better to explicitly write "We don't support upgrading to ~arch".

2. Are there an "official" way to upgrading to ~arch Perl? It seems that you have to append the output of:
$ echo "### Perl keywords"
$ eix '-#' dev-perl
$ eix '-#' perl-core
$ eix '-#' virtual/perl-
$ echo dev-lang/perl
to package.keywords. Because upgrading to ~arch Perl is also a repeated topic in the forum, (There're several threads on upgrading to 5.24.) it's better to document it somewhere. If Project:Perl doesn't prefer to be annoyed with reports by unexperienced users, it should be discouraged, and the instruction should be somewhere on a wiki subpage of Project:Perl. If it's ok, then I can write it on the normal wiki page and/or the forum thread.

Thank you very much for your elaborate replies and for your effort.
Comment 4 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2017-02-09 04:07:14 UTC
(In reply to Teika kazura from comment #3)
> Offtopic, about upgrading to unstable (~arch) Perl:
> 1. If gentoo-perl-helpers fails upgrading to ~arch Perl, is it a bug of
> gentoo-perl-helpers? Otherwise, it's better to explicitly write "We don't
> support upgrading to ~arch".

It *should* work, but if there are any problems with it, more details as to what's happening should be posted on the bug tracker.

There's a few more tools now for advanced users who stumble into any "More than one subslot conflict at a time" problem, but discovery and usage is left as a reading exercise atm :)

> 
> 2. Are there an "official" way to upgrading to ~arch Perl? It seems that you
> have to append the output of:
> $ echo "### Perl keywords"
> $ eix '-#' dev-perl
> $ eix '-#' perl-core
> $ eix '-#' virtual/perl-
> $ echo dev-lang/perl
> to package.keywords. Because upgrading to ~arch Perl is also a repeated
> topic in the forum, (There're several threads on upgrading to 5.24.) it's
> better to document it somewhere. If Project:Perl doesn't prefer to be
> annoyed with reports by unexperienced users, it should be discouraged, and
> the instruction should be somewhere on a wiki subpage of Project:Perl. If
> it's ok, then I can write it on the normal wiki page and/or the forum thread.
> 
> Thank you very much for your elaborate replies and for your effort.

I personally tested this tool myself by upgrading from "arch" to "~arch", and then with a little finangiling, *downgraded* from "~arch" to "arch", which is a miracle in itself. ( But it was a minimal chroot )

But it assumes you already have the relevant keywords set up:

For instance, if you try to coerce installing 5.24 when its not accepted by keywords, you'll just get ignored ( mostly because doing this ourselves is too complex )

So "upgrade" is "Unmask then run tool"
and "downgrade" is "mask, then cringe, then run tool"*

Though presently, to use the tool for a downgrade you need to be an expert, because you have to change the "cleanup" to "nuke all the virtuals"

At presently, the fact that the set upgrade thing takes 2 versions at all is mostly a deception that will be filled out later.

The logic at present is basically "if not build against <targetperl> then mark it for reinstall" and portage sucks up the difference.

There is also related articles here: 

https://wiki.gentoo.org/wiki/Project:Perl
https://wiki.gentoo.org/wiki/Project:Perl/dev-lang/perl

Due to how it works you can pretty much put any version other than the current one in the "target" field and it will create a set for you that references "all the perl things", and portage will under many conditions only try to reinstall the subset of that it feels necessary, and can be abused for other things :)

But yeah, its very painful working on this sort of thing where you've only got bash to run magic. I had to implement a light-weight key-value lookup system with awk already ....
Comment 5 Teika kazura 2017-07-03 23:12:12 UTC
Allow me pushing one more favor into this bug: Could you add something like
------------------------------------------------------------------------
einfo "When upgrading perl, you may encounter difficulty due to \"subslot conflicts\"."
einfo "See https://wiki.gentoo.org/wiki/Perl for instruction."
------------------------------------------------------------------------
to pkg_postinst of the dev-lang/perl ebuilds?
Comment 6 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2017-07-04 03:58:43 UTC
By the time a user sees that, it will be too late.... because they only see that message when they *dont* have the problem.

Its doable, but I don't see the point, its expecting too much of peoples memory-attention, and the writing is very much "if you have a problem", which they won't have, and so they are unlikely to check, and unlikely to remember.