Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 400659

Summary: perl-core/Module-Build and dependencies on it are unnecessary
Product: Gentoo Linux Reporter: Patrick <gentoo>
Component: [OLD] DevelopmentAssignee: Gentoo Perl team <perl>
Status: RESOLVED INVALID    
Severity: major    
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---

Description Patrick 2012-01-25 00:37:43 UTC
I was looking to install dev-perl/DateManip-5.56 but was shocked at the number of dependencies a simple pure-perl module wanted to pull in (was over 30). I copied the ebuild into my local portage overlay, ripped out the dependency on virtual/perl-Module-Build and installed the package. Installed without a single dependency needed, and it works just fine.

The virtual/perl-Module-Build wants to pull in perl-core/Module-Build to provide Module::Build. This is completely unnecessary as Module::Build is already a part of the base perl install.

# qfile /usr/lib/perl5/5.12.3/Module/Build.pm
dev-lang/perl (/usr/lib64/perl5/5.12.3/Module/Build.pm)


There is more though. During the install of my package (the DateManip package with the ripped out dependency), it prints out the following warning:
 * Using Module::Build
 * QA Notice: The ebuild uses Module::Build but doesn't depend on it.
 *            Add virtual/perl-Module-Build to DEPEND!

This indicates that something in portage is broken; assuming that Module::Build doesnt exist without the package virtual/perl-Module-Build.



I did a quick look through all the packages in dev-perl, and there are 158 packages depending on perl-Module-Build. To me this indicates a significant problem in the way the perl dependency system is implemented in portage if all these packages depend on another package that is really not required.





# emerge --info

Portage 2.1.10.11 (hardened/linux/amd64/no-multilib/selinux, gcc-4.4.5, glibc-2.12.2-r0, 3.0.4-hardened-r1 x86_64)
=================================================================
System uname: Linux-3.0.4-hardened-r1-x86_64-Intel-R-_Atom-TM-_CPU_D525_@_1.80GHz-with-gentoo-2.0.3
Timestamp of tree: Tue, 24 Jan 2012 13:00:01 +0000
app-shells/bash:          4.1_p9
dev-lang/python:          2.7.1-r1, 3.1.3-r1
dev-util/cmake:           2.8.4-r1
dev-util/pkgconfig:       0.26
sys-apps/baselayout:      2.0.3
sys-apps/openrc:          0.8.3-r1
sys-apps/sandbox:         2.4
sys-devel/autoconf:       2.68
sys-devel/automake:       1.10.3, 1.11.1
sys-devel/binutils:       2.21.1-r1
sys-devel/gcc:            4.4.5
sys-devel/gcc-config:     1.4.1-r1
sys-devel/libtool:        2.4-r1
sys-devel/make:           3.82-r1
sys-kernel/linux-headers: 2.6.36.1 (virtual/os-headers)
sys-libs/glibc:           2.12.2
Repositories: gentoo x-portage
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe -march=native -fomit-frame-pointer"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/gentoo-release /etc/php/apache2-php5.3/ext-active/ /etc/php/cgi-php5.3/ext-active/ /etc/php/cli-php5.3/ext-active/ /etc/sandbox.d /etc/terminfo"
CXXFLAGS="-O2 -pipe -march=native -fomit-frame-pointer"
DISTDIR="/usr/portage/distfiles"
FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles fixpackages news parallel-fetch protect-owned sandbox selinux sesandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch"
FFLAGS=""
GENTOO_MIRRORS="http://gentoo.mirrors.easynews.com/linux/gentoo http://mirrors.tds.net/gentoo http://ftp.gtlib.cc.gatech.edu/pub/gentoo"
LANG="en_US.utf8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="acl acpi amd64 apache2 berkdb bzip2 cli cracklib crypt cups cxx dbus dri gdbm gpm hardened iconv imap ipv6 justify lm_sensors mmx modules mudflap mysql ncurses nls nptl nptlonly open_perms openldap openmp pam pax_kernel pcre posix pppd readline sasl selinux session smp sse sse2 ssl sysfs syslog tcpd threads udev unicode urandom xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="alias auth_basic autoindex cache deflate dir env filter headers include info log_config mime mime_magic status setenvif userdir vhost_alias authz_host rewrite auth_digest authn_file authn_default authn_dbd authz_default authz_user" CALLIGRA_FEATURES="kexi words flow plan stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel mach64 mga neomagic nouveau nv r128 radeon savage sis tdfx trident vesa via vmware dummy v4l" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS


Reproducible: Always
Comment 1 Jeroen Roovers (RETIRED) gentoo-dev 2012-01-25 14:58:51 UTC
It looks like you ought to update your system - your dev-lang/perl is out of date.
Comment 2 Patrick 2012-01-26 03:28:28 UTC
perl-5.12.3-r1 is still in portage. However just for gits and shiggles I upgraded to the latest stable, 5.12.4-r1. Problem still exists


# emerge -pv =dev-perl/DateManip-5.56

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild  N     ] perl-core/version-0.940.0  103 kB [0]
[ebuild  N     ] virtual/perl-Test-Harness-3.17  0 kB [0]
[ebuild  N     ] virtual/perl-Archive-Tar-1.54  0 kB [0]
[ebuild  N     ] perl-core/ExtUtils-CBuilder-0.27.03  30 kB [0]
[ebuild  N     ] perl-core/Perl-OSType-1.2.0  USE="-test" 12 kB [0]
[ebuild  N     ] perl-core/JSON-PP-2.272.0  41 kB [0]
[ebuild  N     ] perl-core/CPAN-Meta-YAML-0.4.0  29 kB [0]
[ebuild  N     ] virtual/perl-ExtUtils-MakeMaker-6.56  0 kB [0]
[ebuild  N     ] perl-core/Test-Simple-0.980.0  105 kB [0]
[ebuild  N     ] perl-core/IO-1.25  52 kB [0]
[ebuild  N     ] virtual/perl-File-Temp-0.220.0-r1  0 kB [0]
[ebuild  N     ] virtual/perl-version-0.940.0  0 kB [0]
[ebuild  N     ] virtual/perl-ExtUtils-CBuilder-0.27.03  0 kB [0]
[ebuild  N     ] virtual/perl-JSON-PP-2.272.0  0 kB [0]
[ebuild  N     ] virtual/perl-CPAN-Meta-YAML-0.4.0  0 kB [0]
[ebuild  N     ] virtual/perl-Perl-OSType-1.2.0  0 kB [0]
[ebuild  N     ] virtual/perl-Test-Simple-0.980.0-r1  0 kB [0]
[ebuild  N     ] virtual/perl-IO-1.25  0 kB [0]
[ebuild  N     ] perl-core/ExtUtils-ParseXS-2.22.05  40 kB [0]
[ebuild  N     ] perl-core/Module-Metadata-1.0.6  23 kB [0]
[ebuild  N     ] perl-core/Parse-CPAN-Meta-1.440.100  8 kB [0]
[ebuild  N     ] perl-core/Version-Requirements-0.101.20  15 kB [0]
[ebuild  N     ] virtual/perl-Parse-CPAN-Meta-1.440.100-r2  0 kB [0]
[ebuild  N     ] virtual/perl-ExtUtils-ParseXS-2.22.05  0 kB [0]
[ebuild  N     ] virtual/perl-Module-Metadata-1.0.6  0 kB [0]
[ebuild  N     ] virtual/perl-Version-Requirements-0.101.20-r2  0 kB [0]
[ebuild  N     ] perl-core/CPAN-Meta-2.112.621  75 kB [0]
[ebuild  N     ] virtual/perl-CPAN-Meta-2.112.621  0 kB [0]
[ebuild  N     ] perl-core/Module-Build-0.380.0  299 kB [0]
[ebuild  N     ] virtual/perl-Module-Build-0.380.0-r2  0 kB [0]
[ebuild     UD ] dev-perl/DateManip-5.56 [5.56-r1] USE="-test" 0 kB [1=>0]

(the DateManip-5.56-r1 that its saying its downgrading from is my modified version without the dependency)
Comment 3 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2012-01-26 04:58:52 UTC
(In reply to comment #0)
> I was looking to install dev-perl/DateManip-5.56 but was shocked at the number
> of dependencies a simple pure-perl module wanted to pull in (was over 30). I
> copied the ebuild into my local portage overlay, ripped out the dependency on
> virtual/perl-Module-Build and installed the package. Installed without a single
> dependency needed, and it works just fine.
> 

> I did a quick look through all the packages in dev-perl, and there are 158
> packages depending on perl-Module-Build. To me this indicates a significant
> problem in the way the perl dependency system is implemented in portage if all
> these packages depend on another package that is really not required.
> 

Underneath it all, this is because Date-Manip doesn't come with one, but 2 install toolkits.  ( And this is a very common situation for perl modules )

https://metacpan.org/source/SBECK/Date-Manip-6.30

You'll see Build.PL and Makefile.PL here, Makefile.PL being an "older" install tool ( ExtUtils::MakeMaker ) and the Build.PL being the newer ( Module::Build )

I'm not /excusing/ the situation, but we do things this way because upstream can be a bit flippant and just silently remove Makefile.PL in a future release. ( At which point having Module::Build becomes mandatory, not optional )

The proportion of modules that presently depend on virtual/perl-Module-Build will thus fall into 2 categories:

a) Modules that *do* actually need Module::Build, and without will fail to install.
b) Modules that *dont* actually need Module::Build , and can fallback to ExtUtils::MakeMaker 

To make it more annoying, the presence of Makefile.PL is meaningless in itself, because sometimes Makefile.PL is an EUMM Script, and other times its merely a compatibility shim, which just provides an EUMM-like interface , but using Module::Build code.

So, its simpler for upstream, for management reasons, to just see a distribution that has Module::Build as one of 2 options, and make Module::Build a mandatory requirement to make the installation consistent across all Gentoo, otherwise there are possibly differences in behavior when using MB vs EUMM , and having this change happen automatically based on the availability of MB sounds dangerous. 

( And on Perl/CPAN itself, Distributions that ship with Module::Build also specify that Module::Build is a mandatory dependency that must be installed prior to even *trying* to build the distribution , and any legacy ExtUtils::MakeMaker back-compat exists mostly so people who have really, really, really old CPAN clients that don't even *see* the "you need module::build" hint  don't just become a confusing ball of flame when people install it ).

In summary, when a distribution ships with Module::Build, its fair to say "Upstream want their distribution built with Module::Build, but they have a few other options if you're really in a bind because you're on a server that hasn't had an upgrade this decade" , so really, we're just forwarding upstreams installation requirements in the most sensible way we can to portage.

> The virtual/perl-Module-Build wants to pull in perl-core/Module-Build to
> provide Module::Build. This is completely unnecessary as Module::Build is
> already a part of the base perl install.
> 
> # qfile /usr/lib/perl5/5.12.3/Module/Build.pm
> dev-lang/perl (/usr/lib64/perl5/5.12.3/Module/Build.pm)
> 
> 
> There is more though. During the install of my package (the DateManip package
> with the ripped out dependency), it prints out the following warning:
>  * Using Module::Build
>  * QA Notice: The ebuild uses Module::Build but doesn't depend on it.
>  *            Add virtual/perl-Module-Build to DEPEND!
> 
> This indicates that something in portage is broken; assuming that Module::Build
> doesnt exist without the package virtual/perl-Module-Build.
> 

Not really portage so much, the virtual is just there to say "yep, we have Module::Build", and that warning is thrown up by the perl-module eclass if it finds a Build.PL in the distribution.


The core issue why perl-core/Module-Build is required boils down to version specifics. Some modules require specific versions of Module::Build, which could come from either perl itself or a perl-core/ package, and the virtual has to handle this.

If A module requires Module::Build 0.38 , you only have one of 2 choices: upgrade to Perl >5.13.11 , or install Module::Build from CPAN. 

As we have modules in stable that require 0.38 , the respective virtual for 0.38 must exist, and be installed. ( And if you don't like that, you can of course mask virtual/perl-Module-Build-0.380.0 or even perl-core/Module-Build  in entirety, but expect to see a lot more dependencies fail due to that. 

However, The other 2 stable virtuals other problems:

0.36.07 *must* depend on perl-core/Module-Build, because no version of perl we ship contains that version of Module::Build ( only perls 5.13.3 to 5.13.9 have this version )

And 0.34.0201 will only not prompt to install perl-core/Module-Build if you're running perl 5.10.1 , because thats the only version of Perl that has that specific version.

My suggestion for your solution is as follows:

a) NOT remove the dependency on virtual/perl-Module-Build 

b) Create =virtual/perl-Module-Build-0.36.03 with a dependency of 


|| (  ~dev-lang/perl-5.12.4 ~dev-lang/perl-5.12.3 =perl-core/Module-Build-${PV} )

c) user mask >virtual/perl-Module-Build-0.36.03 

This will tell portage that the aforementioned versions of perl satisfy the requirement of providing Module::Build 0.3603 , and it will happily not even try installing perl-core/Module-Build ( unless something else pulls it ) , and when you at some stage in the future upgrade past 5.12.3, it will keep the version of Module::Build at 0.3603 as long as possible.
Comment 4 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2012-01-26 05:07:34 UTC
(In reply to comment #2)
> 
> Calculating dependencies... done!
> [ebuild  N     ] perl-core/version-0.940.0  103 kB [0]
> [ebuild  N     ] virtual/perl-Test-Harness-3.17  0 kB [0]
> [ebuild  N     ] virtual/perl-Archive-Tar-1.54  0 kB [0]
----
> [ebuild  N     ] perl-core/Module-Build-0.380.0  299 kB [0]
> [ebuild  N     ] virtual/perl-Module-Build-0.380.0-r2  0 kB [0]
> [ebuild     UD ] dev-perl/DateManip-5.56 [5.56-r1] USE="-test" 0 kB [1=>0]
> 
> (the DateManip-5.56-r1 that its saying its downgrading from is my modified
> version without the dependency)

Also should point out that portage does something CPAN doesn't/can't, and tries to keep all dependencies up to date until informed otherwise. All these "New" dependencies are essentially "Updates", things that do exist in perl itself, but more recent versions are available to be installed separately ( and the virtual handle whether or not this happens ) 

You'll find there to be a lot in common in this list ( after removing virtual/* , which don't install anything ) with what you'll get if you run cpan-outdated  ( https://metacpan.org/module/cpan-outdated )

You can of course /block/ all these updates from happening, but portage will tell you if the result of blocking those updates results in bad things happening. 

When a CPAN dist specifies "needs Module::Build", CPAN just checks "haz module build? yes => no update, no => install module build", regardless of whether or not there are updates are available, regardless of whether or not the currently installed version might just accidentally be completely broken and *need* updating ( hence why Moose have to ship https://metacpan.org/source/DOY/Moose-2.0401/bin/moose-outdated  to let users know they've got old versions of things that are known broken , and we don't have to resort to such tactics )

The only time CPAN will ever update a distribution is when a dist specifies "need Module::Build vFoo" and the current version is less than Foo. 

Hope that helps explain what you're seeing more clearly.
Comment 6 Patrick 2012-01-26 14:25:59 UTC
I can see the reasoning behind your arguments, but is there no way to say that dev-lang/perl-5.12.4-r1 is a provider of virtual/perl-Module-Build-0.3603? I know you said I can create this virtual myself, but it seems to me the proper way to handle this situation would be some automated scan through dev-lang/perl to see what versions of modules it includes, and then provide the virtual packages for those module versions.
Comment 7 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2012-01-26 16:13:23 UTC
(In reply to comment #6)
> I can see the reasoning behind your arguments, but is there no way to say that
> dev-lang/perl-5.12.4-r1 is a provider of virtual/perl-Module-Build-0.3603? I
> know you said I can create this virtual myself, but it seems to me the proper
> way to handle this situation would be some automated scan through dev-lang/perl
> to see what versions of modules it includes, and then provide the virtual
> packages for those module versions.

You could probably toy with package.provided and just lie to portage and say that the specified virtual is installed, but that'll break lots. 

The easiest solution here is to just create that virtual, and the only reason its not probably already in portage is nobody thought somebody would /want/ to hold back Module::Build to that version.

And as far as automation goes, 'corelist' is really all thats necessary. 

However, a lot of the modules Perl provides are *not* available independently on CPAN, and as there are so very many of them, there are no virtuals or perl-core/* for them at all. ( Though I believe upstream are changing things more so more perl bits are independently available ( dual-life ) , so this may trend towards an increase ).

I think the only way Gentoo can realistically make this problem nicer for people like you is adding the required virtual for people who don't want to upgrade Module::Build past perl stock versions. ( And I think it should be considered a "bug" that there are versions of Perl in Gentoo without a virtual pointing to it for any included module that is already using virtualing )

( I myself don't understand why it is you don't want the latest stable versions of everything, but I hear its a reasonably common request in production environments )
Comment 8 Patrick 2012-01-28 07:38:40 UTC
I just think the 'proper' way is that if perl satisfies a dependency, it should get credit for it. Base perl includes a lot of modules, some of them may not have any updates at all, so it avoids having to install unnecessary packages.
Comment 9 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2012-01-29 00:25:09 UTC
(In reply to comment #8)
> I just think the 'proper' way is that if perl satisfies a dependency, it should
> get credit for it. Base perl includes a lot of modules, some of them may not
> have any updates at all, so it avoids having to install unnecessary packages.

"I just think the 'proper' way is that if perl satisfies a dependency, it should
 get credit for it."

Yes, and this is what the virtuals exist for. Without virtuals, there is no way for perl to tell packages what it provides, and no way for packages to know who provides what.  Virtuals are the glue that hold this together. 

Virtuals are simply an empty package that has a version, which maps to other packages. 

virtual/foo-8.0 depending on  x-0.1 and y-0.2 means that 

"if you want foo version 8.0, you need x-0.1 or y-0.2"

And before you suggest "dev-lang/perl should handle this", keep in mind we often don't know what "provides" records we need to create until after the fact, and we often need to update provide records long after we create each version of dev-lang/perl.

Having this handled by a virtual makes the work involved minimal, but if this were to be handled by dev-lang/perl, then each time we changed this, you'd need to reinstall/recompile perl.

Virtuals are a pain, but we don't have nay good alternatives at present. 

05:17:02 < kent\n> tove: summary of bug 4000659 is "dont want to upgrade everything, but where is the virtual that says 5.12.4 actually has module::build in it? it seems to be missing" 
05:17:19 < kent\n> bug 400659  
05:17:22 < willikins> kent\n: https://bugs.gentoo.org/400659 "perl-core/Module-Build and dependencies on it are unnecessary"; Gentoo Linux, Development; UNCO; gentoo:perl
05:18:21 < kent\n> theres virtuals for 5.10.1 and 5.14* , but nothing for 5.12* 
05:18:38 < kent\n> which means /all/ stable users are getting forced to install perl-core/Module-Build
Comment 10 Kent Fredric (IRC: kent\n) (RETIRED) gentoo-dev 2013-07-22 16:48:37 UTC
Just an update for anyone else viewing this bug ( or suggesting the same again at a later date ):

Module::Build is now for-certain to be removed from a future Perl release, so there is more reason to depend on the virtual than ever before.

And the current heuristics we use for this QA and codepath is now invalid, because Build.PL is no longer an indication of Module::Build, but can instead be an indicator of Module::Build::Tiny, which has more deps not-in-core, and a slightly different interface ( the pure_install target is missing )
Comment 11 Mikle Kolyada (RETIRED) archtester Gentoo Infrastructure gentoo-dev Security 2013-07-25 22:24:48 UTC
(In reply to Kent Fredric from comment #10)

> Module::Build is now for-certain to be removed from a future Perl release,
> so there is more reason to depend on the virtual than ever before.
> 
> And the current heuristics we use for this QA and codepath is now invalid,
> because Build.PL is no longer an indication of Module::Build, but can
> instead be an indicator of Module::Build::Tiny, which has more deps
> not-in-core, and a slightly different interface ( the pure_install target is
> missing )


Yeah. I agree, it's correct. We should not change anything (I mean Module::Build packages and dependencies on it).