Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 3450 - CPAN in Gentoo?
Summary: CPAN in Gentoo?
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: High normal (vote)
Assignee: Nicholas Jones (RETIRED)
URL:
Whiteboard:
Keywords:
: 3540 5474 (view as bug list)
Depends on:
Blocks: 4919
  Show dependency tree
 
Reported: 2002-06-06 06:41 UTC by Seemant Kulleen (RETIRED)
Modified: 2004-03-04 23:28 UTC (History)
17 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
ebuilder.pl (ebuilder.pl,5.53 KB, text/plain)
2002-07-09 16:53 UTC, Leon Brocard
Details
Simple generated ebuild: YAML-0.35.ebuild (YAML-0.35.ebuild,523 bytes, text/plain)
2002-07-09 16:53 UTC, Leon Brocard
Details
Complicated generated ebuild: AxKit-XSP-Util-1.6.ebuild (AxKit-XSP-Util-1.6.ebuild,653 bytes, text/plain)
2002-07-09 16:55 UTC, Leon Brocard
Details
VERY PRIMITIVE NOT WORKING EXAMPLE (gcpan.tgz,162.43 KB, application/octet-stream)
2002-07-15 20:10 UTC, Michael Cummings (RETIRED)
Details
g-cpan.pl (g-cpan.pl,4.25 KB, text/plain)
2002-11-07 05:52 UTC, J Robert Ray
Details
Update to jrray's model (gzpan,5.21 KB, text/plain)
2002-12-03 09:05 UTC, Michael Cummings (RETIRED)
Details
emerges on the fly (gzpan,5.74 KB, text/plain)
2002-12-06 10:11 UTC, Michael Cummings (RETIRED)
Details
cleaned up even more (thanks baz) (g-cpan,8.80 KB, text/plain)
2002-12-07 10:27 UTC, Michael Cummings (RETIRED)
Details
good version (g-cpan,8.79 KB, text/plain)
2002-12-07 16:08 UTC, Michael Cummings (RETIRED)
Details
Further amends to the script [mostly aesthetic] (cpan.pl,8.92 KB, text/plain)
2002-12-09 16:44 UTC, baz
Details
g-cpan.pl (g-cpan.pl,9.18 KB, text/plain)
2003-01-08 02:10 UTC, J Robert Ray
Details
Patch against my previous attachment. (nomd5.txt,1.43 KB, patch)
2003-01-08 23:22 UTC, J Robert Ray
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Seemant Kulleen (RETIRED) gentoo-dev 2002-06-06 06:41:54 UTC
Hi Carpaski,

Just wondering on the possibilities of including CPAN in Gentoo?  Issues to be
concerned about are the interaction of CPAN with the portage system for package
management and of course sandbox installations and system updates for post-install.
Comment 1 Seemant Kulleen (RETIRED) gentoo-dev 2002-07-09 10:14:50 UTC
mcummings, let's carry out the analysis discussion on here, so it's better
documented.
Comment 2 Michael Cummings (RETIRED) gentoo-dev 2002-07-09 10:48:01 UTC
This came up on the -user list this week, and it got me thinking about the
the ramifications of ebuild vs cpan. The below is a pasted email I sent to
Seemant 
 ---*snip*---
                                                                                
                                                                         
 "Problem" (not exactly, but it could get painfull after a while): When a 
 perl module is required for an installation, currently it requires that  
 a seperate ebuild file be built to support it. That ebuild file has a    
 finite life span since it points only to the direct download location    
 (generally at the primary cpan site). For perl users, this can           
 eventually lead to problems when modules are updated. In addition, many  
 modules have other dependancies, either to other perl modules or in some 
 cases to external programs/libraries (any of the image handling modules, 
 archive handling modules (tar, zip, arj, etc.), all of the secure socket 
 modules, etc.).                                                          
                                                                          
 My take on a solution: Avoid the ebuild completely. Perl is              
 distributed with a core set of modules, including the CPAN module. The   
 CPAN module is an interactive and command line accessible module that    
 allows the user to install the current stable version of a module,       
 and is configurable to follow all dependancies and download them as      
 well. It's like portage for perl. Actually, it's a ton like portage for  
 perl, and when I first started using gentoo my thought was "wow, this is 
 like the cpan module...only no interactive shell...". The cpan module    
 can be invoked to start a shell environment to install modules, or it    
 can be a one liner install. Depending on how you have configured cpan,   
 i.e. whether you told it to ask or to follow dependancies, whether to    
 perform tests, etc., it can do everything for you without requiring      
 monitoring/interaction. If an ebuild has a perl component that requires  
 a particular module, you would send that module name (i.e., 
Bundle::LWP,Net::FTP, etc.) to the cpan module 
as                                                                            
       perl -MCPAN -e 'install Net::FTP'   
          The beauty of using the CPAN module is that it will make sure your 
perl installation has all of the dependancies, and it will download from 
mirrors automatically. You can also use the cpan module to confirm not only 
that a version of the module is installed, but which version(in case your 
ebuild has a dependancy on foo-bar.1.4, not foo-
bar<=1.3).                                                                      
 The caveats: the cpan module must be configured at least once (it can always 
be reconfigured later). This configuration includes your target mirrors to use, 
favorite pager, ftp program, etc. It is easy to determine whether it has been 
configured yet or not (it creates a .cpan directory in your (root's generally) 
home directory where it performs all if its builds and stores module listings), 
and it should be possible to create a generic, written on the fly from local 
environment variables, config file (the actual config file is Configure.pm and 
is located in /usr/lib/perl5/5.6.1/CPAN/).                      
       Another caveat, and I throw this out only because it is possible, not 
because I would anticipate any ebuild having a dependancy on the latest version 
of cpan.pm, is that if you tell it to upgrade cpan.pm, it will also upgrade 
your perl version to the latest release (and bypass the emerge -u sys-
devel/perl completely). I am only aware of it upgrading perl during an update 
to the cpan module, however.
                                      
 Does that make any more sense? From what I know, it should  be pretty 
straightforward/simple to set up an eclass to call perl -MCPAN and install the 
required module, but it will require that the cpan module be configured. Would 
it be better to take the java config route, i.e. inform the user that they must 
perform the configuration themselves, preferably sometime just after installing 
perl, or would it be better to come up with a schema to create a generic cpan 
config, tell them about it, remind them that it would work better if they 
reconfig'd,  and then went on our merry way? 
 ---*snip*---

One more caveat is that I am not familiar enough with the inner mechs of the 
build process to address keeping portage up to date with installed 
modules...though as I type this I realize you could add to the (potential) 
perl_cpan eclass a test fetaure to see what the current install is...
Comment 3 Seemant Kulleen (RETIRED) gentoo-dev 2002-07-09 11:18:17 UTC
Leon,

no worries :)  I'd like to hear some of your thoughts about this, in fact.  CPAN
in gentoo is something I've been thinking about for about 2.5 months now, but I
dunno nearly enough about perl to know how to proceed.  So, from what I
understand from Leon's e-mail, is that CPAN does not know how to handle
non-perl-module dependencies.  That correct?  To recap. The idea is to have a
few perl modules in dev-perl (like sdl-perl and gtk-perl) and possibly anything
else with a strict exernal dependencies, and then for the rest, use CPAN.

Question is, how can we have CPAN interact nicely with portage?
Comment 4 Michael Cummings (RETIRED) gentoo-dev 2002-07-09 11:36:33 UTC
Maybe my experience has just been fortunate, but I have seen CPAN grab *non* 
perl dependancies as well when the dependancy in question is a C library. If 
you attempt to install the perl module for ssh via cpan, one of the 
requirements is a C math library (yeah, names escape me right now...math-pari? 
something like that) - CPAN.pm will include that as part of the installation. 
It *won't* grab gtk if you are attempting to instal perl-gtk, however - it 
assumes you've gotten that far already (and portage fits in nicely right 
there). My experience with cpan.pm has been that it will fulfill dependancies 
if they are either other perl modules, or internal/core libraries, but not if 
they are actually external programs. Hope that helps,

Mike
Comment 5 Leon Brocard 2002-07-09 11:49:03 UTC
Right. I've been thinking over dependencies recently and the process where
modules with strict dependencies are ebuilds and everything else is managed
"magically" by  CPAN seems as if it might work.

Little note: I don't really like CPAN.pm. A replacement has been coded which has
much better internals and is intended to have an easily-programmable backend,
which we'd probably have to plug into.

http://www.perl.com/pub/a/2002/03/26/cpanplus.html
http://cpanplus.sourceforge.net/
http://search.cpan.org/search?dist=CPANPLUS

CPANPLUS still needs some configuration when it is first installed, but I'm
working with the developers to make it assume sensible defaults (they're
currently on a road trip but I'll meet up with them at the O'Reilly Open Source
Conference).

So basically, all we need is a mechanism for portage so that when it sees a
dependency like dev-perl/Acme-Buffy and it doesn't actually know about
Acme::Buffy, for it to run:

  use strict;
  use CPANPLUS::Backend;
  my $cp = CPANPLUS::Backend->new();
  $cp->install(modules => ['Acme::Buffy']);

I imagine this might require some portage hacking, but I'm afraid I don't know
much about that. I'm very interested in helping out. If we can get this to work
nicely it'd be great.

[Note that some module installs are interactive, which sucks. There isn't even a
standard "please accept decent defaults". I might have to hound module authors...]

So, where do I have to look to find out more about portage? ;-)
Comment 6 Michael Cummings (RETIRED) gentoo-dev 2002-07-09 12:00:14 UTC
I didn't include CPANPLUS in my post for only one reason: cpan.pm is part of 
the stable build of perl (or, unstable ;) ), and cpanplus is an additional 
module...I know, with portage et al., grabbing and installing cpanplus isn't a 
biggie, was just looking at ways to do this with a minimal of extra work on the 
users (or DEPEND's) part.

Other than that, cpanplus has advantages, and AFAIK (and tested) cpanplus 
accepts the same command line implementation as cpan (with the bonus of being 
able to turn of verbosity), i.e.
     perl -MCPANPLUS -e 'install egg::foo'
Comment 7 Grant Goodyear (RETIRED) gentoo-dev 2002-07-09 12:03:34 UTC
Here's the requirements:

1. Anything that cpan installs must be recorded properly in the portage db,
so that it may be uninstalled if desired.

2. Emerge -u world should also update installed perl packages, even those
that were installed as deps by cpan.

3. ebuild perl-pkg package still needs to work.

My gut instinct is that coupling cpan to portage is going to be such a pain
that it isn't going to be worth while, but I could well be wrong.
Comment 8 teri-aka TrAns13nT 2002-07-09 12:09:57 UTC
Ok, here's my take on CPAN. Whatever is decided upon for the perl modules and 
CPAN should be a nice user friendly idea. From what I've seen  most gentoo 
users are barely adept enough to run linux, let alone gentoo, and I do include 
myself in that category, but I'm trying hard to learn. I have CPAN setup and 
have used it to install some things. How bout having ebuilds for the perl 
modules, but in the ebuild have it error out and tell users to use CPAN 
instead, kinda like a few of the other ebuild that error out and tell u to dl 
the files and then copy them to where they belong. Well, that my opinion. Have 
fun feloow gentoo-ers
Comment 9 Leon Brocard 2002-07-09 13:14:06 UTC
I thought about this on the train home and I have the odd feeling that Grant's
gut instinct is right. Perhaps a better approach is to autogenerate lots of
ebuild files.

I'm going to try and get a program to generate correct ebuilds (with accurate
dependencies) for every CPAN distribution beginning with A. This should give us
a fairly good idea of how hard this is.

A couple of notes I scribbled on the back of my book: there's a difference
between modules and distributions (for example List::Util and Scalar::Util are
both in Scalar-List-Utils). The gentoo name should be the distribution, so
dev-perl/Scalar-List-Utils. Also, a lot of Perl modules come with tests and we
should run the tests - they're there for a reason.

Anyway, I'll let you know if generating ebuild files is feasible.
Comment 10 Michael Cummings (RETIRED) gentoo-dev 2002-07-09 13:19:16 UTC
Leon,

     Yeah, still stuck on using perl to do the actual update, so bear with 
me...what if the ebuild file pointed to cpan/cpanplus/whatever for the actual 
build execution? passing the ${P} name to cpan, yada?
     A is going to be huge...if it gets too big, maybe try Z (so few z's).
     Just a random thought.

Mike
Comment 11 Michael Cummings (RETIRED) gentoo-dev 2002-07-09 14:36:55 UTC
Leon,

     Didn't think of this until I was in the car on the way home. What aboout 
using the modules list file to use for comparing version and current releases? 
It's all cpan does. It can be found at your favorite cpan mirror 
under /pub/CPAN/*module*.tar.gz (exact name escapes me). Off the cuff, could 
the virtual ebuild approach work here? Compare installed module to mod list, do 
a virtual ebuild if necessary? Just a thought, will play with this in a bit,

Mike
Comment 12 Leon Brocard 2002-07-09 16:52:20 UTC
You guys are going to love this. Right, I've just finished up a ebuilder.pl file
that goes through and creates ebuilds for every module on CPAN beginning with Y.
You were right: there are a lot of A's. I've tested it with the As and it takes
a while ;-)

OK, I've commented this but the script still has a couple of dependencies, which
should be obvious from the "use"-es at the top. I've attached the script and an
example dead-simple ebuild and a more complicated ebuild, all autogenerated.

If you run the script it'll write out ebuilds to a directory called "ebuilds" in
your current directory (for all the Ys by default, as there aren't that many).

It does this by: downloading the module, extracting it, running 'perl
Makefile.PL', looking in the generated Makefile and writing out an ebuild in the
right place and with the correct text (hey, even a description) and
dependencies. The generated ebuilds work for me. Are they in the right style?

Note the DEPENDS hash, where we can add dependencies. This way, simple external
library dependencies can be satisfied.

I'm pretty impressed by how far this quick script got. It's generating proper
ebuilds (with correct dependencies) for about 80% of the modules on CPAN. As
with all magic tools, there are a couple of caveats:

a) It doesn't know about external library dependencies. But we can get around
this with DEPENDS: 'XML::LibXML' => 'dev-libs/libxml2'
b) Some module authors get their dependencies wrong. We should tell them to fix
it, but we can work around it by putting it in DEPENDS
c) Some modules have interactive installs. These are very annoying. Some of them
have options which use sensible defaults - we can probably hack ebuilder to use
this, but otherwise we have to do the ebuild by hand (or patch the module)

Cool or what? ;-)

Comment 13 Leon Brocard 2002-07-09 16:53:05 UTC
Created attachment 2089 [details]
ebuilder.pl
Comment 14 Leon Brocard 2002-07-09 16:53:55 UTC
Created attachment 2090 [details]
Simple generated ebuild: YAML-0.35.ebuild
Comment 15 Leon Brocard 2002-07-09 16:55:12 UTC
Created attachment 2091 [details]
Complicated generated ebuild: AxKit-XSP-Util-1.6.ebuild
Comment 16 Michael Cummings (RETIRED) gentoo-dev 2002-07-10 06:57:00 UTC
Would it be possible to get a day or so to compare this to my simplistic test 
model (still being written)? I don't know that this is a must push out today 
issue (could be wrong!!!). A little concerned (perhaps unwarranted) with 
staticly creating ebuilds for *all* perl modules, mostly in the sense of 
overwhelming rsync's with a perl module list that outmatches the rest of 
the /usr/portage tree (unless I miss my guess, there are thousands of modules). 
Thanks,

Mike
Comment 17 Piers Cawley 2002-07-10 07:36:21 UTC
It occurs to me that a hybrid scheme might well be in order so that if one were
to do 'emerge dev-perl/some-module' and there was no ebuild file available, then
portage would use ebuilder.pl (or its moral equivalent) to generate an ebuild
file, which would then be used to build the module. Of course, there will
probably be some 'maverick' modules that have other dependencies, these can be
handled either with a custom ebuild (in the case of a 'pathological' module), or
by a module specific config file for ebuilder.pl.

Given that the majority of packages on CPAN build very straightforwardly, this
should allow the auto building of anything from CPAN without having to rsync
vast amounts of, essentially identical, ebuilds...
Comment 18 Leon Brocard 2002-07-10 08:01:36 UTC
The large number of ebuilds generated does vaguely worry me, but it's necessary
to have portage handle all the CPAN modules. Otherwise, we'd get a situation
like other distributions have where users install some modules using portage and
then install other ones using CPAN.pm or CPANPLUS. This'd lose the advantage of
using portage properly, and would probably fail Grant's requirements. I'll play
some more and see if I think this will work.  
Comment 19 Michael Cummings (RETIRED) gentoo-dev 2002-07-10 08:16:07 UTC
My take would be (and what I'm working in my oh so luxurious free time) that a 
hybrid would be better - ebuilds for those that absolutely demand it, but 
otherwise doing an "ebuild on the fly." In addition to grant's requirements, 
and in order to help make sure that portage is used instead of cpan directly, 
would be to make the portage solution "easier" than cpan/cpanplus.pm. Not only 
so that modules can be easily included as dependancies, but so that it is a 
simple matter of "emerge dev-perl/modname" and letting it go. If the module 
ebuild exists, build, otherwise do a virtual ebuild, and this is where that 
spurious note about the mirror listing would come in handy, because so long as 
that is maintained somewhat, you would have in a few files a listing of all the 
modules available (grep -v or equiv for protected modules, i.e. modules that 
have specific ebuilds) and available cpan mirrors to connect to (the path I 
gave yesterday was off, should have been mirror:/pub/CPAN/modules/*.gz).

At its core, the cpan/cpanplus modules are no more complicated than your 
average ebuild. It grabs the file, runs the make, test, install, and tells you 
its done. Maintenance on a G-CPAN (making it up as I go) would be relatively 
low level - make sure the ./files/ has the current listings available for 
rsync, and tada. Gasp, this is starting to sound easier and feasible the more I 
write. I should stop now.

Scrolling down, I realize that I've pretty much said the same thing you did in 
your last post, Leon ;)
Comment 20 Leon Brocard 2002-07-14 03:17:15 UTC
Update: I've been working on this during the evenings, so I thought I'd let you
know what I'm up to. I've gone through the whole of CPAN and generated 2879
ebuilds and am currently in the process of QA. Unfortunately this means, for
each ebuild: a) install ebuild to see if the dependencies are ok b) remove all
newly installed ebuilds. This is going to take a while to run, and I've had to
skip lots of Perl modules on the way which were too tricky to figure out, but at
least I'm getting somewhere. Once I have a tested set of ebuilds I'll let you
know. Leon
Comment 21 Seemant Kulleen (RETIRED) gentoo-dev 2002-07-14 03:34:15 UTC
Leon, all I have to say is "wow!"
Comment 22 Michael Cummings (RETIRED) gentoo-dev 2002-07-14 15:07:18 UTC
I've been working on this evenings as well, hope to have samples to post soon.
I've taken a different approach than Leon - I'm trying to get the builds to
occur on the fly so that there doesn't need to be an individual ebuild for each
module, but instead a single cach all module that I'm calling gcpan for now
(gentoo-cpan). Since the all the cpan script really does is perform a wget
against a list of mirrors, it would seem to fit into the ebuild schema pretty
well. The only thing I'm not certain about is that I think this would require a
change to the emerge code for emerge requests against the dev-perl/ tree.
Comment 23 Michael Cummings (RETIRED) gentoo-dev 2002-07-15 20:10:27 UTC
Created attachment 2276 [details]
VERY PRIMITIVE NOT WORKING EXAMPLE

This is a very early, primitive, NON-FUNCTIONAL (I think) sample of where I am
thinking. Many bugs yet to work out, but you can get an idea from here of what
I am thinking. If you untar this and type ./g-cpan* yaml (for instance), I echo
a few of the variables before exiting (and not actually downloading and
installing). Still very preliminary/primitive (can I stress that enough?), but
I wanted to have something to show. Opinions and feedback welcome,

Mike (and yes, I know I am missing keywords like KEYWORDS, SLOT, etc. - this
isn't even close to a submission yet)
Comment 24 Seemant Kulleen (RETIRED) gentoo-dev 2002-07-16 16:45:28 UTC
<MaxPerl> I don't know enough about the ebuild system to say.  don't know why
perl modules can't just be translated through to cpan install

MaxPerl> emerge CGI::Cookie would translate to "perl -e'install CGI::Cookie'
-MCPAN" without any other involvement
Comment 25 Leon Brocard 2002-07-17 06:17:12 UTC
We can't just translate emerge calls to CPAN.pm calls as we want proper package
management (updates, unmerge etc.)

BTW I'm at the O'Reilly Open Source conference next week so my work on this will
be delayed for a little bit, although I'd love to meet up with anyone who will
be there.
Comment 26 Nicholas Jones (RETIRED) gentoo-dev 2002-07-17 07:45:57 UTC
Ok... here's what I had intended to do once I got around to this...

Parts:
1. CPAN ebuild
1.a. Modify CPAN to sandbox itself by editing CPAN itself
2. Integrate /etc/portage/
2.a. /etc/portage/on-update/ <-- Scripts for on-update
2.b. Add configuration/tracking of module installation into /etc/portage/perl

CPAN would be setup to use Portage's directories as setup by the admin.
CPAN provides variables for doing this. CPAN would sandbox itself by
executing the perl makefiles inside of the sandbox, and would install
into a sandboxed directory... Might requires some codeplay.

Upon completion of building, sandboxed code would be md5sum'd and appended to
the current perl var/db CONTENTS files so that it will be delete upon unmerge
of existing versions.

The module built would also get it's own CONTENTS file in the var/db tree
so that individual upgrades could be dealt with. Module names would be stored
into the /etc/portage/perl CPAN config/tracker for upgrade/rebuilds.

The /etc/portage/on-update scripts would have to be integrated into portage
to allow for CPAN to execute proper upgrades when a new CPAN version was not
present. Otherwise upgrades would never occur.

CPAN's codifications would also have to preclude certain modules from being
updated via CPAN due to their effect of sandboxing and portage-integration.

I may have more... But I'd have to update as I go...

NOTE ON OTHER ATTEMPTS:
800K files in the portage tree are NOT acceptable. If you wish to continue
with downloading the module info files, do that as a SRC_URI, not as a $FILES.

Also, if you use an automatic generation, remember that QC on every update
will need to be performed by someone.
Comment 27 Michael Cummings (RETIRED) gentoo-dev 2002-07-17 09:26:14 UTC
I'm not sure how the sandboxed install directory would work unless you made all 
of /usr/lib/perl5 a sandbox. Depending on the module, it can install to 
different portions of that tree. And in order to be usable (without unnecessary 
coding by the users), the module must be installed and 'notice' (i.e., added to 
perl's autoload table) sent to perl itself for it to be usable.

Me thinks.

As to modifying the cpan module distribution itself, that was what my (rather 
poor) attempt was aiming for, sort of. The cpan module requires a complete 
configuration before it can be used. My primitive example was to try and get 
the functionality of cpan without relying on the actual perl module involved. 
All cpan sites maintain a core of key, updated files that contain a list of 
valid mirrors, authors, current module packages, and their paths, that might be 
exploitable to some degree. Of course, I completely failed to take into 
consideration QC (honest mistake) for each package. What I was hoping for, 
naively, was a way to build perl modules on the fly, like cpan does, but with 
the ability to add those additions to the portage db so that gentoo hard core 
perl folk could have that extra bit of functionality. Of course, when I first 
looked into this, I was also thinking of an easier way for ebuilds that have a 
dep on a perl module to grab those modules - at least from a basic find/grep, 
none exist (yet).

One caveat, no matter what the approach is, is that there would need to be a 
way to mask certain modules from ever being updateable via a simple ebuild. For 
instance, if you update the cpan module to the latest, it will go ahead and 
update your perl version as well, which without post-install checking of 
modules installed vs modules in the db, would be lost to -u.

In my rather poor example code above, I have a sub that will display all 
installed perl modules on your system (classic textbook example). My thought 
was that this could be used in coordination with portage to determine versions 
installed in relation to available updates.

I do like the idea of a hacked cpan module for gentoo, something i didn't 
attain though an inspiration for my attempt, but it would still require some 
configuration, somewhere along the line, to meet the user's needs. By placing 
an ebuild script around the calls to that script, I was hoping to make some of 
the tasks easier, not sure I did that.

I think I'm babbling at this point. Feedback?
Comment 28 Nicholas Jones (RETIRED) gentoo-dev 2002-07-19 01:31:19 UTC
CPAN provides build and install prefixing variables. Those can be exploited to 
sandbox CPAN. Where it installs in unimportant as we just want _what_ it 
installs; prefix it to the image directory after it builds. I have no idea what 
you mean by 'notice'. I guess I'll have to look into that at some point.

>> a way to mask certain modules from ever being updateable via a simple
>> ebuild. For instance, if you update the cpan module to the latest, it
>> will go ahead and update your perl version as well, which without
>> post-install checking of modules installed vs modules in the db, would
>> be lost to -u.

As I had mentioned... CPAN would not be allowed to update itself. This would
be explicitly defined in the code, or perhaps optionally in the config/module
file that cpan would use.

>> I do like the idea of a hacked cpan module for gentoo, something i didn't
>> attain though an inspiration for my attempt, but it would still require
>> some configuration, somewhere along the line, to meet the user's needs. By
>> placing  an ebuild script around the calls to that script, I was hoping to
>> make some of the tasks easier, not sure I did that.

Any way you look at it, you'd have to force-build it everytime to get the 
updates. Portage would have no way to determine if a module needed to be 
rebuilt. (One exception would be the bazillion-ebuild-no-cpan method.)
Comment 29 Michael Cummings (RETIRED) gentoo-dev 2002-07-19 18:00:59 UTC
'notice' was a poor choice of words on my part. What I meant was "Each time a
module is installed on your system, it appends information like the following to
a file called perllocal.pod which can be found in /usr/local/lib/perl5/version
number/architecture/ or something akin to that. The path for your specific
installation is in your @INC which you can divine with perl -V."
(http://www.cpan.org/misc/cpan-faq.html#How_installed_modules)
Comment 30 Paul Belt 2002-07-19 22:34:38 UTC
From everything that I have read so far, and all my research, here are my
conclusions.  Most, if not all of this, is regurgitated info... just a summary
of my thoughts on the issues involved.  To my understanding...

A package-system (portage,rpm,deb,tgz... pick) needs to have a package built to
satisfy its dependencies.  However, these packages often (and quickly) fall
behind their CPAN counterparts.  When a developer updates a module via CPAN, it
obsoletes the packaging-system info about said module.

I looked at how other distributions deal with this issue.  I hate having to
re-invent the wheel when I can avoid it.  Unfortunately, none of their solutions
are elegant.  In fact, they are all pretty much the same.  All have the same
issue we are experiencing.

CPAN will always know about CPAN updates before the package-management system. 
We don't own it, nor were it's developers kind enough to provide call-backs for
us to use.  However, that doesn't stop us from mimicking it's behavior / doing
it's job.

Hacking CPAN to use portage, or more labor intensive (but worth it), writing our
own drop-in replacement (here in called GPAN) is the only practical solution to
get CPAN to play nicely with portage.

Michael Cummings (mcummings@gentoo.org) in comment #22 stated he's "trying to
get the builds to occur on the fly so that there doesn't need to be an
individual ebuild for each module."   This will avoid having 1000's of ebuilds
and will create /var/db/pkg/* entries when someone updates modules via GPAN. 
This will allow 'CPAN' to function as it expects, and assist portage so portage
can behave as we expect / need.  CPAN/MyConfig.pm might be able to be used in
some manner.  (refer to /usr/lib/perl5/5.6.1/CPAN.pm)

Everything that CPAN expects for a configuration is listed here:  cpan_home,
keep_source_where, build_dir, build_cache, scan_cache, index_expire, gzip, tar,
unzip, make, pager, makepl_arg, make_arg, make_install_arg, urllist,
inhibit_startup_message, ftp_proxy, http_proxy, no_proxy, prerequisites_policy,
cache_metadata

We should modify portage so when dev-cpan/*.ebuild is called, it uses GPAN to
check if said module exists, and if so, check if we already have an ebuild.  If
we have an ebuild, use it, else generate one.  We can compare the CPAN files
list to the /var/db/pkg/dev-perl/* to see if an update is even necessary.

GPAN can convert CPAN 'external' dependencies to 'real' ebuilds.  This will take
care of CPAN dependencies that are not CPAN modules.  CPAN modules that have
broken dependencies can be put in dev-cpan until we can get the authors of the
modules to fix it.  Once fixed, let GPAN deal with it if possible.

Nicholas Jones (carpaski@gentoo.org) in comment #26 stated he intends to get
CPAN to behave nicely with sandboxing.  We can take this functionality and use
it in GPAN, as it would solve that issue.  Futher comments exist on this topic
in comment #28.

NOTE:  ebuilder's functionality would be the main difference between CPAN and GPAN

::> CAVEATS
"Some modules have interactive installs."
This is where dev-cpan/ comes in.  Create sensible defaults if possile.

ebuilder.pl can not generate ebuilds for about 20% of all CPAN modules.
The best we can do for now is either, enhance ebuilder.pl or create those
ebuilds by hand.

"CPAN has the ability to update perl itself, bypassing portage."

We can not make this system fool-proof, but we sure as hell can try.  There will
always be a way around portage.  We just have to accept this.

Did I miss anything?  If GPAN is the way to go, Mr. Cummings, Mr. Jones, anyone
else who's interested, let me know and I'll assist you, if possible.
Comment 31 Michael Cummings (RETIRED) gentoo-dev 2002-07-22 11:39:42 UTC
I hate to be the one to say this (see thread if you doubt that), but I am 
beginning to wonder at the advantages to having a cpan interface in gentoo. I 
broke down and searched through all of the current ebuilds (unmasked, at 
least), and not one outside of the perl tree has a dependancy on any perl 
modules inside the tree. This is beginning to feel like a lot of hacks (old 
fashioned definition) for something that would only be a little bit valuable, 
and could potentially bloat up the portage db even more (even virtual ebuilds 
would need to be listed...). In the end, it would seem like the only folks that 
would find this good are those of that use perl daily, and we already know how 
to run the config tool for cpan and do a one liner install. I'd be happy to 
work with anyone interested in pursuing this, and would welcome encouraging 
words, but that's where my market shares (all two cents worth) lie right now.
Comment 32 Seemant Kulleen (RETIRED) gentoo-dev 2002-07-26 03:29:05 UTC
Michael, is it perhaps worth then to just collect a fairly current list of CPAN
modules with external dependencies (ie non-perl-modules dependencies) and ebuild
them into the tree? 

By the same token, is it worth removing the ebuilds currently in portage which
do depend solely on other perl modules?  Not entirely sure of the policy
regarding pkg_setup() in ebuilds but if policy allows it, perhaps, just a call
to cpan to actually install the perl-module dependencies that each might have. 
In case that is unclear, let me make a small example.

Let's say we have the foo-perl module, which depends on GTK+ and also on
Net::Timer and My::PerlModule.  But, Net::Timer only depends on Net::Clocks, and
My::PerlModule only depends on Your::PerlModule.  Then, foo-perl's ebuild would
just ask cpan to install My::PerlModule (thereby pulling in Your::PerlModule)
and Net::Timer (pulling in Net::Clock).

On the other hand, if Net::Clock or Your::PerlModule have external dependencies,
we're in a web again.  Bleh, I've dizzied myself.
Comment 33 Michael Cummings (RETIRED) gentoo-dev 2002-08-12 11:16:08 UTC
Can anyone with free time (any version of perl will do) check out the following 
eclass files - http://cvs.gentoo.org/~mcummings/new_perl_eclass.tgz - these are 
to replace the current perl-modules and perl-post eclasses and address issues 
with perllocal.pod getting misset or wiped of all local info. The folks on this 
bug list seemed the best group to target given the subject :)

Mike
Comment 34 John Nilsson 2002-10-01 10:13:34 UTC
Automatic dependency resolving by portage is very much wanted. I was thiking of
installing SlashCode for some testing, ended up generating and testing a bunch
of ebuilds for Perl-Modules.

About the issue of bloating the portage tree: Why not generate symlinks for all
identical ebuilds after rsync? Only problem beeing the $header: line.

While generating a small number of ebuild scripts I ran into some problems.
1. A dependency tricked me into generating an ebuild for File:Spec unmerge of
this ebuild would brake perl and module would install.
2. Modules would get installed in diffrent places breaking the Gentoo "FHS".
/usr/lib/Site_Perl/<Module> was popular, however /usr/lib/Site_Perl is not
included int @INC as /usr/lib/perl5/Site_Perl is the official Gentoo location
(my assumption).
3. As already stated some packages required interaction during install. Very
annoying. DBIx::Password in this case. It would be impossible to supply sensible
defaults to this package.

John
Comment 35 Karl Trygve Kalleberg (RETIRED) gentoo-dev 2002-11-02 16:33:13 UTC
*** Bug 3540 has been marked as a duplicate of this bug. ***
Comment 36 SpanKY gentoo-dev 2002-11-03 16:23:27 UTC
*** Bug 9658 has been marked as a duplicate of this bug. ***
Comment 37 Matt Keadle 2002-11-06 20:56:11 UTC
I was gonna read this thread before adding this but JEEZ. I want to see my
(future) wife (whom I haven't met yet) sometime!

Webmin has a perl module installer that downloads the tarballs from CPAN into
/tmp, builds and test them, and then cleans up after itself on install/failure.
It's all dynamic, even down to perl module selection. The list of modules is
grabbed from CPAN each time you use it (or you can enter the module name in
directly). I've had admins use it several times with a very high success rate.

Now, it doesn't keep track of the actuall files installed, so we'd need some
kind of wrapper for that. But I bet there is something in there we could take a
hint from!
Comment 38 J Robert Ray 2002-11-07 05:52:15 UTC
Created attachment 5443 [details]
g-cpan.pl

Here's yet-another-script I threw together.

It leverages the CPAN.pm module.  The upside is CPAN takes care of doing all
the dirty work.  The downside is it does a lot of dirty work.

This script generates ebuilds for perl modules that don't already exist, on
demand.

In my test case, I want to install the Xmms::Perl module.  As root you run
"./g-cpan.pl Xmms::Remote".

First, the script has to find out what dist contains the Xmms::Perl module, it
uses CPAN to discover this can be found in "Xmms-Perl-0.12.tar.gz".

Then, it uses the CPAN module to 'make' this dist.  Here's the ugly part: CPAN
will fetch the file and compile it.  This is necessary to make CPAN figure out
what this module's dependencies are.  But it's convenient because now we have
the file and can create a digest for it.  We could also move the file to
DISTFILES but I didn't do that.

CPAN tells us that Xmms-Perl has a number of depends.  One is 'MPEG::MP3Info',
but the script figures out this is MP3-Info, and an ebuild already exists for
that, so it moves on.

The script moves along, recursively making ebuilds for all the depends.  It
ends up making an ebuild for 'Term-ReadLine-Perl', 'ANSIColor', and finally
'Xmms-Perl'.  It puts these in your portage overlay directory.

Finally, it helpfully suggests you 'emerge Xmms-Perl' to really install the
perl module.

When the script writes out the ebuilds, it fills in the DEPEND variable
appropriately.

Here is the ebuild it makes for Xmms-Perl:

Xmms-Perl-0.12.ebuild
=========================================================================
# Copyright 1999-2002 Gentoo Technologies, Inc.
# Distributed under the terms of the GNU General Public License v2

inherit perl-module

S=${WORKDIR}/${P}
DESCRIPTION="Perl interface to the xmms_remote_* API"
SRC_URI="http://www.cpan.org/modules/by-authors/id/D/DO/DOUGM/${P}.tar.gz"
HOMEPAGE="http://www.cpan.org/modules/by-authors/id/D/DO/DOUGM/${P}.readme"

SLOT="0"
LICENSE="Artistic | GPL-2"
KEYWORDS="x86 ppc alpha"

DEPEND="dev-perl/MP3-Info
	dev-perl/Term-ReadLine-Perl
	dev-perl/ANSIColor"
=========================================================================


I was able to emerge this successfully.

Now, the script falls apart on certain modules that have elaborate config
scripts, one such example is Net::SSH::Perl, it starts prompting the user to
make decisions.  A package like this is going to need someone to make an ebuild
manually, I guess.
Comment 39 Michael Cummings (RETIRED) gentoo-dev 2002-11-07 06:02:09 UTC
We want to avoid a script that merely generates more ebuilds - we are trying to 
get away from a dev-perl tree that is huge, in favor of being able to install 
perl modules "on the fly". The issue at this point (as I see it) isn't with the 
coding of such a tool - CPAN, CPANPLUS, homebrew, webmin, what have you, all 
are servicable ways to do this. IMO (won't say humble, but definitely IMO) the 
issue that needs to be resolve first is integration with the emerge system 
itself. I have a bug open to discuss this, will make this bug a dep of it so 
you all can see it.

Mike, one guy, one opinion, one track mind =:)
Comment 40 Michael Cummings (RETIRED) gentoo-dev 2002-11-07 06:04:34 UTC
OK, so Spanky closed the bug I was thinking of (9658, see above for link) as a 
duplicate of this, which sort of defeats the purpose.
Comment 41 J Robert Ray 2002-11-07 06:09:45 UTC
I could've sworn I was asked to make such a tool.  We obviously don't want an
ebuild for every perl module, but the advantage to using something like my
script over just using CPAN is that the modules you install get in the portage
db and can be uninstalled later.
Comment 42 Michael Cummings (RETIRED) gentoo-dev 2002-11-07 06:35:26 UTC
Sorry jrray =:) Eh, what do I know? The original thinking was to avoid extra 
ebuilds - i.e., consolidate the ebuilds that aren't specific (exceptions 
including the DBI modules), that really are just a perl Makefile.PL -args, 
make, make install and be able to do them from a single source.

I wonder if a hack of what you did (some type of clean up afterwards to remove 
the new ebuild?) would work? Presumably, once your version was done the new 
perl module would be in portage's db, at which point we could do whatever we 
wanted with the temporary/new ebuild that was generated.

Would still need some kind of "hook" to be able to search against the possible 
modules (modules list from cpan, updated regularly?), but what you've done 
would eliminate a lot of steps.

I'm also wondering (top of my head, just thought of this, kind of contradicts 
the need for searches and extra hooks) if the ability to install these perl 
modules shouldn't be some type of function in the emerge process that kicks off 
something like your script, but that isn't an ebuild itself, kind of like an 
embedded ebuild that is autogenerated as you go. For example, ebuild foo needs 
perl module bar. inside that ebuild, there is a line that says something like 
perl_module("bar") which calls a section of code like the one you wrote, runs 
the ebuild, then proceeds with the rest of the parent ebuild. 

Part of my reasoning is that my understanding as inheritor of the dev-perl tree 
is that we will only add ebuilds for perl modules that satisfy deps (even if 
that satisfaction is sixteen layers deep, techincally all ebuilds in dev-perl 
should satisfy a dep elsewhere). So we (from this perspecitve at least) don't 
need to be concerned with being able to emerge module-fooie, just that emerge 
pakg-barrie can meet its perl module deps.

Just a thought. Way too much thinking on my part without enough coffee this 
morning. Good to see this bug getting activity again! =:)
Comment 43 J Robert Ray 2002-11-07 06:56:09 UTC
My understanding is that we want to ship ebuilds for perl modules that have
non-perl module dependencies.

An example would be the GD module that has a dep on media-libs/libgd.

Currently, we would need an ebuild for the situation you describe, when an
external package has a dep on a perl module.  Can you come up with an example
for that?  Imagemagick probably should be an example, but the ebuild for that
just installs the perl module itself.  Michael mentioned above that he couldn't
find any ebuilds that have a perl module as a dependency.

An interesting way that my script could be used would be a way for an ebuild to
specify a perl module dependency without having to provide the ebuild for it. 
Portage could generate the ebuild on demand.

I don't think the ebuilds should be considered temporary and deleted after the
merge.  After you run my script you can 'emerge -s' for them, even if you didn't
install the modules.

As for providing a searchable list of perl modules, the files you can find in
~root/.cpan/sources/modules will help.  03modlist.data.gz is a list of modules
and descriptions.
Comment 44 Matt Keadle 2002-11-10 02:54:34 UTC
In true geek fashion I go out for drinks and end up thinking about perl and
Gentoo. Anyway, I bear my thoughts nekkid for all to see </me beats his chest>:

As devs and maintainers, part of our duties include making things available and
easy for end users (within boundries, of course). Ebuilds, though they serve
other purposes too, are essentially automatic package installion script. A lot
of the work we de involves automation. I often fall into a mindset after dealing
with many ebuilds that if something can be automated it should be automated. I
would be suprised if I'm the only one who slips into this. So what does this
mean in reference to CPAN/Gentoo?

Looking at other distros, you mostly find a static set of available/supported
packages. Take Red Hat as an example: The distribution that is Red Het Linux 8.0
is a kernel and base system, some Red Het specific stuff, and pretty much just a
large collection of applications. Perl modules themselves are not applications,
but you'll still find rpms of many perl modules in the distribution. They are
there because they are dependancies or because they are popular and widely used.
Outside of that, there is so far no tool that I know of that any distro includes
that provides a way to obtain extra perl modules and still contain them within
it's base packaging system, rpm in the case of Red Hat. If a user wants perl
modules that are not included it is up to them to look for 3rd party rpms or
install them through the CPAN module directly. This has become accepted as "the
way it is" with Red Hat.

Gentoo, I believe, could benefit from a simple policy. If a perl module is a
dependancy of an external application, there should an automated (ebuild, if not
otherwise) way to install it. If it is a popular, well used module, there should
also be an automated process. Outside of this it should be left up to the user.
This is where my first main point comes in. It seems that we're feeling
pressured, and perhaps obligated, to overextend ourselves and automate the
entire CPAN archive more than is realistic. From comment #20 we see "I've gone
through the whole of CPAN and generated 2879 ebuilds". How many of those would
actually get used, or, more importantly, *not* used? And, as the comment also
states, QA of those ebuilds is a nightmare. The aforementioned policy is meant
to be a set of guidelines for us as much as it is for the end user. I do not
believe there is any sustainable, maintainable, and predictable way for us to
automate the whole of CPAN and place it into the confines of portage.

Having said that, I'm still a believer in progress towards a worthy goal. So
here is my take on a solution. First, it's based on letting the CPAN module do
most of the work, and letting go of the idea that portage itself will keep track
of every installed module. Second, it's built around the idea that, even if the
system isn't perfect, providing the user with as much information as possible
can make up for a lot. Third, I am not a programmer, at least as much as this
would require, so feel free to be critical of the work involved. So here we go!

Choose your interface, either ncurses, GTK, or whichever. A userspace program
that lets the user see which perl modules are currently installed as well as a
list of currently available modules from CPAN. The CPAN list can be called in
realtime directly from CPAN while online and possibly cached to local media,
letting the user choose to refresh the list at their will. Users could be able
to select modules from the CPAN list that they would like to install. Once
selected the program would be able to discover if the selected modules are
available as ebuilds and then take them directly from portage, easily handled by
externally calling emerge. If the modules are not available from portage the
program would externally call perl -MCPAN -e 'install <modulename>'. Also,
currently installed modules should be able to be selected for removal. It would
also be desireable to show if an installed module was emerged or brought in
through CPAN. As devs, our routine would not have to change much, if at all. If
we add a new ebuild for a perl module, the program should be able to detect that
and act accordingly. As users, they would get a pretty face and a nice way to
keep track of installed modules. As Gentoo, it would be the first distribution
that I know of to have a custom means to aquire allmost any perl module available.

Now, as has already been discussed, not every perl module can be automated.
Also, there would be little in the way of preventing users from removing
important perl modules. I believe both of these points are moot. If a module is
important to include in Gentoo, we, as devs, will find a way. As far as users
removing important pieces of a system, there's nothing to prevent a user from
unmerging the baselayout program either. At some point the user has to be held
accountable, and we cannot be responsible for quick and poor judgement on their
part. We are not their "Guide to Linux".

4:00 am and I need a refill on this Coke and a ciggy. So pick this appart. I'm
not suggesting a best way to handle this, just another idea.
Comment 45 Michael Cummings (RETIRED) gentoo-dev 2002-11-10 05:40:41 UTC
Two notes - and ammendment for jrray - while when I wrote that original post
there were no deps on dev-perl items outside of dev-perl, that has since
changed. There are now a handful or so packages, written perl, outside of
dev-perl, that do dep on dev-perl items.

mkeadle - I think I agree with you. (would favor CPANPLUS over CPAN only because
CPAN always crapped out on me when it attempted LWP transfers whereas CPANPLUS
seems to be more intelligent about its transfers). In fact, I agree enough that
if I find some free time in the next day or so I'm going to play with the curses
and tk modules for perl to see if I can get a draft app together.
Comment 46 Dylan Carlson (RETIRED) gentoo-dev 2002-11-10 16:48:55 UTC
My $.02 -- FreeBSD does this quite successfully with FreeBSD ports, they use 
CPAN without user interaction for both install and uninstall, to resolve 
dependencies on perl modules.   
 
Might be worthwhile to look at those scripts they use and see if they are 
adaptable. 
Comment 47 J Robert Ray 2002-11-11 02:11:25 UTC
To Matt in #44:

A situation I don't think you covered in your message is when you go to install
a perl module with no ebuild, and that perl module depends on other perl modules
that do have ebuilds.

My little script handles this case, and this case needs to be handled when a
perl module such as GD has external dependencies to C libraries.

If you let CPAN have its merry way it will upgrade installed modules,
potentially breaking compatiblity with installed libraries.  I've had this
happen to me with libgd.

The approach I take is to use CPAN to discover available packages and discover
their dependencies, but not use it to install anything.  I think it is better to
leave the installing part to Portage.

Has anyone looked at or tried my script?  It isn't very long.
Comment 48 Seemant Kulleen (RETIRED) gentoo-dev 2002-12-02 13:09:05 UTC
Gentlemen, please test jrray's script.   It is almost exactly what we want.
Comment 49 Michael Cummings (RETIRED) gentoo-dev 2002-12-02 14:35:52 UTC
jrray,

     Just a few comments/notes. These are based on using your script in
production vs as a backend perl module ebuild maker.

     * In addition to checking the portage dir for an existing ebuild, the
script needs to check /var/db/pkg/dev-perl to see if it has generated/installed
an ebuild on the fly for this module before
     * In addition to checking portage in general, needs to do a ExtUtils call
to see if the module has been manually installed before, and if so that the
version is >= the required version.
     * Rather than writing the module to PORTAGE_OVERLAY, why not reset OVERLAY
for the duration of the script? That way, A) We can clean it out once the ebuild
is done running and B) it doesn't matter if they have one set or not :)

     Obviously, my comments are geared towards using this in production vs as a
dev tool. I think I had another thought, but this is what I remember from the
drive home. Have to run now, but basicaly my line of thought was using your
script to create ebuilds that we use to satisfy a dep, then erase when we're
done - but since they were added via an ebuild, portage has a record of them
(and presumably records some stuff in /var/db/pkg).

     I'll try and code a revision along these lines tonight/tomorrow if I can
and you don't object. Can't we source their make.conf and use the $ENV vars
created to set some of this? just a random thought as I hit commit...
Comment 50 Michael Cummings (RETIRED) gentoo-dev 2002-12-03 09:05:57 UTC
Created attachment 6171 [details]
Update to jrray's model

Not completed. I've added checks for /var/db/pkg/dev-perl for modules installed
on the fly (the future for this script?) and for manually installed modules. I
tried using ExtUtils::Installed properly, but that was returning only partial
results, so I had to resort to a eval 'use module;'. Not pretty, but efficient.


I also commented out the overlay_dir checks and replaced them with a mapping to
/tmp/perl-modules. The idea being (per conversation on IRC) that the ebuilds
generated would be installed, then wiped. On a TO-DO list still (either when I
have a second or someone else does):

* Version checking (I have code for that, just need to see how to incorporate
it) so that if necessary, we can assure that version X of the module is
installed
* installing the module from within the script
* other stuff my brain is big enough to remember
* Clean up
Comment 51 Michael Cummings (RETIRED) gentoo-dev 2002-12-06 10:11:20 UTC
Created attachment 6252 [details]
emerges on the fly

OK, this is a tested, working model. I installed a module we don't have in
portage (CGI::Application) - it autocreated the ebuild, installed the deps for
it that were in portage, then installed the module. The module shows up in a
search of portage =:) and shows up in perl as valid module =:D

As far as I know, only two things remain. I need help/time to get it to clean
up the tmp directory it makes for the ebuilds, and we need to see if this is
portable into mainstream. Oh yeah, and tons more testing, and I could really
use a code review on this because I know my coding isn't that great. Thanks
all!
Comment 52 Michael Cummings (RETIRED) gentoo-dev 2002-12-07 10:27:44 UTC
Created attachment 6274 [details]
cleaned up even more (thanks baz)

Ok, this version has some added functionality. It will emerge an ebuild if you
already have one, as well as emerging it's temp ebuilds (one stop shopping).
Added in Digest::MD5 to minimize non-perl calls. Added a section for source
make.conf since there were a few other important vars in there that the script
needed to be aware of. 

ToDO is still post install clean up. Anyone want to take that on? =;) and
system("rm -rf tmp_overlay_dir") is not preferred.

I'll get to it, but I wanted to post an update for those interested.
Comment 53 Michael Cummings (RETIRED) gentoo-dev 2002-12-07 16:08:00 UTC
Created attachment 6286 [details]
good version

Final update for the day. This version cleans up after itself and contains a
few syntax changes. Ready for steady input folks!
Comment 54 Michael Cummings (RETIRED) gentoo-dev 2002-12-07 19:19:45 UTC
Realized there is still one more "feature" to add. Ideally, this will
(hopefully?) be incorporated main stream so that it can be used by ebuilds to
fulfill any perl deps they have. Currently, it requires a configured CPAN on
your box to run. Thinking it shouldn't be to hard to check if CPAN is configured
already, and if not either auto-config it with the no option, or provide a
generic config for it to use.
Comment 55 baz 2002-12-09 16:44:32 UTC
Created attachment 6352 [details]
Further amends to the script [mostly aesthetic]

Just tidied up a little - made code (hopefully) more readable, removed some
surplus code, standardised coding methods.  I'm not even a gentoo developer, so
feel free to flame me if appropriate!
Comment 56 Michael Cummings (RETIRED) gentoo-dev 2002-12-11 08:27:36 UTC
Since it's a slow day on this end, I'm going to see if I can break the dep on 
cpan. Ideally this will be used non-interactively, which is defeated if at any 
point the user needs to interact with cpan for configuration. Even supplying 
a "no" doesn't cut it - we have users in all manner of network setups, 
including behind firewalls, proxies, etc. We really only use CPAN for one or 
two parts - getting the current verison of the ebuild (duplicable), and 
checking deps (also duplicable). I'm going to work based on bas's copy, so if 
anyone else is working on an update ping me please.
Comment 57 J Robert Ray 2003-01-08 02:10:13 UTC
Created attachment 7081 [details]
g-cpan.pl

Poking my head back in this bug, the current incarnation wasn't working for me,
so I did some tweaking and a little cleanup.  Forgive me for that ugly python
bit, but it is probably the easiest way to get that done.
Comment 58 Nicholas Jones (RETIRED) gentoo-dev 2003-01-08 06:20:26 UTC
Absorbed into portage for 2.0.47-r1
Anybody got some docs?
Comment 59 baz 2003-01-08 11:02:29 UTC
Quick sanity check:

> use File::Spec;
> use File::Path;
> use CPAN;
> use Digest::MD5;

These mean that we'll still need at least four perl module ebuilds in portage,
when we can get rid of the dependency on Digest::MD5 by just using eg.

$md5 = `md5sum $file_to_get_sum_for`;

or equivalent.

I'm assuming md5sum is included in the base system, if not then please ignore -
I don't have a gentoo system to hand on which to check at the moment :)

Likewise, there are methods to make this work without using the File:: routines.
 Should we be removing the dependancy on these ebuilds too?

Just my 
Comment 60 baz 2003-01-08 11:02:29 UTC
Quick sanity check:

> use File::Spec;
> use File::Path;
> use CPAN;
> use Digest::MD5;

These mean that we'll still need at least four perl module ebuilds in portage,
when we can get rid of the dependency on Digest::MD5 by just using eg.

$md5 = `md5sum $file_to_get_sum_for`;

or equivalent.

I'm assuming md5sum is included in the base system, if not then please ignore -
I don't have a gentoo system to hand on which to check at the moment :)

Likewise, there are methods to make this work without using the File:: routines.
 Should we be removing the dependancy on these ebuilds too?

Just my £0.02...
Comment 61 J Robert Ray 2003-01-08 23:21:14 UTC
The 4 modules you mention ship with Perl 5.8.  Digest::MD5 doesn't ship with
5.6.1 so I can see where avoiding that may be a good idea.  I'll post a patch.
Comment 62 J Robert Ray 2003-01-08 23:22:07 UTC
Created attachment 7115 [details, diff]
Patch against my previous attachment.

Can someone sanity check this please?
Comment 63 J Robert Ray 2003-01-15 06:40:18 UTC
Dealing with the libperl_rebuilder, I get errors about modules I installed with
g-cpan.pl because the ebuilds don't exist, but the script wants to re-emerge
these modules.

So to make the script succeed, I have to unmerge these "bad" modules, run the
script, and then merge them again with g-cpan.

I still question the real benefit of not leaving the generated ebuild files in a
real overlay tree for later use.  Otherwise portage is always going to treat
these packages as orphaned.
Comment 64 Michael Cummings (RETIRED) gentoo-dev 2003-01-15 07:36:30 UTC
What about a compromise? If they have a portdir_overlay set, use it for the 
generated ebuilds, otherwise, not. 

What is the future of this tool, though? If it's goal is to replace the need 
for a dev-perl tree (as much as possible, granted) then won't this issue 
resolve itself at that date? If it's not, then maybe we should consider 
wrapping this up as an ebuild soon and just letting it go.
Comment 65 J Robert Ray 2003-01-15 07:48:36 UTC
The future is folding this functionality into Portage.

First off, some combination of the source files CPAN uses as its backend
database (ftp://ftp.perl.org/pub/CPAN/authors/01mailrc.txt.gz,
ftp://ftp.perl.org/pub/CPAN/modules/02packages.details.txt.gz,
ftp://ftp.perl.org/pub/CPAN/modules/03modlist.data.gz) will be in our portage
tree somewhere.  This will make all available perl modules visible to Portage,
show up in emerge -[sS], etc.

I think these modules should appear in a virtual category like perl-cpan/... or
something.

emerge can take a special arg to allow modules to be called via their module
name (i.e. Some::Thing) rather than their package name (i.e. perl-cpan/some-thing).

During an emerge of a virtual perl module package, things like what happen in
the g-cpan.pl script will happen.  I feel it wouldn't be a hefty step to avoid
using CPAN.pm at this point.  I used it just so I could resolve 'Some::Thing' to
a package name without having to fetch and parse the aforementioned backend files.

The dependency information is very easily parseable out of the Makefile.PL files.

The steps taken would then be:

Fetch the package, place the tarball in $(DISTFILES).  Extract Makefile.PL from
the tarball and parse the depends.  Generate temporary ebuild and merge.  This
is of course a recursive process.

Now we don't need to keep the temporary ebuild around because we have the
database in the portage tree.  Also, now portage can detect upgraded versions of
modules that are installed.  Portage knows about all the modules without having
to generate ebuilds for them all.

The only sticky part is making the modules we create by hand in dev-perl/* to
mask the ones in the module database.  Perhaps this is something we can do on
the server side when we turn the 3 CPAN database files into something that
Portage will parse.
Comment 66 Seemant Kulleen (RETIRED) gentoo-dev 2003-02-10 08:02:33 UTC
*** Bug 5474 has been marked as a duplicate of this bug. ***
Comment 67 Seth Chandler 2003-03-03 20:42:15 UTC
root@lhotse:/home/sethbc/oggasm-1.4.0# /usr/lib/portage/bin/g-cpan.pl 
Bareword found where operator expected at /usr/lib/portage/bin/g-cpan.pl line 296, near ")
    PORTAGE_DIR"
        (Missing operator before PORTAGE_DIR?)
Can't modify constant item in scalar assignment at /usr/lib/portage/bin/g-cpan.pl line 296, near "portageq portdir_overlay)
    "
syntax error at /usr/lib/portage/bin/g-cpan.pl line 296, near "portageq portdir_overlay)
    "
syntax error at /usr/lib/portage/bin/g-cpan.pl line 311, near "}"
Execution of /usr/lib/portage/bin/g-cpan.pl aborted due to compilation errors.
root@lhotse:/home/sethbc/oggasm-1.4.0# 
****************************
well...someone needs to brush up on perl i guess...me included :-)
Comment 68 Nicholas Jones (RETIRED) gentoo-dev 2003-03-10 20:04:28 UTC
Ok. Fixed in cvs for 47-r9

Please make a new bug... Reference this one.
Don't reopen it. This one is way too big.