Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 317311

Summary: dev-db/cassandra - an open source distributed database management system designed to handle large amounts of data
Product: Gentoo Linux Reporter: Mina Naguib <mina>
Component: New packagesAssignee: Default Assignee for New Packages <maintainer-wanted>
Status: CONFIRMED ---    
Severity: enhancement CC: chris.burroughs, damienkallison, java, JoelKoglin, martinharrigan, patrick, theblackpeter, tokenmathematician
Priority: High Keywords: EBUILD
Version: unspecified   
Hardware: All   
OS: Linux   
URL: http://cassandra.apache.org/
See Also: https://bugs.gentoo.org/show_bug.cgi?id=497904
Whiteboard:
Package list:
Runtime testing required: ---
Attachments: New ebuild: cassandra-0.6.1.ebuild
Supporting files needed by ebuild
Fixes wrong home dir for cassandra user
Fix the location of javadocs (javadocs/ to build/javadoc)
Fix the location of javadocs (javadocs/ to build/javadoc) (yes, actually javadoc)
Ebuild for cassandra 0.6.5
Supporting files needed by the 0.6.5 ebuild
Ebuild for cassandra 0.7.4
Supporting files needed by the 0.7.4 ebuild
Patch for Cassandra 0.7.4 ebuild to fix documentation install (again s/ build/javadocs to build/javadoc)

Description Mina Naguib 2010-04-26 13:59:14 UTC
Attached is a new ebuild for the apache-cassandra database project.

Reproducible: Always
Comment 1 Mina Naguib 2010-04-26 14:00:03 UTC
Created attachment 229251 [details]
New ebuild: cassandra-0.6.1.ebuild
Comment 2 Mina Naguib 2010-04-26 14:04:44 UTC
Created attachment 229253 [details]
Supporting files needed by ebuild
Comment 3 Mina Naguib 2010-04-28 20:13:00 UTC
Created attachment 229567 [details]
Fixes wrong home dir for cassandra user
Comment 4 Robin Kauffman 2010-04-30 00:32:31 UTC
Created attachment 229761 [details, diff]
Fix the location of javadocs (javadocs/ to build/javadoc)

When building with 'doc' USE flag, ebuild looks in the wrong location for generated docs.  When the path is fixed, the package merges.  Unless this is a really weird/bad location for javadocs, it makes more sense to modify the ebuild than to patch the antfile or change the build process.
Comment 5 Robin Kauffman 2010-04-30 01:13:02 UTC
Created attachment 229763 [details, diff]
Fix the location of javadocs (javadocs/ to build/javadoc) (yes, actually javadoc)

Previous patch had a typo and will still fail to merge with USE flag set.  Should be fixed now.
Comment 6 Mina Naguib 2010-10-02 20:52:05 UTC
Created attachment 249359 [details]
Ebuild for cassandra 0.6.5
Comment 7 Mina Naguib 2010-10-02 20:53:35 UTC
Created attachment 249364 [details]
Supporting files needed by the 0.6.5 ebuild
Comment 8 Alistair Bush (RETIRED) gentoo-dev 2010-10-03 07:36:37 UTC
(In reply to comment #6)
> Created an attachment (id=249359) [details]
> Ebuild for cassandra 0.6.5
> 

Just had a quick look am i'm concerned about the "addwrite /root/.ivy2/" line.  This should not be needed and demonstrates that something is wrong.

Also this ebuild downloads jar's during its build.  This needs to be fixed.  Most of those downloaded seem to be already packaged which is good.  There are also bundled jars which will needs to be replaced with system jars.

init.d and conf.d need some work too but they are a good starting place.
Comment 9 Mina Naguib 2010-11-15 03:04:23 UTC
(In reply to comment #8)

Hi Alistair

Cassandra uses Apache ANT + Ivy2 at build time, which manages many things including downloading external dependencies and compiling them.  The dependencies are quite deep as they manage kick-starting other build-time dependencies, not just final run-time dependencies.

The hairiness here is making two build + package environments (portage + ivy) happy.  I did my best since I couldn't find any other ebuilds that do anything similar.

My options to keep ivy happy were to let it use its default cache location (user's HOME/.ivy2), or explicitly set it (to a static location, or a temporary one).

I opted to keep the default, since that means the cached ivy objects are:
1. Re-usable outside the context of portage
2. Re-usable by multiple invocations of portage (different ebuilds, same ebuild across re-installs or upgrades)

I'd be happy to implement concrete suggestions regarding portage + ivy if there are any.

JARs-wise, this ebuild currently ends up installing:
/usr/share/cassandra/lib/antlr-3.1.3.jar
/usr/share/cassandra/lib/cassandra.jar
/usr/share/cassandra/lib/clhm-production.jar
/usr/share/cassandra/lib/commons-cli-1.1.jar
/usr/share/cassandra/lib/commons-codec-1.2.jar
/usr/share/cassandra/lib/commons-collections-3.2.1.jar
/usr/share/cassandra/lib/commons-lang-2.4.jar
/usr/share/cassandra/lib/google-collections-1.0.jar
/usr/share/cassandra/lib/hadoop-core-0.20.1.jar
/usr/share/cassandra/lib/high-scale-lib.jar
/usr/share/cassandra/lib/jackson-core-asl-1.4.0.jar
/usr/share/cassandra/lib/jackson-mapper-asl-1.4.0.jar
/usr/share/cassandra/lib/jline-0.9.94.jar
/usr/share/cassandra/lib/json-simple-1.1.jar
/usr/share/cassandra/lib/libthrift-r917130.jar
/usr/share/cassandra/lib/log4j-1.2.14.jar
/usr/share/cassandra/lib/slf4j-api-1.5.8.jar
/usr/share/cassandra/lib/slf4j-log4j12-1.5.8.jar

If my understanding is correct, each one of these would need to be converted to a dedicated ebuild, SLOTted to that particular version to ensure API/ABI compatibility.  I may attack this if we find a solution to IVY.
Comment 10 Alistair Bush (RETIRED) gentoo-dev 2010-11-15 08:21:45 UTC
(In reply to comment #9)
> (In reply to comment #8)
> 
> Hi Alistair
> 
> Cassandra uses Apache ANT + Ivy2 at build time, which manages many things
> including downloading external dependencies and compiling them.  The
> dependencies are quite deep as they manage kick-starting other build-time
> dependencies, not just final run-time dependencies.
> 
> The hairiness here is making two build + package environments (portage + ivy)
> happy.  I did my best since I couldn't find any other ebuilds that do anything
> similar.
> 
> My options to keep ivy happy were to let it use its default cache location
> (user's HOME/.ivy2), or explicitly set it (to a static location, or a temporary
> one).
> 

Try setting it to ${T} or ${S}

> I opted to keep the default, since that means the cached ivy objects are:
> 1. Re-usable outside the context of portage
> 2. Re-usable by multiple invocations of portage (different ebuilds, same ebuild
> across re-installs or upgrades)
> 

Yeah sadly that is just wrong (against gentoo policy).   Think of it this way.  We manifest the source files that a package declares within SRC_URI so that we can ensure they don't change.  There have been cases where src releases have been infected by malware.  Your current way has no protection against that.   Also as I said,  most of those packages already exist within the tree so you are subjecting users to 3+ copies of the jars.  (1) The portage package (potentially) (2) the /root/.ivy jar and (3) the cassandra copy.

> I'd be happy to implement concrete suggestions regarding portage + ivy if there
> are any.
> 
> JARs-wise, this ebuild currently ends up installing:
> /usr/share/cassandra/lib/antlr-3.1.3.jar

dev-java/antlr:3 and call java-pkg_jar-from --into .... antlr-3

> /usr/share/cassandra/lib/cassandra.jar
> /usr/share/cassandra/lib/clhm-production.jar
> /usr/share/cassandra/lib/commons-cli-1.1.jar

dev-java/commons-cli:1 and call java-pkg_jar-from --into .... commons-cli

> /usr/share/cassandra/lib/commons-codec-1.2.jar

etc

> /usr/share/cassandra/lib/commons-collections-3.2.1.jar

etc

> /usr/share/cassandra/lib/commons-lang-2.4.jar

etc

> /usr/share/cassandra/lib/google-collections-1.0.jar

might not have this one
> /usr/share/cassandra/lib/hadoop-core-0.20.1.jar

or this

> /usr/share/cassandra/lib/high-scale-lib.jar

or this
> /usr/share/cassandra/lib/jackson-core-asl-1.4.0.jar

pretty sure this is an etc

> /usr/share/cassandra/lib/jackson-mapper-asl-1.4.0.jar

as with this

> /usr/share/cassandra/lib/jline-0.9.94.jar

definitely a java-pkg_jar-from candidate

> /usr/share/cassandra/lib/json-simple-1.1.jar

and this as well
> /usr/share/cassandra/lib/libthrift-r917130.jar

mmmm...  not sure on this one

> /usr/share/cassandra/lib/log4j-1.2.14.jar
> /usr/share/cassandra/lib/slf4j-api-1.5.8.jar
> /usr/share/cassandra/lib/slf4j-log4j12-1.5.8.jar

oh more java-pkg_jar-from one

> 
> If my understanding is correct, each one of these would need to be converted to
> a dedicated ebuild, SLOTted to that particular version to ensure API/ABI
> compatibility.  I may attack this if we find a solution to IVY.
> 

We dont slot version so much as slot breaking api changes.

a package with the following ebuilds

slot 1
pkg-1.0.ebuild
pkg-1.1.ebuild

slot 2
pkg-2.0

we can always have a dependency of

>=cat/pkg-1.1:1

which would only match the pkg-1.1 ebuild.
Comment 11 Mina Naguib 2010-11-15 09:04:40 UTC
(In reply to comment #10)

Thanks for the feedback.

> > My options to keep ivy happy were to let it use its default cache location
> > (user's HOME/.ivy2), or explicitly set it (to a static location, or a temporary
> > one).
> 
> Try setting it to ${T} or ${S}

I don't mind, however that will mean that the ivy cache will be cold every invocation.  That means every time it's needed, it'll be re-populated by downloading and building all the dependencies.  See below.

> > I opted to keep the default
>
> Yeah sadly that is just wrong (against gentoo policy).   Think of it this way. 
> We manifest the source files that a package declares within SRC_URI so that we
> can ensure they don't change.  There have been cases where src releases have
> been infected by malware.  Your current way has no protection against that.  
> Also as I said,  most of those packages already exist within the tree so you
> are subjecting users to 3+ copies of the jars.  (1) The portage package
> (potentially) (2) the /root/.ivy jar and (3) the cassandra copy.

Understood, but that leads me to conclude that there's simply no way to mix portage and ivy-built packages.

If we keep the ebuild downloading and compiling from source (as recommended by the gentoo docs), we have to delegate the build step to what the developers define and that uses ivy, which means we lose control over individual remote resources and it has to do its thing.  We therefore have to delegate the verification of signatures to it etc..

As I've mentioned, ivy doesn't "just" download the extra 17 JARs needed at run-time.  Here's a full list (~6Megs) of what the cache looks like, all needed at various stages (build/test/doc/runtime/other..)

/root/.ivy2/cache
/root/.ivy2/cache/ant
/root/.ivy2/cache/ant/ant
/root/.ivy2/cache/ant/ant/jars
/root/.ivy2/cache/ant/ant/jars/ant-1.6.5.jar
/root/.ivy2/cache/ant/ant/ivydata-1.6.5.properties
/root/.ivy2/cache/ant/ant/ivy-1.6.5.xml.original
/root/.ivy2/cache/ant/ant/ivy-1.6.5.xml
/root/.ivy2/cache/asm
/root/.ivy2/cache/asm/asm
/root/.ivy2/cache/asm/asm/ivy-3.2.xml.original
/root/.ivy2/cache/asm/asm/jars
/root/.ivy2/cache/asm/asm/jars/asm-3.2.jar
/root/.ivy2/cache/asm/asm/ivy-3.2.xml
/root/.ivy2/cache/asm/asm/ivydata-3.2.properties
/root/.ivy2/cache/asm/asm-parent
/root/.ivy2/cache/asm/asm-parent/ivy-3.2.xml.original
/root/.ivy2/cache/asm/asm-parent/ivy-3.2.xml
/root/.ivy2/cache/asm/asm-parent/ivydata-3.2.properties
/root/.ivy2/cache/commons-collections
/root/.ivy2/cache/commons-collections/commons-collections
/root/.ivy2/cache/commons-collections/commons-collections/ivy-3.2.xml.original
/root/.ivy2/cache/commons-collections/commons-collections/jars
/root/.ivy2/cache/commons-collections/commons-collections/jars/commons-collections-3.2.jar
/root/.ivy2/cache/commons-collections/commons-collections/ivy-3.2.xml
/root/.ivy2/cache/commons-collections/commons-collections/ivydata-3.2.properties
/root/.ivy2/cache/junit
/root/.ivy2/cache/junit/junit
/root/.ivy2/cache/junit/junit/jars
/root/.ivy2/cache/junit/junit/jars/junit-4.6.jar
/root/.ivy2/cache/junit/junit/ivy-4.6.xml
/root/.ivy2/cache/junit/junit/sources
/root/.ivy2/cache/junit/junit/sources/junit-4.6-sources.jar
/root/.ivy2/cache/junit/junit/ivy-4.6.xml.original
/root/.ivy2/cache/junit/junit/ivydata-4.6.properties
/root/.ivy2/cache/log4j
/root/.ivy2/cache/log4j/log4j
/root/.ivy2/cache/log4j/log4j/jars
/root/.ivy2/cache/log4j/log4j/jars/log4j-1.2.12.jar
/root/.ivy2/cache/log4j/log4j/ivy-1.2.12.xml
/root/.ivy2/cache/log4j/log4j/ivydata-1.2.12.properties
/root/.ivy2/cache/log4j/log4j/ivy-1.2.12.xml.original
/root/.ivy2/cache/org.apache.rat
/root/.ivy2/cache/org.apache.rat/apache-rat
/root/.ivy2/cache/org.apache.rat/apache-rat/ivydata-0.6.properties
/root/.ivy2/cache/org.apache.rat/apache-rat/jars
/root/.ivy2/cache/org.apache.rat/apache-rat/jars/apache-rat-0.6.jar
/root/.ivy2/cache/org.apache.rat/apache-rat/ivy-0.6.xml
/root/.ivy2/cache/org.apache.rat/apache-rat/ivy-0.6.xml.original
/root/.ivy2/cache/org.apache.rat/apache-rat-tasks
/root/.ivy2/cache/org.apache.rat/apache-rat-tasks/ivydata-0.6.properties
/root/.ivy2/cache/org.apache.rat/apache-rat-tasks/jars
/root/.ivy2/cache/org.apache.rat/apache-rat-tasks/jars/apache-rat-tasks-0.6.jar
/root/.ivy2/cache/org.apache.rat/apache-rat-tasks/ivy-0.6.xml
/root/.ivy2/cache/org.apache.rat/apache-rat-tasks/ivy-0.6.xml.original
/root/.ivy2/cache/org.apache.rat/apache-rat-project
/root/.ivy2/cache/org.apache.rat/apache-rat-project/ivydata-0.6.properties
/root/.ivy2/cache/org.apache.rat/apache-rat-project/ivy-0.6.xml
/root/.ivy2/cache/org.apache.rat/apache-rat-project/ivy-0.6.xml.original
/root/.ivy2/cache/org.apache.rat/apache-rat-core
/root/.ivy2/cache/org.apache.rat/apache-rat-core/ivydata-0.6.properties
/root/.ivy2/cache/org.apache.rat/apache-rat-core/jars
/root/.ivy2/cache/org.apache.rat/apache-rat-core/jars/apache-rat-core-0.6.jar
/root/.ivy2/cache/org.apache.rat/apache-rat-core/ivy-0.6.xml
/root/.ivy2/cache/org.apache.rat/apache-rat-core/ivy-0.6.xml.original
/root/.ivy2/cache/commons-cli
/root/.ivy2/cache/commons-cli/commons-cli
/root/.ivy2/cache/commons-cli/commons-cli/jars
/root/.ivy2/cache/commons-cli/commons-cli/jars/commons-cli-1.1.jar
/root/.ivy2/cache/commons-cli/commons-cli/ivydata-1.1.properties
/root/.ivy2/cache/commons-cli/commons-cli/ivy-1.1.xml.original
/root/.ivy2/cache/commons-cli/commons-cli/ivy-1.1.xml
/root/.ivy2/cache/commons-logging
/root/.ivy2/cache/commons-logging/commons-logging
/root/.ivy2/cache/commons-logging/commons-logging/jars
/root/.ivy2/cache/commons-logging/commons-logging/jars/commons-logging-1.1.1.jar
/root/.ivy2/cache/commons-logging/commons-logging/ivydata-1.1.1.properties
/root/.ivy2/cache/commons-logging/commons-logging/ivy-1.1.1.xml
/root/.ivy2/cache/commons-logging/commons-logging/sources
/root/.ivy2/cache/commons-logging/commons-logging/sources/commons-logging-1.1.1-sources.jar
/root/.ivy2/cache/commons-logging/commons-logging/ivy-1.1.1.xml.original
/root/.ivy2/cache/commons-logging/commons-logging/javadocs
/root/.ivy2/cache/commons-logging/commons-logging/javadocs/commons-logging-1.1.1-javadoc.jar
/root/.ivy2/cache/ivy-report.css
/root/.ivy2/cache/ivy-report.xsl
/root/.ivy2/cache/apache-cassandra-cassandra-default.xml
/root/.ivy2/cache/avalon-framework
/root/.ivy2/cache/avalon-framework/avalon-framework
/root/.ivy2/cache/avalon-framework/avalon-framework/jars
/root/.ivy2/cache/avalon-framework/avalon-framework/jars/avalon-framework-4.1.3.jar
/root/.ivy2/cache/avalon-framework/avalon-framework/ivy-4.1.3.xml.original
/root/.ivy2/cache/avalon-framework/avalon-framework/ivydata-4.1.3.properties
/root/.ivy2/cache/avalon-framework/avalon-framework/ivy-4.1.3.xml
/root/.ivy2/cache/commons-lang
/root/.ivy2/cache/commons-lang/commons-lang
/root/.ivy2/cache/commons-lang/commons-lang/jars
/root/.ivy2/cache/commons-lang/commons-lang/jars/commons-lang-2.1.jar
/root/.ivy2/cache/commons-lang/commons-lang/ivy-2.1.xml.original
/root/.ivy2/cache/commons-lang/commons-lang/ivydata-2.1.properties
/root/.ivy2/cache/commons-lang/commons-lang/ivy-2.1.xml
/root/.ivy2/cache/net.java.dev.jna
/root/.ivy2/cache/net.java.dev.jna/jna
/root/.ivy2/cache/net.java.dev.jna/jna/jars
/root/.ivy2/cache/net.java.dev.jna/jna/jars/jna-3.2.7.jar
/root/.ivy2/cache/net.java.dev.jna/jna/sources
/root/.ivy2/cache/net.java.dev.jna/jna/sources/jna-3.2.7-sources.jar
/root/.ivy2/cache/net.java.dev.jna/jna/ivy-3.2.7.xml
/root/.ivy2/cache/net.java.dev.jna/jna/ivy-3.2.7.xml.original
/root/.ivy2/cache/net.java.dev.jna/jna/ivydata-3.2.7.properties
/root/.ivy2/cache/com.thoughtworks.qdox
/root/.ivy2/cache/com.thoughtworks.qdox/qdox
/root/.ivy2/cache/com.thoughtworks.qdox/qdox/jars
/root/.ivy2/cache/com.thoughtworks.qdox/qdox/jars/qdox-1.10.jar
/root/.ivy2/cache/com.thoughtworks.qdox/qdox/ivy-1.10.xml.original
/root/.ivy2/cache/com.thoughtworks.qdox/qdox/ivy-1.10.xml
/root/.ivy2/cache/com.thoughtworks.qdox/qdox/ivydata-1.10.properties
/root/.ivy2/cache/logkit
/root/.ivy2/cache/logkit/logkit
/root/.ivy2/cache/logkit/logkit/jars
/root/.ivy2/cache/logkit/logkit/jars/logkit-1.0.1.jar
/root/.ivy2/cache/logkit/logkit/ivy-1.0.1.xml
/root/.ivy2/cache/logkit/logkit/ivydata-1.0.1.properties
/root/.ivy2/cache/logkit/logkit/ivy-1.0.1.xml.original
/root/.ivy2/cache/resolved-apache-cassandra-cassandra-working@myhostname.xml
/root/.ivy2/cache/org.apache
/root/.ivy2/cache/org.apache/apache
/root/.ivy2/cache/org.apache/apache/ivydata-4.properties
/root/.ivy2/cache/org.apache/apache/ivydata-5.properties
/root/.ivy2/cache/org.apache/apache/ivy-5.xml.original
/root/.ivy2/cache/org.apache/apache/ivy-4.xml
/root/.ivy2/cache/org.apache/apache/ivy-5.xml
/root/.ivy2/cache/org.apache/apache/ivy-4.xml.original
/root/.ivy2/cache/com.thoughtworks.paranamer
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-parent
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-parent/ivy-2.1.xml.original
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-parent/ivydata-2.1.properties
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-parent/ivy-2.1.xml
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-generator
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-generator/jars
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-generator/jars/paranamer-generator-2.1.jar
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-generator/ivy-2.1.xml.original
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-generator/ivydata-2.1.properties
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-generator/ivy-2.1.xml
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-ant
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-ant/jars
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-ant/jars/paranamer-ant-2.1.jar
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-ant/sources
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-ant/sources/paranamer-ant-2.1-sources.jar
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-ant/ivy-2.1.xml.original
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-ant/javadocs
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-ant/javadocs/paranamer-ant-2.1-javadoc.jar
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-ant/ivydata-2.1.properties
/root/.ivy2/cache/com.thoughtworks.paranamer/paranamer-ant/ivy-2.1.xml
/root/.ivy2/cache/org.apache.commons
/root/.ivy2/cache/org.apache.commons/commons-parent
/root/.ivy2/cache/org.apache.commons/commons-parent/ivydata-5.properties
/root/.ivy2/cache/org.apache.commons/commons-parent/ivy-5.xml.original
/root/.ivy2/cache/org.apache.commons/commons-parent/ivy-5.xml
/root/.ivy2/cache/resolved-apache-cassandra-cassandra-working@myhostname.properties
/root/.ivy2/cache/javax.servlet
/root/.ivy2/cache/javax.servlet/servlet-api
/root/.ivy2/cache/javax.servlet/servlet-api/jars
/root/.ivy2/cache/javax.servlet/servlet-api/jars/servlet-api-2.3.jar
/root/.ivy2/cache/javax.servlet/servlet-api/ivydata-2.3.properties
/root/.ivy2/cache/javax.servlet/servlet-api/ivy-2.3.xml
/root/.ivy2/cache/javax.servlet/servlet-api/ivy-2.3.xml.original

I'd imagine letting ivy do its thing, while maintaining portage's validation of remote resource signatures, would be unfeasible without an incredible amount of work to handle the above.

My other option, which I've already pursued but had abandoned, is to have a cassandra-bin instead of cassandra ebuild.  That would skip building (and all the ivy stuff), at the cost of (also) being against the gentoo java docs' recommendations (which I agree with).

> We dont slot version so much as slot breaking api changes.
> 
> a package with the following ebuilds
> 
> slot 1
> pkg-1.0.ebuild
> pkg-1.1.ebuild
> 
> slot 2
> pkg-2.0
> 
> we can always have a dependency of
> 
> >=cat/pkg-1.1:1
> 
> which would only match the pkg-1.1 ebuild.

I'm slightly worried about this one.  Assuming we've solved the IVY issue and are simply attacking the issue of ownership of the 17 run-time JARs

Let's say cassandra, when downloaded and installed regularly (-bin or -src), comes with libfoo-1.1 - I'd like to have the gentoo-installed cassandra use the exact same version to avoid any subtle bugs.  It'd also allow users to contact the Cassandra project with issues they encounter without being told it's a modified/unsupported platform.  That means it should depend on exactly =libfoo-1.1, not just libfoo-1 or >=libfoo-1.1

So, the question is, assuming libfoo-1.1 and libfoo-1.2 are API+ABI compatible and therefore in the same SLOT, what happens when package A requires =libfoo-1.1 and package B requires =libfoo-1.2 ?
Comment 12 Alistair Bush (RETIRED) gentoo-dev 2010-11-16 07:22:10 UTC
(In reply to comment #11)
> 
> Understood, but that leads me to conclude that there's simply no way to mix
> portage and ivy-built packages.

Yes,  as in we don't use them.

> 
> If we keep the ebuild downloading and compiling from source (as recommended by
> the gentoo docs), we have to delegate the build step to what the developers
> define and that uses ivy, which means we lose control over individual remote
> resources and it has to do its thing.  We therefore have to delegate the
> verification of signatures to it etc..
> 
> As I've mentioned, ivy doesn't "just" download the extra 17 JARs needed at
> run-time.  Here's a full list (~6Megs) of what the cache looks like, all needed
> at various stages (build/test/doc/runtime/other..)

I suppose there is a distinction between what is actually need, and what isn't.  
Initially we only need build and runtime deps the others could be safely ignored.

> I'd imagine letting ivy do its thing, while maintaining portage's validation of
> remote resource signatures, would be unfeasible without an incredible amount of
> work to handle the above.

Yes

> 
> My other option, which I've already pursued but had abandoned, is to have a
> cassandra-bin instead of cassandra ebuild.  That would skip building (and all
> the ivy stuff), at the cost of (also) being against the gentoo java docs'
> recommendations (which I agree with).

cassandra-bin will not make it into the tree.  from my perspective there is no point.
I can see you heading into a certain direction which is down a different path than
gentoo's sadly.

> 
> > We dont slot version so much as slot breaking api changes.
> 
> I'm slightly worried about this one.  Assuming we've solved the IVY issue and
> are simply attacking the issue of ownership of the 17 run-time JARs
> 
> Let's say cassandra, when downloaded and installed regularly (-bin or -src),
> comes with libfoo-1.1 - I'd like to have the gentoo-installed cassandra use the
> exact same version to avoid any subtle bugs.  It'd also allow users to contact
> the Cassandra project with issues they encounter without being told it's a
> modified/unsupported platform.  That means it should depend on exactly
> =libfoo-1.1, not just libfoo-1 or >=libfoo-1.1

All existing java packages use this policy.  including ones that have a **LOT** of deps
such as tomcat, eclipse and netbeans.  So far issues like that have been minor 
compared to the one you are facing right now.  Ever package depending on =libfoo-1.1 is
sadly unworkable,  we will have to kill 90% of the "java tree" to be able to maintain it.

There are also some other interesting implications that you may not have thought about.
imagine situation where a package has security issues and is a dependency of other packages.
How difficult would it be to maintain any security fixes across x many version that require it.
Maintainablity would be a nightmare.

> 
> So, the question is, assuming libfoo-1.1 and libfoo-1.2 are API+ABI compatible
> and therefore in the same SLOT, what happens when package A requires
> =libfoo-1.1 and package B requires =libfoo-1.2 ?
> 
Bang! is probably an appropriate response.  Maybe a little over the top,  but im in
that sort of mood atm :)

From what you are saying I would completely stop creating a .ebuild and would rather just
use paludis's cave client which can install a package based off an "image" directory.
see http://paludis.pioto.org/clients/cave-import.html
Comment 13 Serkan Kaba (RETIRED) gentoo-dev 2010-11-22 20:28:57 UTC
Apache ivy tries to do what maven does, but for ant. We've been discussing these for maven build system as well.

To simply say, the downloading of the dependencies should be disabled and should be replaced by java-pkg_jar-from calls (i.e using system jars). Alistair already pointed out some of the existing packages in the tree. The rest needs to be packaged as well.
Comment 14 Joel Koglin 2010-12-29 04:50:42 UTC
Would it be possible to make an x86 version of this? I would be glad to test it.
Comment 15 Mina Naguib 2011-03-26 15:58:25 UTC
Hi

Unfortunately, I haven't had time to investigate which bundled JARs have their own ebuilds, and which will need new ebuilds created.  Realistically speaking, this is not something I see myself having time to do in the near future either.

Internally at my company, I'm maintaining this ebuild (up to cassandra 0.7.4 now) and running it in production.

If someone has enough time to investigate these (0.7.4) bundled JARs and take on the responsibility of ebuild-ifying them, I'd be happy to post my 0.7.4 ebuild and pass on the baton.


/usr/share/cassandra/lib/antlr-3.1.3.jar
/usr/share/cassandra/lib/avro-1.4.0-fixes.jar
/usr/share/cassandra/lib/avro-1.4.0-sources-fixes.jar
/usr/share/cassandra/lib/commons-cli-1.1.jar
/usr/share/cassandra/lib/commons-codec-1.2.jar
/usr/share/cassandra/lib/commons-collections-3.2.1.jar
/usr/share/cassandra/lib/commons-lang-2.4.jar
/usr/share/cassandra/lib/concurrentlinkedhashmap-lru-1.1.jar
/usr/share/cassandra/lib/guava-r05.jar
/usr/share/cassandra/lib/high-scale-lib.jar
/usr/share/cassandra/lib/jackson-core-asl-1.4.0.jar
/usr/share/cassandra/lib/jackson-mapper-asl-1.4.0.jar
/usr/share/cassandra/lib/jetty-6.1.21.jar
/usr/share/cassandra/lib/jetty-util-6.1.21.jar
/usr/share/cassandra/lib/jline-0.9.94.jar
/usr/share/cassandra/lib/json-simple-1.1.jar
/usr/share/cassandra/lib/jug-2.0.0.jar
/usr/share/cassandra/lib/libthrift-0.5.jar
/usr/share/cassandra/lib/log4j-1.2.16.jar
/usr/share/cassandra/lib/servlet-api-2.5-20081211.jar
/usr/share/cassandra/lib/slf4j-api-1.6.1.jar
/usr/share/cassandra/lib/slf4j-log4j12-1.6.1.jar
/usr/share/cassandra/lib/snakeyaml-1.6.jar
Comment 16 Mina Naguib 2011-04-16 16:50:20 UTC
Created attachment 270215 [details]
Ebuild for cassandra 0.7.4

I've received private emails asking for the 0.7.4 ebuild. Decided to update the ticket with the necessary files.
Comment 17 Mina Naguib 2011-04-16 16:51:22 UTC
Created attachment 270217 [details]
Supporting files needed by the 0.7.4 ebuild

I've received private emails asking for the 0.7.4 ebuild. Decided to update the ticket with the necessary files.
Comment 18 Robin Kauffman 2011-04-17 02:30:28 UTC
Created attachment 270235 [details, diff]
Patch for Cassandra 0.7.4 ebuild to fix documentation install (again s/ build/javadocs to build/javadoc)

A small patch to the cassandra 0.7.4 ebuild allowing documentation install.  Obsoleting earlier patch to 0.6.1 ebuild as 0.6.1 ebuild has been obsoleted by 0.7.4 ebuild.
Comment 19 Patrick Lauer gentoo-dev 2013-06-17 05:38:22 UTC
I've had a look at 1.2.5 - big blocker is how ANT/Ivy doesn't care and tries to fetch random crap.

RDEPEND=">=virtual/jdk-1.6"
DEPEND="${DEPEND}
        dev-java/maven-bin
        dev-java/ant-ivy"

^^ that's a first attempt at getting dependencies, but:

>>> Compiling source in /var/tmp/portage/dev-db/cassandra-1.2.5/work/apache-cassandra-1.2.5-src ...
 * Using following ANT_TASKS: ant-nodeps
Buildfile: /var/tmp/portage/dev-db/cassandra-1.2.5/work/apache-cassandra-1.2.5-src/build.xml

maven-ant-tasks-localrepo:

maven-ant-tasks-download:
     [echo] Downloading Maven ANT Tasks...
    [mkdir] Created dir: /var/tmp/portage/dev-db/cassandra-1.2.5/work/apache-cassandra-1.2.5-src/build
      [get] Getting: http://repo2.maven.org/maven2/org/apache/maven/maven-ant-tasks/2.1.3/maven-ant-tasks-2.1.3.jar
      [get] To: /var/tmp/portage/dev-db/cassandra-1.2.5/work/apache-cassandra-1.2.5-src/build/maven-ant-tasks-2.1.3.jar


And that doesn't even work, so MEH! :)