Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 454346 - New package: app-misc/elasticsearch - Open Source, Distributed, RESTful, Search Engine
Summary: New package: app-misc/elasticsearch - Open Source, Distributed, RESTful, Sear...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: Normal enhancement (vote)
Assignee: Default Assignee for New Packages
URL: https://github.com/travisghansen/chao...
Whiteboard: InOverlay
Keywords: EBUILD, InOverlay
Depends on:
Blocks:
 
Reported: 2013-01-27 22:22 UTC by Travis Hansen
Modified: 2013-11-07 16:23 UTC (History)
3 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
app-misc/elasticsearch-bin-0.20.3.ebuild (elasticsearch-bin-0.20.3.ebuild,684 bytes, text/plain)
2013-01-28 00:00 UTC, Tom Wijsman (TomWij) (RETIRED)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Travis Hansen 2013-01-27 22:22:16 UTC
From the homepage:

So we build a web site or an application and want to add search to it, and then it hits us: getting search working is hard. We want our search solution to be fast, we want a painless setup and a completely free search schema, we want to be able to index data simply using JSON over HTTP, we want our search server to be always available, we want to be able to start with one machine and scale to hundreds, we want real-time search, we want simple multi-tenancy, and we want a solution that is built for the cloud.
Comment 1 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2013-01-28 00:00:55 UTC
Created attachment 337060 [details]
app-misc/elasticsearch-bin-0.20.3.ebuild

Hmm, unfortunate that they only provide their versions in binary; which means we either need to create a binary package and/or capture snapshots from their repository to create a package that compiles from source.

Wrote a binary package to start with to see how this works out, not sure if I got it all right so take this with a grain of salt; for that reason KEYWORDS is empty and if you want to test it you need to add app-misc/elasticsearch-bin-0.20.3 to /etc/portage/package.accept.

The first issue I spot is that brings along some libraries, this is often not done and you would want the package to depend on libraries provided through other packages Portage and not through the package itself. There is still some work to do here:

>>> /usr/lib64/elasticsearch-0.20.3.jar
>>> /usr/lib64/jna-3.3.0.jar
>>> /usr/lib64/jts-1.12.jar
>>> /usr/lib64/log4j-1.2.17.jar
>>> /usr/lib64/lucene-analyzers-3.6.2.jar
>>> /usr/lib64/lucene-core-3.6.2.jar
>>> /usr/lib64/lucene-highlighter-3.6.2.jar
>>> /usr/lib64/lucene-memory-3.6.2.jar
>>> /usr/lib64/lucene-queries-3.6.2.jar
>>> /usr/lib64/snappy-java-1.0.4.1.jar
>>> /usr/lib64/spatial4j-0.3.jar
>>> /usr/lib64/libsigar-amd64-linux.so

Still a QA notice left during the ebuild, which I assume we need to report upstream (every file in that folder has this):

 * QA Notice: The following shared libraries lack a SONAME
 * /usr/lib64/libsigar-amd64-linux.so

Gave it a shot, throws three log4j warnings on launch as well:

log4j:WARN No appenders could be found for logger (node).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.

The localhost URL listed in their guide seems accessible so at least the basis of it is confirmed to work.
Comment 2 Tom Wijsman (TomWij) (RETIRED) gentoo-dev 2013-01-28 00:04:45 UTC
Also, do you know of a better category to put this package in?

I assumed by the Applications category that this was an application therefore I have chosen app-misc at first; but from the idea I get of this I think it feels more like a web server, do you think this would fit better in www-servers? Or is it not really a server?
Comment 3 Travis Hansen 2013-01-28 01:15:08 UTC
This is awesome! Thanks for the quick response.

In terms of category I'm not really sure where to put it.  It's not quite couchdb (dev-db) and it's not quite memcached (net-misc).  I'd probably go with net-misc if it were me.

Additionly, I'd like to see it in connection with init/conf scripts for sane startup.  My ultimate goal is to use this in conjunction with logstash (#447192) in a datacenter environment.  Once the data is stored I'd like to get Kibana (no bug yet: http://kibana.org/index.html) as a frontend to all the data stored in elasticsearch.

I'll test this ebuild out and anxiously await an init script.

Thanks!
Comment 4 Travis Hansen 2013-01-28 18:47:24 UTC
I went ahead and put together some ebuilds with init scripts.  Overlay available here:

https://github.com/travisghansen/chaos

Seems to be working great so far.  Now time for logstash.
Comment 5 Tony Vroon (RETIRED) gentoo-dev 2013-11-07 09:19:06 UTC
Travis, Thomas, thank you for your work.
Sorry that it took a while to take this one on.

+*elasticsearch-0.90.6 (07 Nov 2013)
+
+  07 Nov 2013; Tony Vroon <chainsaw@gentoo.org> +elasticsearch-0.90.6.ebuild,
+  +files/elasticsearch.conf, +files/elasticsearch.init,
+  +files/elasticsearch.service, +metadata.xml:
+  Initial commit, initial ebuild by Tom Wijsman with further tweaks by Travis
+  G. Hansen around systemd compatibility and traditional init scripts. Silenced
+  QA warnings for pre-built components.

To confirm, I will be maintaining this package in Gentoo and am open to any further tweaks you wish to make. Could you file these as a new bug please.
Comment 6 Travis Hansen 2013-11-07 16:23:27 UTC
Thanks Tony!

Next up is https://bugs.gentoo.org/show_bug.cgi?id=447192 :)