Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 537314 - app-misc/elasticsearch-1.4.2 - systemd service fails to start: elasticsearch: You must set the ES_CLASSPATH var
Summary: app-misc/elasticsearch-1.4.2 - systemd service fails to start: elasticsearch:...
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo systemd Team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-01-22 05:48 UTC by Austin M. Matherne
Modified: 2015-06-11 15:37 UTC (History)
1 user (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Austin M. Matherne 2015-01-22 05:48:16 UTC
The systemd service for app-misc/elasticsearch-1.4.2 (I'm not sure about earlier versions) fails to start with the error "You must set the ES_CLASSPATH var". 

systemctl status is as follows:

● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib64/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2015-01-21 22:15:25 CST; 1h 29min ago
  Process: 315 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -Des.default.path.conf=/etc/elasticsearch (code=exited, status=1/FAILURE)
 Main PID: 315 (code=exited, status=1/FAILURE)
 
Jan 21 22:15:25 pangodesktop elasticsearch[315]: You must set the ES_CLASSPATH var
Jan 21 22:15:25 pangodesktop systemd[1]: elasticsearch.service: main process exited, code=exited, status=1/FAILURE
Jan 21 22:15:25 pangodesktop systemd[1]: Unit elasticsearch.service entered failed state.
Jan 21 22:15:25 pangodesktop systemd[1]: elasticsearch.service failed.
Comment 1 Ferenc Erki 2015-03-02 23:30:23 UTC
hi Austin,

can you check please if replacing your elasticsearch.in.sh solves this problem? I mean by using something like the following command:

bunzip2 -c /usr/share/doc/elasticsearch-1.4.2/examples/elasticsearch.in.sh.bz2 > /etc/elasticsearch/elasticsearch.in.sh

And/or could you also please try manually adding ES_CLASSPATH to the /usr/lib/systemd/system/elasticsearch.service file, by replacing the Environment line with the following:

Environment=ES_INCLUDE=/etc/elasticsearch/elasticsearch.in.sh ES_CLASSPATH="/usr/share/elasticsearch/lib/elasticsearch-1.4.2.jar:/usr/share/elasticsearch/lib/*:/usr/share/elasticsearch/lib/sigar/*"

Unfortunately I don't have a systemd box at hand currently, but based on the above input I might be able to fix the ebuilds.
Comment 2 Austin M. Matherne 2015-03-04 04:44:22 UTC
Made both changes, it still doesn't work, but the status changed to 3.

● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib64/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2015-03-03 22:38:37 CST; 24s ago
  Process: 3787 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -Des.default.path.conf=/etc/elasticsearch (code=exited, status=3)
 Main PID: 3787 (code=exited, status=3)
Comment 3 Ferenc Erki 2015-03-04 09:49:51 UTC
Thanks for the feedback! Looks like the ES_CLASSPATH problem itself can be solved by one of the methods I suggested above.

Judging by the code, exit code 3 indicates one of the following:
- it could not create the pidfile
- it could not set up logging
- other initialization problem (like the node couldn't start up)

The first two is usually because if improper permission/ownership on the directories. They are forcibly taken care of by the initscript, but I don't think the service file handles it (or how it should exactly handle it). Doing it by hand would be stg like this:
- chown -R elasticsearch /run/elasticsearch /var/lib/elasticseaerch /var/log/elasticsearch
- chmod -R 750 /run/elasticsearch /var/lib/elasticseaerch /var/log/elasticsearch

I'll try to setup a Gentoo VM with systemd a debug it further. If you try some of the above and it works for you or changes the situation, please let us know here.
Comment 4 Austin M. Matherne 2015-03-05 00:14:44 UTC
/run/elasticsearch didn't exist on my system. Creating it and setting permissions still gave the same error (shown below). It was trying to write to /usr/share/elasticsearch/data/ which did not exist either and who's parent was owned by root. If I create the folder and give ownership of it to elasticsearch the service launches.

Here's the log output before I created the folder.
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib64/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2015-03-04 17:57:51 CST; 6s ago
  Process: 3186 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -Des.default.path.conf=/etc/elasticsearch (code=exited, status=3)
 Main PID: 3186 (code=exited, status=3)
 
Mar 04 17:57:51 pangodesktop elasticsearch[3186]: log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Mar 04 17:57:51 pangodesktop elasticsearch[3186]: {1.4.2}: Initialization Failed ...
Mar 04 17:57:51 pangodesktop elasticsearch[3186]: - ElasticsearchIllegalStateException[Failed to obtain node lock, is the following location writable?: [/usr/share/elasticsearch/data/elasticsearch]]
Mar 04 17:57:51 pangodesktop elasticsearch[3186]: IOException[failed to obtain lock on /usr/share/elasticsearch/data/elasticsearch/nodes/49]
Mar 04 17:57:51 pangodesktop elasticsearch[3186]: IOException[Cannot create directory: /usr/share/elasticsearch/data/elasticsearch/nodes/49]
Mar 04 17:57:51 pangodesktop elasticsearch[3186]: org.elasticsearch.ElasticsearchIllegalStateException: Failed to obtain node lock, is the following location writable?: [/usr/share/elasticsearch/data/elasticsearch]
Mar 04 17:57:51 pangodesktop elasticsearch[3186]: at org.elasticsearch.env.NodeEnvironment.<init>(NodeEnvironment.java:116)
Mar 04 17:57:51 pangodesktop systemd[1]: elasticsearch.service: main process exited, code=exited, status=3/NOTIMPLEMENTED
Mar 04 17:57:51 pangodesktop systemd[1]: Unit elasticsearch.service entered failed state.
Mar 04 17:57:51 pangodesktop systemd[1]: elasticsearch.service failed.
Comment 5 Ferenc Erki 2015-03-05 00:27:03 UTC
Looks like I started guessing in the right direction then! :)

When wou could successfully start it, did you use the modified service file, or the extracted elasticsearch.in.sh file? Or both? :) I think using only the latter should be enough if the directory permissions/ownerships are OK.
Comment 6 Austin M. Matherne 2015-03-05 00:43:58 UTC
I used the modified service file (along with creating the data directory). However, I just noticed that it goes to a failed state when I attempt to stop it.

$ systemctl stop elasticsearch.service
$ systemctl status -l elasticsearch.service

● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib64/systemd/system/elasticsearch.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2015-03-04 18:40:48 CST; 6s ago
  Process: 324 ExecStart=/usr/share/elasticsearch/bin/elasticsearch -Des.default.path.conf=/etc/elasticsearch (code=exited, status=143)
 Main PID: 324 (code=exited, status=143)
 
Mar 04 18:38:25 pangodesktop systemd[1]: Started Elasticsearch.
Mar 04 18:38:25 pangodesktop systemd[1]: Starting Elasticsearch...
Mar 04 18:38:25 pangodesktop elasticsearch[324]: log4j:WARN No appenders could be found for logger (common).
Mar 04 18:38:25 pangodesktop elasticsearch[324]: log4j:WARN Please initialize the log4j system properly.
Mar 04 18:38:25 pangodesktop elasticsearch[324]: log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Mar 04 18:40:48 pangodesktop systemd[1]: Stopping Elasticsearch...
Mar 04 18:40:48 pangodesktop systemd[1]: elasticsearch.service: main process exited, code=exited, status=143/n/a
Mar 04 18:40:48 pangodesktop systemd[1]: Stopped Elasticsearch.
Mar 04 18:40:48 pangodesktop systemd[1]: Unit elasticsearch.service entered failed state.
Mar 04 18:40:48 pangodesktop systemd[1]: elasticsearch.service failed.
Comment 7 Ferenc Erki 2015-03-05 11:16:42 UTC
Exit code 143 indicates a process shutdown due to a signal, which is SIGTERM in this case (143-128=15 => SIGTERM, according to kill -l). That seems normal, but of course it would be nice if systemd could report it as a non-failed state..
Comment 8 Austin M. Matherne 2015-03-24 01:26:56 UTC
Can this make its way into portage?
Comment 9 Ferenc Erki 2015-03-24 09:20:27 UTC
I started to work on improving the current initscript/service file. I believe the main problem here is that the elasticsearch.in.sh file should take care of setting up the environment for elasticsearch, but e.g. ES_CLASSPATH is being overridden in the initscript, and being excluded from the service file (while sourcing elasticsearch.in.sh is also missing there).

I think elasticsearch.in.sh needs to be placed to a common location (e.g. upstream deb install it under /usr/share/elasticsearch/bin) and then it need to be made sure that both the initscript and the service file are using it.
Comment 10 Tony Vroon (RETIRED) gentoo-dev 2015-06-11 15:35:46 UTC
+*elasticsearch-1.6.0 (11 Jun 2015)
+
+  11 Jun 2015; Tony Vroon <chainsaw@gentoo.org> -elasticsearch-1.4.4.ebuild,
+  -elasticsearch-1.5.0.ebuild, +elasticsearch-1.6.0.ebuild,
+  +files/elasticsearch.init4, +files/elasticsearch.service3, metadata.xml:
+  Security fix relating to an unspecified arbitrary file modification
+  vulnerability. Ebuild, init script and systemd service file by Ferenc Erki.
+  Closes bug #537314 by Austin M. Matherne and bug #547964 by Tomas Mozes.
+  Adding Ferenc Erki as proxy maintainer. Removing all vulnerable ebuilds for
+  security bug #551776.