Just unmasked and emerged collectl, service fails to start. hbox # /etc/init.d/collectl start * Caching service dependencies ... [ ok ] * Starting collectl ... /sbin/start-stop-daemon: stat /usr/sbin/collectl: No such file or directory (No such file or directory [ !! ] hbox # which collectl /usr/bin/collectl hbox # cp /etc/init.d/collectl /etc/init.d/collectl.old hbox # nano /etc/init.d/collectl hbox # diff /etc/init.d/collectl /etc/init.d/collectl.old 11c11 < --exec /usr/bin/collectl -- -D --- > --exec /usr/sbin/collectl -- -D hbox # /etc/init.d/collectl start * Caching service dependencies ... [ ok ] * Starting collectl ... [ ok ] hbox # Reproducible: Didn't try Steps to Reproduce: 1. echo "=sys-apps/collectl-3.2.1 ~amd64" >> /etc/portage/package.keywords 2. emerge -av collectl 3. /etc/init.d/collectl start Actual Results: hempenbox phaedrus # /etc/init.d/collectl start * Caching service dependencies ... [ ok ] * Starting collectl ... /sbin/start-stop-daemon: stat /usr/sbin/collectl: No such file or directory (No such file or directory [ !! ] Expected Results: /etc/init.d/collectl start * Caching service dependencies ... [ ok ] * Starting collectl ... [ ok ] emerge --info Portage 2.1.6.13 (default/linux/amd64/10.0, gcc-4.3.4, glibc-2.9_p20081201-r2, 2.6.30-gentoo-r5 x86_64) ================================================================= System uname: Linux-2.6.30-gentoo-r5-x86_64-AMD_Athlon-tm-_64_Processor_3000+-with-gentoo-1.12.13 Timestamp of tree: Sun, 08 Nov 2009 06:45:02 +0000 ccache version 2.4 [enabled] app-shells/bash: 4.0_p28 dev-java/java-config: 2.1.9-r1 dev-lang/python: 2.6.2-r1 dev-util/ccache: 2.4-r7 dev-util/cmake: 2.6.4 sys-apps/baselayout: 1.12.13 sys-apps/sandbox: 1.6-r2 sys-devel/autoconf: 2.13, 2.63-r1 sys-devel/automake: 1.9.6-r2, 1.10.2 sys-devel/binutils: 2.18-r3 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.6a virtual/os-headers: 2.6.27-r2 ACCEPT_KEYWORDS="amd64" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -pipe -march=athlon64" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /var/lib/hsqldb" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/env.d/java/ /etc/fonts/fonts.conf /etc/gconf /etc/php/apache2-php5/ext-active/ /etc/php/cgi-php5/ext-active/ /etc/php/cli-php5/ext-active/ /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c /etc/udev/rules.d" CXXFLAGS="-O2 -pipe -march=athlon64" DISTDIR="/usr/portage/distfiles" FEATURES="ccache distlocks fixpackages parallel-fetch protect-owned sandbox sfperms strict unmerge-orphans userfetch" GENTOO_MIRRORS="http://mirror.leaseweb.com/gentoo/ http://gentoo.mirrors.tds.net/gentoo http://mirror.datapipe.net/gentoo http://gentoo-mirror.spb.ru/ http://ftp.udc.es/gentoo/" LDFLAGS="-Wl,-O1" LINGUAS="en_US" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage" USE="X acl alsa amd64 apache2 bash-completion berkdb bzip2 cli consolekit cracklib crypt cups dri fontforge fortran gd gdbm gif gpm gtk iconv java jpeg jpg lm_sensors math mmx modules mp3 mudflap multilib mysql ncurses nls nptl nptlonly nsplugin offensive ogg opengl openmp pam pcre perl png pppd pulseaudio python readline reflection samba session spell spl sse sse2 ssl sysfs tcpd tiff truetype unicode vhosts vorbis xml xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard mouse joystick" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en_US" USERLAND="GNU" VIDEO_CARDS="radeon" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, FFLAGS, INSTALL_MASK, LANG, LC_ALL, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, PORTDIR_OVERLAY
Was this installed using the INSTALL script found in the tarball? I'd seen this sort of thing before and had made some changes to both the start script as well as INSTALL since V3.2.1. Maybe that will fix the problem? If not, let me know exactly what you did to install it and I'll make sure the problem doesn't reoccur. -mark
I'm afraid that I was not aware of said INSTALL script. The only step I failed to mention was having a look at /etc/collectl.conf, which I do not believe I made any changes to. It does seem to be working, but I haven't made time to play with the logs.
Digging a little deeper into this I installed gentoo and then did an INSTALL of collectl from the taball on top of this and though collectl did indeed install and run, I also did see a few problems. There was no specific gentoo section in my INSTALL script and so it made some assumptions in the 'generic' distro processing section that were wrong for gentoo. I've since identified the problems and have enhanced the installation process to put the start script into /etc/init.d as well as to run rc-update to place collectl in /etc/runlevels/default. The other thing I found if you try to use collectl to monitor infiniband is uses lspci to verify there is interconnect hardware and since it can't find lspci it dies. Is lspci not available as part of gentoo or when I did my build did I just leave it off? Just remember, if you do install it and only run interactively it appears to run just fine as is, as long as you don't try to monitor your IB traffic. Please keep me in the loop as I'm getting ready for a new release and will be happy to incorporate any additional changes that will help it install more seamlessly on a gentoo distro. As a final question, is there anything I can do to help make it part of the standard distro? Or doesn't gentoo add that sort of thing? -mark
(In reply to comment #3) > Is lspci not available as part of gentoo or when I did my build > did I just leave it off? I just came back to Gentoo myself within the last couple weeks and was surprised to find lspci not part of the tarball. I'm sure there is a reason, like people wanting to run non-pci machines? Maybe you want a USE flag for infiniband that depends on sys-apps/pciutils > > Just remember, if you do install it and only run interactively it appears to > run just fine as is, as long as you don't try to monitor your IB traffic. > > Please keep me in the loop as I'm getting ready for a new release and will be > happy to incorporate any additional changes that will help it install more > seamlessly on a gentoo distro. > > As a final question, is there anything I can do to help make it part of the > standard distro? Or doesn't gentoo add that sort of thing? Gentoo makes no assumptions about what we want. For example, the handbook has a section on installing a cron deamon and system logger, with a suggestion about installing logrotate. The default is none of the above. I imagine that your best bet on getting onto new setups is to get a mention in the handbook, though I've no clue as to what that would take. Here's hoping that we can pull somebody else into this convo that knows the technicalities and politics of the distro better and can help in that regard.
My philosophy with collectl, right or wrong, has been to start with making sure everything works on redhat and as I find exceptions to deal with them. So in the case of lspsi not being there I'll simply disable supporting features that require it. I also found the same to happen in the case of ethtool and dmidecode. In the case of a missing ethtool it already informs the user that w/o it collectl can't report network speeds in the header. My intent is to go through things in more detail and determine exactly what is lost is a utility is missing and try me best to 'do the right thing' in their absence. For example I use dmidecode to determine the actual product name of the hardware which in turn allows me to figure out how to parse the ipmitool output. However, if it's not there one can still supply their own parsing rules, it's just more painful. Maybe as collectl continues to gain in popularity more users will provide additional ideas to help it run in difference environments. -mark
the error in question is trivial to fix, but i'm kind of inclined to just drop the package. the INSTALL script might work fine on local installs by people, but it doesnt fit with packaging systems at all. nor does the install tree. a properly packaged tree goes into /usr/, not some random /opt/ subdir. not sure how you want to address this Mark.
re packaging system: I really don't know much about this as I'm pretty new to gentoo re /usr: I have no idea it this addresses your point or not, but the root where collectl is installed by a single line in the install script and could be easily changed to /usr/collectl or whatever someone might want BUT there is a lot of stuff under that directory including documentation, install/uninstall scripts and release notes. Furthermore, if someone installs the collectl-utils package you get additional stuff sch as examples and libraries. The thing I personally like about this structure is it puts everything in one place and makes it easier to manage, especially when there's simply no other place to put the other things. I went through this same thing with debian packaging - nobody could tell me where to put the things that didn't have a natural home and they didn't like the idea of a /usr/collectl. Would that work for gentoo instead of /opt/hp/collectl? re /opt/hp/collectl: It's not really that random as HP uses that to install a lot of their utilities. I was under the impression others used it as well. Actually I just found this: http://www.linuxtopia.org/online_books/linux_beginner_books/linux_filesystem/opt.html, admitting just because you find it on the Internet doesn't make it so, but the 1st sentence says, "This directory is reserved for all the software and add-on packages that are not part of the default installation." -mark
Hi Mark, The only time I personally have seen /opt used is for closed source software. In comment #6, Mike is referring to the /usr hierarchy of the fhs, which can be found at http://www.pathname.com/fhs/pub/fhs-2.3.html. Basically you want the binaries installed in /usr/bin (or /usr/sbin if only the system administrator can run them), man pages in the appropriate directory under /usr/share/man, documentation in /usr/share/doc/packagename-version, and config file in /etc. What are the other files that you think do not have a natural home?
my packaging points arent specific to Gentoo. they'll apply to any and all distribution. William pointed out the FHS which is what i was thinking. if you're looking for a site-independent location to store read-only files (i.e. the .ph files), then /usr/share/collectl/ is probably the best location. changing the one line in the INSTALL script isnt sufficient. your /opt quotation is appropriate, iff it is a person doing an install. when it is integrated into a distro's package manager, it is not appropriate. there is also the problem that the script will check and operate on the root fs. a packagable package should respect some variable (the most common being DESTDIR) and use that as the root. so when you create symlinks and such, you should be doing it in $DESTDIR/..... also, the distro checks would be nice if you also backed that up a step to let the packager indicate which distro they are (again, to avoid implicit rootfs checks). perhaps if it started out with: if [[ -z ${DISTRO} ]] ; then ... try and guess what distro we are ... fi ... handle ${DISTRO} behaviors ... or if you dont like the env passing, make it options to the INSTALL script.
Good feedback, but before getting into more details just a few comments: - collectl is already part of the fedora distro so I guess they're ok with my directory structure - I see at least 2 major issues with a gentoo installation, the first just getting it to run correctly independent of where it's installed and I believe I have that working now thanks to this bugzilla. The other is installing it in the directories as have been outlined in earlier comments, which I'd like to explore in more detail because I do know there are the similar issues with debian. - I also need to be very careful not to break anything. My thought is I could have something like INSTALL (to do it the way I currently do) and an INSTALL-gentoo and maybe even an INSTALL-debian, etc... I took a look at all the files that come with collectl and the recommended directories, and while it sounds like most would fit, there are others that I'm still puzzled by. Starting with binaries, while I can certainly install the main collectl script (it really isn't a binary) in /usr/bin, I also have a couple of supporting utilities. Where would I put those? While I could certainly toss them into /usr/bin too, how would one even know it existed OR if they did see it how would they know it was associated with collectl? What do other open source packages do that have utilities? I'm pretty sure they have their own subdirectory structures, though I'm not where where they're rooted. I also have a bunch of documentation that is NOT manpages, but rather html which also includes a FAQ and the GPL and ARTISIC licenses which I thought had to be delivered (but maybe just not installed anywhere?) with the source. There are over 20 of these. There are also release notes too. I assume that would all go into /usr/share/doc/collectl? That's certainly easy enough to to. When you say to put read-only things like ph files into /usr/share/collectl, and I also have a data table as well which sounds appropriate for there, is that sort of a 'catch-all' directory I could put all the stuff that doesn't fit anywhere else in? While I think this addresses all the collectl files, thinking ahead I also have an open source complementary package called collectl-utils which is more-of-less packaged the same way as collectl except that it also includes example data files. If it's legal to have something like /usr/share/collectl/examples/, that would certainly cover that piece of the puzzle. Finally, by design you can have multiple copies of collectl installed on the same system, which I find invaluable for development and to aid those who may want to do some hacking without breaking the original. Being able to run 2 different instances at the same time in 2 different windows is extremely useful and something I don't intend to change. The way I do this is to ALWAYS look for all files in the same directory collectl is in. BUT, I could also see inserting a test to see if its running on gentoo and if so to look in /usr/share/collectl instead. -mark
if you d/l the fedora rpm of your colletcl, you'll see that no, they do not like your directory layout. they install things by themselves into their own directory tree like we're asking (well, i dont entirely agree with the layout they've chosen, but that's minor). i dont think the directory layout we're asking for here is specific to Gentoo. i'd rather have it be called "INSTALL-fhs". having separate install scripts sounds like a bad idea to me as it's easy to have bitrot, but you're the maintainer so that's your call of course. since the scripts are arch-independent, you should be able to use /usr/share/collectl/ for housing them (the *.ph and such). i dont think you have any actual compiled binaries do you ? all your data tables too can go here since they're arch-independent. doc files should all go into /usr/share/doc/collectl/ by default. under Gentoo, we're a little more specific than that, but if the script does something like: DOCDIR=/usr/share/doc/collectl/ it's easy enough for us to `sed` i'm not sure about the examples ... if they're like "contrib" scripts which are meant for people to run, then /usr/share/collectl/examples/ would be fine. if they're example files which people are supposed to read/edit/tinker with, that should be under /usr/share/doc/collectl/exmaples/ running multiple versions independently wouldnt be a Gentoo-specific trick. again, that should apply to any distro that is packaging collectl. i dont know what you mean exactly by having multiple copies *installed* though. typically people test development versions by simply unpacking/compiling and running it in that local directory. they dont install it until they want to use it. or they use a completely different/local install prefix. like /opt/random/foo/ or /usr/local/.
Having swapped some emails with RedHat and some internal users of collectl to make sure I don't break any products they use it with, I'm going to use the following directory hierarchy which I'm hoping with satisfy everyone: /etc/collectl.conf /etc/rc.d/init.d/collectl /usr/bin/collectl /usr/share/collectl/envrules.std /usr/share/collectl/formatit.ph /usr/share/collectl/gexpr.ph /usr/share/collectl/hello.ph /usr/share/collectl/lexpr.ph /usr/share/collectl/misc.ph /usr/share/collectl/proctree.ph /usr/share/collectl/sexpr.ph /usr/share/collectl/vmstat.ph /usr//hare/doc/collectl-3.3.5 /usr/share/doc/collectl-3.3.5/ARTISTIC /usr/share/doc/collectl-3.3.5/COPYING /usr/share/doc/collectl-3.3.5/GPL /usr/share/doc/collectl-3.3.5/RELEASE-collectl /usr/share/doc/collectl-3.3.5/html /usr/share/doc/collectl-3.3.5/html/* /usr/share/man/man1/collectl.1.gz As for my manual INSTALL script, that too will install in this same tree. I'll include an optional parameter (or maybe even a couple) that will allow more flexibility to install some of the components elsewhere. There will still have to be some distro specific code to figure where to put the init.d script since everyone seems to have their own favorite locations. -mark
Hi again Mark, (In reply to comment #12) > /etc/rc.d/init.d/collectl I do not know redhat myself, so is this the init script? If so, that is not the place for init scripts on gentoo, we install them in /etc/init.d. Also, we have optional files with the same names as the corresponding init scripts in /etc/conf.d that allow users to configure which options, if any, they want to pass to the daemon when it starts. Also keep in mind that all init scripts are not alike. They are different depending on which init system the distribution uses, so the one for redhat will not work on gentoo for example. > As for my manual INSTALL script, that too will install in this same tree. I'll > include an optional parameter (or maybe even a couple) that will allow more > flexibility to install some of the components elsewhere. There will still have > to be some distro specific code to figure where to put the init.d script since > everyone seems to have their own favorite locations. I recommend that you add a parameter to support staged installs, similar to what is described here: http://www.gnu.org/prep/standards/html_node/DESTDIR.html This would be very helpful to packagers and users alike. If you have any questions about how it should work, let us know. Thanks, William
your directory structure sounds fine to me. other than the init.d issue as William notes, but we're used to having to workaround that for every package since we use more powerful init.d scripts than the classical hacks.
As for the init script location, yes I understand that that different distros put them in different locations as well as enable them differently too. For that reason I actually package up several different scripts, naming them collectl, collectl-suse, collectl-debian, collectl-generic and at installation time rename/move the right one into the right place. If in a redhat system I further do a 'chkconfig' to enable things whereas on gentoo I do an rc-update to get the right companion directories updated. I assume this is what was referred to. As for a staged install, I don't have a make file but do use prepended directory names in my INSTALL script so I think we're pretty much in sync on this at the concept level. I'm hoping to get out another release in the next few weeks or so and will let you know when it happens. -mark
Just wanted to let anyone following this that I just pushed a new release out to sourceforge that restructures collectl's distribution tree. Specifically collectl is not distrbuted to /usr/bin and it will look for any includes in /usr/share/collectl. Other things such as manpages, etc are now placed in the correct directories avoiding the need for any symlinks. Also, the start script placed in /etc/init.d is gentoo specific and will do the correct things to start/stop/monitor collectl. I hope this gets us closer to something that can be included in the gentoo distro. -mark
the install paths all look fine now, but there's still no DESTDIR support. i slapped together a patch here: http://sources.gentoo.org/viewcvs.py/*checkout*/gentoo-x86/sys-apps/collectl/files/collectl-3.4.0-install.patch the generic init.d script isnt Gentoo-ized ... our init.d scripts arent stand alone scripts that do their own parsing. you define specific functions and openrc handles executing the right thing. you can see our script here: http://sources.gentoo.org/viewcvs.py/*checkout*/gentoo-x86/sys-apps/collectl/files/collectl.initd doesnt matter to us whether you choose to include that in your package. no one else doesnt, so it isnt a big deal for us.
To answer the easy one first, I looked at your init script and see it only supports start/stop - doesn't gentoo do restart? status? I had even added my own 'flush' to tell collectl to flush its I/O buffers, but it's not a biggie as you can also send a signal to it. But more importantly, if someone deploys to a different DESTDIR how will this script work as it has hardcoded paths to directories in it? Maybe I don't fully understand the DESTDIR mechanism. That leads to the more complicated question and that is DESTDIR, as I don't think your patches will work correctly without something in collectl itself changing. As coded, collectl has a couple of hardcoded directories. It always looks in /etc for collectl.com, uses /var/log for its logs and reads additional include/data files from /usr/share/collectl. There ARE ways to override there - I'm not totally clueless ;-) - but I'm guessing that may not be completely consistent with the gentoo model. You can override the directory collectl loads its conf file from via -C. You override the location for /var/log/collectl with -f and you can change the include directory in collectl.conf. So as patched, even if collectl were installed in a different location it would still expect these other files to be where indicated unless otherwise overwritten. BUT maybe using alternative directories and overriding their locations at start time IS ok. I just don't know. But as I said, I still don't know how the start script would deal with all this. Since it sounds like there isn't a big rush to deal with this I'll be happy to see how much closer I can get collectl's next releast to what gentoo expect. At the very least I can use your start script instead of mine. You can always send me the entire script via private email rather than clog up this thread, unless what is pointed to IS the complete script, though I didn't see any LSB headers. -mark
common things like "restart" and "status" are taken care of by the common code. no point in forcing everyone to implement the same cruft everywhere. i can add a "flush" function easy enough. http://sources.gentoo.org/sys-apps/collectl/files/collectl.initd?r1=1.2&r2=1.3 i think you're misinterpreting DESTDIR. it isnt a path to install a package and leave it there, it's a temporary root tree to install things to so that the result can be collected/archived/etc... before installing it into /. you can think of it as doing: cp -a ./ $DESTDIR/ <do stuff in $DESTDIR/> cp -a $DESTDIR/ / and since the package manager is handling the install/upgrade/etc..., none of the `rm` steps are needed/wanted when working with DESTDIR. this is a standard build function that every autotooled package supports (./configure && make) as well as many non-autotooled packages (because of its usefulness).
re: DESTIR - ahh, the light goes on! I will definitely put this in the next release. Thanks for the clarification re: flush - I think it's a good thing to have but I also suspect I'm one of the few people who know about and/or even use it. ;-) As I said before I'd be happy to include whatever you come up with in my next release, though I'm not in any immediate hurry to release anything. -mark
DESTDIR support is the only last "big" thing. after that, it's just cheese. like passing custom paths to the script without having to `sed` it (via env/cmdline). since everything else is resolved in 3.4.0, i think we can close this bug. thanks Mark.
