Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 510718 - dev-util/sysdig - System-level exploration and troubleshooting tool
Summary: dev-util/sysdig - System-level exploration and troubleshooting tool
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: Normal enhancement with 1 vote (vote)
Assignee: Michał Górny
URL: http://www.sysdig.org/
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-05-19 09:19 UTC by Vladimir Pavljuchenkov (SpiderX)
Modified: 2014-11-12 11:28 UTC (History)
5 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
ebuild for last release (sysdig-0.1.82.ebuild,1.17 KB, text/plain)
2014-05-19 09:20 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
ebuild for trunk version (sysdig-9999.ebuild,1.15 KB, text/plain)
2014-05-19 09:21 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
metadata (metadata.xml,834 bytes, text/xml)
2014-05-19 09:22 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
sysdig-9999.ebuild (sysdig-9999.ebuild,2.06 KB, text/plain)
2014-05-22 06:16 UTC, Pavel Volkov
Details
sysdig-0.1.84 (sysdig-0.1.84.ebuild,1.70 KB, text/plain)
2014-06-25 19:34 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
sysdig-9999 (sysdig-9999.ebuild,1.68 KB, text/plain)
2014-06-25 19:35 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
sysdig-0.1.82 (sysdig-0.1.82.ebuild,1.69 KB, text/plain)
2014-06-25 19:44 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
sysdig-0.1.85 (sysdig-0.1.85.ebuild,1.85 KB, text/plain)
2014-07-08 19:16 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
sysdig-9999 (sysdig-9999.ebuild,1.83 KB, text/plain)
2014-07-08 19:18 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
metadata with bundled-libs flag (metadata.xml,913 bytes, text/xml)
2014-07-31 19:49 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
updated 0.1.82 ebuild (sysdig-0.1.82.ebuild,1.84 KB, text/plain)
2014-07-31 19:50 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
updated 9999 ebuild (sysdig-9999.ebuild,2.34 KB, text/plain)
2014-07-31 19:50 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
bump to 0.1.86 (sysdig-0.1.86.ebuild,2.36 KB, text/plain)
2014-07-31 19:51 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
updated 9999 ebuild (sysdig-9999.ebuild,2.32 KB, text/plain)
2014-07-31 20:13 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
bump to 0.1.87 (sysdig-0.1.87.ebuild,2.36 KB, text/plain)
2014-08-07 08:52 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
bump to 0.1.88 (sysdig-0.1.88.ebuild,2.36 KB, text/plain)
2014-08-21 09:42 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
ebuild for sysdig-0.1.89 (sysdig-0.1.89.ebuild,2.36 KB, text/plain)
2014-09-26 17:32 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
ebuild for sysdig-0.1.91 (sysdig-0.1.91.ebuild,2.36 KB, text/plain)
2014-11-01 19:12 UTC, Vladimir Pavljuchenkov (SpiderX)
Details
ebuild for sysdig-0.1.92 (sysdig-0.1.92.ebuild,2.36 KB, text/plain)
2014-11-04 12:25 UTC, Vladimir Pavljuchenkov (SpiderX)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Vladimir Pavljuchenkov (SpiderX) 2014-05-19 09:19:13 UTC
Sysdig is open source, system-level exploration: capture system state and activity from a running Linux instance, then save, filter and analyze.
Think of it as strace + tcpdump + lsof + awesome sauce.
With a little Lua cherry on top.

Reproducible: Always
Comment 1 Vladimir Pavljuchenkov (SpiderX) 2014-05-19 09:20:29 UTC
Created attachment 377224 [details]
ebuild for last release
Comment 2 Vladimir Pavljuchenkov (SpiderX) 2014-05-19 09:21:55 UTC
Created attachment 377226 [details]
ebuild for trunk version
Comment 3 Vladimir Pavljuchenkov (SpiderX) 2014-05-19 09:22:17 UTC
Created attachment 377228 [details]
metadata
Comment 4 Pavel Volkov 2014-05-22 06:16:52 UTC
Created attachment 377404 [details]
sysdig-9999.ebuild

The git ebuild fails to compile .
This is my enhanced version with other improvememnts.
Comment 5 Pavel Volkov 2014-05-22 06:34:49 UTC
And I forgot to explain.
It failed on my amd64 because ARCH was set to "amd64" when calling
/lib/modules/<version>/build/Makefile
resulting in a "file not found" error
The correct value should be x86_64 for that Makefile or unset (so I unset it).

I haven't tried the "last release" version, probably it has the same problem.
Comment 6 Vladimir Pavljuchenkov (SpiderX) 2014-06-25 19:34:55 UTC
Created attachment 379666 [details]
sysdig-0.1.84
Comment 7 Vladimir Pavljuchenkov (SpiderX) 2014-06-25 19:35:15 UTC
Created attachment 379668 [details]
sysdig-9999
Comment 8 Vladimir Pavljuchenkov (SpiderX) 2014-06-25 19:44:06 UTC
Created attachment 379670 [details]
sysdig-0.1.82

Thank you for fixing ebuild and for improvements.

I have also made some improvements:
1. CONFIG_TRACEPOINTS and CONFIG_HAVE_SYSCALL_TRACEPOINTS cannot be set by a user; it is set by the architecture (in arch/*/Kconfig) if syscall tracepoints are actually implemented by that architecture.
So its pointless to have CONFIG_CHECK at all, "checks" should go to keywords. Removed.
Also, eclass linux-info is pointless too. Removed.

2. Dependence on sys-kernel/linux-headers is replaced by virtual/os-headers as more general.

3. USE-flag modules now require kernel_linux through REQUIRE_USE.
This makes ebuild much cleaner.

4. Compile error corrected before by sed is now fixed through 'unset ARCH'

5. Bump to sysdig 0.1.84. I left sysdig 0.1.82 as last one could be compiled with jsoncpp-0.5.0-r1, which is stable.
Comment 9 Vladimir Pavljuchenkov (SpiderX) 2014-07-08 19:16:46 UTC
Created attachment 380462 [details]
sysdig-0.1.85
Comment 10 Vladimir Pavljuchenkov (SpiderX) 2014-07-08 19:18:13 UTC
Created attachment 380464 [details]
sysdig-9999

Ebuild does not install bash, zsh completion anymore.
Comment 11 Pavel Volkov 2014-07-09 10:05:56 UTC
Saving into distfiles as {version}.tar.gz is unnatural, I'd change it to:

SRC_URI="https://github.com/draios/${PN}/archive/${PV}.tar.gz -> ${P}.tar.gz"
Comment 12 Holger Hoffstätte 2014-07-09 19:16:26 UTC
(In reply to SpiderX from comment #9)
> Created attachment 380462 [details]
> sysdig-0.1.85

Hi! Apparently we have duplicated work. I had an ebuild in my overlay for several months now (there was no bug at the time) and also just updated to 0.1.85, can we maybe consolidate our work? 

Mine can be found at:
https://github.com/hhoffstaette/portage/tree/master/sys-apps/sysdig

I'm reasonably sure I'm doing a few things wrong esp. wrt. the kernel module and a few general ebuild practices, but it builds reliably and works, with and espcially without embedded libs for userspace stuff. I also added misc. USE flags, just today for the shell completions, which work great.

It would be great if we can keep these features. What bugs me is the dependency on a configured kernel tree (apparently some modules manage to build without that?) and that - for now - the userspace tools and kernel module need to be installed in lockstep. IMHO having split ebuilds would be nice, but that can wait until the module has stabilized upstream.

Please let me know what you think. :)
Comment 13 Vladimir Pavljuchenkov (SpiderX) 2014-07-31 19:49:57 UTC
Created attachment 381960 [details]
metadata with bundled-libs flag
Comment 14 Vladimir Pavljuchenkov (SpiderX) 2014-07-31 19:50:26 UTC
Created attachment 381962 [details]
updated 0.1.82 ebuild
Comment 15 Vladimir Pavljuchenkov (SpiderX) 2014-07-31 19:50:50 UTC
Created attachment 381964 [details]
updated 9999 ebuild
Comment 16 Vladimir Pavljuchenkov (SpiderX) 2014-07-31 19:51:11 UTC
Created attachment 381966 [details]
bump to 0.1.86
Comment 17 Pavel Volkov 2014-07-31 20:10:36 UTC
Why did you change keywords in 9999?
Comment 18 Vladimir Pavljuchenkov (SpiderX) 2014-07-31 20:13:02 UTC
Created attachment 381968 [details]
updated 9999 ebuild

fixed, thanks.
Comment 19 Vladimir Pavljuchenkov (SpiderX) 2014-07-31 20:40:32 UTC
Holger Hoffstätte

Hello,

> Apparently we have duplicated work.

That's for sure.

> I had an ebuild in my overlay for several months now (there was no bug at the time) and also just updated to 0.1.85, can we maybe consolidate our work? 

Sure.

> I'm reasonably sure I'm doing a few things wrong esp. wrt. the kernel module and a few general ebuild practices, but it builds reliably and works, with and espcially without embedded libs for userspace stuff. I also added misc. USE flags, just today for the shell completions, which work great.

I have take a look at your ebuild. It's not good enough now to be added to portage, which is my goal.

> It would be great if we can keep these features. What bugs me is the dependency on a configured kernel tree (apparently some modules manage to build without that?) and that - for now - the userspace tools and kernel module need to be installed in lockstep. IMHO having split ebuilds would be nice, but that can wait until the module has stabilized upstream.

Yes, we should support both completions and possibility to build ebuild with bundled-libs.
I don't see reasons to split ebuild. Portage has a lot of such ebuilds (net-dialup/accel-ppp for example).
There is a reason to use this ebuild with and without kernel module.
You can, for example, build ebuild with module on one host and capture events to file, and later build sysdig without module on another one and read this file on it. So "dependency on a configured kernel" we should leave to user.

I have considered all main features of your ebuild, so take a look at my the latest ebuild.
Comment 20 Vladimir Pavljuchenkov (SpiderX) 2014-08-07 08:52:53 UTC
Created attachment 382436 [details]
bump to 0.1.87
Comment 21 Holger Hoffstätte 2014-08-10 10:13:17 UTC
(In reply to SpiderX from comment #19)

Hello Vladimir, thanks for your efforts. I finally got some time to install your ebuild and was happy to see that it does everything that mine does - thanks for porting all USE flags over. I agree that your ebuild is a bit cleaner than mine; I couldn't get the cmake eclasses to work and had to fiddle around instead.

Since this works so well I've now switched over to your ebuild and deleted mine. :)

Thanks and +1 to get this into portage!
Comment 22 Vladimir Pavljuchenkov (SpiderX) 2014-08-21 09:42:37 UTC
Created attachment 383260 [details]
bump to 0.1.88

Tested under amd64 and x86.
Comment 23 Vladimir Pavljuchenkov (SpiderX) 2014-09-26 17:32:19 UTC
Created attachment 385554 [details]
ebuild for sysdig-0.1.89

bump to 0.1.89
Comment 24 Bruce Guenter 2014-10-21 23:05:57 UTC
Should this not go in sys-apps instead of dev-util?
Comment 25 Vladimir Pavljuchenkov (SpiderX) 2014-10-22 10:44:20 UTC
The closest analog of sysdig is systemtap.
Systemtap is in dev-util section in portage, so it will be logical that sysdig goes in same category.
P.S. strace is in dev-util too.
Comment 26 Vladimir Pavljuchenkov (SpiderX) 2014-11-01 19:12:07 UTC
Created attachment 388060 [details]
ebuild for sysdig-0.1.91

bump to 0.1.89
Comment 27 Vladimir Pavljuchenkov (SpiderX) 2014-11-04 12:25:51 UTC
Created attachment 388524 [details]
ebuild for sysdig-0.1.92

bump to 0.1.92
Comment 28 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-11-11 21:51:34 UTC
Thanks for your ebuild. I have simplified it a bit and adjusted to fit into Gentoo standards a little better. It seems to work great for me. I'm just wondering a bit about CONFIG_TRACEPOINTS since AFAICS that can be enabled only indirectly. Maybe we should check for something more direct instead.

+*sysdig-0.1.92 (11 Nov 2014)
+
+  11 Nov 2014; Michał Górny <mgorny@gentoo.org> +metadata.xml,
+  +sysdig-0.1.92.ebuild:
+  Introduce initial version of sysdig, bug #510718.
Comment 29 Pavel Volkov 2014-11-12 06:44:12 UTC
Something I noticed during install phase:

depmod: WARNING: //lib/modules/3.17.2-gentoomelf/misc/sysdig-probe.ko needs unknown symbol getnstimeofday
Comment 30 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-11-12 08:42:48 UTC
(In reply to Pavel Volkov from comment #29)
> Something I noticed during install phase:
> 
> depmod: WARNING: //lib/modules/3.17.2-gentoomelf/misc/sysdig-probe.ko needs
> unknown symbol getnstimeofday

What kernel version are you using? This may be either a missing CONFIG_CHECK or minimal KV dep, and I'm leaning towards the latter.
Comment 31 Holger Hoffstätte 2014-11-12 09:30:05 UTC
(In reply to Michał Górny from comment #28)
> Thanks for your ebuild. I have simplified it a bit

Thanks for committing this, but unfortunately your "simplifications" are quite the opposite.

I notied that the zsh-completion is gone for no apparent reason; I added that specifically because I use zsh, and now the chisel completion is useless. I care less about the possible use of the bundled deps, but that also was added to prevent possible future problems with luajit, which I discussed/fixed with upstream.
Not building the kernel module was added for systems where only data analysis is performed.

Please add the USE flags back, at least the zsh completion.
Comment 32 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2014-11-12 09:38:47 UTC
(In reply to Holger Hoffstätte from comment #31)
> I notied that the zsh-completion is gone for no apparent reason; I added
> that specifically because I use zsh, and now the chisel completion is
> useless.

$ qlist sysdig | grep zsh
/usr/share/zsh/vendor-completions/_sysdig
/usr/share/zsh/site-functions/_sysdig

I don't see any conditionals controlling their installation, so I don't understand how can you end up not having them.

> I care less about the possible use of the bundled deps, but that
> also was added to prevent possible future problems with luajit, which I
> discussed/fixed with upstream.

We don't really want bundled deps in Gentoo. If we hit issues with Lua, I'll try to devise a solution then. But even in the worst case, I'd go for bundling Lua only rather than all three libraries.

> Not building the kernel module was added for systems where only data
> analysis is performed.

I didn't think of that use case. Will restore in a few hours since I have to leave now.
Comment 33 Holger Hoffstätte 2014-11-12 09:54:15 UTC
(In reply to Michał Górny from comment #32)
> (In reply to Holger Hoffstätte from comment #31)
> > I notied that the zsh-completion is gone for no apparent reason; I added
> > that specifically because I use zsh, and now the chisel completion is
> > useless.
> 
> $ qlist sysdig | grep zsh
> /usr/share/zsh/vendor-completions/_sysdig
> /usr/share/zsh/site-functions/_sysdig
> 
> I don't see any conditionals controlling their installation, so I don't
> understand how can you end up not having them.

Yes, OK. I didn't actually install it, I just saw that the USE flag was gone and did not see explicit installation in the build. If it's always installed then fine.

> I didn't think of that use case. Will restore in a few hours since I have to
> leave now.

Thanks!
Comment 34 Pavel Volkov 2014-11-12 11:28:59 UTC
(In reply to Michał Górny from comment #30)
> > depmod: WARNING: //lib/modules/3.17.2-gentoomelf/misc/sysdig-probe.ko needs
> > unknown symbol getnstimeofday
> 
> What kernel version are you using? This may be either a missing CONFIG_CHECK
> or minimal KV dep, and I'm leaning towards the latter.

Look at the path above :) It's gentoo-sources 3.17.2.