Summary: | ebuild for bootchart | ||
---|---|---|---|
Product: | Gentoo Linux | Reporter: | Paul Pacheco <paulpach> |
Component: | [OLD] baselayout | Assignee: | Gentoo's Team for Core System packages <base-system> |
Status: | RESOLVED FIXED | ||
Severity: | enhancement | CC: | arpad.borsos, centic, enzo, eva, gentoo-bugs, gentoo-bugzilla, gentoo.bugzilla, ikelos, jaak, jesse, joris, karsten.elfenbein, kensington, kkrizka, langthang, m.debruijne, mabi, mafteah, mal, mmueller12345, moixa, philantrop, plate, radek, shaun.hill, spock, tar, viper, will, wschlich, ziga.mahkovec |
Priority: | High | Keywords: | EBUILD |
Version: | unspecified | ||
Hardware: | All | ||
OS: | Linux | ||
Whiteboard: | |||
Package list: | Runtime testing required: | --- | |
Bug Depends on: | |||
Bug Blocks: | 69579, 141114 | ||
Attachments: |
bootchart.tar.gz
hooks.diff hooks.diff bootchart.tar.gz hooks.diff bootchart.tar.gz bootchart.tar.gz bootchart-0.6.tar.gz bootchart-0.7.tar.gz bootchart-0.8.tar.gz hooks_for_shapshot.diff bootchart-0.8-r1.tar.gz bootchart-0.8-r2.tar.gz hooks-baselayout-1.12.0_pre6.diff bootchart-0.8-r3.tar.gz bootchart-0.8-r4.tar.gz bootchart-0.9.tar.gz |
Description
Paul Pacheco
2004-12-14 13:40:41 UTC
Created attachment 45989 [details]
bootchart.tar.gz
ebuild for bootchart. Requires the other patch to be applied first.
Created attachment 45990 [details, diff]
hooks.diff
patch that adds the required hooks to system scripts.
Apply with
cd /
patch -p0 < hooks.diff
Created attachment 45995 [details, diff]
hooks.diff
Now RC_USE_BOOTCHART can be turned on/off in /etc/conf.d/rc
Created attachment 46083 [details]
bootchart.tar.gz
Logs are now generated in /var/run/bootchart
and then moved to /var/log/bootchart
as vapier requested
any chance we can move all the 'api' logic into bootchart-functions.sh (btw, BC_DIR wasnt update in that file) ? that way if the semantics of the bootchart java app itself changes, we just have to update bootchart-functions.sh instead of roll a new baselayout There should be a message about RC_USE_BOOTCHART in pkg_postinst. Any I would add a ${PV} to the patches in ${FILESDIR} to avoid confusion for new versions. Created attachment 46132 [details, diff]
hooks.diff
hooks are much simpler now and hidden away in boot_profile_functions.sh as
suggested by SpanKY.
I renamed bootchart-function.sh to boot_profile_functions.sh because some time
in the future I might make a port of bootchart in python and I would like to be
able to use the same hooks.
Created attachment 46134 [details]
bootchart.tar.gz
This new ebuild has the boot_profile_functions.sh with all the logic, and
pkg_postinst now mentions RC_USE_BOOTCHART
I would suggest to set RC_USE_BOOTCHART="yes" by default because it is totally irrelevant and harmless before bootchart is emerged. At that point, the user most likelly wants to actually generate the chart. bootchart is somewhat of a debugging feature ... the defaut on the baselayout side of things will be off 1. The current setup won't work for users that have a seperate /usr and/or /var partition. /usr/bin/top and /usr/bin/iostat are started before /usr is mounted. With a /var partition it's impossible to mount the tmpfs at /var/run/bootchart for the logfiles. Possible solutions: /usr: copy /usr/bin/top and /usr/bin/iostat to a location where bootchart can reach them when the computer is restarted/shutdown. This way you'll always have a current binary. /var: use a unique mountpoint in / that nobody else in the world uses? 2. in process_name() the process-data is written to $BC_DIR/mnt/boot.pidname.log The directory $BC_DIR/mnt does not exist (so this will fail) and the other logfiles are written to $BC_DIR This is inconsistant and should be the same. 3:30am: still no bootchart and a hard time debugging this to get one :+ Just a thought: computer shutdown: if RC_USE_BOOTCHART="yes" create the workdirectory for bootchart in / for example /bootchart (if it doesn't already exists!) copy the environment for bootchart to /bootchart (top/iostat/etc. + mountpoint for tmpfs /bootchart/mnt) computer startup: mount tmpfs on /bootchart/mnt (this should be accessible) run top and iostat from /bootchart/bin etc. end of startup: kill top/iostat move logfiles to /var/log/bootchart unmount tmpfs remove /bootchart removing /bootchart might be to much but I like my root clean :) Invisible to the user and should be possible with any partition layout. Created attachment 46582 [details]
bootchart.tar.gz
fixed the $BC_DIR/mnt/boot.pidname.log issue.
Added a warning about /, /usr and /var having to be on the same device
Sorry, but the devs would not hear about /bootchart. here is a work around
though:
WORK AROUND:
if you have /, /usr and /var in separate partitions, follow these steps:
1) create directories /lib/bootchart and /lib/bootchart/mnt (or any other
location that you prefer)
2) copy /usr/bin/top and /usr/bin/iostat to /bin
3) edit /sbin/boot_profile_functions.sh and change BC_TOP_DIR and BC_DIR to
point to /lib/bootchart and /lib/bootchart/mnt respectivelly.
4) copy /usr/share/bootchart/.toprc in /lib/bootchart
I meant to say BC_TOP_HOME I expected that much, it probably would break any standard. like I said: just a thought Created attachment 46645 [details] bootchart-0.6.tar.gz version bump. plus /var/run/bootchart was being cleaned up by /etc/init.d/bootmisc, so moved BC_DIR to /var/log/bootchart. I have no idea how it ever worked. This version now depends on java tar library, an ebuild for that is available at bug 75343 Created attachment 48313 [details]
bootchart-0.7.tar.gz
version bump
does not use javatar any more I suggest app-benchmarks/bootchart. I am very much interested in integrating support for this in baselayout. Created attachment 50832 [details]
bootchart-0.8.tar.gz
* Version bump.
* Now it does not require top or iostat.
* show in tooltips stack trace where subshells are created. It will tell you
what bash function, script and line the boot proceses were started. You must
set svg format for this and use batik or adobesvg to view the svg
* Create the image by itself. So the only thing that the user has to do is
enable RC_USE_BOOTCHART in /etc/conf.d/rc, reboot and voila, the image is in
/var/log/bootchart.png (default) or /var/log/bootchart.svgz depending on the
format.
Created attachment 51297 [details, diff]
hooks_for_shapshot.diff
This patch is a port of hooks.diff for the 1.7.0_pre20050209 snapshot provided
by agriffis.
Ebuild is the same.
Created attachment 51549 [details]
bootchart-0.8-r1.tar.gz
process_name can be called by several proceses at the same time when parallel
boot is enabled. This corrupted the file init_pidname.log because bash's >> is
not atomic.
Now I synchronize access to the file.
Thanks to Mehdi Salem Naraghi who found the problem.
How about a -java flag? The log files can be rendered at www.bootchart.org so the java stuff is not strictly needed. Thanks Anders The RDEPENDency on sys-apps/acct needs to be changed to sys-process/acct. And I would also appreciate the possibility to USE=-java to disable the building of the Java applications. Created attachment 53544 [details]
bootchart-0.8-r2.tar.gz
* Changed RDEPEND sys-apps/acct to sys-process/acct
* Added a java use flag. If emerged with -java it will not compile the java
renderer
This is working nicely on my x86 with baselayout-1.9.4-r6. But I think the ebuild should tell me that I need to add "init=/sbin/bootchartd" to my kernel options. but you don't: To enable bootchart you just set RC_USE_BOOTCHART="yes" in /etc/conf.d/rc as the ebuild say at the end. (provided you applied the hooks which should be merged some time in the future) @ Paul Pacheco, You might want to fix function stop_boot_profile in boot_profile_functions.sh. I don't have java-config installed. Other than that, nice work. I got my bootchart ;) not working... I emerged th ebuild and applied the hooks patch and set RC_USE_BOOTCHART="yes" in /etc/conf.d/rc ... after reboot /var/log/bootchart was mounted and had the log files inside and there was no tgz file. I proceeded to tar the files by hand and used the web renderer form , but the resulting chart stopped right after starting the bootchartd process. I don't where it is failing .. can anyone point me to how to find out the problem? Paul I have replied to your email, if you didn't get it , I will resend it. can you rework this to: install the addon functions as /lib/rcscripts/addons/profiling-functions.sh redo the patch for baselayout to use the form 'profiling blahblahblah' that way baselayout just has to define the one function 'profiling' and your code will handle the rest ... i have the same problem as "pharon@gmail.com", my chart stops right after bootchartd. any ideas? What version of baselayout do you have? 1.11.12-r1, latest ~amd64. heres my emerge --info as well: Portage 2.0.51.22-r1 (default-linux/amd64/2005.0/no-multilib, gcc-3.4.3-20050110, glibc-2.3.5-r0, 2.6.11-gentoo-r9 x86_64) ================================================================= System uname: 2.6.11-gentoo-r9 x86_64 AMD Athlon(tm) 64 Processor 3200+ Gentoo Base System version 1.6.12 distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled] ccache version 2.4 [enabled] dev-lang/python: 2.3.5 sys-apps/sandbox: 1.2.8 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.5 sys-devel/binutils: 2.15.92.0.2-r9 sys-devel/libtool: 1.5.18 virtual/os-headers: 2.6.11 ACCEPT_KEYWORDS="amd64 ~amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O3 -march=athlon64 -mtune=athlon64 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O3 -march=athlon64 -mtune=athlon64 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig ccache distlocks sandbox strict" GENTOO_MIRRORS="http://gentoo.inode.at/ ftp://gentoo.inode.at/source/ http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/ ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/" LANG="de_DE@euro" LC_ALL="de_DE@euro" LDFLAGS="-Wl,-O1" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="amd64 X acpi alsa apache2 berkdb bitmap-fonts cdr crypt curl dba dvd dvdr esd fam fbcon flac font-server fortran gd gif gnome gpm gstreamer gtk gtk2 hal howl imagemagick imlib innodb ipv6 jp2 jpeg lzw lzw-tiff mp3 mysql mysqli ncurses nls no-htdocs no_wxgtk1 nptl nptlonly ogg oggvorbis opengl oss pam perl png python qt readline sdl session simplexml ssl symlink tcpd tiff tokenizer truetype truetype-fonts type1-fonts unicode usb userlocales vorbis xine xml2 xpm xrandr xv zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LINGUAS, MAKEOPTS it's not working here either. I have baselayout-1.11.12-r4. If I manually make the tgz with the files in /var/log/bootchart/ and I use the web form, I still get an exception. $ emerge --info Portage 2.0.51.22-r1 (default-linux/x86/2005.0, gcc-3.4.4, glibc-2.3.4.20041102-r1, 2.6.12-gentoo-r2 i686) ================================================================= System uname: 2.6.12-gentoo-r2 i686 Intel(R) Pentium(R) 4 CPU 2.80GHz Gentoo Base System version 1.6.12 ccache version 2.3 [enabled] dev-lang/python: 2.3.5 sys-apps/sandbox: 1.2.9 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.5 sys-devel/binutils: 2.15.92.0.2-r10 sys-devel/libtool: 1.5.16 virtual/os-headers: 2.6.8.1-r2 ACCEPT_KEYWORDS="x86" AUTOCLEAN="yes" CBUILD="i686-pc-linux-gnu" CFLAGS="-march=pentium4 -O2 -pipe -fomit-frame-pointer" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/splash /etc/terminfo /etc/texmf/web2c /etc/env.d" CXXFLAGS="-march=pentium4 -O2 -pipe -fomit-frame-pointer" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs autoconfig ccache confcache distlocks fixpackages sandbox sfperms strict" GENTOO_MIRRORS="http://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo http://distfiles.gentoo.org http://www.ibiblio.org/pub/Linux/distributions/gentoo" LINGUAS="nl en no fr de" MAKEOPTS="-j3" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/overlays/my_portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="x86 X alsa apache2 apm arts avi bash-completion berkdb bitmap-fonts bzip2 cddb cdparanoia cdr clamd crypt cups curl directfb dvd dvdr dvdread emboss encode fam ffmpeg foomaticdb fortran gcj gd gdbm gif gimpprint gpgme gphoto2 gpm gstreamer hal icu imagemagick imap imlib innodb ipv6 java jpeg junit kde kdeenablefinal kdexdeltas libclamav libg++ libvisual libwww lm_sensors lua mad mikmod mmx mono mozilla moznomail moznoxft mp3 mpeg mplayer mysql ncurses network nls nntp nptl nvidia ogg oggvorbis openexr opengl oss pam pdflib perl php pic png python qt quicktime rar readline samba sdl slang slp smtp spell sql sqlite sse sse2 ssl stencil-buffer subject-rewrite subtitles subversion svg svga symlink tcpd tetex threads tiff truetype truetype-fonts type1-fonts unicode visualization vorbis win32codecs wmf xine xml xml2 xscreensaver xsl xv xvid zeroconf zlib linguas_nl linguas_en linguas_no linguas_fr linguas_de userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS I emerged it (with -java) and patch the hooks.diff file. /var/log/bootchart is created but stays empty. During boot I saw message "accton: function not implemented" which I don't think I had before. I saw the notice about BSD requirements - but it's marked just as for better results. Is there absolute kernel requirements ? Portage 2.0.51.22-r2 (default-linux/amd64/2005.0, gcc-3.4.3, glibc-2.3.5-r0, 2.6.11-gentoo-r11 x86_64) ================================================================= System uname: 2.6.11-gentoo-r11 x86_64 AMD Athlon(tm) 64 Processor 3000+ Gentoo Base System version 1.6.13 dev-lang/python: 2.3.5 sys-apps/sandbox: 1.2.11 sys-devel/autoconf: 2.13, 2.59-r6 sys-devel/automake: 1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.5 sys-devel/binutils: 2.15.92.0.2-r10 sys-devel/libtool: 1.5.18-r1 virtual/os-headers: 2.6.11-r2 ACCEPT_KEYWORDS="amd64" AUTOCLEAN="yes" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=k8 -O2 -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config /usr/lib/X11/xkb /usr/share/config /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-march=k8 -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoconfig distlocks fixpackages sandbox sfperms strict" GENTOO_MIRRORS="http://gentoo.osuosl.org/ ftp://gentoo.risq.qc.ca/ ftp://ftp.ussg.iu.edu/pub/linux/gentoo ftp://gentoo.ccccom.com" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/disk0/gentoo/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage" USE="amd64 X adns alsa avi berkdb bitmap-fonts bonobo cdr cups curl dbus dvd dvdr dvdread dvi encode fam fftw flac font-server foomaticdb fortran gd gdbm gif gimp gimpprint glitz glut gnome gnomedb gpm gstreamer gtk gtk2 gtkhtml hal imagemagick imlib ipv6 java jpeg kerberos libwww lm_sensors lua lzw lzw-tiff mad motif mozilla mozsvg mp3 mpeg mysql ncurses network nptl nptlonly nvidia odbc ogg oggvorbis opengl oss pam pdflib perl plotutils png postgres ppds python quicktime readline samba sdl sensord slang spell ssl svg tcltk tcpd theora tiff truetype truetype-fonts type1-fonts usb userlocales v4l v4l2 vorbis xine xml xml2 xmms xpm xprint xscreensaver xv zlib userland_GNU kernel_linux elibc_glibc" Unset: ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS, LINGUAS from the ebuild: Change TMPFS_MNT in /etc/bootchartd.conf if / and /var are not on the same device. Tmpfs must be enabled in the kernel. so you need / and /var in the same device, that means /var can not be a separate partition. and you need tmpfs enabled in the kernel (not module) /var/log/bootchart will remain empty. It is mounted at boot time as a tmpfs, written to, and then unmounted when boot finishes. All the logs should be in /var/log/bootchart.tgz, which should be ready to be posted here: http://www.bootchart.org/download.html Paul, are you working on the changes requested in comment #31? Yes I am, I have just been very busy. ( sorry, my family just won't stop eating :) ). I have it mostly done, I will soon post a new patch and ebuild. Cool. If you hurry, it make be able to make it into baselayout-1.12.0. Created attachment 67783 [details, diff] hooks-baselayout-1.12.0_pre6.diff Hooks against baselayout-1.12.0_pre6.diff with the stuff from comment #31 Created attachment 67784 [details] bootchart-0.8-r3.tar.gz This one requires hooks-baselayout-1.12.0_pre6.diff so it is intended to be run only in baselayout-1.12.0_pre6.diff Should comply with comment #31 Created attachment 67788 [details]
bootchart-0.8-r4.tar.gz
* Fix autorender
* Fix RC_USE_BOOTCHART to correctly check for "yes"
Could we get this updated to the latest build? And possibly add to portage? hello guys, no more activities on this ? Is there a show stopper for the ebuild to be included in the tree. What about version 0.9 ? I'll try to come with an ebuild as soon as I as get it running. It should be trivial to update to 0.9, I just dont have time and patience to bug the developers on IRC to get this merged. Created attachment 92256 [details]
bootchart-0.9.tar.gz
Updated to 0.9. Cleaned up the ebuild a bit. Marked baselayout.diff obsoleted because I will file another bug for bootchart support in baselayout.
(In reply to comment #37) > from the ebuild: > > Change TMPFS_MNT in /etc/bootchartd.conf if / and /var > are not on the same device. > Tmpfs must be enabled in the kernel. > > so you need / and /var in the same device, that means /var can not be a > separate partition. and you need tmpfs enabled in the kernel (not module) 1. There's no TMPFS_MNT in /etc/bootchartd.conf so "changing" it is not an option. 2. To what or where should I set it? Does bootchart work *now* without the obsoleted baselayout patch? (In reply to comment #49) > 1. There's no TMPFS_MNT in /etc/bootchartd.conf so "changing" it is not an > option. > 2. To what or where should I set it? Well, it's not critical - you just won't get nice names on the bootchart. I've got a patch for that in the works though. Basically we mount --bind the bootchart log dir to /lib/bootchart and the unmount it when we're done. Hopefully have this in portage later today, if not then sometime early next week. bootchart is integrated now Re-opening sa bootchart itself isn't in the tree until I get the a small issue with the binary for process accounting being available in / resolved. *** Bug 143811 has been marked as a duplicate of this bug. *** bootchart is in portage. Note that process accounting will fail unless /usr is on the same partition as / (In reply to comment #54) > Note that process accounting will fail unless /usr is on the same partition as > / Is there a fix planned or at least a known workaround? (In reply to comment #55) > Is there a fix planned or at least a known workaround? Just wanted to clarify that process accounting is an optional feature, the bootchartd logger will be able to get a good enough process tree without it. Details here: http://www.bootchart.org/docs.html In any case, the way this is solved in Fedora is by installing the accton executable in /sbin and symlinking it from /usr/sbin/. users can copy it themselves, but i see no reason to punish everyone by moving acct to / simply because of an optional *development* addon like bootchart (In reply to comment #57) > users can copy it themselves, but i see no reason to punish everyone by moving > acct to / simply because of an optional *development* addon like bootchart Spanky, out of curiosity: What would be the problem with moving it to /sbin? Maybe another einfo line to inform the users about copying accton would be ok? (Ziga, thanks, it works nicely for me like that.) Looking at other distributions, accton/acctoff does actually need to move to / as it is applicable in some cases for NFS-mounted /usr. It's a very small, 5k on my x86 machine. However those with a seperate /var partition should be even more careful, as they might be data written to the /var structure on the / partition, instead of the /var partition. To avoid this, early during the boot, they should accton to an accessible location, and then later use the existing acct init script to switch to /var/account/. |