Bugzilla Bug 74425

this is an ebuild for bootchart from 
http://www.klika.si/ziga/bootchart/


Reproducible: Always
Steps to Reproduce:
1.
2.
3.

Created an attachment (id=45989) [details]
bootchart.tar.gz

ebuild for bootchart. Requires the other patch to be applied first.

Created an attachment (id=45990) [details]
hooks.diff

patch that adds the required hooks to system scripts.
Apply with 

cd /
patch -p0 < hooks.diff

Created an attachment (id=45995) [details]
hooks.diff

Now RC_USE_BOOTCHART can be turned on/off in /etc/conf.d/rc

Created an attachment (id=46083) [details]
bootchart.tar.gz

Logs are now generated in /var/run/bootchart
and then moved to /var/log/bootchart
as vapier requested

any chance we can move all the 'api' logic into bootchart-functions.sh (btw,
BC_DIR wasnt update in that file) ?

that way if the semantics of the bootchart java app itself changes, we just
have to update bootchart-functions.sh instead of roll a new baselayout

There should be a message about RC_USE_BOOTCHART in pkg_postinst.
Any I would add a ${PV} to the patches in ${FILESDIR} to avoid confusion for new versions.

Created an attachment (id=46132) [details]
hooks.diff

hooks are much simpler now and hidden away in boot_profile_functions.sh as
suggested by SpanKY.

I renamed bootchart-function.sh to boot_profile_functions.sh because some time
in the future I might make a port of bootchart in python and I would like to be
able to use the same hooks.

Created an attachment (id=46134) [details]
bootchart.tar.gz

This new ebuild has the boot_profile_functions.sh with all the logic, and
pkg_postinst now mentions RC_USE_BOOTCHART

I would suggest to set RC_USE_BOOTCHART="yes" by default because it is totally
irrelevant and harmless before bootchart is emerged. At that point, the user
most likelly wants to actually generate the chart.

bootchart is somewhat of a debugging feature ... the defaut on the baselayout
side of things will be off

1. The current setup won't work for users that have a seperate /usr and/or /var
partition.

/usr/bin/top and /usr/bin/iostat are started before /usr is mounted.

With a  /var partition it's impossible to mount the tmpfs at /var/run/bootchart
for the logfiles.

Possible solutions:
/usr: copy /usr/bin/top and /usr/bin/iostat to a location where bootchart can
reach them when the computer is restarted/shutdown. This way you'll always have
a current binary.

/var: use a unique mountpoint in / that nobody else in the world uses?

2. in process_name() the process-data is written to
$BC_DIR/mnt/boot.pidname.log
The directory $BC_DIR/mnt does not exist (so this will fail) and the other
logfiles are written to $BC_DIR
This is inconsistant and should be the same.

3:30am: still no bootchart and a hard time debugging this to get one :+

Just a thought:

computer shutdown:
if RC_USE_BOOTCHART="yes"
create the workdirectory for bootchart in /
for example /bootchart (if it doesn't already exists!)
copy the environment for bootchart to /bootchart (top/iostat/etc. + mountpoint for tmpfs /bootchart/mnt)

computer startup:
mount tmpfs on /bootchart/mnt (this should be accessible)
run top and iostat from /bootchart/bin etc.

end of startup:
kill top/iostat
move logfiles to /var/log/bootchart
unmount tmpfs
remove /bootchart

removing /bootchart might be to much but I like my root clean :)
Invisible to the user and should be possible with any partition layout.

Created an attachment (id=46582) [details]
bootchart.tar.gz

fixed the $BC_DIR/mnt/boot.pidname.log issue.
Added a warning about /, /usr and /var having to be on the same device

Sorry, but the devs would not hear about /bootchart. here is a work around
though:

WORK AROUND:

if you have /, /usr and /var in separate partitions, follow these steps:

1) create directories /lib/bootchart and /lib/bootchart/mnt (or any other
location that you prefer)
2) copy /usr/bin/top and /usr/bin/iostat to /bin
3) edit /sbin/boot_profile_functions.sh and change BC_TOP_DIR and BC_DIR to
point to /lib/bootchart and /lib/bootchart/mnt respectivelly.
4) copy /usr/share/bootchart/.toprc in /lib/bootchart

I meant to say BC_TOP_HOME

I expected that much, it probably would break any standard.
like I said: just a thought

Created an attachment (id=46645) [details]
bootchart-0.6.tar.gz

version bump. plus

/var/run/bootchart was being cleaned up by /etc/init.d/bootmisc, so moved
BC_DIR to /var/log/bootchart. I have no idea how it ever worked.

This version now depends on java tar library, an ebuild for that is available
at bug 75343

Created an attachment (id=48313) [details]
bootchart-0.7.tar.gz

version bump

does not use javatar any more

I suggest app-benchmarks/bootchart. I am very much interested in integrating
support for this in baselayout.

Created an attachment (id=50832) [details]
bootchart-0.8.tar.gz

* Version bump.

* Now it does not require top or iostat.

* show in tooltips stack trace where subshells are created. It will tell you
what bash function, script and line the boot proceses were started. You must
set svg format for this and use batik or adobesvg to view the svg

* Create the image by itself. So the only thing that the user has to do is
enable RC_USE_BOOTCHART in /etc/conf.d/rc, reboot and voila, the image is in
/var/log/bootchart.png (default) or /var/log/bootchart.svgz depending on the
format.

Created an attachment (id=51297) [details]
hooks_for_shapshot.diff

This patch is a port of hooks.diff for the 1.7.0_pre20050209 snapshot provided
by agriffis.

Ebuild is the same.

Created an attachment (id=51549) [details]
bootchart-0.8-r1.tar.gz

process_name can be called by several proceses at the same time when parallel
boot is enabled. This corrupted the file init_pidname.log because bash's >> is
not atomic.

Now I synchronize access to the file.

Thanks to Mehdi Salem Naraghi who found the problem.

How about a -java flag? The log files can be rendered at www.bootchart.org so
the java stuff is not strictly needed.

Thanks
Anders

The RDEPENDency on sys-apps/acct needs to be changed to sys-process/acct.

And I would also appreciate the possibility to USE=-java to disable the building of the Java applications.

Created an attachment (id=53544) [details]
bootchart-0.8-r2.tar.gz

* Changed RDEPEND sys-apps/acct to sys-process/acct

* Added a java use flag. If emerged with -java it will not compile the java
renderer

This is working nicely on my x86 with baselayout-1.9.4-r6.

But I think the ebuild should tell me that I need to add "init=/sbin/bootchartd" to my kernel options.

but you don't:

To enable bootchart you just set 
RC_USE_BOOTCHART="yes"
in /etc/conf.d/rc

as the ebuild say at the end.

(provided you applied the hooks which should be merged some time in the future)

@ Paul Pacheco,
You might want to fix function stop_boot_profile in boot_profile_functions.sh. I don't have java-config installed. Other than that, nice work. I got my bootchart ;)

not working...

I emerged th ebuild and applied the hooks patch and set RC_USE_BOOTCHART="yes"
in /etc/conf.d/rc ...

after reboot /var/log/bootchart was mounted and had the log files inside and there was no tgz file. I proceeded to tar the files by hand and used the web renderer form , but the resulting chart stopped right after starting the bootchartd process.

I don't where it is failing .. can anyone point me to how to find out the problem?
 

Paul I have replied to your email, if you didn't get it , I will resend it.

can you rework this to:

install the addon functions as /lib/rcscripts/addons/profiling-functions.sh

redo the patch for baselayout to use the form 'profiling blahblahblah'

that way baselayout just has to define the one function 'profiling' and your code will handle the rest ...

i have the same problem as "pharon@gmail.com", my chart stops right after
bootchartd. any ideas?

What version of baselayout do you have? 

1.11.12-r1, latest ~amd64.
heres my emerge --info as well:

Portage 2.0.51.22-r1 (default-linux/amd64/2005.0/no-multilib,
gcc-3.4.3-20050110, glibc-2.3.5-r0, 2.6.11-gentoo-r9 x86_64)
=================================================================
System uname: 2.6.11-gentoo-r9 x86_64 AMD Athlon(tm) 64 Processor 3200+
Gentoo Base System version 1.6.12
distcc 2.18.3 x86_64-pc-linux-gnu (protocols 1 and 2) (default port 3632) [disabled]
ccache version 2.4 [enabled]
dev-lang/python:     2.3.5
sys-apps/sandbox:    1.2.8
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.5
sys-devel/binutils:  2.15.92.0.2-r9
sys-devel/libtool:   1.5.18
virtual/os-headers:  2.6.11
ACCEPT_KEYWORDS="amd64 ~amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O3 -march=athlon64 -mtune=athlon64 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config
/usr/lib/X11/xkb /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-O3 -march=athlon64 -mtune=athlon64 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig ccache distlocks sandbox strict"
GENTOO_MIRRORS="http://gentoo.inode.at/ ftp://gentoo.inode.at/source/
http://linux.rz.ruhr-uni-bochum.de/download/gentoo-mirror/
ftp://linux.rz.ruhr-uni-bochum.de/gentoo-mirror/"
LANG="de_DE@euro"
LC_ALL="de_DE@euro"
LDFLAGS="-Wl,-O1"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="amd64 X acpi alsa apache2 berkdb bitmap-fonts cdr crypt curl dba dvd dvdr
esd fam fbcon flac font-server fortran gd gif gnome gpm gstreamer gtk gtk2 hal
howl imagemagick imlib innodb ipv6 jp2 jpeg lzw lzw-tiff mp3 mysql mysqli
ncurses nls no-htdocs no_wxgtk1 nptl nptlonly ogg oggvorbis opengl oss pam perl
png python qt readline sdl session simplexml ssl symlink tcpd tiff tokenizer
truetype truetype-fonts type1-fonts unicode usb userlocales vorbis xine xml2 xpm
xrandr xv zlib userland_GNU kernel_linux elibc_glibc"
Unset:  ASFLAGS, CTARGET, LINGUAS, MAKEOPTS

it's not working here either. I have baselayout-1.11.12-r4. If I manually make 
the tgz with the files in /var/log/bootchart/ and I use the web form, I still 
get an exception. 
 
$ emerge --info 
Portage 2.0.51.22-r1 (default-linux/x86/2005.0, gcc-3.4.4, 
glibc-2.3.4.20041102-r1, 2.6.12-gentoo-r2 i686) 
================================================================= 
System uname: 2.6.12-gentoo-r2 i686 Intel(R) Pentium(R) 4 CPU 2.80GHz 
Gentoo Base System version 1.6.12 
ccache version 2.3 [enabled] 
dev-lang/python:     2.3.5 
sys-apps/sandbox:    1.2.9 
sys-devel/autoconf:  2.13, 2.59-r6 
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.5 
sys-devel/binutils:  2.15.92.0.2-r10 
sys-devel/libtool:   1.5.16 
virtual/os-headers:  2.6.8.1-r2 
ACCEPT_KEYWORDS="x86" 
AUTOCLEAN="yes" 
CBUILD="i686-pc-linux-gnu" 
CFLAGS="-march=pentium4 -O2 -pipe -fomit-frame-pointer" 
CHOST="i686-pc-linux-gnu" 
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3.4/env /usr/kde/3.4/share/config /usr/kde/3.4/shutdown /usr/kde/3/share/config /usr/lib/X11/xkb /usr/lib/mozilla/defaults/pref /usr/share/config /var/qmail/control" 
CONFIG_PROTECT_MASK="/etc/gconf /etc/splash /etc/terminfo /etc/texmf/web2c /etc/env.d" 
CXXFLAGS="-march=pentium4 -O2 -pipe -fomit-frame-pointer" 
DISTDIR="/usr/portage/distfiles" 
FEATURES="autoaddcvs autoconfig ccache confcache distlocks fixpackages sandbox 
sfperms strict" 
GENTOO_MIRRORS="http://ftp.belnet.be/mirror/rsync.gentoo.org/gentoo 
http://distfiles.gentoo.org 
http://www.ibiblio.org/pub/Linux/distributions/gentoo" 
LINGUAS="nl en no fr de" 
MAKEOPTS="-j3" 
PKGDIR="/usr/portage/packages" 
PORTAGE_TMPDIR="/var/tmp" 
PORTDIR="/usr/portage" 
PORTDIR_OVERLAY="/usr/local/overlays/my_portage" 
SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" 
USE="x86 X alsa apache2 apm arts avi bash-completion berkdb bitmap-fonts bzip2 
cddb cdparanoia cdr clamd crypt cups curl directfb dvd dvdr dvdread emboss 
encode fam ffmpeg foomaticdb fortran gcj gd gdbm gif gimpprint gpgme gphoto2 
gpm gstreamer hal icu imagemagick imap imlib innodb ipv6 java jpeg junit kde 
kdeenablefinal kdexdeltas libclamav libg++ libvisual libwww lm_sensors lua mad 
mikmod mmx mono mozilla moznomail moznoxft mp3 mpeg mplayer mysql ncurses 
network nls nntp nptl nvidia ogg oggvorbis openexr opengl oss pam pdflib perl 
php pic png python qt quicktime rar readline samba sdl slang slp smtp spell sql 
sqlite sse sse2 ssl stencil-buffer subject-rewrite subtitles subversion svg 
svga symlink tcpd tetex threads tiff truetype truetype-fonts type1-fonts 
unicode visualization vorbis win32codecs wmf xine xml xml2 xscreensaver xsl xv 
xvid zeroconf zlib linguas_nl linguas_en linguas_no linguas_fr linguas_de 
userland_GNU kernel_linux elibc_glibc" 
Unset:  ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS 
 

I emerged it (with -java) and patch the hooks.diff file. /var/log/bootchart is
created but stays empty. During boot I saw message "accton: function not
implemented" which I don't think I had before.

I saw the notice about BSD requirements - but it's marked just as for better
results. Is there absolute kernel requirements ?


Portage 2.0.51.22-r2 (default-linux/amd64/2005.0, gcc-3.4.3, glibc-2.3.5-r0,
2.6.11-gentoo-r11 x86_64)
=================================================================
System uname: 2.6.11-gentoo-r11 x86_64 AMD Athlon(tm) 64 Processor 3000+
Gentoo Base System version 1.6.13
dev-lang/python:     2.3.5
sys-apps/sandbox:    1.2.11
sys-devel/autoconf:  2.13, 2.59-r6
sys-devel/automake:  1.4_p6, 1.5, 1.6.3, 1.7.9-r1, 1.8.5-r3, 1.9.5
sys-devel/binutils:  2.15.92.0.2-r10
sys-devel/libtool:   1.5.18-r1
virtual/os-headers:  2.6.11-r2
ACCEPT_KEYWORDS="amd64"
AUTOCLEAN="yes"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=k8 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/kde/2/share/config /usr/kde/3/share/config
/usr/lib/X11/xkb /usr/share/config /var/qmail/control"
CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d"
CXXFLAGS="-march=k8 -O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FEATURES="autoconfig distlocks fixpackages sandbox sfperms strict"
GENTOO_MIRRORS="http://gentoo.osuosl.org/ ftp://gentoo.risq.qc.ca/
ftp://ftp.ussg.iu.edu/pub/linux/gentoo ftp://gentoo.ccccom.com"
MAKEOPTS="-j2"
PKGDIR="/usr/portage/packages"
PORTAGE_TMPDIR="/disk0/gentoo/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage"
SYNC="rsync://rsync.namerica.gentoo.org/gentoo-portage"
USE="amd64 X adns alsa avi berkdb bitmap-fonts bonobo cdr cups curl dbus dvd
dvdr dvdread dvi encode fam fftw flac font-server foomaticdb fortran gd gdbm gif
gimp gimpprint glitz glut gnome gnomedb gpm gstreamer gtk gtk2 gtkhtml hal
imagemagick imlib ipv6 java jpeg kerberos libwww lm_sensors lua lzw lzw-tiff mad
motif mozilla mozsvg mp3 mpeg mysql ncurses network nptl nptlonly nvidia odbc
ogg oggvorbis opengl oss pam pdflib perl plotutils png postgres ppds python
quicktime readline samba sdl sensord slang spell ssl svg tcltk tcpd theora tiff
truetype truetype-fonts type1-fonts usb userlocales v4l v4l2 vorbis xine xml
xml2 xmms xpm xprint xscreensaver xv zlib userland_GNU kernel_linux elibc_glibc"
Unset:  ASFLAGS, CTARGET, LANG, LC_ALL, LDFLAGS, LINGUAS

from the ebuild: 
 
 Change TMPFS_MNT in /etc/bootchartd.conf if / and /var 
 are not on the same device. 
 Tmpfs must be enabled in the kernel. 
 
so you need / and /var in the same device, that means /var can not be a 
separate partition. and you need tmpfs enabled in the kernel (not module) 

/var/log/bootchart will remain empty. It is mounted at boot time as a tmpfs, 
written to, and then unmounted when boot finishes. 
All the logs should be in /var/log/bootchart.tgz, which should be ready to be 
posted here: http://www.bootchart.org/download.html 
 

Paul, are you working on the changes requested in comment #31?

Yes I am, I have just been very busy. ( sorry, my family just won't stop 
eating :) ). 
 
I have it mostly done, I will soon post a new patch and ebuild. 
 

Cool. If you hurry, it make be able to make it into baselayout-1.12.0.

Created an attachment (id=67783) [details]
hooks-baselayout-1.12.0_pre6.diff

Hooks against baselayout-1.12.0_pre6.diff
with the stuff from comment #31

Created an attachment (id=67784) [details]
bootchart-0.8-r3.tar.gz

This one requires hooks-baselayout-1.12.0_pre6.diff so it is intended to be run

only in baselayout-1.12.0_pre6.diff

Should comply with comment #31

Created an attachment (id=67788) [details]
bootchart-0.8-r4.tar.gz

* Fix autorender
* Fix RC_USE_BOOTCHART to correctly check for "yes"

Could we get this updated to the latest build? And possibly add to portage?

hello guys,

no more activities on this ? Is there a show stopper for the ebuild to be
included in the tree. What about version 0.9 ? I'll try to come with an ebuild
as soon as I  as get it running.

It should be trivial to update to 0.9, I just dont have time and patience to
bug the developers on IRC to get this merged.

Created an attachment (id=92256) [details]
bootchart-0.9.tar.gz

Updated to 0.9. Cleaned up the ebuild a bit. Marked baselayout.diff obsoleted
because I will file another bug for bootchart support in baselayout.

(In reply to comment #37)
> from the ebuild: 
>  
>  Change TMPFS_MNT in /etc/bootchartd.conf if / and /var 
>  are not on the same device. 
>  Tmpfs must be enabled in the kernel. 
>  
> so you need / and /var in the same device, that means /var can not be a 
> separate partition. and you need tmpfs enabled in the kernel (not module) 

1. There's no TMPFS_MNT in /etc/bootchartd.conf so "changing" it is not an
option.
2. To what or where should I set it?

Does bootchart work *now* without the obsoleted baselayout patch?

(In reply to comment #49)
> 1. There's no TMPFS_MNT in /etc/bootchartd.conf so "changing" it is not an
> option.
> 2. To what or where should I set it?

Well, it's not critical - you just won't get nice names on the bootchart. I've
got a patch for that in the works though. Basically we mount --bind the
bootchart log dir to /lib/bootchart and the unmount it when we're done.

Hopefully have this in portage later today, if not then sometime early next
week.

bootchart is integrated now

Re-opening sa bootchart itself isn't in the tree until I get the a small issue
with the binary for process accounting being available in / resolved.

*** Bug 143811 has been marked as a duplicate of this bug. ***

bootchart is in portage.

Note that process accounting will fail unless /usr is on the same partition as
/

(In reply to comment #54)
> Note that process accounting will fail unless /usr is on the same partition as
> /

Is there a fix planned or at least a known workaround?

(In reply to comment #55)
> Is there a fix planned or at least a known workaround?

Just wanted to clarify that process accounting is an optional feature, the
bootchartd logger will be able to get a good enough process tree without it. 
Details here: http://www.bootchart.org/docs.html

In any case, the way this is solved in Fedora is by installing the accton
executable in /sbin and symlinking it from /usr/sbin/.

users can copy it themselves, but i see no reason to punish everyone by moving
acct to / simply because of an optional *development* addon like bootchart

(In reply to comment #57)
> users can copy it themselves, but i see no reason to punish everyone by moving
> acct to / simply because of an optional *development* addon like bootchart

Spanky, out of curiosity: What would be the problem with moving it to /sbin?
Maybe another einfo line to inform the users about copying accton would be ok?

(Ziga, thanks, it works nicely for me like that.)

Looking at other distributions, accton/acctoff does actually need to move to /
as it is applicable in some cases for NFS-mounted /usr. It's a very small, 5k
on my x86 machine.

However those with a seperate /var partition should be even more careful, as
they might be data written to the /var structure on the / partition, instead of
the /var partition. To avoid this, early during the boot, they should accton to
an accessible location, and then later use the existing acct init script to
switch to /var/account/.