svscan | * Caching service dependencies ...
Service `readahead-list-early' needs non existant service `checkroot' [ ok ]
svscan | * Stopping service scan services ... [ ok ]
svscan | * Stopping service scan logging ... [ ok ]
svscan | * Stopping service scan ... [ ok ]
svscan | * Starting service scan ... [ ok ]
Service `readahead-list-early' needs non existant service `checkroot'
# this should start as early as possible
# we can't do 'before *' as that breaks it
before hostname modules domainname crypto-loop checkfs localmount
i think need fsck ??
# this should start as early as possible
# we can't do 'before *' as that breaks it
before hostname modules domainname crypto-loop checkfs localmount
(In reply to comment #0)
> Expected Results:
> i think need fsck ??
May be better: need root
because in depend this script was called before checkfs (that should be changed to fsck?).
with need root system will not boot...
Created attachment 157363 [details]
Created attachment 157365 [details]
This is working good on me with openrc-2.* but i'm not sure if it's ok.
Why did you close this bug? I never said that I wouldn't fix it.
Just that I hadn't gotten to it yet.
Currently my "readahead-list-early" only has a "before localmount root fsck clock" and works nice.
My "readahead-list" only has "need localmount"
This becouse my readahead-list-early is scheduled to run before *anything*.
My /usr and /var has their own partitions, and are not avaible at that time, so "readahead-list" is scheduled to only "need localmount" and readahead the rest that was not avaible when "readahead-list-early" ran.
I would post my files that works well, but they have some in this case cruft from me trying to implement "readahead-watch" from debian. All other ehaviour is original
Is this '!' there for some reason in "READAHEAD_LIST_*" from conf.d/readahead-list? They only prevented the scripts from working here.
(i.e. only thing that prevented these scripts from working for me was changing the init-files and remove the '!'s from the conf.d file.)
for me it only failed to start in one case:
root needs fsck which has "after clock modules"
but in the original "readahead-list-early" it wants to run BEFORE modules, makeing them block each other. So if you do not clean up the "before" part (it seems like you have that in these new ersions) OpenRC will fail to start anything since it can't start it's deps.
Created attachment 168928 [details]
merged early and main init.d script (with ionice)
Here my setup:
- OpenRC automatically read conf.d/service for init.d/service.foo so I renamed readahead-list-early to readahead-list.early.
- Then I merged "early" and "main" init.d script, readahead-list.early is now a symlink to readahead:
-rwxr-xr-x 1 root root 818 oct 18 15:24 readahead-list
lrwxrwxrwx 1 root root 14 oct 1 14:52 readahead-list.early -> readahead-list
- early have only before * and it works for me, but I have no special setup like raid, lvm, cryptfs...
I think dependencies should be kept minimal for early script (it should be really an early script!). Special setup could use extra dependencies from the config script conf.d/readahead-list.early like rc_need="lvm"...
I haven't heavy tested fsck and readahead-list running at the same time, 'cause i switched fsck at shutdown level
- with the incoming real sysinit-runlevel [https://bugs.gentoo.org/show_bug.cgi?id=240984], it will be possible to run it before udev! (I tried to hard /sbin/readahead-list in /lib/rc/sh/init.sh, and it worked fine).
- I run readahead-list in background and IO idle with ionice -c3 so it shouldn't slowdown the startup, (this may be a config option).
I think you should add somthing like
if mountpoint -q /usr; then
if mountpoint -q /var; then
[ -n "$dirs" ] || exit 0
for the script if it is not early.
This becouse the later script is only really needed if /usr, /opt, /var and / is not on the same partition (or mounted in some other strange way).
And that is another problem (but that maybe should be fixed in "linux-utils") and that is that ionice resides in /usr/bin and may not be avaible at early boot time.
Not really baselayout2 related but some mechanism to automatically generate the file list like the profile boot option and readahead-watch in Ubuntu would be nice. https://launchpadlibrarian.net/18941971/readahead-list_0.20050517.0220-1ubuntu4.diff.gz
Their readahead-watch is basically the same as the Python inotify watcher on gentoo wiki. However we want to implement it, all the bits and pieces are there. We just need to package them into a nice solution.
We can do something like this:
1. Read /proc/cmdline in our init scripts. If there's a profile option there, all init scripts launch the watcher instead of readahead-list.
2. In the early script, launch watcher and direct the output to the runlevel-boot file list.
3. In the readahead-list init script, kill the watcher from the early script. Start a new watcher and direct output to the runlevel-default file list.
4. Add a new script. Make it run after xdm or somehow make it after everybody. Wait for a few seconds to let xdm fully load first. Now kill the watcher.
Of course we might have to tweak and change a few details in this scheme to fit the Gentoo baselayout environment. But that will finally give us a sane, usable way for mere mortals to generate their own file lists.
I wanted to answer here for a better way to generate the lists.
It IS mentioned in the contrib/ directory of my package, but worth noting again.
I would consider the following as the BEST way to generate the lists.
1a. Build an initramfs with the following:
1a.1. Add in auditctl from sys-process/audit.
1a.2. Use auditctl to: set a large audit buffer (32mb or so should be plenty)
1a.3. Use auditctl to: load rules that log all exec and file open+read calls.
1b. If you don't need to log all of the early boot, you can do one of the following, just taking note that only file access AFTER that point will be recorded for optimization. This amounts to just running 1a.2 and 1a.3 from your real root, as early as you like.
2. Fairly late in the boot runlevel, start up the auditd half to dump the contents of the buffer, and continue recording up until the end of the default runlevel.
3. The logs generated by auditd now contain ALL files accessed for boot, in a manner such that they can be reasonably associated with processes, and a list can be generated for sorting.
The above has the additional benefit that since you have process ordering data, you can also do BETTER sorting than just on-disk order. A good starting point for evaluation would be group by process, and then use on-disk order for files accessed within each process.
Re the ubuntu patch:
1. The 10-readahead-list-sort.patch file is rejected for duplicating functionality already in the package.
2. The readahead-watch program has some large flaws:
2.1. inotify is only good for a single directory, so they traverse the ENTIRE tree from the given mountpoint, adding in every single directory. On my home box, that's 5328 inotify's AND I have /usr+/var on separate partitions.
2.2. needs to drop ALL ioctls and sorting of it's own.
2.3. Tracking a single mountpoint only :-(.
I created a new readahead-list.audit initscript (symlink like btarquini did) executing this commands:
# auditctl -b 65536 -r 1024
# auditctl -a exit,always -S open -S read -k bootup -F dir=/etc
# auditctl -a exit,always -S open -S read -k bootup -F dir=/bin
# auditctl -a exit,always -S open -S read -k bootup -F dir=/sbin
# auditctl -a exit,always -S open -S read -k bootup -F dir=/usr
# auditctl -a exit,always -S open -S read -k bootup -F dir=/var
# auditctl -a exit,always -S open -S read -k bootup -F dir=/lib
# auditctl -a exit,always -S open -S read -k bootup -F dir=/lib32
# auditctl -a exit,always -S open -S read -k bootup -F dir=/lib64
# # alternative:
# #auditctl -w /etc/ -p rx -k bootup
# #auditctl -w /bin/ -p rx -k bootup
# #auditctl -w /sbin/ -p rx -k bootup
# #auditctl -w /usr/ -p rx -k bootup
# #auditctl -w /var/ -p rx -k bootup
# #auditctl -w /lib/ -p rx -k bootup
# #auditctl -w /lib32/ -p rx -k bootup
# #auditctl -w /lib64/ -p rx -k bootup
(I also tried running it from initramfs, but setting watches on not-yet-available files did not work for me. (I cant preload that early anyway))
I've put it into sysinit runlevel (where early is).
To stop the recording after booting has finished, i've put the following commands into local_start() in /etc/conf.d/local:
# auditctl -D -k bootup
# auditctl -b 256 -r 0
# /etc/init.d/auditd stop # optional
Add auditd to default runlevel and make sure to adjust the config/rules (all rules are deleted on startup by default. -> comment out all lines in /etc/audit/audit.rules)
Before rebooting, please remove all readahead(but audit)/preload/prefetch initscripts from all runlevels to get resonable results. You might also want to delete old audit logs to avoid filtering afterwards.
from AUDITD(8) NOTES: A boot param of audit=1 should be added to ensure that all processes that run before the audit daemon starts is marked as auditable by the kernel. Not doing that will make a few processes impossible to properly audit.
Reboot with 'shutdown -rf now' to avoid fsck during audit.
Boottime might be slightly increaѕed while auditing.
After booting (please boot without readahead/preload/etc. to get resonable results) you can get the results with:
# ausearch -i -k bootup
Now we need to make a list for readahead-list of it (i'm sure this can be improved):
# ausearch -i -k bootup | tr '\n' ' ' | sed -e 's/----/\n/g' | sed -n 's/.*name=\([-_A-Za-z0-9\/\.]*\).*exe=\([-_A-Za-z0-9\/\.]*\).*/\2\n\1/p' | awk '!x[$1]++'
I searched for the line in the list where the first files after the readahead-list.main initscript would be started is and put all lines before into the early list, all files after into the main list. (adjust to your line numbers)
# cat list | head -n 165 > list.early
# cat list | tail -n +166 > list.main
Configure readahead-list to use your new lists.
Now just remove readahead-list.audit as well as auditd from runlevels and add the readahead/preload initscripts again to their runlevels.
Unfortunately the resulting lists are not as good as hoped for. The following quote from auditctl man pages might give the reason for this:
"The way that watches work is by tracking the inode internally. This means that if you put a watch on a directory, you will see what appears to be file events, but it is really just the updating of meta data. You might miss a few events by doing this. If you need to watch all files in a directory, its recommended to place an individual watch on each file."
Maybe there is a way to improve this situation?
Since i couldn't find any, and readahead-list doesnt seem to read in whole directorie contents, i was looking for an addition/another method generating the lists.
I found this method here: http://forums.gentoo.org/viewtopic-t-478491.html
So i created a readahead-list.lsof initscript symlink running
# /usr/sbin/readahead-list-lsof /etc/readahead-list/lsof.list &
This method needs binaries from /usr/bin, which is available quite late in my case. So i put it in boot runlevel, where main is. You might put it earlier (sysinit runlevel where early is) if your setup has /usr available at this time.
Reboot without readahead/preload services and fsck like above.
Then i merged /etc/readahead-list/lsof.list with my normal lists.
I'll attach my initscript aswell as the readahead-list-lsof script.
For gdm users: append /etc/X11/gdmprefetchlist to READAHEAD_LIST_main
I cant tell how much each single step decreased the bootup time, my recent bootup time (without X) is 38sec.
Created attachment 238355 [details]
/etc/init.d/readahead-list with early, audit and lsof
Created attachment 238361 [details]
/usr/sbin/readahead-list-lsof list-generation-script based on lsof
Created attachment 239903 [details]
/etc/init.d/readahead-list with early, audit and ionice
Here is a correction for the list generation:
# ausearch -i -k bootup | tr '\n' ' ' | sed -e 's/----/\n/g' | sed -n 's/.*name=\([-_A-Za-z0-9\/\.]*\).*pid=\([0-9]*\).*exe=\([-_A-Za-z0-9\/\.]*\).*/\2 \3\n\2 \1/p' | sort -n | sort -u | cut -d' ' -f2 | awk '!x[$1]++' > list
Consider disabling rc_parallels in /etc/rc.conf while auditing, to get better readable lists.
I reduced the amount of LVM logical volumes to further increase boot speed.
35sec with X(gdm) now.
(In reply to comment #16)
> Here is a correction for the list generation:
sry, this one:
# ausearch -i -k bootup | tr '\n' ' ' | sed -e 's/----/\n/g' | sed -n 's/.*name=\([-_A-Za-z0-9\/\.]*\).*pid=\([0-9]*\).*exe=\([-_A-Za-z0-9\/\.]*\).*/\2 \3\n\2 \1/p' | sort -n | cut -d' ' -f2 | awk '!x[$1]++' > list
This is looking good, but can we cleanup the lsof stuff and integrate it with the audit mode (if we even keep it at all, it seems a bit hackish to me).
I think the ausearch final command also needs packaging into a script of some sort.
Created attachment 249377 [details]
Script for the ausearch step.
1. Splits into early and main list automatically right where the bootmisc rc script is run.
2. Handles the case where the file is in relative path. (A few of them become '.' in the original sed script.)
3. Checks that the file actually exists. (Maybe it's a temporary file or maybe a program is trying to open an optional file that doesn't exist on this box. Either way, we can't preload that. Trim it from the list.)
4. Excludes certain patterns. This includes things related to auditd, which we're not going to run normally. And temporary files in /var/run, /var/tmp, etc. These don't exist until we run the program that generates it. We can't load them early.
I still get Service `readahead-list-early' needs non existant service `checkroot'. Could someone commit a correct (and safe) version of the initscript to the tree?
So a year since last comment and what is the status?
Anything in need of testing, anything in need of coding, anything holding this to be committed?
Two years. TWO YEARS. And still nothing ("In progress"? You got to be kidding). Is actually anybody working on this?
(In reply to comment #21)
> So a year since last comment and what is the status?
> Anything in need of testing, anything in need of coding, anything holding
> this to be committed?
Yes, the stuff I listed in comment #18 needs implementing still.
(In reply to comment #23)
> Yes, the stuff I listed in comment #18 needs implementing still.
(In reply to comment #18)
> This is looking good, but can we cleanup the lsof stuff and integrate it
> with the audit mode (if we even keep it at all, it seems a bit hackish to
> I think the ausearch final command also needs packaging into a script of
> some sort.
I think dropping lsof completely sounds reasonable for now.
And that ausearch-command, how about a script placed in /etc/local.d executing that command if it finds readahead-audit running?