Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 22822 - sys-kernel/ck-sources: proposals for a tweakable CK kernel
Summary: sys-kernel/ck-sources: proposals for a tweakable CK kernel
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: [OLD] Core system (show other bugs)
Hardware: x86 Linux
: High enhancement (vote)
Assignee: x86-kernel@gentoo.org (DEPRECATED)
URL:
Whiteboard:
Keywords: EBUILD
: 23248 23709 (view as bug list)
Depends on:
Blocks:
 
Reported: 2003-06-14 09:57 UTC by TGL
Modified: 2004-02-17 04:49 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
ck-sources-2.4.21-r1.ebuild (ck-sources-2.4.21-r1.ebuild,5.28 KB, text/plain)
2003-06-14 09:59 UTC, TGL
Details
ck-sources-full-2.4.21-r1.ebuild (ck-sources-full-2.4.21-r1.ebuild,6.25 KB, text/plain)
2003-06-14 10:01 UTC, TGL
Details
ck-sources-2.4.21-r1.ebuild (ck-sources-2.4.21-r1.ebuild,3.28 KB, text/plain)
2003-06-15 11:28 UTC, TGL
Details
ck-sources-2.4.21-r1.ebuild (ck-sources-2.4.21-r1.ebuild,3.28 KB, text/plain)
2003-06-15 13:24 UTC, TGL
Details
ck-sources-2.4.21-r1.ebuild (ck-sources-2.4.21-r1.ebuild,3.29 KB, text/plain)
2003-06-16 03:38 UTC, Dominic Böttger
Details
ck-sources-2.4.21-r1.ebuild (ck-sources-2.4.21-r1.ebuild,5.51 KB, text/plain)
2003-06-16 15:45 UTC, TGL
Details
ck-sources-2.4.21-r1.ebuild (ck-sources-2.4.21-r1.ebuild,5.93 KB, text/plain)
2003-06-17 08:10 UTC, Dominic Böttger
Details
ck-sources-2.4.21-r1.ebuild (ck-sources-2.4.21-r1.ebuild,5.81 KB, text/plain)
2003-06-17 17:03 UTC, TGL
Details
ck-sources-2.4.21-r1.ebuild (ck-sources-2.4.21-r1.ebuild,5.71 KB, text/plain)
2003-06-18 15:13 UTC, TGL
Details
ck-sources-2.4.21-r1.ebuild (ck-sources-2.4.21-r1.ebuild,5.58 KB, text/plain)
2003-06-18 17:01 UTC, TGL
Details
ck-sources-2.4.21-r1.ebuild (ck-sources-2.4.21-r1.ebuild,5.58 KB, text/plain)
2003-06-19 03:29 UTC, TGL
Details
ck-sources-2.4.21-r1.ebuild (ck-sources-2.4.21-r1.ebuild,5.52 KB, text/plain)
2003-06-19 13:09 UTC, TGL
Details
ck-sources-2.4.21-r2.ebuild (ck-sources-2.4.21-r2.ebuild,5.47 KB, text/plain)
2003-06-20 08:11 UTC, TGL
Details
ck-sources-2.4.21-r2.ebuild (ck-sources-2.4.21-r2.ebuild,5.80 KB, text/plain)
2003-06-22 16:32 UTC, TGL
Details
ck-sources-2.4.21-r2.ebuild (ck-sources-2.4.21-r2.ebuild,5.78 KB, text/plain)
2003-06-22 17:23 UTC, TGL
Details
ck-sources-2.4.21-r3.ebuild (ck-sources-2.4.21-r3.ebuild,5.32 KB, text/plain)
2003-06-23 02:39 UTC, TGL
Details
ck-sources-2.4.21-r3.ebuild (ck-sources-2.4.21-r3.ebuild,5.68 KB, text/plain)
2003-07-05 15:58 UTC, TGL
Details
optional ck-sources ebuild (ck-sources-2.4.21-r3.ebuild,873 bytes, text/plain)
2003-08-08 20:08 UTC, Luis Medinas (RETIRED)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description TGL 2003-06-14 09:57:19 UTC
Hi,

I've started to make some new ebuilds for the CK kernel. The goal is to provide 
not only the core performance patches but also the optional features listed 
here: http://members.optusnet.com.au/ckolivas/kernel/index2.html

I have written two ebuilds which allow, depending on several USE flags, the
following:
 - base CK patchset (O(1) scheduler, preemptible, low latency)
 - ACPI 
 - Supermount 
 - XFS
 - nForce2 patchs
 - Bootsplash
 - Swap prefetching
 - CPU Frequency scaling for supported laptops
 - Grsec 
 - AAVM, as a replacement of the CK autoregulation VM (only in the 2nd ebuild)
 - Rmap, as a replacement of the CK autoregulation VM (only in the 2nd ebuild)

This ebuilds are here as a submission but more as a request for comments, for
several reasons:
 - the official CK patch for 2.4.21 is not out yet, patches names will probably
change, etc.
 - the two alternatives VM patches are currently buggy and doesn't apply
 - I've used some conditional SRC_URI, which is something I've never seen
before, and I don't know if this is a good idea
 - I use a lot of local use flags, maybe this should be replace by another
variable? (like the ALSA_CARDS variable in alsa-driver)
 - this will probably be really boring to test, there are so many possible
combinations... I've personnaly assumed that CK knows his job.
 - I know there is already a lot of available kernels, and a new one (not really
new, but more complicated) is more work for maintainers, and if you think this
one is not really needed, I can understand.

-- 
TGL.

Reproducible: Always
Steps to Reproduce:
Comment 1 TGL 2003-06-14 09:59:27 UTC
Created attachment 13267 [details]
ck-sources-2.4.21-r1.ebuild

The first ebuild, without the choice of the VM. (the -r1 is to match -ck
revision)
Comment 2 TGL 2003-06-14 10:01:49 UTC
Created attachment 13268 [details]
ck-sources-full-2.4.21-r1.ebuild

The second ebuild, with the choice of the VM. Patches seem to be broken
anyway...
Comment 3 FL 2003-06-14 12:36:54 UTC
as i know from my experience with ck, the system stops working during the init process when one script tries to "activate possibly more swap". this is in course of ck's own vm hack. so maybe it is better to use the *full* sources and change to rmap or aa. also some standalone patches of this website won't apply correctly to the kernel.
Comment 4 TGL 2003-06-15 02:25:32 UTC
> the system stops working during the init process when one script tries to 
> "activate possibly more swap".

I've never had such an issue. But this probably depends on the what swap devices you have in fstab (because its a "swapon -a" which is executed at this point). I only have one partition, and you?

> so maybe it is better to use the *full* sources and change to rmap or aa
> also some standalone patches of this website won't apply correctly to the 
> kernel.

I've tried quite a lot of combinations without reject but for the aa and rmap patches, so the -full ebuild is not useable right now. The aa patch is obviously corrupted, and I've not looked in depth at the rmap patch yet, but it doesn't apply neither. I've only left this ebuild hoping this patches will get fixed in a next release. 
Comment 5 TGL 2003-06-15 11:28:18 UTC
Created attachment 13316 [details]
ck-sources-2.4.21-r1.ebuild

Here is a new version of the ebuild. 
Changes:
 - use the standard full patch instead of patches 1000 to 1041 (since it's now
available and apply flawlessly)
 - use some "flag? (  )" in the SRC_URI. Don't ask me why I've not done it this
way the first time, I have no idea...
 - updated the URL.
Comment 6 TGL 2003-06-15 13:24:28 UTC
Created attachment 13327 [details]
ck-sources-2.4.21-r1.ebuild

Oops, their was a typo in the previous one...
Comment 7 Dominic Böttger 2003-06-16 03:38:38 UTC
Created attachment 13359 [details]
ck-sources-2.4.21-r1.ebuild

XFS - patch had the wrong URL, now I fixed it.
Comment 8 Dominic Böttger 2003-06-16 03:55:01 UTC
XFS - Patch could not be applied
Comment 9 Dominic Böttger 2003-06-16 04:43:38 UTC
If xfs is patched into kernel, grsec and supermount will fail. If supermount is in kernel, grsec and xfs will fail. I have no solution yet.
Comment 10 TGL 2003-06-16 06:10:26 UTC
With the previous release of the xfs patch, this different combinations applied fine. Since it has been removed from CK page, I've mirrored it here if you need it: http://tdegreni.free.fr/gentoo/1060_XFS_0305311351_2.4.21-ck1.patch.bz2

Now, to use the new version, we have to wait for a new version of the grsec patch. I think CK has thought about that and will release it when he have time. And I will try to  make the required changes to supermount cause this ones seems obvious.
Comment 11 TGL 2003-06-16 07:07:18 UTC
Here is a modified supermount patch that won't conflict with the new version of the xfs patch:
http://tdegreni.free.fr/gentoo/1050_SM1.2.7_030616_2.4.21-ck1.patch.bz2
Comment 12 Chris Gianelloni (RETIRED) gentoo-dev 2003-06-16 12:52:32 UTC
This would be a lot better if you keep it as seperate patches versus the one large patch.  The reason for this is that the variable Hz and Desktop Tuning1 patches break PCMCIA.  You would rather want to drop those two patches if pcmcia is in USE.
Comment 13 TGL 2003-06-16 15:43:13 UTC
Chris: you're right. I've made the changes to take care of this in the following ebuild.
Comment 14 TGL 2003-06-16 15:45:04 UTC
Created attachment 13388 [details]
ck-sources-2.4.21-r1.ebuild

Changes:
 - back to the separate patches for the main CK patch set.
 - disable some patches if pcmcia is in USE.
 - added an updated Rmap patch. It can't be used together with XFS.
 - use an updated XFS_PATCH (see comment #7).
 - use an updated SM_PATCH (see comment #11).
 - use an updated BASE_PATCH.
 - use an updated CFS_PATCH.
 - various cleanups.
Comment 15 Dominic Böttger 2003-06-17 08:10:57 UTC
Created attachment 13416 [details]
ck-sources-2.4.21-r1.ebuild

Based on the newest fixes from TGL I added the athlon superpage patch. It's
only for testing purposes yet.
Here you can get som infos about athlon superpage:
http://shimizu-lab.dt.u-tokai.ac.jp/lsp.html

The patch was originally developed for ck-2.4.20-r6
I hope it will run, because I got a great performance boost.
What do you think about it?
Comment 16 TGL 2003-06-17 16:57:29 UTC
Dominic, I don't think you can apply this patch that way. There have been several changes to some files (mm/memory.c for instance) that the patch will break. That said, I've looked at the webpage you've pointed, and it really looks interresting. I've made a first cleanup of the patch, but there are still some issues. I will try to finish that later if I can. 


Today there was again some updates to rmap and xfs patches. I've updated my ebuild, tried again several combinations, and have reported the success and failures to CK. So far, I've successfully run:
 * ck1
 * ck1+xfs
 * ck1-ckvm+rmap
(and also this ones +acpi and/or +supermount)
The minor patches nforce2, bootsplash and freqscale seem to merge fine in cases I've tried so far, but I've not compiled/tested them yet (I will be able to test bootsplash, but not freqscale or nforce2).
And there are still some rejects on rmap+xfs, on grsec, and on aavm.


Comment 17 TGL 2003-06-17 17:03:47 UTC
Created attachment 13436 [details]
ck-sources-2.4.21-r1.ebuild

Changes:
 - XFS update 
 - Rmap update 
 - reverse to the previous Supermount patch, since CK has solved all conflicts
in his new XFS patch
 - better level of confidence in the different patch combinations allowed by
the ebuild :)
Comment 18 Dominic Böttger 2003-06-18 02:14:46 UTC
It's a pity, but it was a test.
I had this patch in the last ck-2.4.20-r6 kernel and it was great. In the german forum we talked about a great performance boost. I will ask the developer of this patch for an update to 2.4.21.
I think it will be perfect for the performance optimized ck-sources.
I hope the patches from con will soon be final for 2.4.21.
Comment 19 Philipp H 2003-06-18 12:36:48 UTC
Hi, 
 
I build a kernel from your cg-surces ebuild, everything seems to be fine expect the 
emu10k1 driver ebuild. I get a Memory Access Violation (translated from german 
"Speicherzugriffsfehler") when I want to load the modul. 
 
please fix that error, if you need further information please send me a Mail or contact 
me on icq. 
Comment 20 TGL 2003-06-18 15:13:19 UTC
Created attachment 13488 [details]
ck-sources-2.4.21-r1.ebuild

Changes:
 - updated to the latest BASE patch
 - removed DT3 (now redundant)
 - small correction to ST patch to match the BASE changes
Comment 21 TGL 2003-06-18 15:35:47 UTC
Hi Philipp,

 1) does this emu10k1 drivers still works with vanilla 2.4.21? If it doesn't work, wait at least for a new version or cvs snapshot. There has been a small update in the last few days, it's rare enough to seem important ;)

 2) have you tried the kernel emu10k1 drivers with 2.4.21-ck1? Appart that they lack the utility scripts, they are almost the same, and they are more up to date on the ac97 codecs which is all that has changed in 2.4.21 anyway. (Maybe you can even use the scripts from the ebuild with this drivers, I've not tried.) If they doesn't work, tell me and I will make my tests and report to ck.

Thanks.
Comment 22 TGL 2003-06-18 17:01:04 UTC
Created attachment 13497 [details]
ck-sources-2.4.21-r1.ebuild

Changes:
 - AAVM included (it was really trivial to cleanup in fact, I should have given
it a close look earlier...)
Comment 23 TGL 2003-06-19 03:29:50 UTC
Created attachment 13528 [details]
ck-sources-2.4.21-r1.ebuild

Changes:
 - URI updates
Comment 24 TGL 2003-06-19 13:09:59 UTC
Created attachment 13562 [details]
ck-sources-2.4.21-r1.ebuild

Changes:
 - cleaned XFS to make it apply also on rmap based sources
 - added a pkg_postinst()
Comment 25 TGL 2003-06-20 08:11:58 UTC
Created attachment 13598 [details]
ck-sources-2.4.21-r2.ebuild

Changes:
 - Bumped the main patches to -ck2pre (additional patches are still okay)
 - Swap prefetch is now default (no more USE flag)
 - Minor ebuild cleanups
Comment 26 TGL 2003-06-22 16:32:26 UTC
Created attachment 13698 [details]
ck-sources-2.4.21-r2.ebuild

Changes:
 - various patches revision bumps
 - added Grsec (USE: grsec)
 - added Packet Writing for cdrw/dvdr (USE: packetwrit)
 - ebuild cleanups (added some utility functions to help dealing with patches)
 - mirrored the patches on my own webspace because Con's page is moving fast...
Comment 27 TGL 2003-06-22 17:23:54 UTC
Created attachment 13703 [details]
ck-sources-2.4.21-r2.ebuild

Oops, there was a trailing test code line in the previous one.
Comment 28 TGL 2003-06-22 18:15:53 UTC
Maybe it's time for a small "status" comment:

* Things I've successfully used in many combination:
 - The standard -ck2 patchset (default). Some parts may be disabled if USE contains 'pcmcia' or 'aavm' or 'rmap'.
 - ACPI (USE: 'acpi')
 - XFS (USE: 'xfs')
 - Supermount (USE: 'supermount')
 - Bootsplash (USE: 'bootsplash')
 - AAVM, as a replacement to standard CKVM (USE: 'aavm')
 - Rmap, as a replacement to standard CKVM (USE: 'rmap')

* Things that are included, apply and compile fine, but I've not tested yet by lack of time:
 - Grsec (USE: 'grsec'). Can't be used with rmap or aavm. Implies XFS.

* Things that are included, apply and compile fine, but I've not tested yet by lack of hardware. I really need comments on this ones:
 can't test:
 - Nforce2 patch (USE: 'nforce2')
 - Frequency scaling for laptops (USE: 'freqscal')
 - Packet writing for CDRW/DVDR (USE: 'packetwrit')

To summarize my experience, I was quite sceptical at the beginning about this ebuild. I'm not anymore, it works fine and is not such a mess to maintain. Con Kolivas really does a great job. Things I still wonder, and I would like some comments on:
 - do you think there is some need for this kind of customizable kernel?
 - what would you think of a standard way to include user's own patches? [1]
 - is USE with quite a lot of local flags the right way to take user's choices, or should I use another variable?
 - is it an issue that some USE flags may conflicts? If it is, I can replace conflicts by a few overridings. 


[1]:
In fact, I personnaly use a version with this line:
   [ -n "${CUSTOM_PATCHES}" ] && add_patches ${CUSTOM_PATCHES}
And then "CUSTOM_PATCHES='1200_foo.bz2 1201_bar.bz2' emerge ck-cources" to add one or two very specific things that are not in -ck.
Comment 29 Chris Gianelloni (RETIRED) gentoo-dev 2003-06-22 18:54:27 UTC
Well, I will be honest with you, I think you could remove some of the optional USE flags and either decide if you want them or not.  I would stick with the USE flags that are in use already in Gentoo, such as aavm (and rmap), acpi, and xfs.  Personally, I would include the supermount, bootsplash, nforce2 and packet writing, without a need for user intervention.  I would also split out the grsec patch into multiple patches, such as how the Gentoo kernel does so, with and without xfs (it's only one part that rejects) and under the various VM selections.  This could still be a local USE variable.  I would add the frequency scaling to the pcmcia USE flag, since it only applies to laptops, and is very useful.

I guess my point is that having a huge number of local USE flags, many of which are redundant, seems to be a bit counterproductive.  This is only my opinion, but I felt I would share since I am following this ebuild and you requested comments.
Comment 30 Markus Nigbur (RETIRED) gentoo-dev 2003-06-22 22:09:05 UTC
You may handle it, like the fluxbox ebuild maintainer does.
Adding the patches to the ebuild, but comment out some rarely needed.
So the enduser can edit the ebuild to his needs by commenting out or removing a comment without having to deal with the rather complex ebuild syntax.
Comment 31 TGL 2003-06-23 02:28:00 UTC
Hi Chris,

Thanks a lot for your comments, they all makes sense. I'll submit soon en update to -ck3 which will use less USE flags. I was myself also thinking of splitting grsec, but it's not done yet, this will come in the next few days. 
Comment 32 TGL 2003-06-23 02:31:11 UTC
Hi Markus,

I would prefer to let the user out of the ebuild if possible. But thanks for the suggestion.
Comment 33 TGL 2003-06-23 02:39:25 UTC
Created attachment 13719 [details]
ck-sources-2.4.21-r3.ebuild

Changes:
 - bumped base patches to -ck3
 - following Chris' suggestions, removed several use flags:
    - Are now also included by default: supermount, nforce2, bootsplash 
      and packet writing
    - 'pcmcia' flag is used in place of 'freqscal'

Todo:
 - split grsec in at least two parts (common and xfs), maybe more if needed to
apply on rmap or ckvm.
Comment 34 TGL 2003-07-05 15:58:25 UTC
Created attachment 14179 [details]
ck-sources-2.4.21-r3.ebuild

Changes:
 - updated ACPI patch
 - updated scheduler tuning
Comment 35 Jay Pfeifer (RETIRED) gentoo-dev 2003-07-09 21:27:40 UTC
*** Bug 23709 has been marked as a duplicate of this bug. ***
Comment 36 Jay Pfeifer (RETIRED) gentoo-dev 2003-07-09 21:28:17 UTC
*** Bug 23248 has been marked as a duplicate of this bug. ***
Comment 37 Luis Medinas (RETIRED) gentoo-dev 2003-08-08 20:08:06 UTC
Created attachment 15783 [details]
optional ck-sources ebuild

Well this is my ebuild this only an update from ck-sources-2.4.20... its is
very simple,clean and ...it works
Comment 38 Tim Yamin (RETIRED) gentoo-dev 2004-01-18 02:26:34 UTC
Would somebody like to update this to 2.4.24?
Comment 39 Michal Januszewski (RETIRED) gentoo-dev 2004-02-17 04:09:26 UTC
I'm closing this as it doesn't appear to receive much attention since Sep 2003. Besides, we're now moving to 2.6  with the -ck patchset and there are no optional patches for 2.6 yet. 
Comment 40 TGL 2004-02-17 04:49:59 UTC
Yep, I should have closed it myself long time ago but i forgot, sorry about that. The idea was only good for choosing the vm in fact, but in the last version i made i had to do several glue patch depending of the combinations of the other patch, so i realized it was really too much bloat and would be a pain to maintain. 

In current 2.6-ck, it would probably make sense to add some flags for the hyperthreading tweaks or supermount-ng (using the individual patches), but that's not really needed since they can simply be ignored at config time, so using the all-in-one patch is enough.