Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 266383 - cmake should probably always bootstrap
Summary: cmake should probably always bootstrap
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: New packages (show other bugs)
Hardware: All Linux
: High major (vote)
Assignee: Gentoo KDE team
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-04-16 14:32 UTC by Zdenek Behan
Modified: 2009-05-02 23:36 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
cmake-2.6.3-r1.patch (cmake.patch,1.04 KB, patch)
2009-04-19 13:24 UTC, Tomáš Chvátal (RETIRED)
Details | Diff
Modified patch (cmake-check-usable.patch,881 bytes, patch)
2009-04-19 17:30 UTC, Zdenek Behan
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Zdenek Behan 2009-04-16 14:32:06 UTC
When emerging CMake, and no previous version of cmake is installed, cmake bootstraps itself with some sort of minimalist version during configuration phase, and then proceeds with the build.

If the system already contains a previous version of cmake, it uses this version. This is a problem if you're upgrading from a rather old version of cmake to rather new version of cmake. New cmake builds require new cmake features.

Example versions that broke things in my case:
upgrading from: 2.4.6-r1
upgrading to: 2.6.3-r1

Configuration phase went smoothly, with just a few warnings, and immediately at the start of build phase, i got this error:
arm-unknown-linux-gnu-gcc: DEFINES: No such file or directory

Quick look into the cmake generated Makefile says that the old cmake produced a string:
<compiler> DEFINES <more defines> -o <file> <sourcefile>
instead of string:
<compiler> ${CXX_DEFINES} <more_defines> -o <file> <sourcefile>

Unmerging the old version first, then emerging the new one fixes the problem.

Principially, I'd say that in enviroments so dynamic as gentoo, CMake should be always bootstrapped, eliminating influence of any previous versions installed. The extra time overhead is really microscopic, compared to the whole build. I'm pretty sure cmake should have a configuration option for forcing bootstrap, and if it doesn't, it's probably an upstream bug.
Other chance to fix this is set new versions as blocker for old versions that are known to be incompatible, which is a 1) huge task to test in the first place, 2) even more huge task to maintain as new versions come.

Reproducible: Always
Comment 1 Zdenek Behan 2009-04-16 22:30:03 UTC
Okay, looking into the ebuild, the bootstrap vs. normal configure logic is actually IN the ebuild.

However, the check 'if ! has_version ">=dev-util/cmake-2.4.1" ; then' is probably not sufficient. Hard to tell why, maybe there was a regression after 2.4.1 causing this.

Either way, no check can really make sure this does not happen again with future versions or future regressions. The only certain way to fix this is to always bootstrap.

Here's simple benchmarks from my quad core system:

Incremental build (with previous version installed):
real    1m57.453s
user    4m32.818s
sys     0m30.666s

Bootstrap:
real    2m36.869s
user    6m18.117s
sys     0m33.954s

So bootstrapping gives about 30% of increase in total time and 40% in user time. With the total time really being still in the single digit minutes domain anyway.
Comment 2 Tomáš Chvátal (RETIRED) gentoo-dev 2009-04-16 23:16:10 UTC
this issue was already fixed.
I forget on that the new eclass supports only 2.6 series.
Otherwise it would not fail.

Next time sinc before reporting bugs ;]
That way you will be sure if some bug is fixed or not.
Anyway thanks for the info, it is nice that people are actualy looking on what the heck i am doing ;]

Cheers
Comment 3 Tomáš Chvátal (RETIRED) gentoo-dev 2009-04-16 23:17:38 UTC
(In reply to comment #1)
> Okay, looking into the ebuild, the bootstrap vs. normal configure logic is
> actually IN the ebuild.
> 
> However, the check 'if ! has_version ">=dev-util/cmake-2.4.1" ; then' is
> probably not sufficient. Hard to tell why, maybe there was a regression after
> 2.4.1 causing this.
> 
> Either way, no check can really make sure this does not happen again with
> future versions or future regressions. The only certain way to fix this is to
> always bootstrap.
> 
> Here's simple benchmarks from my quad core system:
> 
> Incremental build (with previous version installed):
> real    1m57.453s
> user    4m32.818s
> sys     0m30.666s
> 
> Bootstrap:
> real    2m36.869s
> user    6m18.117s
> sys     0m33.954s
> 
> So bootstrapping gives about 30% of increase in total time and 40% in user
> time. With the total time really being still in the single digit minutes domain
> anyway.
> 

The speedup on quad is ince. Take look on my poor p3 500MHz, it was not really enjoying the bootstrap :]
Specialy when i spent month testing all cmake ebuilds in the tree for regressions ;D
Comment 4 Zdenek Behan 2009-04-16 23:44:43 UTC
(In reply to comment #2)
> Next time sinc before reporting bugs ;]
> That way you will be sure if some bug is fixed or not.
> Anyway thanks for the info, it is nice that people are actualy looking on what
> the heck i am doing ;]

I usually do. Unfortunately, I was fixing a very old gentoo install on a very slow arm computer, and it took me 4 days to progress through all the issues to fixing cmake. ;) And that's exactly how old my portage tree should be there.

I synced and it's indeed fixed. Thanks and sorry!

> The speedup on quad is ince. Take look on my poor p3 500MHz, it was not really
> enjoying the bootstrap :]

Well, as i said, for single-cpu machines, the slowdown with bootstrap is approximately a factor of 1.4x. Means that if you have a slow machine, the whole build will be slow, bootstrap or not, that's more like a minor difference in time requirements. It's not like gcc which is compiled 3 times around during bootstrap, _that_ sure makes a difference.

I also yesterday compiled cmake on a 233MHz ppc, and still would prefer bootstrap over incremental build, simply because bootstrap will only fail if something is wrong with the package, rather than with your system being outdated. The time spent bootstrapping is still way shorter than the time i spent trying to figure out what the hell is wrong. ;)

Just curious: How about adding a local use flag to force bootstrap? Maybe enabled by default, to give slow computer owners a choice to speed things up.
Comment 5 Zdenek Behan 2009-04-18 10:59:18 UTC
(In reply to comment #2)
> this issue was already fixed.
> I forget on that the new eclass supports only 2.6 series.
> Otherwise it would not fail.

OK, take this:

System with:
~cmake-2.6
~curl-7.15

1) upgrade curl to the most recent stable version (7.19?)
2) run cmake - "cmake: error while loading shared libraries: libcurl.so.3: cannot open shared object file: No such file or directory"
3) emerge cmake - same error

The soname from curl-7.15 changed from libcurl.so.3 to libcurl.so.4, so this is completely normal linking dependency breakage.

Upgrading any dependency to an incompatible ABI or different soname is a trap that cannot be escaped any other way than unmerging cmake and emerging it clean (forcing bootstrap). Exactly the same issue as before, only different flavour...

Looking at the soname list:
libdl.so.2,libexpat.so.1,libz.so.1,libcurl.so.4,libstdc++.so.6,libm.so.6,libgcc_s.so.1,libc.so.6 /usr/bin/cmake 
At the very least, libcurl and libexpat occasionally do change enough to break dynamic linking, so the issue seems serious enough to me.

So let me reiterate, i _STILL_ think that there should at least be a useflag to always force bootstrap, and in fact it should be negated (ie. no-bootstrap), to allow people who want to speed things up skip the bootstrap, as long as they're aware what the use-flag does and why it can break things. This would be the definite solution even for all future errors of this kind.


Principles aside, there's a simple way to hotfix this. Just make a check if cmake executes properly right within the ebuild, something like:
cmake --version &> /dev/null
HAVE_CMAKE=$?
if [ "${HAVE_CMAKE}" = "1" ]; then ./bootstrap ........; fi
Comment 6 Tomáš Chvátal (RETIRED) gentoo-dev 2009-04-19 13:24:14 UTC
Created attachment 188878 [details, diff]
cmake-2.6.3-r1.patch

Ok, good thinking, on this i kinda forgot :P
Try this patch it should incorporate the version check and the usability check so it should bootstrap when needed.
Report if it works as expected so i will apply it to the main tree.
Comment 7 Zdenek Behan 2009-04-19 17:30:31 UTC
Created attachment 188898 [details, diff]
Modified patch

Only problem that "cmake --version" will return 127 if a library is missing. I've normalized that to 0/1 value. Check the new patch.
Comment 8 Zdenek Behan 2009-04-19 18:37:02 UTC
Btw. are you still not convinced that having default behaviour of "always bootstrap" and having a local use flag for "no boostrap if conditions are good" would be easier, and play-it-safe solution for the future, yet still allowing faster builds for the people who need them? It's pretty much the same as "fastbuild" use flag for php, experimental, may be dangerous, but if you need it because of time constraints, turn it on, suffer the warning, and maybe the consequences.

Comment 9 Tomáš Chvátal (RETIRED) gentoo-dev 2009-04-19 20:17:19 UTC
(In reply to comment #8)
> Btw. are you still not convinced that having default behaviour of "always
> bootstrap" and having a local use flag for "no boostrap if conditions are good"
Well actualy i am lazy :D
Hm adding useflag... Might be good idea :]
Ok i will think about it when i wont feel so lazy.
Adjusting the priority because without it we cant stable 2.6.3
Comment 10 Zdenek Behan 2009-04-19 21:02:12 UTC
(In reply to comment #9)
> Well actualy i am lazy :D
> Hm adding useflag... Might be good idea :]
> Ok i will think about it when i wont feel so lazy.

You should have said so. I'd gladly do it myself, and send a patch. ;)
Comment 11 Marcus D. Hanwell (RETIRED) gentoo-dev 2009-05-02 23:36:40 UTC
CMake 2.6 can be build with CMake 2.4, but the Gentoo eclasses break that. CMake 2.6 can certainly be built with earlier CMakes and that is tested. So I think if a use flag was added it should be a bootstrap use flag to request that CMake is always bootstrapped. Having CMake build itself is a good default in my opinion with minimal risk.

I bumped to CMake 2.6.4 and applied this patch. Please open a bug if there are any further issues. Hopefully this is now working to everyone's satisfaction.