Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 757513 - Re-enable building newer cmake on clang-based darwin prefix
Summary: Re-enable building newer cmake on clang-based darwin prefix
Status: RESOLVED WONTFIX
Alias: None
Product: Gentoo/Alt
Classification: Unclassified
Component: Prefix Support (show other bugs)
Hardware: All OS X
: Normal enhancement (vote)
Assignee: Gentoo Prefix
URL:
Whiteboard:
Keywords: PullRequest
: 757516 (view as bug list)
Depends on:
Blocks: 758167 755644
  Show dependency tree
 
Reported: 2020-11-28 23:10 UTC by Jacob Floyd
Modified: 2021-01-01 13:38 UTC (History)
0 users

See Also:
Package list:
Runtime testing required: ---


Attachments
darwin-isysroot patch for cmake-3.16.* (cmake-3.16.5-darwin-isysroot.patch,2.06 KB, text/plain)
2020-11-28 23:12 UTC, Jacob Floyd
Details
darwin-isysroot patch for cmake-3.17+ (cmake-3.17.4-darwin-isysroot.patch,2.32 KB, text/plain)
2020-11-28 23:15 UTC, Jacob Floyd
Details
Patch cmake-3.19.1.ebuild (cmake-3.19.1.ebuild.patch,1.91 KB, patch)
2020-11-30 00:46 UTC, Jacob Floyd
Details | Diff
Modified prefix-dirs patch for cmake 3.16+ (cmake-3.14.0_rc3-prefix-dirs.patch,4.64 KB, patch)
2020-11-30 00:52 UTC, Jacob Floyd
Details | Diff
bootstrap-prefix.sh modifications for bootstrapping cmake with clang (bootstrap-prefix.sh.patch,3.76 KB, patch)
2020-12-03 02:24 UTC, Jacob Floyd
Details | Diff
bootstrap-prefix.sh modifications for bootstrapping cmake with clang (0001-Fix-bootstrapping-cmake-with-clang-on-darwin.patch,4.47 KB, patch)
2020-12-10 04:10 UTC, Jacob Floyd
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jacob Floyd 2020-11-28 23:10:13 UTC
I'm looking into what it would take to use newer versions of cmake on darwin prefix. The first issue I see is that the darwin-isysroot.patch is commented out in all newer ebuilds and it does not apply cleanly. So, I have updated the patch so that it applies to all versions that are currently in tree.

Reproducible: Always

Steps to Reproduce:
Build a newer version of cmake than 3.13.5.
Actual Results:  
Masked

Expected Results:  
build newer

The mask comes from bug#689234:

> # Fabian Groffen <grobian@gentoo.org> (2019-03-22)
> # Newer versions break because they insist on sysroot usage lacking
> # -lSystem as looked for by clang/llvm.  If your system got this
> # package, you have to emerge -C cmake and re-emerge it.
> >dev-util/cmake-3.13.4
Comment 1 Jacob Floyd 2020-11-28 23:12:09 UTC
Created attachment 675634 [details]
darwin-isysroot patch for cmake-3.16.*

This only applies to ~dev-util/cmake/cmake-3.16.5
Comment 2 Jacob Floyd 2020-11-28 23:15:43 UTC
Created attachment 675637 [details]
darwin-isysroot patch for cmake-3.17+

This patch applies cleanly to >=dev-util/cmake-3.17.4-r1, including versions: 3.17.4-r1, 3.18.4, 3.18.5, 3.19.0, 3.19.1.

It also disables some additional logic in the Darwin-Initialize.cmake file that I would describe as: "try really hard to find an SDK to set sysroot", which of course we don't want.
Comment 3 Jacob Floyd 2020-11-28 23:23:56 UTC
I'm not sure if additional changes are required to prevent cmake from setting sysroot. I have read through a bunch of the cmake code and tested that these patches apply, but I have not compiled cmake with them yet.

I will file another issue shortly to fix building building cmake-3.13.5.ebuild during darwin prefix bootstrapping. I have a related patch that handles adjusting Framework path handling to check in a prefix-specific symlink to the SDK without using the sysroot flag.
Comment 4 Fabian Groffen gentoo-dev 2020-11-29 09:05:19 UTC
This is a shame you spent time on this, I fixed this one already in a different way for GCC profiles.  I guess the fix applies just about the same way for Clang usage.
Comment 5 Fabian Groffen gentoo-dev 2020-11-29 09:10:14 UTC
I just did this in the cmake ebuild:

>        sed -i -e '/cmake_gnu_set_sysroot_flag/d' \
>            Modules/Platform/Apple-GNU-C{,XX}.cmake || die

in the darwin specific hook.

Could you test patching the relevant Apple-Clang-C*.cmake in the same way to see if that makes it work?

Regarding the build fix during stage2, that may be necessary, but on the other hand the if the native toolchain is used, it shouldn't bother.  Not sure.
Comment 6 Jacob Floyd 2020-11-29 23:57:43 UTC
(In reply to Fabian Groffen from comment #5)
> I just did this in the cmake ebuild:
> 
> >        sed -i -e '/cmake_gnu_set_sysroot_flag/d' \
> >            Modules/Platform/Apple-GNU-C{,XX}.cmake || die
> 
> in the darwin specific hook.
> 
> Could you test patching the relevant Apple-Clang-C*.cmake in the same way to
> see if that makes it work?
> 
> Regarding the build fix during stage2, that may be necessary, but on the
> other hand the if the native toolchain is used, it shouldn't bother.  Not
> sure.

OK. I added my bootstrap fix to cmake-3.19.1.ebuild so that I could try patching Apple-Clang-C*.cmake. However, Apple-Clang-C*.cmake is not quite analogous to Apple-GNU-C*.cmake, so the sed needs to hit somewhere a bit different:

> sed -i -e '/SYSROOT_FLAG/d' Modules/Platform/Apple/Apple-Clang.cmake || die

One possible problem here is that does all languages, not just C/CXX. Is it problematic if this change applies to Apple-Clang-{OBJC,OBJCXX}? Why didn't you also adjust Apple-GNU-{OBJC,OBJCXX,Fortran}? And if applying to Apple-Clang-{OBJC,OBJCXX}.cmake isn't a problem, then why not have the GNU sed remove the sysroot line from Apple-GNU-*.cmake (instead of just C/CXX)?

Also, it looks like the GCC build doesn't build cmake during stage2. But with llvm+clang, I need cmake first, which means I can't configure clang with the correct search paths before building cmake as described in your comment:

> we've properly instructed where things are via GCC configuration and ldwrapper

And, finally, it looks like the newer cmake has additional deps, because src_configure is failing to find JsonCpp:

> -- Could NOT find JsonCpp: Found unsuitable version "", but required is at least "1.4.1" (found JsonCpp_LIBRARY-NOTFOUND)
> CMake Error at CMakeLists.txt:629 (message):
>   CMAKE_USE_SYSTEM_JSONCPP is ON but a JsonCpp is not found!
> Call Stack (most recent call first):
>   CMakeLists.txt:769 (CMAKE_BUILD_UTILITIES)
> 
> 
> -- Configuring incomplete, errors occurred!
> See also "/Users/jafloyd/Gentoo/tmp/var/tmp/portage/dev-util/cmake-3.19.1/work/cmake-3.19.1_build/CMakeFiles/CMakeOutput.log".
> See also "/Users/jafloyd/Gentoo/tmp/var/tmp/portage/dev-util/cmake-3.19.1/work/cmake-3.19.1_build/CMakeFiles/CMakeError.log".
>  * ERROR: dev-util/cmake-3.19.1::gentoo_prefix failed (configure phase):
>  *   cmake failed
Comment 7 Jacob Floyd 2020-11-30 00:00:41 UTC
Ah. There used to be a system-jsoncpp use flag in cmake 3.13.5. Maybe I can reintroduce it...
Comment 8 Jacob Floyd 2020-11-30 00:46:20 UTC
Created attachment 675796 [details, diff]
Patch cmake-3.19.1.ebuild

This patch drops -isysroot with Clang, and adds the changes necessary to fix bootstrapping with cmake-3.19.1

The bootstrap changes include re-introducing the system-jsoncpp use flag.
It requires adding the following in bootstrap-prefix.sh:

> 	# cmake needs to bootstrap its own jsconcpp
>	echo "dev-util/cmake -system-jsoncpp" >> "${ROOT}"/tmp/etc/portage/package.use
Comment 9 Jacob Floyd 2020-11-30 00:49:48 UTC
*** Bug 757516 has been marked as a duplicate of this bug. ***
Comment 10 Jacob Floyd 2020-11-30 00:52:01 UTC
Created attachment 675799 [details, diff]
Modified prefix-dirs patch for cmake 3.16+

I built 3.19.1 with this modified patch (which adds ${EPREFIX}/MacOSX.sdk to the framework search path). See bug#757516 for more details on why this is needed.
Comment 11 Fabian Groffen gentoo-dev 2020-12-02 19:27:25 UTC
updated work on bootstrap-prefix: https://bugs.gentoo.org/757381#c34
Comment 12 Jacob Floyd 2020-12-02 19:33:08 UTC
I've got a lot farther, now it bombs after compiling clang during emerge -e system.

I'll file a bunch of bugs + patches later to get more help debugging what's up with clang.
Comment 13 Jacob Floyd 2020-12-03 02:24:02 UTC
Created attachment 676357 [details, diff]
bootstrap-prefix.sh modifications for bootstrapping cmake with clang

This is how I adjusted the tree for cmake:

>	echo "->=dev-util/cmake-3.14" >> "${ROOT}"/tmp/etc/portage/package.mask
>	echo "->=dev-util/cmake-3.14" >> "${ROOT}"/etc/portage/package.mask
>	# changes to support new location for SDK frameworks
>	cp -f ~/p/gentoo/new/cmake.files/* "${PORTDIR}"/dev-util/cmake/files/
>	cp -f ~/p/gentoo/new/cmake-3.19.1.ebuild "${PORTDIR}"/dev-util/cmake/
>	ebuild "${PORTDIR}"/dev-util/cmake/cmake-3.19.1.ebuild manifest

So, we need to:

1- replace the patch with attachment 675799 [details, diff]
2- adjust the ebuild to attachment 675796 [details, diff]
     - reintroduce useflag system-jsoncpp (only disable for bootstrapping)
     - add SDK Frameworks to the search path during cmake bootstrap
     - disable adding the sysroot flag for clang
3- adjust bootstrap-prefix.sh to with this patch
     - install the MacOSX.sdk symlink for clang installs (not just for gcc)
     - set env vars to use SDK headers (for stage1 and stage2 only):
       C_INCLUDE_PATH, CPLUS_INCLUDE_PATH
       These vars are more reliable across multiple packages built with clang
       than trying to mess with command line args like: -isystem, -I, etc
     - disable the ssl useflag for rhash (a cmake dep)
       This disables using optimized algorithms from openssl libs (which are
       not available yet). Instead rhash uses internal implementations.
     - disable system-jsoncpp useflag for dev-util/cmake
4- drop the mask


I have more fixes, but they should go in other bugs.
Comment 14 Jacob Floyd 2020-12-10 04:10:43 UTC
Created attachment 677521 [details, diff]
bootstrap-prefix.sh modifications for bootstrapping cmake with clang

Updated patch to be an export (git format-patch) of the first commit in https://github.com/gentoo/prefix/pull/1
Comment 15 Jacob Floyd 2020-12-10 05:40:15 UTC
Comment on attachment 677521 [details, diff]
bootstrap-prefix.sh modifications for bootstrapping cmake with clang

Please see the PR on github. I missed a few changes, so I rebased and force pushed there.
Comment 16 Larry the Git Cow gentoo-dev 2020-12-18 20:19:46 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=3d7bc76da7f2d1f408e6ac7467697ac4aed5a1d3

commit 3d7bc76da7f2d1f408e6ac7467697ac4aed5a1d3
Author:     Fabian Groffen <grobian@gentoo.org>
AuthorDate: 2020-12-18 20:16:35 +0000
Commit:     Fabian Groffen <grobian@gentoo.org>
CommitDate: 2020-12-18 20:19:42 +0000

    dev-util/cmake-3.19.2: fix some issues when building with Clang, #757513
    
    Changes by Jacob Floyd, to respect the Prefix local MacOSX.sdk and to
    drop any isysroot crap when using Clang.
    
    Bug: https://bugs.gentoo.org/757513
    Package-Manager: Portage-3.0.9, Repoman-3.0.2
    Signed-off-by: Fabian Groffen <grobian@gentoo.org>

 dev-util/cmake/cmake-3.19.2.ebuild                      | 3 +++
 dev-util/cmake/files/cmake-3.14.0_rc3-prefix-dirs.patch | 8 +++++---
 2 files changed, 8 insertions(+), 3 deletions(-)
Comment 17 Jacob Floyd 2020-12-27 20:39:31 UTC
OK. The cmake ebuild in ::gentoo needs a couple more of my fixes. PR coming shortly...
Comment 18 Jacob Floyd 2020-12-28 02:14:57 UTC
OK. https://github.com/gentoo/gentoo/pull/18845 is the last change in ::gentoo required to fix cmake build during prefix bootstrap.

There's also a matching change to bootstrap-prefix.sh which is the first commit in https://github.com/gentoo/prefix/pull/1
Comment 19 Fabian Groffen gentoo-dev 2021-01-01 13:38:15 UTC
this is handled much simpler by just bootstrapping cmake in stage1