Currently, dev-libs/boost has multiple build problems on arm64-macos, and arm64-macos is currently masked. Since Boost is a critical package that is used by a huge portion of C++ programs, a port is urgently needed. I did some initial troubleshooting. After fixing two problems, I hit a roadblock and couldn't fix it further. Help wanted. Problem 1: Boost wants to build static library, which is unsupported on macOS. Using the upstream ebuild boost-1.82.0.ebuild without modification, Boost fails to build because the build system wants to pass the "-static" flag to ar, but this option (and static linking in general) is unsupported by macOS. > arm64-apple-darwin22-ar: only one of -a and -[bi] options allowed > usage: ar -d [-TLsv] archive file ... > ar -m [-TLsv] archive file ... > ar -m [-abiTLsv] position archive file ... > ar -p [-TLsv] archive [file ...] > ar -q [-cTLsv] archive file ... > ar -r [-cuTLsv] archive file ... > ar -r [-abciuTLsv] position archive file ... > ar -t [-TLsv] archive [file ...] > ar -x [-ouTLsv] archive [file ...] > ...skipped <p/Users/ec2-user/gentoo/var/tmp/portage/dev-libs/boost-1.82.0/work/boost_1_82_0-.arm64/stage/lib>libboost_test_exec_monitor.a for lack of <pbin.v2/libs/test/build/darwin-12.1.0/gentoorelease/link-static/pch-off/threading-multi/visibility-hidden>libboost_test_exec_monitor.a... After some digging, it turned out that "toolset=darwin" is actually broken in Boost [1] and has even been removed from Boost since 2019 [2] - so it raises the question of whether Boost even work on ~x64-macos (which the KEYWORD claims to support). The solution is avoiding "darwin" and just use GCC instead, so I removed the Darwin check from the ebuild: > if [[ ${CHOST} == *-darwin* ]]; then > compiler="darwin" > compiler_version="$(gcc-fullversion)" Problem 2: O_LARGEFILE / D_LARGEFILE64_SOURCE is unsupported After making this change, the Boost was able to continue its build procedure, before it immediately hit another problem: > "arm64-apple-darwin22-g++" -fvisibility-inlines-hidden > -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE > -O2 -pipe -std=c++17 -fPIC -finline-functions -Wno-inline -Wall > -fvisibility=hidden -DBOOST_ALL_NO_LIB=1 -DBOOST_IOSTREAMS_DYN_LINK=1 > -DBOOST_IOSTREAMS_USE_DEPRECATED -DNDEBUG -I"." -c -o "bin.v2/libs/iostreams > /build/gcc-12.1/gentoorelease/pch-off/threading-multi/visibility-hidden > /file_descriptor.o" "libs/iostreams/src/file_descriptor.cpp" > libs/iostreams/src/file_descriptor.cpp: In member function _void > boost::iostreams::detail::file_descriptor_impl::open(const > boost::iostreams::detail::path&, std::ios_base::openmode)_: > libs/iostreams/src/file_descriptor.cpp:225:18: error: _O_LARGEFILE_ > was not declared in this scope > 225 | oflag |= O_LARGEFILE; > | ^~~~~~~~~~~ It appears that on modern macOS, 64-bit file I/O is already the default, so there's no support for a special option like O_LARGEFILE, so we should not declare the build option D_LARGEFILE64_SOURCE. To work around the problem I added this hack in the ebuild function create_user-config.jam(): > if [[ ${CHOST} == *-darwin* ]]; then > sed \ > -e 's/-D_LARGEFILE64_SOURCE//g' \ > -i "${BUILD_DIR}"/user-config.jam || die > fi There's also the question of how OS X on vintage PPC machines should be handled. Problem 3: invalid install_name found, your application or library will crash at runtime With both hacks applied, Boost can now finish building. Unfortunately it still cannot be installed as it's rejected by Portage due to invalid install_name. At this point I've completely running out of ideas. > * Working around completely broken build-system(tm) > * correcting install_name of /usr/lib/libboost_atomic.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_chrono.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_container.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_contract.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_date_time.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_filesystem.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_graph.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_iostreams.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_json.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_locale.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_log.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_log_setup.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_math_c99.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_math_c99f.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_math_c99l.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_math_tr1.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_math_tr1f.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_math_tr1l.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_nowide.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_prg_exec_monitor.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_program_options.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_random.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_regex.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_serialization.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_system.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_thread.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_timer.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_type_erasure.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_unit_test_framework.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_url.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_wave.dylib ... [ ok ] > * correcting install_name of /usr/lib/libboost_wserialization.dylib ... [ ok ] > >> Completed installing dev-libs/boost-1.82.0 into /Users/ec2-user/gentoo/var/tmp/portage/dev-libs/boost-1.82.0/image > > * Final size of build directory: 1835076 KiB ( 1.7 GiB) > * Final size of installed tree: 191580 KiB (187.0 MiB) > > > * QA Notice: Package triggers severe warnings which indicate that it > * may exhibit random runtime failures. > * ./boost/archive/iterators/wchar_from_mb.hpp:103:30: warning: member _boost::archive::iterators::wchar_from_mb<boost::archive::iterators::xml_escape<const char*> >::sliding_buffer<char>::m_buffer_ is used uninitialized [-Wuninitialized] > * ./boost/archive/iterators/wchar_from_mb.hpp:103:30: warning: member _boost::archive::iterators::wchar_from_mb<boost::archive::iterators::xml_escape<const char*> >::sliding_buffer<wchar_t>::m_buffer_ is used uninitialized [-Wuninitialized] > > * Please do not file a Gentoo bug and instead report the above QA > * issues directly to the upstream developers of this software. > * Homepage: https://www.boost.org/ > * QA Notice: invalid reference to bin.v2/libs/thread/build/gcc-12.1/gentoorelease/pch-off/threadapi-pthread/threading-multi/visibility-hidden/libboost_thread.dylib in //Users/ec2-user/gentoo/usr/lib/libboost_wave.dylib > * QA Notice: invalid reference to bin.v2/libs/serialization/build/gcc-12.1/gentoorelease/pch-off/threading-multi/visibility-hidden/libboost_serialization.dylib in //Users/ec2-user/gentoo/usr/lib/libboost_wserialization.dylib > * QA Notice: invalid reference to bin.v2/libs/thread/build/gcc-12.1/gentoorelease/pch-off/threadapi-pthread/threading-multi/visibility-hidden/libboost_thread.dylib in //Users/ec2-user/gentoo/usr/lib/libboost_type_erasure.dylib > * QA Notice: invalid reference to bin.v2/libs/log/build/gcc-12.1/gentoorelease/pch-off/threadapi-pthread/threading-multi/visibility-hidden/libboost_log.dylib in //Users/ec2-user/gentoo/usr/lib/libboost_log_setup.dylib > * QA Notice: invalid reference to bin.v2/libs/thread/build/gcc-12.1/gentoorelease/pch-off/threadapi-pthread/threading-multi/visibility-hidden/libboost_thread.dylib in //Users/ec2-user/gentoo/usr/lib/libboost_log_setup.dylib > * QA Notice: invalid reference to bin.v2/libs/filesystem/build/gcc-12.1/gentoorelease/pch-off/threading-multi/visibility-hidden/libboost_filesystem.dylib in //Users/ec2-user/gentoo/usr/lib/libboost_log_setup.dylib > * QA Notice: invalid reference to bin.v2/libs/chrono/build/gcc-12.1/gentoorelease/pch-off/threading-multi/visibility-hidden/libboost_chrono.dylib in //Users/ec2-user/gentoo/usr/lib/libboost_timer.dylib > * QA Notice: invalid reference to bin.v2/libs/thread/build/gcc-12.1/gentoorelease/pch-off/threadapi-pthread/threading-multi/visibility-hidden/libboost_thread.dylib in //Users/ec2-user/gentoo/usr/lib/libboost_log.dylib > * QA Notice: invalid reference to bin.v2/libs/filesystem/build/gcc-12.1/gentoorelease/pch-off/threading-multi/visibility-hidden/libboost_filesystem.dylib in //Users/ec2-user/gentoo/usr/lib/libboost_log.dylib > * ERROR: dev-libs/boost-1.82.0::gentoo_prefix failed: > * invalid install_name found, your application or library will crash at runtime > * > * Call stack: > * misc-functions.sh, line 837: Called install_qa_check > * misc-functions.sh, line 173: Called install_qa_check_macho > * misc-functions.sh, line 432: Called die > * The specific snippet of code: > * has allow_broken_install_names ${FEATURES} || \ > * die "invalid install_name found, your application or library will crash at runtime" > * > * If you need support, post the output of `emerge --info '=dev-libs/boost-1.82.0::gentoo_prefix'`, > * the complete build log and the output of `emerge -pqv '=dev-libs/boost-1.82.0::gentoo_prefix'`. > * The complete build log is located at '/Users/ec2-user/gentoo/var/tmp/portage/dev-libs/boost-1.82.0/temp/build.log'. > * The ebuild environment file is located at '/Users/ec2-user/gentoo/var/tmp/portage/dev-libs/boost-1.82.0/temp/environment'. > * Working directory: '/Users/ec2-user/gentoo/var/tmp/portage/dev-libs/boost-1.82.0/image' > * S: '/Users/ec2-user/gentoo/var/tmp/portage/dev-libs/boost-1.82.0/work/boost_1_82_0' [1] https://web.archive.org/web/20160713132921/https://svn.boost.org/trac/boost/ticket/9772#comment:19 [2] https://github.com/boostorg/build/issues/528
I found the culprit of the last problem. Boost' ebuild system generates incorrect dynamic library paths due to technical limitations, described by its packager as "completely broken build-system". So the ebuild contains fixups to work around the problem. The first part is manually changing the install_name of the dynamic libraries: > ebegin " correcting install_name of ${d#${ED}}" > install_name_tool -id "/${d#${D}}" "${d}" > eend $? This is working as expected. The next part is searching for references from one library to another and fix these pathes: > # fix references to other libs > refs=$(otool -XL "${d}" | \ > sed -e '1d' -e 's/^\t//' | \ > grep "^libboost_" | \ > cut -f1 -d' ') > local r > for r in ${refs}; do > ebegin " correcting reference to ${r}" > install_name_tool -change \ > "${r}" \ > "${EPREFIX}/usr/lib/${r}" \ > "${d}" > eend $? > done This hack is no longer working. The sed expression attempts to find all paths that starts with "libboost" and add them into a list of paths to be fixed. But it's currently no longer matching anything. Right now references to other libraries don't start with libboost but instead starts with a path prefix. I'm not sure what's going on... > $ otool -XL libboost_wave.dylib > //Users/ec2-user/gentoo/usr/lib/libboost_wave.dylib (compatibility version 0.0.0, current version 0.0.0) > bin.v2/libs/thread/build/gcc-12.1/gentoorelease/pch-off/threadapi-pthread/threading-multi/visibility-hidden/libboost_thread.dylib (compatibility version 0.0.0, current version 0.0.0) > /Users/ec2-user/gentoo/usr/lib/gcc/arm64-apple-darwin22/12.1.0/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.30.0) > /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 0.0.0)
I propose the following new workaround. Instead of "grep ^libboost" to match the beginning of the line, we use "grep libboost" to allow lines with a path prefix to be matched as well. We then call basename to extract the filename from the path. The new code is: > # fix references to other libs > # these paths look like this: > # bin.v2/libs/thread/build/gcc-12.1/gentoorelease/pch-off/threadapi-pthread/threading-multi/visibility-hidden/libboost_thread.dylib > refs=$(otool -XL "${d}" | \ > sed -e '1d' -e 's/^\t//' | \ > grep "libboost_" | \ > cut -f1 -d' ') > local r > for r in ${refs}; do > # strip path prefix from references, leaving only something like: > # libboost_thread.dylib > local r_basename=$(basename ${r}) > > ebegin " correcting reference to ${r_basename}" > install_name_tool -change \ > "${r}" \ > "${EPREFIX}/usr/lib/${r_basename}" \ > "${d}" > eend $? > done It allows Portage to finally install Boost. But I still need to check whether it's functional or not.
I don't know which application is the best for testing Boost. As the ebuild said, the Boost's own test is difficult to run and doesn't really test much. I found a random Boost programming tutorial at https://github.com/sprinfall/boost-asio-study. I ran some Boost::Asio demos, and it appears to run okay. So it means linking and runtime behaviors more or less is functioning as expected. I'll submit a patch later.
I just opened a Pull Request at https://github.com/gentoo/gentoo/pull/30758 to solve this problem.
Follow-up: with this patched Boost, I was able to run openEMS, an electromagnetic simulation program that solves Maxwell's equations in 3D space. This program uses Boost for string manipulation and multithreading. So Boost definitely works, at least for the commonly-used parts.
The bug has been closed via the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=e442bb64a57259f89e12d901c8fe23a5abe62a76 commit e442bb64a57259f89e12d901c8fe23a5abe62a76 Author: Yifeng Li <tomli@tomli.me> AuthorDate: 2023-05-16 21:11:24 +0000 Commit: David Seifert <soap@gentoo.org> CommitDate: 2023-05-16 21:11:24 +0000 dev-libs/boost: fix build on macOS / Apple Silicon This commits fix three problems in existing Boost 1.82.0 ebuilds, allowing one to install Boost on macOS, including Apple Silicon. 1. Boost wants to build static library, which is unsupported on macOS. Using the unmodified ebuild, Boost fails because the build system wants to pass the "-static" flag to ar, but this option (and static linking in general) is unsupported by macOS, creating this error message: arm64-apple-darwin22-ar: only one of -a and -[bi] options allowed It turned out that "toolset=darwin" is actually broken in Boost [1] and has even been removed from Boost since 2019 [2]. Thus, the fix is to remove the option compiler="darwin". 2. error: O_LARGEFILE was not declared in this scope It appears that on modern macOS, 64-bit file I/O is already the default, and there's no special support for options like O_LARGEFILE. Thus, on on Darwin, we avoid running the command append-lfs-flags. 3. invalid install_name found, your application or library will crash at runtime To fix the broken install_name and references, a hack is used on Darwin to find a list paths. The original command began with "grep ^libboost" to match a library name at the beginning of the output of "otool -XL". But for some reason, the library names now include a path prefix, such as: bin.v2/libs/thread/build/gcc-12.1/gentoorelease/pch-off/ threadapi-pthread/threading-multi/visibility-hidden/ libboost_thread.dylib Thus, matching at the beginning of the line no longer works. To fix the problem, we instead use "grep libboost" to allow lines with a path prefix to be matched as well. We then extract the basename the filename from the path. [1] https://web.archive.org/web/20160713132921/https://svn.boost.org/trac/boost/ticket/9772#comment:19 [2] https://github.com/boostorg/build/issues/528 Closes: https://github.com/gentoo/gentoo/pull/30758 Closes: https://bugs.gentoo.org/904983 Signed-off-by: Yifeng Li <tomli@tomli.me> Signed-off-by: David Seifert <soap@gentoo.org> dev-libs/boost/boost-1.82.0.ebuild | 36 +++++++++++++++++++++++------------- 1 file changed, 23 insertions(+), 13 deletions(-)
