Hello everyone, on request from https://github.com/gentoo/gentoo/pull/39857#issuecomment-2563810769 I'd like to discuss with you way to use system libraries in Zig packages. Some of them contain special toggles for their build scripts (called "system integration options" in Zig terminology), which is often used to choose between: 1) Dynamically or statically linking to system library using found compiled objects, or 2) Compiling static (possibly modified) version of same library using Zig build system. In zig.eclass this is controlled by adding "-fsys=[name]" or "-fno-sys=[name]" flags during src_configure. zig-ebuilder currently emits USE-flags "+system-[lib_name]" for every such option found, and based on them force-enables or force-disables them in ebuild. It was mentioned that: > We generally do unconditional system libraries unless there's a good reason > not to, e.g. upstream sometimes modify the bundled copy. Which suggests removing these USE-flags and always enabling "system integrations" in ebuild options. I have tried USE-flags approach due to these factors: * Projects which are converted to Zig Build system usually follow one of the 4 routes: * * (Likely no modifications) Creating repository with build system part and adding tarball with upstream source as dependency in build.zig.zon or as git submodule. In this case you can be sure that it was not modified and likely close to the vanilla variant. * * (Can be patched) Same but tarball is changed to forked version, * * (Can be patched) Full forking of upstream source * * (Likely no modification) upstreamed back to C project, like in `dev-libs/libpcre2`: https://github.com/PCRE2Project/pcre2/blob/pcre2-10.44/build.zig Sometimes modifications are unavoidable, because vanilla variant does not build correctly in zig-static case: `zig build` uses `zig cc` under hood when compiling C/C++ source files, which differs from regular `clang` compiler by 1) enabling UBSAN in "Debug" mode, 2) using compiler_rt written in zig instead of LLVM one. One such case happened with GLFW and its fork for game engine in Zig: https://devlog.hexops.com/2021/perfecting-glfw-for-zig-and-finding-undefined-behavior/ (they have improved since then and nowadays also supply "verify.sh" script to show what files (apart from build files) differ from upstream) * It allows more reproducibility and easier troubleshooting for both users and upstream developers. If they are asked to try with bundled dependencies and check if some bug persist, it is very easy to do by disabling them in package.use (and re-enabling after that). For ebuild authors it's also useful for checking ZBS_DEPENDENCIES completeness in case they filled it manually (idk why would they but anyway) or zig-ebuilder messed up somewhere. * Without USE-flags, it would be more difficult for users to disable system dependency for previously mentioned troubleshootings, they would need to handle this in /etc/portage/env like so: ZBS_ARGS_EXTRA="-fno-sys=[some_library]" And also ensure that it applies only to package which have this option, since Zig will not ignore non-existant option and would instead fail with: > system library name not recognized by build script: 'some_library' > access the help menu with 'zig build -h' What do you think?
(In reply to Eric Joldasov from comment #0) > > * It allows more reproducibility and easier troubleshooting for both users > and upstream developers. If they are asked to try with bundled dependencies > and check if some bug persist, it is very easy to do by disabling them in > package.use (and re-enabling after that). For ebuild authors it's also > useful for checking ZBS_DEPENDENCIES completeness in case they filled it > manually (idk why would they but anyway) or zig-ebuilder messed up somewhere. > * Without USE-flags, it would be more difficult for users to disable system > dependency for previously mentioned troubleshootings, they would need to > handle this in /etc/portage/env like so: The issue with this argument is that it applies in general to bundled libraries. But we don't add USE=system-XXX for all bundled libraries, we default to using system libraries, and we only add a USE (in general, not just for zig) if there's a compelling reason, e.g. sometimes upstream make modifications to the bundled copy, or they're perhaps very hard to work with and really really want you to test with their bundled copy. I appreciate that last part is similar to what you're saying, but doing it for an entire ecosystem is something else.
I was linked here from https://github.com/gentoo/gentoo/pull/39924#discussion_r1900398547 To quote myself from there... It strikes me as incredibly wrong to first flout the rules and then open a bug report asking whether the rules should be respected. Instead, you should respect the rules and then open a bug report asking whether the rules should be broken. > zig-ebuilder currently emits USE-flags "+system-[lib_name]" for > every such option found IMO you need to immediately fix this to not do so, and *then* have a discussion about whether to start doing so again.
After some thinking I think you are right: * For reproducing part: user can always build it by themselves toggling flags as you need, * other parts don't scale well for "entire ecosystem", I expect number of system integration options to only grow in the future, and adding all these as a flags is too much. I have checked ghostty dependencies and all of them seem to be identical to their "vanilla" counterparts, excluding difference in version pinned here vs Gentoo (in meantime I have also found some small omissions in ebuild dependencies, and will open PR soon). I'll disable "system-*" USE-flag generation in zig-ebuilder now.
The bug has been referenced in the following commit(s): https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=5c8b025773af642a12ad84f94931b604e488addd commit 5c8b025773af642a12ad84f94931b604e488addd Author: Eric Joldasov <bratishkaerik@landless-city.net> AuthorDate: 2025-01-15 19:13:43 +0000 Commit: Joonas Niilola <juippis@gentoo.org> CommitDate: 2025-01-17 13:35:59 +0000 x11-terms/ghostty: add 1.0.1-r3 * Remove `system-*` USE-flags due to linked bug Also noticed when reviewing upstream build.zig files: * Enable `truetype` USE-flag on harfbuzz; Ghostty needs freetype-related functions as can be seen in `createFont` function from "pkg/harfbuzz/freetype.zig" * Remove libpng dependency and enable `png` USE-flag on freetype instead, as a separate dependency it is used in upstream only in static build case as a transient dependency of freetype (mentioned in build.zig comment). PNG is decoded with wuffs version instead. * Same with zlib dependency, it's only used as dependency of freetype and libpng. Decompression is done with Zig standard module version instead (`std.compress.zlib`). * Same with libxml2 dependency, for fontconfig. Bug: https://bugs.gentoo.org/947033 Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net> Signed-off-by: Joonas Niilola <juippis@gentoo.org> x11-terms/ghostty/ghostty-1.0.1-r3.ebuild | 125 ++++++++++++++++++++++++++++++ 1 file changed, 125 insertions(+)
zig-ebuilder enabled all system integrations for some time now, and no ghostty version now has `system-*`, so I'll mark it as resolved now
