Bug 940360 – dev-ml/* - vendored csexp and pp in Dune's internal libraries should be removed before the build process

• Product: Gentoo Linux
• Status: UNCONFIRMED ---

[Description Hiroki Tokunaga 2024-09-28 07:28:48 UTC]

Created attachment 904056 [details]
emerge --info

Currently, packages of Dune's internal libraries, such as dev-ml/dune-configurator and dev-ml/dune-private-libs, are built using vendored versions of csexp and pp instead of the system-wide packages dev-ml/csexp and dev-ml/pp. However, these vendored versions should be removed before the build process. The .opam files of packages dune-configurator contain lines instructing to remove these vendored ones (see [1]).

If the vendored ones are not removed, it leads to potential issues. For example, if a package depends on both dev-ml/dune-configurator and dev-ml/csexp, it may end up using two different versions of csexp. This conflict results in the "make inconsistent assumptions" error. I encountered this problem when I was writing an ebuild for GURU (see [2]).

[1]: https://github.com/ocaml/dune/blob/e4380ffddbdf924b3ec4c56048cd8331e1bf39ed/dune-private-libs.opam#L31-L32
[2]: https://forums.gentoo.org/viewtopic-p-8835262.html#8835262

[Comment 1 Larry the Git Cow 2024-09-28 14:30:36 UTC]

The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=2bdf5a887a2a4bac72846b7d2bdedcd506f39dfe

commit 2bdf5a887a2a4bac72846b7d2bdedcd506f39dfe
Author:     Hiroki Tokunaga <[e-mail]>
AuthorDate: 2024-09-28 07:53:21 +0000
Commit:     Alfredo Tupone <[e-mail]>
CommitDate: 2024-09-28 14:27:56 +0000

    dev-ml/*: remove vendored `csexp` and `pp` before the build process
    
    Closes: https://bugs.gentoo.org/940360
    Signed-off-by: Hiroki Tokunaga <[e-mail]>
    Closes: https://github.com/gentoo/gentoo/pull/38801
    Signed-off-by: Alfredo Tupone <[e-mail]>

 ...nfigurator-3.16.0.ebuild => dune-configurator-3.16.0-r1.ebuild} | 4 +---
 ...te-libs-3.16.0-r3.ebuild => dune-private-libs-3.16.0-r4.ebuild} | 7 +++++++
 2 files changed, 8 insertions(+), 3 deletions(-)

[Comment 2 Hiroki Tokunaga 2024-10-05 14:43:39 UTC]

This bug is again alive as the PR reverted[1,2].

The strange thing is that building dev-ml/dune fails with the following error after emerging dev-ml/dune-private-libs that is not built with the vendored csexp and pp (see [3].)

Done: 1480/1542 (jobs: 1)cd _boot && /usr/bin/ocamlopt.opt -o ../_boot/dune.exe -g -I +threads unix.cmxa threads.cmxa readdir.o wait4_stubs.o signal_stubs.o platform_stubs.o copyfile_stubs.o xdg_stubs.o sha512_stubs.o sha256_stubs.o sha1_stubs.o custom_opamStubs.o spawn_stubs.o dune_stats_stubs.o dune_flock.o dune_digest_stubs.o csexp_rpc_stubs.o fsevents_stubs.o inotify_stubs.o fswatch_win_stubs.o winsize.o -args compiled_ml_files -alert -unstable
Internal error, please report upstream including the contents of _build/log.
Description:
  ("Unexpected find result", { found = Not_found; lib.name = "pp" })
Raised at Stdune__Code_error.raise in file
  "otherlibs/stdune/src/code_error.ml", line 10, characters 30-62
Called from Fiber__Scheduler.exec in file "vendor/fiber/src/scheduler.ml",
  line 76, characters 8-11
-> required by ("<unnamed>", ())
-> required by ("<unnamed>", ())
-> required by ("<unnamed>", ())
-> required by ("load-dir", In_build_dir "default/test/blackbox-tests/utils")
-> required by ("toplevel", ())

I must not crash.  Uncertainty is the mind-killer. Exceptions are the
little-death that brings total obliteration.  I will fully express my cases. 
Execution will pass over me and through me.  And when it has gone past, I
will unwind the stack along its path.  Where the cases are handled there will
be nothing.  Only I will remain.

[1]: https://github.com/gentoo/gentoo/commit/e23364ef7dcff6e143e505d1979a121f521a0036
[2]: https://github.com/gentoo/gentoo/commit/3442c744414ac4009ab318670dbca8586f935280
[3]: https://github.com/gentoo/gentoo/pull/38801/files#diff-1e932d3d81aa79a29c616a8f2933bbe53f0b01aac6af94f038f36eb8e9b16364