Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 704146 - dev-ml/extlib-1.7.6: Error: Syntax error: 'end' expected
Summary: dev-ml/extlib-1.7.6: Error: Syntax error: 'end' expected
Status: IN_PROGRESS
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal with 3 votes (vote)
Assignee: Gentoo Team for the ML programming language family
URL:
Whiteboard:
Keywords:
Depends on:
Blocks: 627946
  Show dependency tree
 
Reported: 2019-12-29 07:27 UTC by Michał Górny
Modified: 2021-01-27 07:50 UTC (History)
13 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
/var/log/portage/dev-ml:extlib-1.7.2:20191229-072327.log (dev-ml:extlib-1.7.2:20191229-072327.log,2.76 KB, text/x-log)
2019-12-29 07:27 UTC, Michał Górny
Details
/var/log/portage/dev-ml:extlib-1.7.6:20191229-130147.log (dev-ml:extlib-1.7.6:20191229-130147.log,2.54 KB, text/plain)
2019-12-29 13:03 UTC, Michał Górny
Details
extlib-1.7.6-no-line-directive.patch (no-line-directive.patch,517 bytes, patch)
2020-01-28 06:52 UTC, Naohiro Aota
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2019-12-29 07:27:36 UTC
Created attachment 601626 [details]
/var/log/portage/dev-ml:extlib-1.7.2:20191229-072327.log

ocamlfind ocamlc -pp "cppo -D OCAML4  -D OCAML4_02  -D OCAML4_03  -D OCAML4_04  -D OCAML4_05  -D WITH_BYTES" -g -package bytes -c extArray.mli extA
rray.ml
File "extArray.mli", line 114, characters 2-5:
114 | # 114
        ^^^
Error: Syntax error: 'end' expected
File "extArray.mli", line 29, characters 0-3:
29 | unctions} *)
     ^^^
  This 'sig' might be unmatched
make: *** [Makefile:45: extArray.cmo] Error 2


Normally, I'd say it looks like parallel make issue, expect the ebuild is already doing -j1.
Comment 1 Mikle Kolyada archtester Gentoo Infrastructure gentoo-dev Security 2019-12-29 11:44:54 UTC
Bumped.
Comment 2 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2019-12-29 13:02:49 UTC
Fails with the same error.
Comment 3 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2019-12-29 13:03:05 UTC
Created attachment 601678 [details]
/var/log/portage/dev-ml:extlib-1.7.6:20191229-130147.log
Comment 4 Mark Wright gentoo-dev 2020-01-16 02:17:36 UTC
Will push a fix soon.
Comment 5 Mark Wright gentoo-dev 2020-01-16 06:04:13 UTC
The problem appears to be caused by some change in quoting behaviour
of ocamlfine, ocamlc, bash, make or cppo.  The quoting insanity is
caused by cppo requiring defines on the command line to use quoted
spaces, single quotes and double quotes:

https://github.com/ocaml-community/cppo/issues/22

From cppo --help:

Usage: cppo [OPTIONS] [FILE1 [FILE2 ...]]
Options:
  -D DEF
          Equivalent of interpreting \'#define DEF\' before processing the
          input, e.g. `cppo -D \'VERSION "1.2.3"\'` (no equal sign)

In dev-ml/extlib it creates a command line for the -D DEF in the ocaml file
src/configure.ml, which needs to be quoted.  Then this is parsed to bash in
src/Makfile $(shell ocaml configure.ml -compile-args), then this is passed
to ocamlfind, which calls ocamlc -pp "quoted_string"

I can patch out this quoted insanity stuff to instead use:

xargs -a file command

However I need to somehow break the circular dependency loop on trying
to build dev-ml/cppo which has switched to the dune build system.
In the ml-overlay the jbuilder/dune eclass is using opam for the
install in the (not yet committed) jbuilder/dune eclass:

dune.eclass
-> opam.eclass for src_install
-> dev-ml/opam
-> dev-ml/opam-client
-> dev-ml/opam-solver
-> dev-ml/cudf
-> dev-ml/extlib
-> dev-ml/cppo
-> dune.eclass

So I tried to break this circular dependency loop by patching out extlib
to use (the not yet committed) dev-util/gpp program, then dev-ml/extlib
builds, but its broken, the tests fail and trying to build its reverse
dependency dev-ml/cudf fails:

File "cudf_types.ml", line 1:
Error: The files /usr/lib64/ocaml/extlib/extLib.cmi
       and /usr/lib64/ocaml/extlib/extList.cmi make inconsistent assumptions
       over interface ExtList

So I'm still working on trying to fix this.
Comment 6 Mark Wright gentoo-dev 2020-01-16 12:07:55 UTC
Browsing through the opam source trying to figure out how to build and
install opam without opam, I found a patch to extlib, which looks like
it is to fix this extlib problem.  I can try it later, my 10 year old
machine is very compiling stuff at the moment.

https://github.com/ocaml/opam/blob/master/src_ext/patches/extlib/extlib-1.7.5.patch
Comment 7 Mark Wright gentoo-dev 2020-01-16 13:03:07 UTC
The patch from the opam repo for extlib:

https://github.com/ocaml/opam/raw/master/src_ext/patches/extlib/extlib-1.7.5.patch

does not work either:

>>> Compiling source in /var/tmp/portage/dev-ml/extlib-1.7.6/work/ocaml-extlib-1.7.6 ...
make -j8 -j1 all 
ocamlfind ocamlc -pp "cppo -D OCAML 409 -D WITH_BYTES" -g -bin-annot -package bytes -i extBytes.ml > extBytes.mli
Fatal error: exception Sys_error("409: No such file or directory")
ESC[1mFile "extBytes.ml", line 1ESC[0m:
ESC[1;31mErrorESC[0m: Error while running external preprocessor
Command line: cppo -D OCAML 409 -D WITH_BYTES 'extBytes.ml' > /var/tmp/portage/dev-ml/extlib-1.7.6/temp/ocamlpp7d8175

make: *** [Makefile:58: extBytes.mli] Error 2
 * ERROR: dev-ml/extlib-1.7.6::x-portage failed (compile phase):

I'm guessing that there has been a change to the quoting in cppo, which might require bumping dev-ml/cppo, which is difficult as cppo changed its build system
to dune using opam as the installer which then has circular deps that I need
to break somehow.
Comment 8 Naohiro Aota gentoo-dev 2020-01-28 06:52:36 UTC
Created attachment 605876 [details, diff]
extlib-1.7.6-no-line-directive.patch

I could build dev-ml/extlib-1.7.6 with the attached patch (still got the similar inconsistent assumptions as below building the reverse dep dev-ml/cudf though). 

+ ocamlfind ocamlc -linkpkg -package extlib cudf_types.cmo cudf_conf.cmo cudf_type_parser.cmo cudf_type_lexer.cmo cudf_types_pp.cmo cudf.cmo cudf_checker.cmo cudf_822_parser.cmo cudf_822_lexer.cmo cudf_parser.cmo cudf_printer.cmo main_cudf_check.cmo -o main_cudf_check.byte
File "_none_", line 1:
Error: Files cudf_types.cmo and /usr/lib64/ocaml/extlib/extLib.cma(Std)
       make inconsistent assumptions over interface Std

The point of the patch is adding "-n" to cppo's argument to drop "# <line number>" directives from the preprocessed output.

It looks like ocaml-4.09.0 (?) only accepting "# <line number> <file name>" directive but not accepting "# <line number>" directive.
For example, I tried:

$ cat foo.ml
let () = print_endline "Hello, Debug World!"
$ ocamlbuild foo.native
Finished, 4 targets (4 cached) in 00:00:00.

# No directive. This must be OK.

$ cat foo.ml
# 1 "foo.ml"
let () = print_endline "Hello, Debug World!"
$ ocamlbuild foo.native
Finished, 4 targets (4 cached) in 00:00:00.

# With line directive with a file name. Still, it's fine.

$ cat foo.ml
# 1
let () = print_endline "Hello, Debug World!"
$ ocamlbuild foo.native
+ ocamldep.opt -modules foo.ml > foo.ml.depends
File "foo.ml", line 1, characters 2-3:
1 | # 1
      ^
Error: Syntax error
Command exited with code 2.
Compilation unsuccessful after building 1 target (0 cached) in 00:00:00.

# Without file name, ocaml failed to parse the line.

I cannot find a relevant upstream bug in ocaml's issue tracker.
Comment 9 Naohiro Aota gentoo-dev 2020-01-29 04:16:01 UTC
I just noticed OCaml 4.07 banned "# <line number>" directive [1]. And, cppo 1.6.1 has the support for it [2]. So, dev-ml/cppo-1.5.0 should have RDEPEND="!>=dev-lang/ocaml-4.07" first...

[1] https://github.com/ocaml/ocaml/pull/931
[2] https://github.com/ocaml-community/cppo/blob/master/Changes#L17

And, anyway, we need to break the circular dependency.
Comment 10 Reva Denis 2020-05-11 10:50:57 UTC
What is current status of that issue?
Comment 11 Danny 2020-05-24 08:41:18 UTC
I am also curious about it. I am unable to install opam due to this bug :-(
Comment 12 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-08-21 19:08:41 UTC
I'm really starting to think we should just last rite ocaml and stop pretending that we can maintain a matching working version of even the base set of packages.
Comment 13 Reva Denis 2020-08-21 19:11:28 UTC
@Mgorny So do I. 
Also I might say that about Lua.
Comment 14 Larry the Git Cow gentoo-dev 2020-10-13 12:54:13 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=ee112288e79ef448ea6ca90381e5829b258fa521

commit ee112288e79ef448ea6ca90381e5829b258fa521
Author:     Mark Wright <gienah@gentoo.org>
AuthorDate: 2020-10-13 05:24:59 +0000
Commit:     Mark Wright <gienah@gentoo.org>
CommitDate: 2020-10-13 12:53:25 +0000

    dev-ml/extlib: Bump to 1.7.7, try to fix bug 704146
    
    Try to fix bug 704146, thanks to mgorny for reporting.
    
    The problem appears to be caused by some change in quoting behaviour
    of ocamlfind, ocamlc, bash, make or cppo.  The quoting insanity is
    caused by cppo requiring defines on the command line to use quoted
    spaces, single quotes and double quotes:
    https://github.com/ocaml-community/cppo/issues/22
    
    Bug: https://bugs.gentoo.org/show_bug.cgi?id=704146
    Package-Manager: Portage-3.0.8, Repoman-3.0.1
    Signed-off-by: Mark Wright <gienah@gentoo.org>

 dev-ml/extlib/Manifest                        |  1 +
 dev-ml/extlib/extlib-1.7.7.ebuild             | 59 +++++++++++++++++++++++++++
 dev-ml/extlib/files/extlib-1.7.7-no-git.patch |  9 ++++
 dev-ml/extlib/metadata.xml                    |  5 ++-
 4 files changed, 73 insertions(+), 1 deletion(-)
Comment 15 Michael Haubenwallner gentoo-dev 2020-10-22 08:32:38 UTC
(came here because upgrading my stable amd64 workstation - after a maybe too long time - fails at extlib-1.7.1)

dev-ml/extlib-1.7.7 still breaks with the same error here.
Comment 16 Sam James archtester gentoo-dev Security 2020-10-22 11:19:28 UTC
(In reply to Michael Haubenwallner from comment #15)
> (came here because upgrading my stable amd64 workstation - after a maybe too
> long time - fails at extlib-1.7.1)
> 
> dev-ml/extlib-1.7.7 still breaks with the same error here.

Could you tell us which cppo, ocamlfind, ocamlc are installed?
Comment 17 Michael Haubenwallner gentoo-dev 2020-10-22 11:24:13 UTC
The packages providing these are:
 dev-lang/ocaml-4.09.0
 dev-ml/cppo-1.4.1
 dev-ml/findlib-1.8.1
They were updated yesterday as part of "emerge -uDN @world", while the dev-ml/extlib failure was today.
Comment 18 Sam James archtester gentoo-dev Security 2020-10-22 11:29:47 UTC
(In reply to Michael Haubenwallner from comment #17)
> The packages providing these are:
>  dev-lang/ocaml-4.09.0
>  dev-ml/cppo-1.4.1
>  dev-ml/findlib-1.8.1
> They were updated yesterday as part of "emerge -uDN @world", while the
> dev-ml/extlib failure was today.

Fancy trying dev-ml/cppo-1.6.6 and possibly dev-ml/findlib-1.8.1-r1? Ideally, do cppo first because it's more likely to fix it.
Comment 19 Michael Haubenwallner gentoo-dev 2020-10-22 11:42:47 UTC
While I've no idea why dev-ml/extlib actually was in my world file,
using (unstable) cppo-1.6.6 indeed allows to merge (unstable) extlib-1.7.7 (but not stable extlib-1.7.1).
Comment 20 Sam James archtester gentoo-dev Security 2020-10-22 12:02:53 UTC
(In reply to Michael Haubenwallner from comment #19)
> While I've no idea why dev-ml/extlib actually was in my world file,
> using (unstable) cppo-1.6.6 indeed allows to merge (unstable) extlib-1.7.7
> (but not stable extlib-1.7.1).

Excellent, thanks haubi for getting back so quickly. I'll use this to influence needed stabilisations. It might be a little bit because we're going to have to do a huge number of them for _other_ bugs too, but I'll poke at how it works out.
Comment 21 Larry the Git Cow gentoo-dev 2020-10-22 13:32:31 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=9b5152cc5afeb696ec55aac8b6c0ca1b4934f99d

commit 9b5152cc5afeb696ec55aac8b6c0ca1b4934f99d
Author:     Sam James <sam@gentoo.org>
AuthorDate: 2020-10-22 13:32:23 +0000
Commit:     Sam James <sam@gentoo.org>
CommitDate: 2020-10-22 13:32:28 +0000

    dev-ml/extlib: add lower bound on cppo
    
    Bug: https://bugs.gentoo.org/704146
    Package-Manager: Portage-3.0.8, Repoman-3.0.2
    Signed-off-by: Sam James <sam@gentoo.org>

 dev-ml/extlib/extlib-1.7.7.ebuild | 13 ++++++-------
 1 file changed, 6 insertions(+), 7 deletions(-)