Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!

Bug 565364

Summary: app-text/pandoc-bin - binary package of app-text/pandoc
Product: Gentoo Linux Reporter: Jan Chren (rindeal) <dev.rindeal+gentoo>
Component: New packagesAssignee: Default Assignee for New Packages <maintainer-wanted>
Status: UNCONFIRMED ---    
Severity: normal CC: haskell
Priority: Normal    
Version: unspecified   
Hardware: All   
OS: Linux   
Whiteboard:
Package list:
Runtime testing required: ---

Description Jan Chren (rindeal) 2015-11-10 13:14:05 UTC
As many of you certainly know, trying to compile pandoc is a tiresome job. There are many compilation issues, it pulls more than 100 haskell packges in, all together occupying several hundreds of disk space and several thousands of inodes. It's also the only package written in haskell I'm interested in.

So it comes to my mind naturally that we need this thing precompiled by some wizard (or extracted from the official .deb package) and included in the portage tree.
Comment 1 Sergei Trofimovich gentoo-dev 2015-11-10 22:06:02 UTC
(In reply to rindeal from comment #0)
> As many of you certainly know, trying to compile pandoc is a tiresome job.
> There are many compilation issues, it pulls more than 100 haskell packges
> in, all together occupying several hundreds of disk space and several
> thousands of inodes. It's also the only package written in haskell I'm
> interested in.
> 
> So it comes to my mind naturally that we need this thing precompiled by some
> wizard (or extracted from the official .deb package) and included in the
> portage tree.

You've listed many things here. What is a real problem for you?

I can help mitigating some of them:

- ghc build time: 'USE=binary ghc' disables building ghc completely

- pandoc (+depends) build time:
  whole pandoc (with it's 100 depends) builds in (stable ghc-7.8.4 is already installed):
  $ time FEATURES=-test emerge -1 pandoc -j5
  >>> Installing (100 of 100) app-text/pandoc-1.15.0.6::gentoo
  >>> Jobs: 100 of 100 complete                       Load avg: 2.31, 3.97, 2.82
  real    10m39.121s
  user    33m1.502s
  sys     9m29.643s

  Is it too much?

   HCFLAGS=-O0 for some slow packages can make it faster.

- "many compilation issues": if pandoc does not build it's a bug and must be fixed.
- disk space: how space-constrained your machine is? ghc-7.10.2 says it's about 700MB, pandoc itself is about 70MB

The problem could also be mitigated by BINHOST: https://wiki.gentoo.org/wiki/Binary_package_guide .
For haskell packages it requires a bit more care than typical C application.

For a binary-only package I don't think there is a sane way to deliver user
binary-only version of pandoc in such a way that it can be used with
other haskell depends like pandoc-citeproc.

I can help by advicing how exactly various problems can be worked around
if you really want to go that path and create binary-only package but I
personally think it's not worth the result:

- pandoc is a dynamically linked binary if you build it with ghc-7.10+ on gentoo:
    $ ldd /usr/bin/pandoc | wc -l
    102
  Tracking updates for these deps will not be fun.
- pandoc is also a haskell library. Do you want it to work (say, by loading it in 'ghci -package pandoc')
  if you install binary-only version?

For one-time uses http://pandoc.org/try/ might be enough.
Comment 2 Jan Chren (rindeal) 2016-04-04 13:56:34 UTC
First of all thank you for your comprehensive reply.

Also let's convert `ul` to `ol`:

1. binary ghc decreases the number of issues, but a lot still remains

2. pandoc build time is not an issue

3. "many compilation issues" - pandoc itself is not such a problem as its 100+ dependencies, because these fail to update almost everytime while asking for `haskell-updater` to be run

An example from the last time I tried:
```
Dependency transformers -any: using transformers-0.3.0.0
setup: The following installed packages are broken because other packages they
depend on are missing. These broken packages must be rebuilt before they can
be used.
package cookie-0.4.0.1 is broken due to missing package
blaze-builder-0.3.3.2-3234ac3c9598f6039d4acc6a162a1692
 * ghc-pkg check: 'checking for other broken packages:'
WARNING: cache is out of date: /usr/lib64/ghc-7.8.4/package.conf.d/package.cache
Use 'ghc-pkg recache' to fix.
There are problems in package cookie-0.4.0.1:
  Warning: haddock-interfaces: /usr/share/doc/cookie-0.4.0.1/html/cookie.haddock doesn't exist or isn't a file
  Warning: haddock-html: /usr/share/doc/cookie-0.4.0.1/html doesn't exist or isn't a directory
  dependency "blaze-builder-0.3.3.2-3234ac3c9598f6039d4acc6a162a1692" doesn't exist
There are problems in package pandoc-1.15.0.6:
  Warning: haddock-interfaces: /usr/share/doc/pandoc-1.15.0.6/html/pandoc.haddock doesn't exist or isn't a file
  Warning: haddock-html: /usr/share/doc/pandoc-1.15.0.6/html doesn't exist or isn't a directory
  dependency "blaze-markup-0.5.2.1-b1c2791805aab4079ef40ccff880a48a" doesn't exist
  dependency "http-types-0.8.6-59bce2a4f12a076e1584cb399354e41a" doesn't exist
There are problems in package http-client-0.4.18.1:
  Warning: haddock-interfaces: /usr/share/doc/http-client-0.4.18.1/html/http-client.haddock doesn't exist or isn't a file
  Warning: haddock-html: /usr/share/doc/http-client-0.4.18.1/html doesn't exist or isn't a directory
  dependency "blaze-builder-0.3.3.2-3234ac3c9598f6039d4acc6a162a1692" doesn't exist
  dependency "http-types-0.8.6-59bce2a4f12a076e1584cb399354e41a" doesn't exist
  dependency "streaming-commons-0.1.12.1-e827fbf90db79f52f6afd4fd82c90dfa" doesn't exist
There are problems in package blaze-html-0.6.1.2:
  Warning: haddock-interfaces: /usr/share/doc/blaze-html-0.6.1.2/html/blaze-html.haddock doesn't exist or isn't a file
  Warning: haddock-html: /usr/share/doc/blaze-html-0.6.1.2/html doesn't exist or isn't a directory
WARNING: cache is out of date: /usr/lib64/ghc-7.8.4/package.conf.d/package.cache
Use 'ghc-pkg recache' to fix.
 * Detected broken packages: blaze-html-0.6.1.2 cookie-0.4.0.1 highlighting-kate-0.6 http-client-0.4.18.1 http-client-tls-0.2.2 pandoc-1.15.0.6
 * ERROR: dev-haskell/http-client-0.4.18.1::gentoo failed (configure phase):
 *   //==-- Please, run 'haskell-updater' to fix broken packages --==//
 *
 * Call stack:
 *     ebuild.sh, line  133:  Called src_configure
 *   environment, line 2973:  Called haskell-cabal_src_configure '--flags=network-uri'
 *   environment, line 2097:  Called cabal-configure '--flags=network-uri'
 *   environment, line  707:  Called cabal-show-brokens-and-die 'setup configure failed'
 *   environment, line  803:  Called cabal-show-brokens
 *   environment, line  799:  Called cabal-die-if-nonempty 'broken' 'blaze-html-0.6.1.2' 'cookie-0.4.0.1' 'highlighting-kate-0.6' 'http-client-0.4.18.1' 'http-client-tls-0.2.2' 'pandoc-1.15.0.6'
 *   environment, line  729:  Called die
 * The specific snippet of code:
 *       die "//==-- Please, run 'haskell-updater' to fix ${breakage_type} packages --==//"
```

4. disk-space: I do have a disk space, still I consider nearly 800MB (with thousands of inodes) to be a huge number for a FOSS CLI doc converter

5. dll: deb package on github contains a statically linked binary
```
$ ldd pandoc
        linux-vdso.so.1 (0x00007ffe84ed4000)
        libz.so.1 => /lib64/libz.so.1 (0x0000003395c00000)
        librt.so.1 => /lib64/librt.so.1 (0x0000003396c00000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00000033b4200000)
        libdl.so.2 => /lib64/libdl.so.2 (0x0000003395800000)
        libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00000033ad200000)
        libm.so.6 => /lib64/libm.so.6 (0x0000003395000000)
        libgcc_s.so.1 => /usr/lib/gcc/x86_64-pc-linux-gnu/5.3.0/libgcc_s.so.1 (0x0000003396000000)
        libc.so.6 => /lib64/libc.so.6 (0x0000003394c00000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x0000003395400000)
        /lib64/ld-linux-x86-64.so.2 (0x0000003394800000)
```

6. haskell-library: most people are definitely using it only as a CLI tool not as a library, thus pandoc-bin would cover this type of use and certainly wouldn't try to replace the source based package for more advanced uses like this

7. I'm aware of http://pandoc.org/try/, but this is just a very length-limited demo
Comment 3 Frédéric Pierret 2020-09-08 08:12:37 UTC
As I need to have all the pandoc Haskel dependencies and pandoc to be built fast for CI, I've made a pandoc-bin (https://github.com/fepitre/qubes-gentoo/blob/master/app-text/pandoc-bin/.pandoc-bin.ebuild.0). If the pandoc maintainers are interested, I can open a PR to provide the bindist flag into app-text/pandoc directly. That would help in workaround issue like https://bugs.gentoo.org/738802.
Comment 4 Sergei Trofimovich gentoo-dev 2020-09-08 08:25:28 UTC
(In reply to Frédéric Pierret from comment #3)
> As I need to have all the pandoc Haskel dependencies and pandoc to be built
> fast for CI, I've made a pandoc-bin
> (https://github.com/fepitre/qubes-gentoo/blob/master/app-text/pandoc-bin/.
> pandoc-bin.ebuild.0). If the pandoc maintainers are interested, I can open a
> PR to provide the bindist flag into app-text/pandoc directly.

haskell@ would prefer not to have prebuilt binaries.

> That would help in workaround issue like https://bugs.gentoo.org/738802.

By not installing any haskell libraries? I don't see how it would help otherwise.
Comment 5 Frédéric Pierret 2020-09-08 08:50:44 UTC
> > That would help in workaround issue like https://bugs.gentoo.org/738802.
> 
> By not installing any haskell libraries? I don't see how it would help
> otherwise.

I was emerging pandoc from binpkg as you know on the other issue (BTW, thank you for your latest comment enlightening the issue with pandoc). As there is currently no other alternative up to rebuild all from sources, for CI jobs this is not very acceptable due to limited resources from my side. So this was my "workaround" to install pandoc within a whole binpkg Gentoo pipeline install.
Comment 6 mimosinnet 2021-03-10 23:43:01 UTC
(In reply to Frédéric Pierret from comment #3)
> As I need to have all the pandoc Haskel dependencies and pandoc to be built
> fast for CI, I've made a pandoc-bin

Thanks a lot! I have had issues installing and updating pandoc and, as Jan Chren pointed out, pandoc was the only Haskell package I used. Your ebuild installed perfectly. Cheers!