I have written an eclass for ebuilds that offer libraries for the GHC (Glasgow Haskell Compiler). The idea is that the eclass handles administration stuff such as * detecting the library path * registering packages with the ghc-pkg command * unregistering packages upon uninstallation Currently, I have rewritten a number of ebuilds to test the eclass, and they seem to work fine: * c2hs * WASH * wxhaskell I have not yet rewritten gtk2hs, but I will try to do so. With the eclass in place, all ebuild maintainers should try to get the package to use a *local* package.conf file, which should be named $(get-localpkgconf) and placed in ${S}. If required, an empty local file can be generated in place by calling ghc-setup-pkg. During installation, the file will be moved to its final location by calling ghc-install-pkg. That's it. All the rest is handled by the eclass. Please provide comments/feedback, and if possible, test. I won't have much time before next Monday, but I will try to write an email to gentoo-dev soon, asking for approval of the eclass. Cheers, ks
Created attachment 42775 [details] haskell.eclass Should probably be renamed to ghc-package.eclass, reserving haskell.eclass for something of wider scope (like haskell-cabal support).
Created attachment 42777 [details] c2hs-0.13.1-r1.ebuild (using the eclass)
Created attachment 42778 [details] WASH-2.0.5-r1.ebuild (using the eclass)
Created attachment 42779 [details] wxhaskell-0.8-r1.ebuild (using the eclass)
Copyright header is wrong. It's 2004 and we're no longer Gentoo Technologies, Inc:) If the ghc-listpkg should ever return a package with spaces in its name, the functions calling won't work, as they are currently implemented. Nasty (ba)shism:(
I think ghc-register-pkg should be conditional on $(ghclocalpkg) existing. For my buddha ebuild, it installs things in versioned dirs (ie it wants $(ghc-libdir)) but does not register any packages (the buddha script calls ghc specifying the buddha.pkg.conf file).
re Karl's comments: The copyright header was fixed already in my local version. About spaces in package names: I think I'll worry about this issue once someone decides to have such a package. re Duncan's comment: I agree, checking for the presence of the file is definitely a good idea.
I've just committed the eclass to CVS as ghc-package.eclass. I will soon start adapting ebuilds to make use of it. ks
Created attachment 43232 [details] buddha-1.2.ebuild new ebuild for buddha. It gives an non-fatal error message during ghc-register-pkg because there is no package to register.
Oops, that buddha-1.2.ebuild says Gentoo Technologies, Inc. rather than Gentoo Foundation
Created attachment 43234 [details] gtk2hs-0.9.6-r1.ebuild updated version of gtk2hs-0.9.6.ebuild to use new ghc-package.eclass also adds a 'doc' USE flag. (Note my haddock segfaulted building the docs, I re-emerged it and it worked fine - something we may have to watch for. It was haddock-0.8-r2.ebuild compiled by ghc-6.2.1.ebuild)
Ok, I've added buddha and made it the first package in the tree to use ghc-package.eclass. BTW, with the committed version of the eclass, no warning should occur. Thanks, ks
I've got a problem with gtk2hs using the eclass: registering the package generates the .o ghci files from the .a lib. However this happens after merging the package so the .o file is orphaned. We had better not do the ghc-pkg --auto-ghci-libs during package post install. Perhaps we should provide a command to convert the .a to the .o in the right way. A more general issue is that perhaps it would be better, or as an alternative to the $(ghc-localpkg) to provide a way to add given .conf files to the packages to be registered. So some builds put their stuff into an 'installed' package file, eg wash, while ones like c2hs, gtk2hs (and buddha if it registered its package) produce 'uninstalled' .conf files. For c2hs you've we just slap [] around the file and it is fine. For gtk2hs you can see I'm using ghc-pkg to build an 'isntalled' package file. So perhaps a better way is for the ebuild to specify a list of .conf files which will get registered.
Ok, I didn't think about it, but I agree that it's bad. I therefore removed --auto-ghci-libs from the eclass, which makes it the individual ebuilds responsibility to ensure that the ghci libraries are present. If you want to have a function in the eclass to assist, feel free to submit a patch. Regarding the other problem, I'm not sure I agree. I'm inclined to say that a package is broken if it does not offer using a local package config file. In the case of gtk2hs, you should probably submit a patch upstream so that the ghc-pkg calls do not have to occur within the ebuild. I haven't looked at the .tar.gz, but possibly an alternative would be to sed replace "ghc-pkg" to "ghc-pkg -f ${S}/$(ghc-localpkgconf)" in the Makefiles, which would also remove the need for the manual calls to ghc-pkg in the ebuild. I do not want to make the eclass unnecessarily complicated, but if you can convince me that the additional complexity is justified, we can do it ... ks
> Regarding the other problem, I'm not sure I agree. I'm inclined > to say that a package is broken if it does not offer using a local > package config file. As it happens gtk2hs can use a local pkg file but I don't think it'll work for what we want because it contains a dummy package and build tree paths. But I don't think it's unreasonable for a package to produce a collection of .conf files which it then registeres when you make install. gtk2hs has make install-without-pkg which is what I use. Instead of using ghc-pkg to add these to the $(ghc-localpkgconf) I could do what you do in the c2hs ebuild: echo '[' > ${S}/$(ghc-localpkgconf) cat ${D}/$(ghc-libdir)/gtk2hs/gtk2/gtk2.conf >> ${S}/$(ghc-localpkgconf) echo ',' >> ${S}/$(ghc-localpkgconf) cat ${D}/$(ghc-libdir)/gtk2hs/mogul/mogul.conf >> ${S}/$(ghc-localpkgconf) etc... Would that be better?
First gtk2hs: Ok, if there are incompatible paths in the package configuration file upon use of a local file, I can see why it goes wrong. Common situation: The [ , , ... ] solution might be a simple way of having one configuration file per package in the end (something I somehow would prefer, because it makes it easier for possible external tools to analyze quickly from which package a certain file stems). This could get eclass support, though. Atm, there are the two functions ghc-setup-pkg ghc-install-pkg The function ghc-setup-pkg does nothing but to create ${S}/$(ghc-localpkgconf) with content []. If we'd make ghc-setup-pkg take an arbitrary number of package configuration files as argument, with the idea that all of these are uninstalled, singular package descriptions, ghc-setup-pkg could concatenate them into ${S}/$(ghc-localpkgconf), separated by commas. The relevant part of the gtk2hs ebuild would then become something like ghc-setup-pkg /gtk2hs/gtk2/gtk2.conf /gtk2hs/mogul/mogul.conf ... which is almost what you wanted originally, only with slightly different semantics. Acceptable or bad idea? Cheers, ks
That sounds like a good idea. It allows both styles of build systems to work without too much fuss. I'm currently testing this function which could be added to the eclass: # make a ghci foo.o file from a libfoo.a file ghc-makeghcilib() { local outfile outfile=`echo $1 | sed 's:lib\(.*\)\.a$:\1.o:'` ld --relocatable --discard-all --output=$outfile $1 }
Created attachment 43257 [details] gtk2hs-0.9.6-r1.ebuild updated to not use ghc-pkg just for combining several .conf files into $(ghc-localpkgconf). Also uses new eclass function (updated from previous definition): # make a ghci foo.o file from a libfoo.a file ghc-makeghcilib() { local outfile outfile=`dirname $1`/`basename $1 | sed 's:^lib\(.*\)\.a$:\1.o:'` ld --relocatable --discard-all --output=$outfile $1 }
Last version honnestly! This is what it should be: # make a ghci foo.o file from a libfoo.a file ghc-makeghcilib() { local outfile outfile=`dirname $1`/`basename $1 | sed 's:^lib\(.*\)\.a$:\1.o:'` ld --relocatable --discard-all --output=$outfile --whole-archive $1 }
I've committed a version of your gtk2hs ebuild using ghc-package.eclass. Please test, because I've made a couple of modifications. ks
All those changes look fine. I've tried it and it seems to install and register fine. BTW why the extra \? in sed 's:^lib\?\(.*\)\.a$:\1.o:' One thing I forgot to change before attaching the ebuild here was to use optimisations (I'd been testing without optimisations as it takes much longer to build otherwise) so that just involves changing: --with-hcflags="-H180m" \ to --with-hcflags="-O -H180m" \ in the econf bit
About the \? ... I'm not sure, I guess I wanted to make it more unlikely that the function actually destroys the .a file if it doesn't conform to the correct naming convention. Anyway, it should make no difference on correct function calls. I'll try to remember adding the -O tomorrow, tonight is HCAR time ... ks
These all emerge ok and seem to work (though I've not done much testing), except for gtk2hs. Everytime I try to emerge gtk2hs, X ends up being killed, as far as I can tell because I run out of RAM (this is on a machine with 256MB of ram, and 1GB of swap, and only running fluxbox and an aterm). The offending program seems to be c2hs (according to top), regardless of whether or not I have the c2hs package emerged or not. Is anyone else experiencing a lot of memory being used when compiling gtk2hs? Here's my emerge info: Portage 2.0.51-r3 (default-linux/x86/2004.2/gcc34/2.6, gcc-3.4.2, glibc-2.3.4.20041102-r0, 2.6.9-rc2-mm1 i686) ================================================================= System uname: 2.6.9-rc2-mm1 i686 Mobile Intel(R) Pentium(R) 4 - M CPU 1.80GHz Gentoo Base System version 1.6.6 Autoconf: sys-devel/autoconf-2.59-r5 Automake: sys-devel/automake-1.8.5-r1 Binutils: sys-devel/binutils-2.15.92.0.2-r1 Headers: sys-kernel/linux26-headers-2.6.8.1-r1 Libtools: sys-devel/libtool-1.5.2-r7 ACCEPT_KEYWORDS="x86 ~x86" AUTOCLEAN="yes" CFLAGS="-O2 -march=pentium4 -pipe" CHOST="i686-pc-linux-gnu" COMPILER="" CONFIG_PROTECT="/etc /usr/X11R6/lib/X11/xkb /usr/kde/2/share/config /usr/kde/3.3/env /usr/kde/3.3/share/config /usr/kde/3.3/shutdown /usr/kde/3/share/config /usr/lib/mozilla/defaults/pref /usr/share/config /usr/share/texmf/dvipdfm/config/ /usr/share/texmf/dvips/config/ /usr/share/texmf/tex/generic/config/ /usr/share/texmf/tex/platex/config/ /usr/share/texmf/xdvi/ /var/qmail/control" CONFIG_PROTECT_MASK="/etc/gconf /etc/terminfo /etc/env.d" CXXFLAGS="-O2 -march=pentium4 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="autoaddcvs ccache distlocks sandbox sfperms" GENTOO_MIRRORS="http://194.117.158.28 http://194.117.158.29 ftp://194.117.158.28/mirrors/gentoo" MAKEOPTS="-j2" PKGDIR="/usr/portage/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="X aalib alsa apm atm avi berkdb bitmap-fonts cdparanoia cdr crypt cups directfb divx4linux dvd encode f77 fam fbcon flac foomaticdb fortran gdbm ggi gif gpm gstreamer gtk gtk+ gtk2 imagemagick imlib java jpeg ldap libg++ libwww live lzo mad mikmod mmx mmx2 motif mozilla mpeg mysql ncurses network nls oggvorbis pam pdflib perl png python quicktime readline sdl slang spell sqlite sse sse2 ssl svga tcltk tcpd tetex theora tiff truetype usb v4l v4l2 x86 xine xml2 xmms xv xvid zlib"
It's a known issue with gtk2hs. Building gtk2hs uses a preprocessor tool called c2hs which uses a huge amount of memory. I reckon it needs about 400Mb of RAM. I'm a gtk2hs developer and I'm actually working on this issue at the moment. There will not be a quick fix however, you'll have to wait for the next release of gtk2hs. Perhaps I should put a check in the ebuild to make it bail out if the machine has less than 512Mb of RAM.
Andres, what's the most sensible way of checking the emount of system memory in an ebuild? A couple people have hit this issue now.
Created attachment 44121 [details] c2hs-0.13.4.ebuild updated c2hs ebuild, updated to version 0.13.4 make sure ghc package actually gets registered ghc package missing haskell98 dependency hack no longer needed as it was fixed in 0.13.4 new hack to stop c2hs from using versioned dirs, as we're already doing that ourselves.
Created attachment 44122 [details] buddha-1.2-r1.ebuild Just a small change to where interface files get installed. ghc .hi files get installed in a dir under ghc-libdir. buddha .i files get put under /usr/share/buddha/ifaces/ (previously they were mixed in with the .hi files)
One way to test whether or not someone has enough ram would be along the lines of mem=`cat /proc/meminfo | grep MemTotal | sed -e 's/\(.*: *\)\(.*\)\( .*\)/\2/'` if [[ $(mem)/1024 < 512 ]] ; then die "need more ram" ; fi Though maybe it's better to do a similar thing using free, in case the user isn't using /proc (or maybe there's an even better way that I don't know about, but no-one else has suggested anything here)
eclipse-sdk_3.1_pre2 uses the following, very similar, construct: function get-memory-total() { cat /proc/meminfo | grep MemTotal | sed -r "s/[^0-9]*([0-9]+).*/\1/" } function check-ram() { local mem=$(get-memory-total) [ $(get-memory-total) -lt 775000 ] && ( echo ewarn "To build Eclipse, at least 768MB of RAM is recommended." ewarn "Your machine has less RAM. Continuing anyway." echo ) } I don't like it. I don't think it is the job of individual ebuilds to check the available memory. If at all, we should do something similar and print a warning, but no more. ks
People often miss such warnings though. I'd much rather return to my computer after I've told it to emerge something(s) and find that the emerge failed rather than find that X got shutdown because I ran out of memory (which will stop the emerge anyway). Perhaps a combination of this with something like the BREAKME variable being used in xorg ebuilds [1] so that the emerge fails if the user doesn't have enough memory, unless this variable is set. Still not a very good solution, but if people miss the warning and then find that X has been killed they'll probably think that it's a bug in X, when clearly it's not. I agree that ebulids checking whether or not the user has enough memory is a bad thing, but I think that it's the lesser of 2 evils What do you think? This makes it easy for the user to try emerging anyway, and makes sure that they don't be making any bug reports about X (though maybe that needs to be explained in the warning) On a different subject, is there any reason why the other packages that depend on ghc don't have ebuilds using the eclass? If not then I can probably write new ebuilds for them over the weekend. [1] from xorg-x11-6.8.0-r2.ebuild if [ -z "${BREAKME}" ]; then die "Set the BREAKME variable to emerge this. It's in development. Stop using it." fi
I'm still not convinced, but more so than before ;) Are there so many ebuilds that need to be ghc-packages and aren't yet? I know I haven't added wxhaskell and WASH yet, but which others did you have in mind? ks
Alex and Haddock both use PATH="${PATH}:/opt/ghc/bin", other than that I didn't notice anything else when I just gave the ebuilds a quick look over. Is it worth inheriting the eclass just for that?
I'd vote no. I think this ugly hack can be removed anyway, but I've been too lazy to test until now. If you have spare time during the weekend, you could write a fresh ebuild for some Haskell library you want in portage. I've added the wash ebuild today, and renamed WASH to wash. That was a bit of a mess, but I hope everything works alright now. ks
Andres, I'll implement whichever mem check method you think is more appropriate / tastefull. As the Gentoo Haskell Tsar you get the privilege of deciding :-). I think it needs something doing however since quite a few people have hit this issue.
Ok. I'd suggest writing a mail to the gentoo-dev mailing list in this case. Point out the path taken by the eclipse-sdk ebuild as a possibility, and possibly the BUILD_ANYWAY variable ... Perhaps we've overlooked other possibilities -- I ran a few more greps on /usr/portage, but couldn't find anything. I'd write the mail myself (and will do), but probably won't have time for it before Monday. ks
Created attachment 44443 [details, diff] gtk2hs-0.9.6-r1.ebuild.patch Following discussion on the gentoo-dev mailing list an eclass "check-reqs" was added to the portage tree. This patch modifies the gtk2hs-0.9.6-r1.ebuild to use this eclass to implement a memory check. I have set the memory requirement at 350Mb.
Duncan, I finally applied your patch. ks
I think this can be considered fixed. The eclass is in portage for a long time now, and seems to work nicely.
I meant to close the bug.
