Hi, I put together an ebuild for geant-4.9.1 over the weekend. Geant[3,4] is a widely used physics simulation code from cern; 3.* is in the tree, 4.* is not, which is bad (for some). Even debian users can have geant4 ... This ebuild seems to work on x86 using the g** compilers. There are also two patches needed to make the build work. There is a DEPEND on sci-physics/clhep from the science overlay, and an optional RDEPEND on sci-geant4/data (see bug report for new ebuild elsewhere). Since this is my first ebuild for the public and the geant4 install is extra nasty I would love to have suggestions for improvements, flames ... I am particularly interested in suggestions concerning 1) Ebuild * no checking of CFLAGS and MAKEOPTS: Is it too bad? I have no idea ... * relying solely on Geant4 ./Configure to figure out platform/compiler: Maybe somebody on a different platform or with a different compiler can tell me if the build is still working. * name: Now its a newer version of sci-physics/geant-3.*.* currently in the tree, maybe causing trouble. * License: Geant4 license not in /usr/portage/license; not sure about licenses for data files, but one could ask ... * USE: I have no idea how to switch some of them on by default ... 2) Geant4 infrastructure * do your projects compile and run successfully? * geant and geant4-data install lots of stuff like data files, config scripts, example apps into /usr/share/geant4. Is this the right place? Is it working for your use case? * even worse: libraries are installed into /usr/lib/geant4/<platform><compiler>/. This is how geant4 does it by default. No versions are recorded. No archives are installed. How much does this violate gentoo policy? Do you need the archives? How much would another solution break your application? I would really really (really) love to have this package in gentoo, maybe in the science overlay: geant4's build system is painful and scary, but portage can conveniently hide all that once and forever ;) b.
Created attachment 145230 [details] First shoot ebuild
Created attachment 145231 [details, diff] Patch 1 for files/
Created attachment 145232 [details, diff] Patch 2 for files/
Created attachment 145246 [details] Second try
Created attachment 145247 [details, diff] Patch 1 for files Remove pointers to tmp build files in /var/tmp/portage/... from geant build scripts
Ok, this first ebuild was pretty much crap. Now it has some reasonable defaults for the USE flags, and the dependencies can actually be meet with portage ;) The libraries are still installed into /usr/lib/geant4/<platform>-<compiler>, but after talking to some people around here, I am pretty sure that this should go away. Geant4 does this to enable compiling binaries for multiple platforms and keeping them around simultanously. You work on some machine in a network, your $HOME comes over e.g. NFS and you magically get the right binary and libraries via enviroment variable settings. I tried some stuff to remove this clutter (see ebuild), but it breaks some other stuff (G4SYSTEM related). Static libraries require some more patching of the geant4 build scripts, which could be avoided, since IMHO gentoo should not be primarly targeting batch queue systems in big research centers (they run some sort of SL, RHLE or antique Debian anyway). But that's politics. I would be happy if I could develop on my machine and still be able to deploy on a production system with the same Makefile. The USE flags I still did not checked are marked in the ebuild. Concering patching: the patches of http://wiki.debian.org/DebianScienceGeant4 approach the mental level of the geant4 build scripts and I do not know who would like to maintain such ... so hopefully over-patching can be avoided by moving stuff around in the end and updating environment variables. b.
Hi Benjamin, Thanks for sharing your efforts! Geant4 is quite different than it 3.21 predecessor, imho worth a whole new package sci-physics/geant4 Take a look at the http://devmanual.gentoo.org if you haven't already. 1) Comments to improve the ebuild: * USE flags: you can probably group some. Ex: opengl for all opengl*. Then it will depend on the gui: motif? ( opengl? ( virtual/motif ) ) * data use flag: since the geant4-data ebuild is straight forward, why not including its install inside this one? * use the qt4 eclass * can you use a trick for unpack for *.gtar.gz? * use doenvd for environment variables * instead of the complex scheme for the -D / MYOPTS I would rather define a simple bash function before the src_unpack and use it within the ./Configure so that you don't need the MYOPTS and all the if's. * all the mv "${D}"/usr/src/geant4 /usr/share/geant4 could use a sed in src_unpack * use spaces for dependencies around the (), ex: ( media-libs/openinventor ) * FEATURE does not belong here but in your make.conf * do not dodoc LICENSE, we will install it 2) Answers to more questions: > no checking of CFLAGS and MAKEOPTS leave it to the user, and that's also why it goes through a testing phase > relying solely on Geant4 ./Configure to figure out platform/compiler we have to trust them. If not, our users or devs will pick up the bugs. We can check the compiler with the toolchain-funcs eclass and tests like [[ $(tc-getCXX) = g++ ]] > data files, config scripts, example apps into /usr/share/geant4 docs and examples in /usr/share/doc/${PF}, see the devmanual and FHS. > could be avoided, since IMHO gentoo should not be primarly targeting batch > queue systems in big research centers (they run some sort of SL, RHLE or > antique Debian anyway). And have many users complaining about it ;-)
Created attachment 145527 [details] Include data files in ebuild Ok, I am very busy at the moment, but here's a dump of a not working version of the ebuild that incorporates the data ebuild. I also learned about doins -r, put things into somewhat LFS conforming places (yet not the geant libs) and did minor cleanups. I changed the section for use flag checking which was totally silly before. Unfortunately, the use flags I tested up to now are not working since the Geant ./Configure assumes some silly things about the layout of include/ and lib/ for the required libraries. This requires some further patching of ./Configure. When I have time again (in maybe a week) I will mainly be looking at making sure the required stuff for a build with optional use flags is found. Later: installing env stuff into /etc/env.d/ could be a bad idea, since Geant tends to extremely pollute the environment, so one could include a wrapper for env dependent commands like the one included in the debian package (maybe just steal it). http://wiki.debian.org/DebianScienceGeant4 So: This ebuild works for building without any display functionality. If you do play around, make sure the examples you use for testing are not broken. b.
Created attachment 147566 [details] geant-4.9.1.ebuild with ${D} removed from Some ${D}'s were used in helper script args (dodir, doins), and these caused the build to bomb on the QA check. I fixed these, and now it builds. See if it does what you intended... For example, I see that no "examples" appears in /usr/share/doc/geant-4.9.1 after the build, but I have not looked into it further. I also added ~amd64 to keywords, since that is the platform on which I built.
Created attachment 147577 [details] geant-4.9.1.ebuild with data, examples, and environments, log message fixed Fixed a few more things: 1) Changed "{PF}" to "${PF}" for environments destination (this had caused two different environments dirs to be created, one in a literal "{PF}" dir). 2) In doins -r, changed the absolute paths to relative (they were not doing anything before, so the optional environments and the examples were not getting installed). 3) The data dirs were being created in the top level "work" dir, and their install was not working. I assume this is because of the parameters passed to "Configure" (e.g. -D g4levelgammadata="${DATA}/PhotonEvaporation"). This should be fixed, so it installs somewhere else below the work dir, but for now, I just did "cd .." and changed the find command so this would do what you expect. 4) Changed log message to give correct path to env.sh file
Hi, I committed an experimental ebuild in the science overlay, where I rewrote a lot of stuff to make it more robust for dependencies. Upstream really needs to change the build system, this is really not friendly. The ebuild still needs work for: * g4py: install for a python use flag * momo: install with java deps * qt4: was too buggy, so it needs patching * work out env variables. Someone could give/describe a small procedure to test the monster?
Created attachment 149260 [details, diff] Patch against science-overlay geant r1007 Hi Joe, hi Sébastien, thank you both for looking into this, you really helped making gentoo and the world as a whole a better place. I wanted to draw you ascii flowers but you wouldn't have liked the result ... I tried to compile+merge the science version but I am still having trouble to get even motif working (which would be the minimum for any X stuff). Ebuild threw me to the command line of ./Configure somewhere at libXm include directory questions, and this is so bad that the USE flag should be disabled for now. As for a build system, I read somewhere they use something proprietary I had never heard of and I cannot recall now. Their forum "Problems installing etc" was and is always very popular and they seem to have no intention to ever release anything. Nice letterhead anybody? I am attaching a patch against r1007 that fixes some regressions. b.
Hi Benjamin, Could you specify how motif failed, i.e. openmotif version, etc...? It works fine here with openmotif-2.3* without intervention. Concerning your patch, there is the move.sh.SH thing, but I failed to see what was modified in the Configure script. Version newer than r1007 took care of tar ball names different than their expanded counterpart. A few remarks: - let's keep MAKEOPTS in the ebuild (with sed), it's gentoo specific - data names should be easy to maintain. we don't want to specify their names and versions in 3 different places in the ebuild. - why removing the shared libraries building? If we really want to not build both static and shared, then a use flag could be adequate. Note we can't specify them both at the same time otherwise it will build static libs with PIC, which we really want to avoid. I updated the ebuild in the overlay with the move.sh fix and a env variable fix. Best
Sébastien, below the output of an ebuild geant* compile. I just realized I do not have /usr/include/Xm, but usr/include/openmotif-2.3/Xm, but maybe I messed it up (since I cannot equery the package the dir belongs to). I've got openmotif-2.3.0-r1 installed. But e.g. ddd is linked against libXm[u] here just fine. Shared library and static library are also perfectly fine with me, but do we really expect people to run this monster build twice *by default*? I believe it should be an optional thing at least ... Now I also understand the data file magic you used; my change is really pointless. But do you mind keeping the data files in a separate data/ under /usr/share/geant? Or maybe even better: use geant4 for everything in share/ to avoid confusion on systems with geant3, too (if I understand the slotting used right). I apologize since I also cannot see what I did in the Configure patch, probably my automagic whitespace policy turned against me after midnight ... What do you have in mind for testing? We could just try if some trusted set of examples do compile and run to some extend -- I can put that together. That's for the install part only though. For everything else we need to rely on physics experiments ;) b. ### ebuild output snippet for compile ################################### If this variable is set, no UI sessions nor any UI libraries are built. This can be useful when running a pure batch job or in a user framework having its own UI system. Do you want to set this variable ? [n] G4UI_BUILD_XAW_SESSION G4UI_USE_XAW Specifies to include and use the XAW interfaces in the application to be built. The XAW (X11 Athena Widget set) extensions are required to activate and build this driver. [n] G4UI_BUILD_XM_SESSION G4UI_USE_XM Specifies to include and use the XM Motif based user interfaces. The XM Motif extensions are required to activate and build this driver. [y] You have selected to use the XM Motif based user interfaces. Specify the correct path where Xm is installed in your system. It was found in /usr/X11R6/include. Press [Enter] to set this path or type the correct one. You can set '-' (without quotation) to CANCEL the XM flag at all: [/usr/X11R6/include] You have selected to use XM Motif based user interfaces. But XM Motif was not found in /usr/X11R6/include. Please specify the correct path where Xm is installed in your system. You can set '-' (without quotation) to CANCEL the XM flag at all: PROMPT! ####### Please understand my frustration: there's an ebuild and I still needed to talk to ./Configure. Arg. No, please.
Hello again, sorry for the noise, but I asked in the Geant4 forums about their plans for the buildsystem and I got a very helpful OTR answer. Actually when I wrote the initial ebuild I set us on the wrong track since *there is no need to use nasty Configure*. Virtually everything can be set through environment variables to fine and plain (g)make. And this is what some people doing lots of installs actually seem to use. The magic is (hidden) in the manual in http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/InstallationGuide/html/ch02s03.html#sect.InstManually.ReqEnvVar and even more on flags in http://geant4.web.cern.ch/geant4/UserDocumentation/UsersGuides/ForApplicationDeveloper/html/apas05.html#sect.MkflEnvVar.EnvVar An actual ebuild could work like the something below in src_compile(): #### pseudo code/untested #### export G4SYSTEM=Linux-g++ export G4INSTALL=$USEFUL_DIR export CLHEP_BASE_DIR=/usr/ export G4LIB=/tmp/usr/lib/geant4 export G4INCLUDE=/tmp/usr/include/geant4 export G4LIB_BUILD_SHARED=1 export G4LIB_BUILD_STATIC=1 export G4LIB_BUILD_ZLIB=1 export G4UI_USE_TERMINAL=1 export G4UI_USE_TCSH=1 export G4UI_BUILD_XM_SESSION=1 export G4UI_BUILD_XAW_SESSION=1 export G4UI_USE_XM=1 export G4UI_USE_XAW=1 export G4UI_BUILD_QT_SESSION=1 export G4UI_USE_QT=1 [...] cd source/ && gmake ${MAKEOPTS} ######## If I understood correctly, autoconf or similar is not going to come anytime soon (if ever). I'll have a closer look over the weekend weather (beautiful spring) permitting. Cheers, b.
Hi Benjamin, Thanks for looking this up, the docs could help. Setting env variables could be a better idea than using this heavy-weight Configure script, and then using our emake system. The small g4*_use functions in the ebuild could be rewritten to set env variables instead. Concerning motif, could you try to re-emerge openmotif? I'll see what I can do about the ebuild probably next week. > If I understood correctly, autoconf or similar is not going to come anytime > soon (if ever). I understand that autotools are not really friendly, but they are robust and maintainable. There are also good alternatives such as CMake or SCons. The actual build system reached a level of fermentation that only my uncle's booze did.
Created attachment 149647 [details, diff] patch against science-overlay r1013 Hi Sébastien, attached a nearly working version of your ebuild not using ./Configure but emake and environment variables. It is longer than the older version, but it does not need patches anymore. Since we manage the enviroment variables on our own, we need to create the env scripts and it should be pretty straight forward to create env.d scripts if we want (though I personally do not like the idea too much). I unset all user env variables in pkg_setup(), so if I understood you at all the USE flags could be fine thereafter. If not I need some pointer to further information and will fix it. THE ONLY BIG PROBLEM left is that when making a user project make tries to create a directory in /usr/lib/geant/ (unsuccessfully when run as user), which is probably caused by something deep in config/binmake.gmk's bowels but I am blind by now. This is particularly bad if anybody ever has the stupid idea to make as root. I will try to find out where this is coming from, but if this is obvious to you I am be happy to know, too. I marked points of interest with TODO, but they are not critical. Best, b.
Created attachment 149703 [details, diff] patch against science-overlay r1013 Sorry for the lousy patch, a fixed version attached: * fix the dir creation issue with a sed in config/architecture.gmk * create a valid env.csh b.
What's left to do to include it in the main tree?
Created attachment 153827 [details, diff] Detailed data dirs for env script (In reply to comment #19) > What's left to do to include it in the main tree? > Two points: (1) I am getting access violations when the libraries are created ACCESS DENIED mkdir: /usr/lib/geant4/Linux-g++ mkdir: cannot create directory `/usr/lib/geant4/Linux-g++': Permission denied make[2]: *** [/usr/lib/geant4/Linux-g++/libG4globman.so] Error 1 Result is fine though, probably just one of the awesome G4DUMMY_VARIABLE lines in config/architecture.gmk. (2) The data directories have to be specified explicitly (yes, for every single one of them). I will attach a patch that fixes this. Sorry, I have been very busy lately. Maybe I will find time to look into (1) in the next month, but I do not want to make promises. Asking for help on gentoo-sh-masochists-wanted would help now ;)
Created attachment 154867 [details, diff] Fix access violations and use explicit data location env variables (in reply to comment #19 and comment #20) Hi, here's a patch against science r1021 that fixes the two issues mentioned. Regarding the access violation fix, I am all but 100% sure it is doing absolutely the right thing. If you go through the patch, you can literally follow the things I did to get rid of the access violation. Maybe something we do will backfire some day, but it I could successfully emerge geant4, and build projects with it. Should we go to the next step, i.e. putting it in the wild to stand or fail? b.
Created attachment 155109 [details] New ebuild for patchlevel 2 Just a copy of geant-4.9.1_p01.ebuild, but works just as well. The data files are the same as for p01.
Bumped in the science overlay with minor fixes. Thanks for keeping up! What should we do about g4py and momo? Are these used?
(In reply to comment #23) > What should we do about g4py and momo? Are these used? *I* do not use them. G4py sounds nice, but I've never seen anybody working with it. There also seems to be Geant4py (something different?) from KEK, but again, I have no idea. Momo seems to be even more strange; never met anybody in HEP using java. If any actual users exist, I can take care of the ebuilds, but I think it's OK to avoid looking into it right now, even more, since the code doesn't seem to be very much worked on.
Let's keep g4py/momo for another round then. I'm planning on pushing the geant build (with dawns and clhep) to the tree soon, but I'm still fighting with deps (openinventor is one nasty package). It's a good time for all to review/tests those 3 packages. Thanks
Now in main portage tree. Thanks much for your effort, Benjamin.
Created attachment 159963 [details] new ebuild Hi, the first beta for the new geant4.9.2 is out. As major build-related changes they claim improved qt4 and geant4py/g4py compiles. The release notes also claim that now sci-physics/clhep-2.0.3.3 was required for the build, but clhep 2.0.3.3 just seems to be a bug fix release for 2.0.3.2, and geant-4.9.2_beta1 is building fine here with 2.0.3.2, so i think a bit loser requirements are fine for this ebuild. I am working on geant4py, and will dump it elsewhere (though it seems to be another interesting build, and I am not sure if we can reach a satisfactory solution). I might also eventually look into qt4 support, though I won't promise. The general approach is outlined in an older version of geant-4.9.1_p01 in science's svn, if anyone else feels like it. One of the data files also changed in this new release. Please also see bug 231314. Cheers, Benjamin
I am installing a package which depends on GEANT4, and it is expecting a file that: ${G4INSTALL}/Configure -incflags which prints out the analagous include directory for the Geant4 code. Take a look at that directory to make sure that the Geant4 header files are located there. My install 4.9.2_p02 does not have this file.
(In reply to comment #28) > I am installing a package which depends on GEANT4, and it is expecting a file > that: > > ${G4INSTALL}/Configure -incflags > > My install 4.9.2_p02 does not have this file. Please open a new bug for that. Also, note that we are building geant4 bypassing completely the very clumsy Configure script shipped with the geant4 tar ball. Thanks.
