Go is a new systems programming language developped at google by Rob Pike. It has garbage collection, coroutines, communication channels and a clean syntax. Reproducible: Always Steps to Reproduce: emerge golang Actual Results: emerge: there are no ebuilds to satisfy "golang". Expected Results: install golang The directory hierarchy for golang is difficult to package. The whole directory is self-contained, and it's built using a shell script (that uses make). This ebuild installs that directory into /usr/golang, with some additional support files (i.e. bash completion, syntax highlighting files, zsh completion) installed into their respective places.
Created attachment 251819 [details] golang-9999.ebuild first iteration of the golang ebuild, live version
Created attachment 251821 [details] files/golang-env environment file for golang, setting the needed PATH and other environment variables
Created attachment 251827 [details] golang-9999.ebuild golang mercurial live ebuild
Created attachment 251829 [details] golang-20101020.ebuild golang release ebuild, releases are just tags in the mercurial repository, so i distribute the tarball for now.
Created attachment 251831 [details] golang-9999.ebuild indentation fix, fixes the postinst and postrm functions to only uninstall the elisp module if it was installed in the first place
Created attachment 251833 [details] golang-20101020.ebuild indentation fix, fixes the postinst and postrm functions to only uninstall the elisp module if it was installed in the first place
Created attachment 251865 [details] golang-9999.ebuild add parallelization to the kencc-based compilers
Created attachment 251867 [details] golang-20101020.ebuild add parallelization to the kencc-based compilers
Hello John, I am willing to take a look at getting this into the tree; it is an interesting programming language. There are several changes which need to be made to the ebuild before it is committed to the tree. I will refer to your live ebuild for now, since all of the changes I will suggest can be easily made to both ebuilds. I would like to call the ebuild "go" instead of golang, since it will be installed in the dev-lang category. I want to rework the src_compile and src_install functions a bit. I think we can make the code in the src_compile function more clear, and based on what I have seen in how another distribution installs go, I believe we can have a more fhs compliant installation than just dropping everything into /usr/golang. Would you like me to tell you what needs to be changed here so you can rework the ebuild, or should I make the changes and attach the new ebuild here so you can test it before I commit it to the tree?
hey William, I'm okay with whatever works best for you, i'm really interested in getting this into the tree, and also about how you are going to resolve the FHS compliance issue. Cheers! John
Created attachment 252191 [details] LICENSE the go license, it's not yet in the tree so it needs to be added if the ebuild is added to the tree.
Comment on attachment 252191 [details] LICENSE I don't need this attachment since I can get the license from the repository.
I have discovered that the build system go uses does not respect user's CFLAGS and LDFLAGS. I filed a bug upstream[1]. [1] http://code.google.com/p/go/issues/detail?id=1234
Created attachment 252463 [details] golang-9999.ebuild This adds two patches from go bug http://code.google.com/p/go/issues/detail?id=1234. It seems to work fine and gets rid of portage warnings about using different CFLAGS and LDFLAGS.
Created attachment 252465 [details] issue2735043_1_2.diff first patch to enable user CFLAGS and LDFLAGS
Created attachment 252467 [details] issue2735043_1_3.diff secont patch to fix CFLAGS and LDFLAGS
Comment on attachment 252463 [details] golang-9999.ebuild Please see my comments on the upstream bug. I don't think those patches have been accepted upstream yet, and I am waiting to see if they will be before I apply them. Also, I have asked a question about their CFLAGS they are using since -O2 and -ggdb do not really make sense together.
Comment on attachment 252465 [details] issue2735043_1_2.diff My concern about applying patches to a live ebuild is if they apply patches or change something else upstream it will break the live ebuild.
Comment on attachment 252467 [details] issue2735043_1_3.diff I'm marking this obsolete for the same reason as the previous patch.
I have filed another bug upstream [1] regarding an fhs install. I want to see what advice I get from upstream for how to do this. [1] http://code.google.com/p/go/issues/detail?id=1239
I have opened two more issues upstream due to qa issues that I found when attempting the build: http://code.google.com/p/go/issues/detail?id=1245 http://code.google.com/p/go/issues/detail?id=1246
upstream issue http://code.google.com/p/go/issues/detail?id=1234 closed, the host's cflags are now not ignored by the golang installer ( revision http://code.google.com/p/go/source/detail?r=9b582a7a30 )
John, I'm not sure why you are adding your email address to the cc list. Since you are the reporter, you will always get emails automatically when anything changes on this bug.
Created attachment 253075 [details] go-9999.ebuild latest version of the ebuild.
Created attachment 253077 [details] files/go.envd go environment settings
To build dev-lang/go on my system I had to patch the ebuild this way: 16c16 < KEYWORDS="" --- > KEYWORDS="~amd64 ~arm ~x86" 32c32,41 < export GOROOT="$(pwd)" --- > export GOROOT="$S" > export GOBIN="$GOROOT/bin" > case ${ARCH} in > x86) > export GOARCH="386" > ;; > *) > export GOARCH="${ARCH}" > ;; > esac
I can verify the bit about $GOROOT and $GOBIN from the patch above. Not setting them was OK on first install, but on subsequent installs it causes the build system to use the values from the environment defined in /etc/env.d/90go, causing a sandbox violation when trying to access /usr/bin/quietgcc. I also noticed that the vim files weren't being installed. Using the following works: if use vim-syntax; then rm misc/vim/readme.txt insinto /usr/share/vim/vimfiles/ doins -r misc/vim/* fi Finally, I get this QA message: QA: other QA Notice: The following files contain insecure RUNPATHs Please file a bug about this at http://bugs.gentoo.org/ with the maintaining herd of the package. /var/tmp/portage/dev-lang/go-9999/work/hg/pkg/linux_amd64 usr/bin/govet /var/tmp/portage/dev-lang/go-9999/work/hg/pkg/linux_amd64 usr/bin/godoc /var/tmp/portage/dev-lang/go-9999/work/hg/pkg/linux_amd64 usr/bin/hgpatch /var/tmp/portage/dev-lang/go-9999/work/hg/pkg/linux_amd64 usr/bin/gofmt /var/tmp/portage/dev-lang/go-9999/work/hg/pkg/linux_amd64 usr/bin/goinstall /var/tmp/portage/dev-lang/go-9999/work/hg/pkg/linux_amd64 usr/bin/cgo /var/tmp/portage/dev-lang/go-9999/work/hg/pkg/linux_amd64 usr/bin/goyacc /var/tmp/portage/dev-lang/go-9999/work/hg/pkg/linux_amd64 usr/bin/ebnflint Running any of those programs results in segmentation faults. In case it's useful, below is the output of emerge --info. ------------------------------------------------------------- marcec go # emerge --info go Portage 2.1.9.25 (default/linux/amd64/10.0, gcc-4.4.4, glibc-2.11.2-r3, 2.6.36-gentoo-r5 x86_64) ================================================================= System Settings ================================================================= System uname: Linux-2.6.36-gentoo-r5-x86_64-AMD_Athlon-tm-_64_X2_Dual_Core_Processor_4200+-with-gentoo-2.0.1 Timestamp of tree: Thu, 03 Feb 2011 21:45:02 +0000 app-shells/bash: 4.1_p9 dev-lang/python: 2.6.6-r1, 3.1.2-r4 dev-util/cmake: 2.8.1-r2 sys-apps/baselayout: 2.0.1-r1 sys-apps/openrc: 0.7.0 sys-apps/sandbox: 2.4 sys-devel/autoconf: 2.13, 2.65-r1 sys-devel/automake: 1.9.6-r3, 1.10.3, 1.11.1 sys-devel/binutils: 2.20.1-r1 sys-devel/gcc: 4.4.4-r2 sys-devel/gcc-config: 1.4.1 sys-devel/libtool: 2.2.10 sys-devel/make: 3.81-r2 virtual/os-headers: 2.6.30-r1 (sys-kernel/linux-headers) ACCEPT_KEYWORDS="amd64" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-O2 -march=native -pipe" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/kde/3.5/share/config/kdm /usr/share/config" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/fonts/fonts.conf /etc/games/angband/edit/ /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-O2 -march=native -pipe" DISTDIR="/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps=y" FEATURES="assume-digests binpkg-logs buildpkg collision-protect distlocks fixlafiles fixpackages metadata-transfer news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox" GENTOO_MIRRORS="http://de-mirror.org/distro/gentoo/ ftp://ftp-stud.fht-esslingen.de/pub/Mirrors/gentoo/ ftp://mirror.muntinternet.net/pub/gentoo/" LANG="de_DE.UTF-8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="en_US en en_GB de" MAKEOPTS="-s -j3" PKGDIR="/usr/portage/packages" PORTAGE_COMPRESS="xz" PORTAGE_COMPRESS_FLAGS="-9" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/usr/local/portage/layman/sunrise /usr/local/portage/layman/pcsx2 /usr/local/portage/layman/science /usr/local/portage" SYNC="rsync://rsync.europe.gentoo.org/gentoo-portage" USE="3dnow 3dnowext X a52 aac aalib accessibility acl acpi alsa amd64 audiofile avahi bash-completion berkdb blas branding bzip2 cairo caps cdda cdinstall cdr chipcard cjk cli consolekit cracklib crypt css cups cxx dbus dga djvu dri dssi dts dvd dvdnav dvdr dvi encode exif fbcon ffmpeg fftw flac fortran ftp fuse gdbm gif gimp glitz glut gmp gnuplot gnutls gpm gtk hbci iconv idn imlib inotify ipv6 jack jpeg jpeg2k kipi ladspa lapack lash latex lcms libcaca libnotify libsamplerate lm_sensors logrotate lzma mad matroska mikmod mjpeg mmx mmxext mng modplug modules mp3 mp4 mpeg mudflap multilib musepack musicbrainz ncurses nfs nls nntp nptl nptlonly nsplugin ntp offensive ogg openal openexr opengl openmp pam pango pcre pdf plotutils png policykit ppds pppd pulseaudio python qt4 quicktime rar readline rtsp samba sasl schroedinger sdl session sid slang slp smp sndfile speex spell sse sse2 sse3 ssl startup-notification svg sysfs taglib tcpd theora threads tiff timidity truetype unicode usb vaapi vcd vim-syntax vorbis vpx webkit wma x264 xattr xcb xcomposite xface xft xml xmp xorg xpm xscreensaver xulrunner xv xvid xvmc zeroconf zlib zsh-completion" ALSA_CARDS="ice1724 hda-intel usb-audio" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en_US en en_GB de" PHP_TARGETS="php5-3" RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="radeon vesa" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_RSYNC_EXTRA_OPTS ================================================================= Package Settings ================================================================= dev-lang/go-9999 was built with the following: USE="bash-completion (multilib) vim-syntax zsh-completion -emacs"
All, it appears that gcc 4.6 will support the Go programming language. So do you think we still need this compiler or would you be able to use the implementation in gcc 4.6 when it is released? I have no idea of the difference between the implementations, so I am requesting input. Thanks much.
The compiler is just a part of the whole, with that (gcc) compiler you still don't have the runtime, GC or the base packages, so it's not extremely useful on it's own
Doesn't build on my system. I've opened issue upstream: http://code.google.com/p/go/issues/detail?id=1669 Please someone check this on your system, because I'm afraid this may happens because of Hardened (but there nothing from grsec/pax in kernel log).
Created attachment 269527 [details] fixed ebuild Ebuild fixed: - added ~x86 to KEYWORDS - added USE-flag "hardened" with patch - fixed "ACCESS DENIED unlinkat: /usr/bin/quietgcc" error on re-emerge - fixed installing vim-related files
Created attachment 269529 [details, diff] patch for hardened
What is the status of the hardened patch? looking over the upstream issues, it looks like they didn't accept it. Is that correct? If so, it doesn't make sense to apply it with a live ebuild, because there is no garantee that it will apply for everyone whenever they try to install go. Also, I had to remove the ~x86 keyword since this is a live ebuild. I may need to use REQUIRED_USE on this to make it not work on hardened systems. Also, since upstream doesn't release tarballs I will need to figure out how to manage releases.
(In reply to comment #33) > looking over the upstream issues, it looks like they didn't accept it. Is that correct? Yeah, looks like this. > If so, it doesn't make sense to apply it with a live ebuild, because > there is no garantee that it will apply for everyone whenever they try to > install go. No, I don't think so. On hardened choice is simple: it's either doesn't work at all, or this patch applies and everything works ok. So, just leave patch in place, this way people will at least have a chance to install Go. > Also, I had to remove the ~x86 keyword since this is a live ebuild. I don't get it. Without x86 or ~x86 it doesn't emerge on x86 at all, so one of these keywords should be in ebuild. > Also, since upstream doesn't release tarballs I will need to figure out how to > manage releases. You can hardcode some hg revision changeset id to ensure exactly this version will be fetched and installed instead of latest.
(In reply to comment #34) > (In reply to comment #33) > > looking over the upstream issues, it looks like they didn't accept it. Is that correct? > Yeah, looks like this. > > If so, it doesn't make sense to apply it with a live ebuild, because > > there is no garantee that it will apply for everyone whenever they try to > > install go. > No, I don't think so. On hardened choice is simple: it's either doesn't work at > all, or this patch applies and everything works ok. So, just leave patch in > place, this way people will at least have a chance to install Go. See my comments below about this. > > Also, I had to remove the ~x86 keyword since this is a live ebuild. > I don't get it. Without x86 or ~x86 it doesn't emerge on x86 at all, so one of > these keywords should be in ebuild. That would be true if this ebuild were based on a tarball. But, since it builds based on the most current checkout of a vcs repository, there is absolutely zero stability. There is no guarantee that it will always work. That is why there are no keywords in a live ebuild. This is also why I'm hesitant about applying the hardened patch to this ebuild. It may work today, but there is absolutely no garantee that it will patch successfully tomorrow. > > Also, since upstream doesn't release tarballs I will need to figure out how to > > manage releases. > You can hardcode some hg revision changeset id to ensure exactly this version > will be fetched and installed instead of latest. Since upstream doesn't release tarballs, I may have to go further and create tarballs of the source tree for each revision and put them on our mirrors.
wow, we have almost 9 month here, so, when the baby is born?
Take heart; I am still interested in working on this. :-) I've been working on some other projects lately, but I will get back to this as soon as I can. :-)
Created attachment 280243 [details] go-9999.ebuild All, I did some testing with mercurial and found that it does produce tarballs which are reproducible in the same way that git does, so the attached ebuild will work either as a live ebuild or to pull a tarball from the gentoo mirrors for a snapshot. The hardened patch only works for one architecture, so I can't use it as it stands. I believe we can handle it in the ebuild without patching the build system. I will look into that next. If you are not using hardened, feel free to test this ebuild and let me know if it works for you.
(In reply to comment #38) > Created attachment 280243 [details] > go-9999.ebuild > Got some warning at the install phase: >>> Completed installing go-9999 into /var/tmp/portage/dev-lang/go-9999/image/ * QA Notice: command not found: * * /var/tmp/portage/dev-lang/go-9999/temp/environment: line 2432: dobashcompletion: command not found strip: x86_64-pc-linux-gnu-strip --strip-unneeded -R .comment -R .GCC.command.line /usr/bin/goyacc /usr/bin/govet /usr/bin/gotype /usr/bin/gotest /usr/bin/gopack /usr/bin/goinstall /usr/bin/gofmt /usr/bin/gofix /usr/bin/godoc /usr/bin/goapi /usr/bin/go /usr/bin/ebnflint /usr/bin/cgo /usr/bin/6prof /usr/bin/6nm /usr/bin/6l /usr/bin/6g /usr/bin/6cov /usr/bin/6c /usr/bin/6a usr/lib/go/pkg/linux_amd64/runtime.a x86_64-pc-linux-gnu-strip:/var/tmp/portage/dev-lang/go-9999/image/usr/lib/go/pkg/linux_amd64/runtime.a: File format not recognized usr/lib/go/pkg/linux_amd64/errors.a x86_64-pc-linux-gnu-strip:/var/tmp/portage/dev-lang/go-9999/image/usr/lib/go/pkg/linux_amd64/errors.a: File format not recognized usr/lib/go/pkg/linux_amd64/sync.a x86_64-pc-linux-gnu-strip:/var/tmp/portage/dev-lang/go-9999/image/usr/lib/go/pkg/linux_amd64/sync.a: File format not recognized [TRUNCATED: The above message repeat for each .a file] ecompressdir: bzip2 -9 /usr/share/doc scanelf: Invalid ar entry scanelf: Invalid ar entry scanelf: Invalid ar entry scanelf: Invalid ar entry scanelf: Invalid ar entry scanelf: Invalid ar entry scanelf: Invalid ar entry scanelf: Invalid ar entry scanelf: Invalid ar entry scanelf: Invalid ar entry scanelf: Invalid ar entry
Created attachment 302167 [details] updated ebuild
Created attachment 302169 [details, diff] hardened patch for amd64
Created attachment 307393 [details] go-1.ebuild I have successfully used this ebuild (based on one from this bug) to install Go 1. The go tool and associated sub-commands seem to work. godoc works, with full access to the package documentation (now that source files are installed). The Vim integration also works now. I also had to pick the tip from earlier in this bug about setting GOBIN to prevent access violation on rebuilds.
(In reply to comment #42) > Created attachment 307393 [details] > go-1.ebuild > > I have successfully used this ebuild (based on one from this bug) to install > Go 1. > > The go tool and associated sub-commands seem to work. godoc works, with > full access to the package documentation (now that source files are > installed). The Vim integration also works now. > > I also had to pick the tip from earlier in this bug about setting GOBIN to > prevent access violation on rebuilds. Some go sub-commands (build,run...etc) require source fils to works, but the files only install with USE flag "doc" enabled. In order to get the sub-commands works without enable USE flag "doc". I have patch the ebuild like this. --- a/go-1.ebuild +++ b/go-1.ebuild @@ -57,10 +57,11 @@ chmod -R +x "${D}/usr/lib/go/pkg/tool" + insinto /usr/lib/go + doins -r src + if use doc; then - insinto /usr/lib/go doins -r doc - doins -r src insinto /usr/lib/go/lib doins -r lib/godoc else
(In reply to comment #43) > (In reply to comment #42) > > Created attachment 307393 [details] > > go-1.ebuild > > > > I have successfully used this ebuild (based on one from this bug) to install > > Go 1. > > > > The go tool and associated sub-commands seem to work. godoc works, with > > full access to the package documentation (now that source files are > > installed). The Vim integration also works now. > > > > I also had to pick the tip from earlier in this bug about setting GOBIN to > > prevent access violation on rebuilds. > > Some go sub-commands (build,run...etc) require source fils to works, but the > files only install with USE flag "doc" enabled. In order to get the > sub-commands works without enable USE flag "doc". I have patch the ebuild > like this. Yeah - it's a known issue. I thought they had fixed it, but apparently the fix was taken out due to problems before the Go 1 release. So for now the src folder needs to always be installed. There is a bug about this: http://code.google.com/p/go/issues/detail?id=2775
Created attachment 308225 [details] go-1.ebuild This is an updated version of my earlier attempt. Changes: - always install src directory (upstream bug 2775) - only run tests with FEATURES="test" (quite a bit faster without tests) - "fix" file time-stamps for packages & tools - use updated bash-completion (the original eclass has been deprecated) The time-stamp "fix" is necessary because the go tool compares time-stamps in order to determine if packages are stale. If a package depends on something newer than itself then it will be rebuilt. This check also applies to the compiler and linker. Since a normal user can't rebuild the installed packages/compiler/linker the packages used by their code will be built from source each time they build their code. Forcing all files under /usr/lib/go/pkg to have the same time-stamp prevents this problem. In order for this fix to work, I have had to disable the stripping of binaries - as this updated the time-stamps of the tools after the src_install phase. It looks like some of the QA check aren't working - but they don't seem to cause any problems so I haven't done anything about them.
(In reply to comment #45) > Created attachment 308225 [details] > go-1.ebuild > > This is an updated version of my earlier attempt. > > Changes: > > - always install src directory (upstream bug 2775) > - only run tests with FEATURES="test" (quite a bit faster without tests) > - "fix" file time-stamps for packages & tools > - use updated bash-completion (the original eclass has been deprecated) > > The time-stamp "fix" is necessary because the go tool compares time-stamps > in order to determine if packages are stale. If a package depends on > something newer than itself then it will be rebuilt. This check also > applies to the compiler and linker. Since a normal user can't rebuild the > installed packages/compiler/linker the packages used by their code will be > built from source each time they build their code. Forcing all files under > /usr/lib/go/pkg to have the same time-stamp prevents this problem. In order > for this fix to work, I have had to disable the stripping of binaries - as > this updated the time-stamps of the tools after the src_install phase. > > It looks like some of the QA check aren't working - but they don't seem to > cause any problems so I haven't done anything about them. There might be something missing here: $ go test.go 2012/04/24 08:21:16 unsupported GOARCH GENTOOARCH $ env | grep GO GOBIN=GENTOO_GOROOT/bin PATH=/usr/local/bin:/usr/bin:/bin:/opt/bin:/usr/i686-pc-linux-gnu/gccbin/4.5.3:GENTOO_GOROOT/bin GOARCH=GENTOOARCH GOROOT=GENTOO_GOROOT GOOS=linux
Created attachment 310261 [details] go-1.0.1.ebuild All, I have definitely not forgotten about this; please accept my appology for taking so long. Here is the ebuild I'm considering using for go1.0.1. Please test and let me know if this works for you.
>>> Install go-1.0.1 into /var/tmp/portage/dev-lang/go-1.0.1/image/ category dev-lang >>> Completed installing go-1.0.1 into /var/tmp/portage/dev-lang/go-1.0.1/image/ ecompressdir: bzip2 -9 /usr/share/doc scanelf: Invalid ar entry scanelf: Invalid ar entry scanelf: Invalid ar entry scanelf: Invalid ar entry scanelf: Invalid ar entry scanelf: Invalid ar entry >>> Installing (1 of 1) dev-lang/go-1.0.1 When I try and build something I get this: go build mypackage: exec: "/usr/lib/go/pkg/tool/linux_amd64/6g": permission denied In addition, if I try building with a modified GOARCH, that fails too: $ GOARCH=386 GOPATH=${PWD}/env go build go build runtime: exec: "/usr/lib/go/pkg/tool/linux_amd64/8g": stat /usr/lib/go/pkg/tool/linux_amd64/8g: no such file or directory
* Package: dev-lang/go-1.0.1 * Repository: myself * USE: bash-completion elibc_glibc emacs kernel_linux userland_GNU x86 * FEATURES: sandbox >>> Unpacking source... >>> Unpacking go1.0.1.src.tar.gz to /var/tmp/portage/dev-lang/go-1.0.1/work >>> Source unpacked in /var/tmp/portage/dev-lang/go-1.0.1/work >>> Preparing source in /var/tmp/portage/dev-lang/go-1.0.1/work/go ... >>> Source prepared. >>> Configuring source in /var/tmp/portage/dev-lang/go-1.0.1/work/go ... >>> Source configured. >>> Compiling source in /var/tmp/portage/dev-lang/go-1.0.1/work/go ... # Building C bootstrap tool. cmd/dist go tool dist: unknown $GOARCH GENTOOARCH # Building compilers and Go bootstrap tool for host, /. go tool dist: unknown $GOARCH GENTOOARCH * Compiling GNU Emacs Elisp files ... Wrote /var/tmp/portage/dev-lang/go-1.0.1/work/go/misc/emacs/go-mode.elc Wrote /var/tmp/portage/dev-lang/go-1.0.1/work/go/misc/emacs/go-mode-load.elc [ ok ] >>> Source compiled. >>> Test phase [not enabled]: dev-lang/go-1.0.1 >>> Install go-1.0.1 into /var/tmp/portage/dev-lang/go-1.0.1/image/ category dev-lang !!! dobin: bin/* does not exist * ERROR: dev-lang/go-1.0.1 failed (install phase): * dobin failed * * If you need support, post the output of 'emerge --info =dev-lang/go-1.0.1', * the complete build log and the output of 'emerge -pqv =dev-lang/go-1.0.1'. * This ebuild is from an overlay named 'myself': '/usr/local/portage/' * The complete build log is located at '/var/tmp/portage/dev-lang/go-1.0.1/temp/build.log'. * The ebuild environment file is located at '/var/tmp/portage/dev-lang/go-1.0.1/temp/environment'. * S: '/var/tmp/portage/dev-lang/go-1.0.1/work/go' * QA Notice: file does not exist: * * dobin: bin/* does not exist =================================================== =================================================== $ emerge --info Portage 2.1.10.52 (default/linux/x86/10.0, gcc-4.5.3, glibc-2.14.1-r2, 3.3.0-gentoo i686) ================================================================= System uname: Linux-3.3.0-gentoo-i686-Intel-R-_Core-TM-2_Duo_CPU_P8600_@_2.40GHz-with-gentoo-2.1 Timestamp of tree: Sat, 14 Apr 2012 11:30:01 +0000 app-shells/bash: 4.2_p24 dev-lang/python: 2.7.2-r3, 3.2.2-r1 dev-util/cmake: 2.8.7-r5 dev-util/pkgconfig: 0.26 sys-apps/baselayout: 2.1 sys-apps/openrc: 0.9.9.3 sys-apps/sandbox: 2.5 sys-devel/autoconf: 2.13, 2.68 sys-devel/automake: 1.11.3 sys-devel/binutils: 2.22-r1 sys-devel/gcc: 4.5.3-r2 sys-devel/gcc-config: 1.6 sys-devel/libtool: 2.4.2 sys-devel/make: 3.82-r3 sys-kernel/linux-headers: 3.3 (virtual/os-headers) sys-libs/glibc: 2.14.1-r2 Repositories: gentoo science myself ACCEPT_KEYWORDS="x86 ~x86" ACCEPT_LICENSE="* -@EULA" CBUILD="i686-pc-linux-gnu" CFLAGS="-march=core2 -mtune=core2 -O2 -pipe" CHOST="i686-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/splash /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=core2 -mtune=core2 -O2 -pipe" DISTDIR="/usr/portage/distfiles" FEATURES="assume-digests binpkg-logs distlocks ebuild-locks fixlafiles news parallel-fetch protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="" GENTOO_MIRRORS="http://mirrors.xmu.edu.cn/gentoo" LANG="zh_CN.utf8" LDFLAGS="-Wl,-O1 -Wl,--as-needed" LINGUAS="zh_CN" MAKEOPTS="-j3" PKGDIR="/usr/portage/packages" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/var/tmp" PORTDIR="/usr/portage" PORTDIR_OVERLAY="/var/lib/layman/science /usr/local/portage" SYNC="rsync://rsync.gentoo.org/gentoo-portage" USE="R X aac acl acpi alsa bash-completion bluetooth bzip2 cairo cdr cjk cli cracklib crypt cups cxx dbus djvu dri dvd eds emacs flac fontconfig fortran gdbm gif gnome gnome-keyring gnuplot go gstreamer gtk gtk3 iconv id3 introspection ios ipv6 jpeg lame latex mmx mmxext modules mp3 mpd mudflap nautilus ncurses networkmanager nls nptl nptlonly ogg opengl openmp pam pcre pdf png pppd pulseaudio python3 readline sendto session sse sse2 ssl svg sysfs systemd tcpd theora truetype unicode usb v4l vorbis wayland x86 xft xml xorg zeitgeist zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1 emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop empty extplug file hooks iec958 ioplug ladspa lfloat linear meter mmap_emul mulaw multi null plug rate route share shm softvol" APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" DRACUT_MODULES="plymouth" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" GRUB_PLATFORMS="pc" INPUT_DEVICES="evdev synaptics" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="zh_CN" PHP_TARGETS="php5-3" QEMU_SOFTMMU_TARGETS="i386" QEMU_USER_TARGETS="i386" RUBY_TARGETS="ruby19" USERLAND="GNU" VIDEO_CARDS="radeon" XTABLES_ADDONS="cui gfw zhang ipset" Unset: CPPFLAGS, CTARGET, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
using the kate useflag results in a file collision... # qfile /usr/share/apps/katepart/syntax/go.xml kde-base/katepart (/usr/share/apps/katepart/syntax/go.xml) # emerge -pv kde-base/katepart [ebuild R ] kde-base/katepart-4.8.2 so better remove that useflag or check the version of katepart
Currently, if the doc use flag is not on, we remove the godoc tool. I'm not sure this is really what we want since godoc can be used to generate documentation for any package regardless of whether the go documentation itself is installed.
(In reply to comment #46) > There might be something missing here: > > $ go test.go > 2012/04/24 08:21:16 unsupported GOARCH GENTOOARCH > > $ env | grep GO > GOBIN=GENTOO_GOROOT/bin > PATH=/usr/local/bin:/usr/bin:/bin:/opt/bin:/usr/i686-pc-linux-gnu/gccbin/4.5. > 3:GENTOO_GOROOT/bin > GOARCH=GENTOOARCH > GOROOT=GENTOO_GOROOT > GOOS=linux You are using the wrong environment settings. Use the file attached to comment #25 and this issue will go away.
Created attachment 310653 [details] go-1.0.1.ebuild All, this is the latest ebuild. Let me know how this version works for you. The main change here is to always install the godoc tool. The "doc" use flag is not meant to be a general-purpose use flag for installing documentation or not; it is used for documentation that takes a long time to build or that is considered "extra", like api documentation. Given that, I am considering dropping the doc use flag from this ebuild and always installing the documentation. Does anyone have any thoughts either way?
William, thanks for your work! just a small comment regarding your two last attachments, they both have application/octet-stream as mime-type instead of text/plain. This leads to following message when I want to view them in bugzilla: "Attachment is not viewable in your browser because its MIME type (application/octet-stream) is not one that your browser is able to display."
Created attachment 310905 [details] go-1.0.1.ebuild All, this is the latest rendition of the go-1.0.1 ebuild. I moved the timestamp fix to pkg_postinst which allows us to strip the binaries again. I need to know if unnecessary rebuilds will happen if we do this. Thanks, William p.s. this attachment is forced to have type text/plain, but that is not the default for the tool I use to interact with bugzilla. If you feel that is an issue this issue should be opened at http://github.com/williamh/pybugz/issues.
(In reply to comment #55) > Created attachment 310905 [details] > go-1.0.1.ebuild > > All, > > this is the latest rendition of the go-1.0.1 ebuild. > > I moved the timestamp fix to pkg_postinst which allows us to strip the > binaries again. I need to know if unnecessary rebuilds will happen if we > do this. A couple of quick builds worked fine. You should drop the go.envd file though. Setting GOBIN is actively harmful as it is the default install destination for all binaries and causes the go install command to try and install to /usr/bin: jp3@vayl: h>go install go install h: open /usr/bin/h: permission denied GOROOT isn't needed at run time either, it is compiled into the tools. jp3@vayl: h>unset GOBIN GOROOT jp3@vayl: h>go install jp3@vayl: h>
Created attachment 311201 [details] go-1.0.1.ebuild All, this is the latest rendition of the go ebuild. Can you please test with it and let me know if things still work? - We no longer install go.envd. - We also now tell portage to skip stripping some .a filess because they aren't libraries.
Created attachment 311351 [details, diff] hardened patch for x86 & amd64
Created attachment 311353 [details] ebuild for 1.0.1 with hardened support
Hi Alex, (In reply to comment #58) > Created attachment 311351 [details, diff] [details, diff] > hardened patch for x86 & amd64 Suppose I am building binary packages which I intend to install on a different machine and the hardened status of the target machine is not the same as the one I am using to build the packages. In that scenario your patch doesn't work, so I don't think I can use it in this form. The better patch would be to have the linkers (6l, 8l and 5l) mark the binaries correctly when they are generated. Also, this patch should not depend on the hardened use flag. Thanks, William
(In reply to comment #60) > Suppose I am building binary packages which I intend to install on a > different machine and the hardened status of the target machine is not the > same as the one I am using to build the packages. > > In that scenario your patch doesn't work, so I don't think I can use it in > this form. > > The better patch would be to have the linkers (6l, 8l and 5l) mark the > binaries correctly when they are generated. Also, this patch should not > depend on the hardened use flag. No, I don't think so. See: - Binary compiled on hardened system will work just fine on non-hardened. - Binary compiled on non-hardened system won't work on hardened system, but this is very usual situation with external binaries well known for all owners of hardened systems, and all of them know how to use paxctl to work around this issue. So, this isn't Go-specific. (Also, non-hardened systems usually don't have paxctl installed.) - This patch required not only for running Go apps, but also to compile Go itself. - This patch should depend on hardened USE-flag because systems without it usually doesn't have paxctl and because all other similar changes and all other ebuilds depend on this USE flag. - As for 5l - I don't have this arch to test my patch.
(In reply to comment #61) > - This patch should depend on hardened USE-flag because systems without it > usually > doesn't have paxctl and because all other similar changes and all other > ebuilds > depend on this USE flag. Patches are not typically applied based on whether or not a use flag is set; they are applied to the code uncondissionally so that everyone is using the same code. Can you not detect whether you are running on a hardened system inside the patch and skip installing the wrappers if you aren't?
(In reply to comment #62) > Patches are not typically applied based on whether or not a use flag is set; > they are applied to the code uncondissionally so that everyone is using the > same code. Quick grep on /usr/portage show 4 packages which apply hardened patches unconditionally and 5 packages which apply hardened patches only when `use hardened`. > Can you not detect whether you are running on a hardened system inside the > patch and skip installing the wrappers if you aren't? That's more complicated than just check `use hardened` in ebuild. Anyway, why you think this is important?
(In reply to comment #63) > That's more complicated than just check `use hardened` in ebuild. Anyway, > why you think this is important? I just checked with one of our QA team members, and he told me that we should avoid applying patches based on use flags where possible. But, tell me more about what you would have to test for. How can you know if you are on a hardened system?
(In reply to comment #64) > I just checked with one of our QA team members, and he told me that we > should avoid applying patches based on use flags where possible. I understood this, it make testing easier. But that's not always possible. > But, tell me more about what you would have to test for. How can you know if > you are on a hardened system? Main reason for this patch is because PaX in kernel will kill Go in the middle of build process and when building user apps. So, probably only correct way to detect hardened system outside of portage and USE flags is check kernel. If it built with /proc/config.gz we can do this: zgrep -q CONFIG_PAX=y /proc/config.gz && echo hardened I've no idea how we can check for hardened without /proc/config.gz, and there is no guarantee all system will provide it.
Ok, thanks for the explanation. Let me check with the hardened team and see if they know of another way to check for this.
Ok, I got some information from the hardened team: - We should use the pax_kernel use flag instead of the hardened use flag. - I found out a way also to test to see if we are running under a PaX kernel. You should be able to enclose your patch in this block: if [ -e /proc/self/status ] && grep -q PaX /proc/self/status; then # your patch here fi - The use flag will now be used for a dependency on sys-apps/paxctl. What are your thoughts?
(In reply to comment #67) > - We should use the pax_kernel use flag instead of the hardened use flag. > - The use flag will now be used for a dependency on sys-apps/paxctl. This make sense. Rename use flag and add DEPEND on paxctl. I'll do this. WRT checking is we running under PAX kernel I didn't see how and why this can be used in this ebuild/patch.
Created attachment 311385 [details] go-1.0.1.ebuild Here is an ebuild that uses the pax_kernel use flag. I'll look at the patch soon.
Add pax_kernel to iuse in the ebuild I just posted. Sorry I missed that.
Created attachment 311387 [details] go-1.0.1.ebuild with hardened support change use flag 'hardened' with 'pax_kernel' add DEPEND and RDEPEND on paxctl
Hi Alex, The more I think about the hardened support, I think I would rather have patches that we can push upstream. To do that we need to meet their requirements [1], and I don't think this patch does that. Is there any way you can look into fixing the linkers and the bootstrap program as they suggest? If not, would you agree to me putting go in the tree without pax support and immediately opening another bug and getting the hardened team involved? (you would be cc'd on that bug as well). [1] http://code.google.com/p/go/issues/detail?id=47#c9
(In reply to comment #72) > The more I think about the hardened support, I think I would rather have > patches that we can push upstream. To do that we need to meet their > requirements [1], and I don't think this patch does that. > [1] http://code.google.com/p/go/issues/detail?id=47#c9 > > Is there any way you can look into fixing the linkers and the bootstrap > program as they suggest? Russ Cox said there: >> If someone wants to do the work to make 6l generate >> binaries that are blessed enough to do W+X mappings, >> patches welcome. This is exactly what my patch does… for PaX (SeLinux probably need something else)… and using wrapper script for linker instead of copy&paste paxctl source into Go linker source. BTW, what's wrong with wrappers except from aesthetic point of view? > If not, would you agree to me putting go in the tree without pax support > and immediately opening another bug and getting the hardened team > involved? (you would be cc'd on that bug as well). I can't stop you from doing this. :) But for me it makes a little sense. Hardened is officially supported part of Gentoo, and there is correct working patch to make Go work with Hardened. So, why do you wanna drop hardened support? If you think hardened patch can and should be improved - that's ok, but that's true for nearly anything in this world, so why not add it to portage first and improve next (when someone better than me will implement something which will be accepted by upstream)?
Hi Alex, I checked with upstream about your patch and they seem to be ok with the idea. So I have tweaked it slightly so that it can always be applied. I am now attaching the new ebuild and patch. Please test and report back.
Created attachment 311873 [details] go-1.0.1.ebuild Here is the new ebuild.
Created attachment 311875 [details] go-1.0.1-hardened.patch Here is the new patch. The difference between this and the previous patch is that the new patch is activated by a command line option.
(In reply to comment #74) > I am now attaching the new ebuild and patch. Please test and report > back. Please add ~amd64 into KEYWORDS (I'm using it and it works). I've tested it on x86 and amd64 (both hardened) and it works ok.
Gentoo policy only allows me to keyword ebuilds for arch's on which I can personally test. When this is in the tree, I will open another bug asking the amd64 and arm teams to add their keywords.
I am checking with our licensing team regarding adding the license to the tree. I read it over again and it is almost identical to the BSD-2 license, but it seems to have an extra clause.
(In reply to comment #79) > I am checking with our licensing team regarding adding the license to > the tree. I read it over again and it is almost identical to the BSD-2 > license, but it seems to have an extra clause. That would be because Go is licensed under 3-clause BSD, not 2-clause. It's /usr/portage/licenses/BSD, not /usr/portage/licenses/BSD-2.
All, Go has been officially added to the tree. Thanks for your help.
