ebuilds that use the git-2.eclass fail during src_unpack() with sandbox enabled. If sandbox is disabled, the ebuild works fine. Reproducible: Always Steps to Reproduce: 1. Add app-portage/gentoolkit-9999 ** to $EPREFIX/etc/portage/package.keywords 2. emerge -v =gentoolkit-9999 3. Actual Results: >>> Emerging (3 of 4) app-portage/gentoolkit-9999 >>> Unpacking source... Cloning into bare repository '/Users/pvarner/Library/Gentoo/usr/portage/distfiles/egit-src/gentoolkit.git'... /Users/pvarner/Library/Gentoo/usr/portage/distfiles/egit-src/gentoolkit.git: Operation not permitted * ERROR: app-portage/gentoolkit-9999 failed (unpack phase): * git-2_initial_clone: can't fetch from git://git.overlays.gentoo.org/proj/gentoolkit.git * * Call stack: * ebuild.sh, line 93: Called call-ebuildshell 'src_unpack' * environment, line 862: Called src_unpack * environment, line 4085: Called git-2_src_unpack * environment, line 1794: Called git-2_fetch * environment, line 1595: Called git-2_initial_clone * environment, line 1688: Called die * The specific snippet of code: * [[ -n ${EGIT_REPO_URI_SELECTED} ]] || die "${FUNCNAME}: can't fetch from ${EGIT_REPO_URI}" * * If you need support, post the output of `emerge --info '=app-portage/gentoolkit-9999'`, * the complete build log and the output of `emerge -pqv '=app-portage/gentoolkit-9999'`. * The complete build log is located at '/Users/pvarner/Library/Gentoo/var/tmp/portage/app-portage/gentoolkit-9999/temp/build.log'. * The ebuild environment file is located at '/Users/pvarner/Library/Gentoo/var/tmp/portage/app-portage/gentoolkit-9999/temp/environment'. * Working directory: '/Users/pvarner/Library/Gentoo/var/tmp/portage/app-portage/gentoolkit-9999/work' * S: '/Users/pvarner/Library/Gentoo/var/tmp/portage/app-portage/gentoolkit-9999/work/gentoolkit-9999' Expected Results: emerge completes successfully I have only tested this on an OS X prefix install, I don't know if the problem exists on other prefix installs. Portage 2.2.01.21313-prefix (prefix/darwin/macos/10.8/x64, gcc-4.2.1, unavailable, 12.2.0 x86_64) ================================================================= System uname: Darwin-12.2.0-x86_64-i386-64bit Timestamp of tree: Thu, 15 Nov 2012 15:48:57 +0000 app-shells/bash: 4.2_p37::gentoo_prefix dev-lang/python: 2.7.3-r2::gentoo_prefix dev-util/pkgconfig: 0.27.1::gentoo_prefix sys-devel/autoconf: 2.69::gentoo_prefix sys-devel/automake: 1.12.4::gentoo_prefix sys-devel/gcc-config: 1.5-r2::gentoo_prefix sys-devel/libtool: 2.4.2::gentoo_prefix sys-devel/make: 3.82-r4::gentoo_prefix Repositories: gentoo_prefix local ACCEPT_KEYWORDS="~x64-macos" ACCEPT_LICENSE="* -@EULA" CBUILD="x86_64-apple-darwin12" CFLAGS="-march=nocona -O2 -pipe" CHOST="x86_64-apple-darwin12" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf /etc/portage /etc/revdep-rebuild /etc/terminfo" CXXFLAGS="-march=nocona -O2 -pipe" DISTDIR="/Users/pvarner/Library/Gentoo/usr/portage/distfiles" EMERGE_DEFAULT_OPTS="--with-bdeps=y" FCFLAGS="" FEATURES="assume-digests binpkg-logs collision-protect config-protect-if-modified distlocks ebuild-locks fixlafiles force-prefix news nostrip preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch" FFLAGS="" GENTOO_MIRRORS="http://distfiles.gentoo.org" LDFLAGS="-Wl,-dead_strip_dylibs" MAKEOPTS="-j8" PKGDIR="/Users/pvarner/Library/Gentoo/usr/portage/packages" PORTAGE_CONFIGROOT="/Users/pvarner/Library/Gentoo/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages" PORTAGE_TMPDIR="/Users/pvarner/Library/Gentoo/var/tmp" PORTDIR="/Users/pvarner/Library/Gentoo/usr/portage" PORTDIR_OVERLAY="/Users/pvarner/Library/Gentoo/usr/local/portage/local" SYNC="rsync://rsync.prefix.freens.org/gentoo-portage-prefix" USE="coreaudio cracklib cxx ipv6 mmx mmxext modules ncurses nls objc objc++ prefix readline sse sse2 ssl svg unicode x64-macos zlib" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump" CAMERAS="ptp2" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" ELIBC="Darwin" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf superstar2 timing tsip tripmate tnt ubx" INPUT_DEVICES="keyboard mouse" KERNEL="Darwin" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" PHP_TARGETS="php5-3" PYTHON_TARGETS="python2_7 python3_2" RUBY_TARGETS="ruby18 ruby19" USERLAND="GNU" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CPPFLAGS, CTARGET, INSTALL_MASK, LANG, LC_ALL, LINGUAS, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
hmmm, perhaps the OSX sandbox is a bit too strict here (disallowing network access)
It also fails with creating the directories under $EPREFIX/usr/portage/distfiles: $ rm -rf egit-src emerge -v1 gentoolkit These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R *] app-portage/gentoolkit-9999::gentoo [9999::gentoo_prefix] 0 kB Total: 1 package (1 reinstall), Size of downloads: 0 kB >>> Verifying ebuild manifests >>> Emerging (1 of 1) app-portage/gentoolkit-9999 >>> Unpacking source... mkdir: cannot create directory ‘/Users/pvarner/Library/Gentoo/usr/portage/distfiles/egit-src’: Operation not permitted * ERROR: app-portage/gentoolkit-9999 failed (unpack phase): * git-2_prepare_storedir: can't mkdir "/Users/pvarner/Library/Gentoo/usr/portage/distfiles/egit-src" * * Call stack: * ebuild.sh, line 93: Called call-ebuildshell 'src_unpack' * environment, line 862: Called src_unpack * environment, line 4085: Called git-2_src_unpack * environment, line 1792: Called git-2_prepare_storedir * environment, line 1758: Called die * The specific snippet of code: * mkdir -m 775 -p "${EGIT_STORE_DIR}" || die "${FUNCNAME}: can't mkdir \"${EGIT_STORE_DIR}\""; * * If you need support, post the output of `emerge --info '=app-portage/gentoolkit-9999'`, * the complete build log and the output of `emerge -pqv '=app-portage/gentoolkit-9999'`. * The complete build log is located at '/Users/pvarner/Library/Gentoo/var/tmp/portage/app-portage/gentoolkit-9999/temp/build.log'. * The ebuild environment file is located at '/Users/pvarner/Library/Gentoo/var/tmp/portage/app-portage/gentoolkit-9999/temp/environment'. * Working directory: '/Users/pvarner/Library/Gentoo/var/tmp/portage/app-portage/gentoolkit-9999/work' * S: '/Users/pvarner/Library/Gentoo/var/tmp/portage/app-portage/gentoolkit-9999/work/gentoolkit-9999' >>> Failed to emerge app-portage/gentoolkit-9999, Log file: >>> '/Users/pvarner/Library/Gentoo/var/tmp/portage/app-portage/gentoolkit-9999/temp/build.log' $ mkdir /Users/pvarner/Library/Gentoo/usr/portage/distfiles/egit-src $ emerge -v1 gentoolkit These are the packages that would be merged, in order: Calculating dependencies... done! [ebuild R *] app-portage/gentoolkit-9999::gentoo [9999::gentoo_prefix] 0 kB Total: 1 package (1 reinstall), Size of downloads: 0 kB >>> Verifying ebuild manifests >>> Emerging (1 of 1) app-portage/gentoolkit-9999 >>> Unpacking source... Cloning into bare repository '/Users/pvarner/Library/Gentoo/usr/portage/distfiles/egit-src/gentoolkit.git'... /Users/pvarner/Library/Gentoo/usr/portage/distfiles/egit-src/gentoolkit.git: Operation not permitted * ERROR: app-portage/gentoolkit-9999 failed (unpack phase): * git-2_initial_clone: can't fetch from git://git.overlays.gentoo.org/proj/gentoolkit.git * * Call stack: * ebuild.sh, line 93: Called call-ebuildshell 'src_unpack' * environment, line 862: Called src_unpack * environment, line 4085: Called git-2_src_unpack * environment, line 1794: Called git-2_fetch * environment, line 1595: Called git-2_initial_clone * environment, line 1688: Called die * The specific snippet of code: * [[ -n ${EGIT_REPO_URI_SELECTED} ]] || die "${FUNCNAME}: can't fetch from ${EGIT_REPO_URI}" * * If you need support, post the output of `emerge --info '=app-portage/gentoolkit-9999'`, * the complete build log and the output of `emerge -pqv '=app-portage/gentoolkit-9999'`. * The complete build log is located at '/Users/pvarner/Library/Gentoo/var/tmp/portage/app-portage/gentoolkit-9999/temp/build.log'. * The ebuild environment file is located at '/Users/pvarner/Library/Gentoo/var/tmp/portage/app-portage/gentoolkit-9999/temp/environment'. * Working directory: '/Users/pvarner/Library/Gentoo/var/tmp/portage/app-portage/gentoolkit-9999/work' * S: '/Users/pvarner/Library/Gentoo/var/tmp/portage/app-portage/gentoolkit-9999/work/gentoolkit-9999' >>> Failed to emerge app-portage/gentoolkit-9999, Log file: >>> '/Users/pvarner/Library/Gentoo/var/tmp/portage/app-portage/gentoolkit-9999/temp/build.log'
I think the macos sandbox doesn't do addwrite, which is kind of nasty
@michael: do you have any idea how to support addwrite stuff? It seems sandbox-exec has nothing to change its permissions at runtime to me.
Created attachment 331582 [details, diff] fix mac os x sandboxing for git/svn sourced ebuilds Good news is that the mac os sandbox does not limit network access - it only affects file access. Bad news is that it currently limits write access to $PORTAGE_BUILDDIR which is usually something like $EPREFIX/var/tmp/portage and does not include $DISTDIR i.e. $EPEREFIX/usr/portage/distfiles. The attached patch makes it work for me[tm] by also allowing writing to $DISTDIR (taking into account that $DISTDIR is overridden with a fake distdir for builds that get their sources via git/svn). Thinking about this now, I'm not sure this actually is the cleanest possible solution since normally $DISTDIR does not need to be writeable during the unpack phase. Only when doing git/svn source fetching, it needs to be writeable when "unpacking" the sources. I think I'll take another crack at this. Anyway, can you try that this patch actually helps migitate the problem and tell me if it's a move in the right direction from your point of view? BTW: Applies cleanly to current prefix portage 21418 as well.
Applied the patch and tested an emerge of gentoolkit-9999 and it worked as expected.
Created attachment 331655 [details, diff] fix mac os x sandboxing for git/svn sourced ebuilds - take two This is my second approach which only allows write access to the actual DISTDIR if PORTAGE_ACTUAL_DISTDIR is set. It seems however, that PORTAGE_ACTUAL_DISTDIR is always set in phases install and unpack even when no git/svn eclass is in use. So the presence of this variable is not really a good indicator. Is there another way to detect that such an eclass is in use and will need write access to DISTDIR?
sandbox disabled on OSX
The fix I proposed works and can be applied as far as I'm concerned. It just is a bit overly permissive by always allowing access to DISTDIR even if the ebuild doesn't need to write there. I can live with that.
pushed, thanks
