Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 812977 - go-module.eclass: fails to clean dependencies folder on Gentoo Prefix
Summary: go-module.eclass: fails to clean dependencies folder on Gentoo Prefix
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Eclasses (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: William Hubbs
URL:
Whiteboard:
Keywords:
: 800872 (view as bug list)
Depends on:
Blocks:
 
Reported: 2021-09-13 20:06 UTC by Raffaele
Modified: 2021-09-23 21:13 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Patch to make the go dependencydirectory writable during the installation (go-modules.eclass.patch,573 bytes, patch)
2021-09-16 19:42 UTC, Raffaele
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Raffaele 2021-09-13 20:06:47 UTC
Whenever Gento prefix is bing used, any attempt at installing a go package based on `go-module.eclass` will fail while attempting to remove the dependencies folder (AKA "${HOME}"/go/pkg/mod/).

The reason lies in this upstream issue: https://github.com/golang/go/issues/27161
This was initially reported here, as a bug with an ebuild. Link: https://bugs.gentoo.org/800872

If this is confirmed to happen on any Gentoo Prefix installation, the recommended solution is to use `-modcachere` as described in the upstream issue.
Possibly, by enabling the flag only if it is a prefix.

Reproducible: Always

Steps to Reproduce:
1. Run Gentoo according to Gentoo Prefix https://wiki.gentoo.org/wiki/Project:Prefix
2. Install a package that depends on go-modules.eclass. For instance, app-shells/fzf: `emerge -v app-shells/fzf` 
Actual Results:  
```
>>> Installing (1 of 1) app-shells/fzf-0.27.2::gentoo
Process Process-3:
Traceback (most recent call last):
  File "/home/xxxxx/gentoo/usr/lib/python3.9/site-packages/portage/package/ebuild/prepare_build_dirs.py", line 41, in prepare_build_dirs
    shutil.rmtree(clean_dir)
  File "/home/xxxxx/gentoo/usr/lib/python3.9/site-packages/portage/__init__.py", line 231, in __call__
    rval = self._func(*wrapped_args, **wrapped_kwargs)
  File "/home/xxxxx/gentoo/usr/lib/python3.9/shutil.py", line 718, in rmtree
    _rmtree_safe_fd(fd, path, onerror)
  File "/home/xxxxx/gentoo/usr/lib/python3.9/shutil.py", line 655, in _rmtree_safe_fd
    _rmtree_safe_fd(dirfd, fullname, onerror)
  File "/home/xxxxx/gentoo/usr/lib/python3.9/shutil.py", line 655, in _rmtree_safe_fd
    _rmtree_safe_fd(dirfd, fullname, onerror)
  File "/home/xxxxx/gentoo/usr/lib/python3.9/shutil.py", line 655, in _rmtree_safe_fd
    _rmtree_safe_fd(dirfd, fullname, onerror)
  [Previous line repeated 3 more times]
  File "/home/xxxxx/gentoo/usr/lib/python3.9/shutil.py", line 675, in _rmtree_safe_fd
    onerror(os.unlink, fullname, sys.exc_info())
  File "/home/xxxxx/gentoo/usr/lib/python3.9/shutil.py", line 673, in _rmtree_safe_fd
    os.unlink(entry.name, dir_fd=topfd)
PermissionError: [Errno 13] Permission denied: 'grapheme.go'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/xxxxx/gentoo/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/home/xxxxx/gentoo/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/xxxxx/gentoo/usr/lib/python3.9/site-packages/portage/util/_async/ForkProcess.py", line 156, in _bootstrap
    sys.exit(self._run())
  File "/home/xxxxx/gentoo/usr/lib/python3.9/site-packages/portage/dbapi/_MergeProcess.py", line 193, in _run
    rval = mylink.merge(self.pkgloc, self.infloc,
  File "/home/xxxxx/gentoo/usr/lib/python3.9/site-packages/portage/dbapi/vartree.py", line 1752, in wrapper
    return f(self, *args, **kwargs)
  File "/home/xxxxx/gentoo/usr/lib/python3.9/site-packages/portage/dbapi/vartree.py", line 5372, in merge
    retval = self.treewalk(mergeroot, myroot, inforoot, myebuild,
  File "/home/xxxxx/gentoo/usr/lib/python3.9/site-packages/portage/dbapi/vartree.py", line 4229, in treewalk
    prepare_build_dirs(settings=self.settings, cleanup=cleanup)
 File "/home/xxxxx/gentoo/usr/lib/python3.9/site-packages/portage/package/ebuild/prepare_build_dirs.py", line 53, in prepare_build_dirs
    _raise_exc(oe)
  File "/home/xxxxx/gentoo/usr/lib/python3.9/site-packages/portage/util/install_mask.py", line 150, in _raise_exc
    raise wrapper
portage.exception.PermissionDenied: [Errno 13] Permission denied: 'grapheme.go'

>>> Failed to install app-shells/fzf-0.27.2, Log file:

>>>  '/home/xxxxx/gentoo/var/tmp/portage/app-shells/fzf-0.27.2/temp/build.log'
```

Expected Results:  
Installation successful
Comment 1 Yuan Liao (Leo3418) 2021-09-14 06:33:37 UTC
I can reproduce this bug on two Prefix installations:
- On CentOS 8.4 x86_64 w/ kernel 4.18
- On Ubuntu 20.04 aarch64 w/ kernel 5.11

And with three packages:
- net-misc/rclone
- net-p2p/syncthing
- net-vpn/tailscale
Comment 2 Raffaele 2021-09-16 19:42:36 UTC
Created attachment 739614 [details, diff]
Patch to make the go dependencydirectory writable during the installation

```
diff --git a/eclass/go-module.eclass b/eclass/go-module.eclass
index d1e81babf1f..101eee5389f 100644
--- a/eclass/go-module.eclass
+++ b/eclass/go-module.eclass
@@ -385,7 +385,7 @@ _go-module_src_unpack_verify_gosum() {
        # This will print 'downloading' messages, but it's accessing content from
        # the $GOPROXY file:/// URL!
        einfo "Tidying go.mod/go.sum"
-       go mod tidy >/dev/null
+       go mod tidy -modcacherw >/dev/null

        # This used to call 'go get' to verify by fetching everything from the main
        # go.mod. However 'go get' also turns out to recursively try to fetch
```

This should fix it. Thanks Chang Liu for his investigation in https://bugs.gentoo.org/800872
Comment 3 Raffaele 2021-09-21 18:27:34 UTC
*** Bug 800872 has been marked as a duplicate of this bug. ***
Comment 4 Larry the Git Cow gentoo-dev 2021-09-23 21:13:57 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=13e742b871c4a90a7545545cd0a04cc838b938ed

commit 13e742b871c4a90a7545545cd0a04cc838b938ed
Author:     William Hubbs <williamh@gentoo.org>
AuthorDate: 2021-09-23 21:09:39 +0000
Commit:     William Hubbs <williamh@gentoo.org>
CommitDate: 2021-09-23 21:10:23 +0000

    go-modules.eclass: allow the module cache to be read/write
    
    This allows the modules to be cleaned up on prefix systems.
    
    More information is available here:
    https://github.com/golang/go/issues/27161
    
    I would like to thank Raffaele Spinelli <rafspiny@gmail.com> for the
    original patch this was based on.
    
    Closes: https://bugs.gentoo.org/812977
    Closes: https://github.com/gentoo/gentoo/pull/22365
    
    Signed-off-by: William Hubbs <williamh@gentoo.org>

 eclass/go-module.eclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)