Bug 434924 – dev-lang/go - static libraries in /usr/lib/go/pkg* with older mtimes than /usr/lib/go/src causes user-compiled packages to fail

• Product: Gentoo Linux
• Status: RESOLVED FIXED

[Description Jeff Mitchell 2012-09-13 16:34:42 UTC]

When Go is built, it compiles various built-in packages (from source in /usr/lib/go/src into static libraries in /usr/lib/go/pkg....).

When being merged, these static libraries can be merged after the source files. This causes the mtimes for the libraries to be earlier than that of the files, as seen here:

[12:15:02] jmitchell:~ $ stat /usr/lib/go/src/pkg/encoding/hex/hex.go 
  File: ‘/usr/lib/go/src/pkg/encoding/hex/hex.go’
  Size: 5062      	Blocks: 16         IO Block: 4096   regular file
Device: fd03h/64771d	Inode: 6052504     Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2012-08-02 15:11:40.000000000 -0400
Modify: 2012-08-02 15:11:40.000000000 -0400
Change: 2012-08-02 15:11:45.132957971 -0400
 Birth: -

[12:16:10] jmitchell:~ $ stat /usr/lib/go/pkg/linux_amd64/encoding/hex.a
  File: ‘/usr/lib/go/pkg/linux_amd64/encoding/hex.a’
  Size: 67548     	Blocks: 136        IO Block: 4096   regular file
Device: fd03h/64771d	Inode: 4593530     Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2012-08-02 15:11:38.000000000 -0400
Modify: 2012-08-02 15:11:38.000000000 -0400
Change: 2012-08-02 15:11:49.563924616 -0400
 Birth: -

The problem is that when you want to install packages as a non-root user, which is the only way recommended by the Go developers, "go get" will see that the used libraries are out-of-date and attempt to recompile them, which will fail due to missing permissions.

As a workaround, you can do "cd /usr/lib/go; find | sudo xargs touch" and things work. But this should be fixed in one way or another in the ebuild; either by updating the library timestamps after merging, or ensuring a specific merge order.

[Comment 1 Jeff Mitchell 2012-09-13 16:59:19 UTC]

I'm not sure why you changed the bug description, which was more correct earlier. The static libraries are coming from the package building and the mtimes from installation/merging; it's not just random static libraries having old mtimes.

And it's rather major as it pretty much makes Go unusable for non-root users  :-)

[Comment 2 William Hubbs 2012-09-13 20:56:39 UTC]

There are a couple of things that come to mind for this bug.

First, a non-root user can use go by setting GOPATH.
I would use something like this personally:

export GOPATH=/home/william/go:

If GOPATH is set, and you have write permission in the first directory
mentioned in GOPATH, things should work.

The question is, should I ewarn about it, or force a default setting?

The other issue is that this shouldn't be happening for things in
/usr/lib/go, so I will take a look at that as well.

[Comment 3 Jeff Mitchell 2012-09-14 13:35:30 UTC]

"If GOPATH is set, and you have write permission in the first directory
mentioned in GOPATH, things should work."

This is what you'd expect (and what I expected) but it's not correct. Go appears to try to build "updated" source files in GOROOT into GOROOT, so files with newer timestamps in /usr/lib/go/src/pkg don't get built into GOPATH, they get built into GOROOT -- which normal users do not have permissions for.

You should be able to see this for yourself quite easily by running touch on all of the files in /usr/lib/go/src/pkg. Then clear out some custom library (like mgo for instance) and try to go get it. It'll fetch the source into GOPATH but then try to rebuild the system libraries in GOROOT.

[Comment 4 William Hubbs 2013-04-16 07:38:07 UTC]

Jeff,

first, I'm sorry it has taken me this long to get back to this bug.

I'm slightly confused to be honest. Here is the code from pkg_postinst
that is supposed to take care of this issue.

	# If the go tool sees a package file timestamped older than a dependancy it
	# will rebuild that file.  So, in order to stop go from rebuilding lots of
	# packages for every build we need to fix the timestamps.  The compiler and
	# linker are also checked - so we need to fix them too.
	ebegin "fixing timestamps to avoid unnecessary rebuilds"
	tref="usr/lib/go/pkg/*/runtime.a"
	find "${ROOT}"usr/lib/go -type f \
		-exec touch -r "${ROOT}"${tref} {} \;
	eend $?

This is supposed to set the date/time of everything in /usr/lib/go to
match the file pointed to by tref. Does that not fix  the mtimes?

[Comment 5 Jeff Mitchell 2013-04-16 14:36:39 UTC]

Honestly, I'm not sure -- I haven't had an issue with this since I rebuilt the package. I don't know why the first time the package was merged in I had those wonky mtimes, though.

[Comment 6 William Hubbs 2013-04-16 15:30:31 UTC]

In that case, I'll go ahead and close this as resolved. However, feel
free to re-open if this happens again.