Long story short: # tar -xf stage3-amd64-20141204.tar.bz2 ./bin/ping # ls -lh bin/ping -rwx--x--x 1 root root 39K Dec 4 04:21 bin/ping # getcap bin/ping [empty] So the stage includes net-misc/iputils[filecaps] which means files are built for capabilities use and without setuid. However, the tarball doesn't include their caps so you get ping both without caps and without setuid.
And this is because fcaps.eclass has IUSE=+filecaps.
I've had ping fail for users other than root because of caps recently, which might have been caused by this. I'm wondering if we need to update the tar call to preserve caps.
I believe we need something along the following applied to catalyst: diff --git a/modules/catalyst_support.py b/modules/catalyst_support.py index 8112ed0..0e9faac 100644 --- a/modules/catalyst_support.py +++ b/modules/catalyst_support.py @@ -108,9 +108,9 @@ contents_map={ # 'find' is disabled because it requires the source path, which is not # always available #"find" :[calc_contents,"find %(path)s"], - "tar-tv":[calc_contents,"tar tvf %(file)s"], - "tar-tvz":[calc_contents,"tar tvzf %(file)s"], - "tar-tvj":[calc_contents,"tar -I lbzip2 -tvf %(file)s"], + "tar-tv":[calc_contents,"tar --xattrs --acls tvf %(file)s"], + "tar-tvz":[calc_contents,"tar --xattrs --acls tvzf %(file)s"], + "tar-tvj":[calc_contents,"tar --xattrs --acls -I lbzip2 -tvf %(file)s"], "isoinfo-l":[calc_contents,"isoinfo -l -i %(file)s"], # isoinfo-f should be a last resort only "isoinfo-f":[calc_contents,"isoinfo -f -i %(file)s"], I suspect we don't need --acls at this point, but I don't think it hurts and it could help in the future.
I think it would be actually safer to build with USE=-filecaps for now. This way the stage will work fine for people without capability/xattr support. People can rebuild iputils afterwards getting caps tested properly. Otherwise, there's still risk that the filecaps will be lost at unpack stage and effectively people will have 'broken' ping.
(In reply to Michał Górny from comment #4) > I think it would be actually safer to build with USE=-filecaps for now. This > way the stage will work fine for people without capability/xattr support. > People can rebuild iputils afterwards getting caps tested properly. > Otherwise, there's still risk that the filecaps will be lost at unpack stage > and effectively people will have 'broken' ping. We (releng) don't control the use flags, so unless you can convince maintainers to drop the IUSE defaults, we'll have to fix catalyst and update the docs. I'm already testing a patch to catalyst (committed to the 2.X branch), that should fix catalyst.
Well, that's a big problem since as I see it, the proper solution would be to use USE=-filecaps for stage build but default to USE=filecaps afterwards, since the flag depends on runtime capabilities of target filesystem. But I don't care that much.
I'd like to note that the Hardened project uses Xattrs to save the PaX marks sometimes (it currently defaults to writing them in the elf header but will change to xattrs by default) so building with USE=-filecaps does not help whereas adding xattr's to hte tarballs fixes any issues the hardened project would have too.
(In reply to Michał Górny from comment #6) > Well, that's a big problem since as I see it, the proper solution would be > to use USE=-filecaps for stage build but default to USE=filecaps afterwards, > since the flag depends on runtime capabilities of target filesystem. But I > don't care that much. As this has proven to be harder to fix than I expected, I'm going to disable USE="filecaps" for the default stages. We're also going to continue testing with catalyst to fix it there.
See also bug #536762 about tar --acls.
*** Bug 540842 has been marked as a duplicate of this bug. ***
commit b6ca4792761dde68e20a46fc259e8b9e88717d18 Author: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org> Date: Sun Feb 22 16:51:41 2015 +0000 Disable caps/filecaps for iputils - one more fix for bug 531788. Signed-off-by: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org> This together with some updates to get stages building again, should fix this issue as soon as we have new stages.
(In reply to Jorge Manuel B. S. Vicetto from comment #11) > commit b6ca4792761dde68e20a46fc259e8b9e88717d18 > Author: Jorge Manuel B. S. Vicetto (jmbsvicetto) <jmbsvicetto@gentoo.org> > Date: Sun Feb 22 16:51:41 2015 +0000 > > Disable caps/filecaps for iputils - one more fix for bug 531788. I do not think it is necessary to explicitly disable "caps" -- this just links agains libcap for caps manipulation at runtime. Stage tarball packing/unpacking should have no effect on this. All we care about here is "filecaps".
This change has added a few more issues we need to fix (providing non-empty /etc/portage/package* dirs), but the bug issues should be fixed now, so I'm going to close this bug.
I'd like to reopen this to track this bug to proper completion. The caps flag doesn't need to be turned off, and once we are building stages with catalyst-3 the -filecaps workaround will no longer be needed either.
I have just tested tar's behavior with xattrs (=app-arch/tar-1.29-r1) TL;DR To extract filecaps info from a tarball. Run tar with --xattrs-include='*' . tar supports --xattrs-include since 1.27 (year 2013) according to its NEWS file. ------ On creating archives: without --xattrs: Archive none of file's xattrs with --xattrs: Archive all file's xattrs On extracting archives: without --xattrs: Extract none of file's xattrs with --xattrs: Extract file's xattrs under user namespace (aka user.*) with --xattrs-include='*': Extract all file's xattrs archived Notes: 1. pax marks are under user namespace. So --xattrs fulfills hardened profile's need. Dump of my /usr/bin/python3.4 xattrs shows user.pax.flags="E" 2. tar(1) says --xattrs-include accepts a value of POSIX regexp. Actually it accepts globbing patterns. :'( 3. when --xattrs-include present, --xattrs seems redundant. Source can be found at src/xattrs.c:xattrs_kw_included() ------ Test script #1: (Preserving full $PS1 to avoid confusion with getfattr's output) localhost tmp # tar -xf test.tar --xattrs localhost tmp # getfattr -dm. test # file: test user.xiami="user" localhost tmp # tar -xf test.tar --xattrs-include='*' localhost tmp # getfattr -dm. test # file: test security.xiami="security" trusted.xiami="trusted" user.xiami="user" Test script #2: # tar -xf ping.tar --xattrs # getcap ping (empty) # tar -xf ping.tar --xattrs-include 'security.*' # getcap ping ping = cap_net_raw+ep
catalyst3 uses pyDeComp, a lib I created to handle all different kinds of tarballs while maintaining a simple common python interface. see the xattrs options here: https://github.com/dol-sen/pyDeComp/blob/master/DeComp/definitions.py#L70 If you recommend setting them differently, than please make a pull request, and I'll make another release.
Emmm.. I mean we can store file capabilities into stage3 tarball and guide users to extract them correctly by adding --xattrs-include='*' to Handbook/Installation.
We can/should get the proper xattrs-include option added to the handbook asap. As soon as we get a catalyst-3 on the releng autobuild setup, the xattrs/caps will make it to the tarball, and things will work.
Catalyst-3.0.0 and pydecomp-0.2 have been released. These have all the required code to include xattrs.
A quick update-- we have the right handbook command, we have catalyst support. All that's left to do is remove the package.use entry/entries disabling this.
...except it is infinitely silly to have stages that are broken on systems that do not have capabilities enabled.
That's a valid point, but then isn't it infinitely silly to have profile & package defaults turning this on?
I should be a bit more constructive. If we're happy with the status quo where stage3 is shipped with packages having filecaps OFF, but with a pending USE change that will turn them back on @ next world upgrade, I'm ok with that and we can close this bug. I assumed we wanted to get filecaps out there on the stage3 itself.
I don't have a strong opinion on what the default should be. However, the difference is that USE=filecaps involves a fallback if the system doesn't support them whereas stage3 unpacking has no clear fallback and involves straightaway broken system.
That itself might be enough reason to maintain the -filecaps setting longterm for stage3 builds, even globally like bindist, and not consider it to be a temporary workaround. That is, to let the filecaps introduction happen later where the ebuild/eclass/PM logic can make sure it's handled properly.
(In reply to Ben Kohler from comment #23) > I should be a bit more constructive. If we're happy with the status quo > where stage3 is shipped with packages having filecaps OFF, but with a > pending USE change that will turn them back on @ next world upgrade, I'm ok > with that and we can close this bug. > > I assumed we wanted to get filecaps out there on the stage3 itself. I think until caps get more "mainstream" support from tar and friends, we should keep things as they are.
(In reply to Ben Kohler from comment #23) > If we're happy with the status quo > where stage3 is shipped with packages having filecaps OFF, but with a > pending USE change that will turn them back on @ next world upgrade, I'm ok > with that and we can close this bug. And that's how it is for ages now. :)
