Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 512780 - app-emulation/qemu - files/qemu-binfmt.initd-r1 sets P flag which confuses qemu
Summary: app-emulation/qemu - files/qemu-binfmt.initd-r1 sets P flag which confuses qemu
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Gentoo QEMU Project
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-06-08 21:42 UTC by Bertrand Jacquin
Modified: 2014-08-04 06:45 UTC (History)
2 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Bertrand Jacquin 2014-06-08 21:42:26 UTC
The init script provided by app-emulation/qemu, /etc/init.d/qemu-binfmt, can be used to define the needed prerequisites to be able to launch other arch (supported by QEmu) binaries like and so for example by chroot, bash etc.

With a qemu is builded with USE="static static-softmmu static-user" QEMU_SOFTMMU_TARGETS="arm i386 x86_64" QEMU_USER_TARGETS="arm i386 x86_64" the binary /usr/bin/qemu-arm is installed.

Then you can launch for example armhf binaries

# /etc/init.d/qemu-binfmt start
# /usr/bin/qemu-arm -L /data/build/armhf /data/build/armhf/bin/uname  -a
Linux otis.scabb 3.10.41-longterm #1 SMP Sat Jun 7 17:01:06 CEST 2014 armv7l armv7l armv7l GNU/Linux

But you cannot use chroot :

# chroot /data/build/armhf /bin/bash
/bin/bash: /bin/bash: cannot execute binary file

This is because the qemu-binfmt helper script define the flag 'P' (last position) :

  if [ $cpu != "arm" -a -x "/usr/bin/qemu-armeb" ] ; then
    echo ':armeb:M::\x7fELF\x01\x02\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff:/usr/bin/qemu-armeb:P' > /proc/sys/fs/binfmt_misc/register
  fi

That flag is documented by the kernel (https://www.kernel.org/doc/Documentation/binfmt_misc.txt) as :

 - 'flags' is an optional field that controls several aspects of the invocation
   of the interpreter. It is a string of capital letters, each controls a certain
   aspect. The following flags are supported -
      'P' - preserve-argv[0].  Legacy behavior of binfmt_misc is to overwrite the
            original argv[0] with the full path to the binary.  When this flag is
            included, binfmt_misc will add an argument to the argument vector for
            this purpose, thus preserving the original argv[0].
      'O' - open-binary. Legacy behavior of binfmt_misc is to pass the full path
            of the binary to the interpreter as an argument. When this flag is
            included, binfmt_misc will open the file for reading and pass its
            descriptor as an argument, instead of the full path, thus allowing
            the interpreter to execute non-readable binaries. This feature should
            be used with care - the interpreter has to be trusted not to emit
            the contents of the non-readable binary.
      'C' - credentials. Currently, the behavior of binfmt_misc is to calculate
            the credentials and security token of the new process according to
            the interpreter. When this flag is included, these attributes are
            calculated according to the binary. It also implies the 'O' flag.
            This feature should be used with care as the interpreter
            will run with root permissions when a setuid binary owned by root
            is run with binfmt_misc.

When using flag 'OC', everything is then correct. This is the way Debian and Ubuntu use it.

# /usr/bin/qemu-arm -L /data/build/armhf /data/build/armhf/bin/uname  -a
Linux otis.scabb 3.10.41-longterm #1 SMP Sat Jun 7 17:01:06 CEST 2014 armv7l armv7l armv7l GNU/Linux
# chroot /data/build/armhf /bin/bash
root@localhost:/# 

That 'P' flag is present to all arch define in init script

Should we change this ?

Reproducible: Always
Comment 1 Bertrand Jacquin 2014-06-08 23:32:23 UTC
@jer, ARM is just an example here, it's the same for every arch
Comment 2 Steev Klimaszewski (RETIRED) gentoo-dev 2014-06-14 23:42:34 UTC
I too am running into this issue, and it's killing me because qemu-user was removed from the tree.

To replicate:
echo 'QEMU_USER_TARGETS="aarch64 arm"' >> /etc/portage/make.conf
echo "app-emulation/qemu static-user" >> /etc/portage/package.use/qemu

emerge debootstrap qemu

/etc/init.d/qemu-binfmt start

cd tmp
debootstrap --verbose --arch armel --variant=minbase --foreign squeeze rootfs/ http://ftp.debian.org/debian
mkdir -p rootfs/dev/pts
mount -t devpts devpts rootfs/dev/pts
mount -t proc proc rootfs/proc
cp /usr/bin/qemu-arm rootfs/usr/bin
chroot rootfs

will then fail with

/bin/bash: /bin/bash: cannot execute binary file

So something is missing, and qemu with static-user is NOT a drop-in replacement for qemu-user as was claimed in the mask message.

Completely aside from this, could we also maybe move towards the same as other distributions and name the file qemu-arm-static to differentiate?
Comment 3 Steev Klimaszewski (RETIRED) gentoo-dev 2014-06-14 23:54:55 UTC
And, I'm unsure why exactly, but when switching to OC as Bertrand suggests, I can no longer launch new processes on my machines, so OC doesn't seem to be 100% correct either.
Comment 4 Joakim Tjernlund 2014-07-15 14:05:11 UTC
Check my patch to qemu-user in bug:
https://bugs.gentoo.org/show_bug.cgi?id=516454

Should sort out this, just don't forget to change binfmt flag from P to PO
too.
Comment 5 Nathan Shearer 2014-07-25 21:12:10 UTC
I can confirm that this bug still exists. I was trying to chroot into an arm environment after following http://wiki.gentoo.org/wiki/Crossdev_qemu-static-user-chroot and http://www.gentoo.org/proj/en/base/embedded/handbook/?part=1&chap=5

I would get the same error, and modifying the /etc/init.d/qemu-binfmt script to use OC instead of P fixed it for me.
Comment 6 Joakim Tjernlund 2014-08-02 15:18:22 UTC
(In reply to Nathan Shearer from comment #5)
> I can confirm that this bug still exists. I was trying to chroot into an arm
> environment after following
> http://wiki.gentoo.org/wiki/Crossdev_qemu-static-user-chroot and
> http://www.gentoo.org/proj/en/base/embedded/handbook/?part=1&chap=5
> 
> I would get the same error, and modifying the /etc/init.d/qemu-binfmt script
> to use OC instead of P fixed it for me.

Removing P is a fragile fix. Some apps require argv0 to be diffrent.
You are better off using the patch in bug 516454 and fixing the 
binfmt flag to be PO instead.
Comment 7 SpanKY gentoo-dev 2014-08-04 06:42:46 UTC
there isn't any info in this bug that covers the lower level problems, so i'll summarize them here

binfmt_misc passes the full resolved path of the emulated program.  this way the interp (qemu-xxx) can open & run it.  the problem is that the original argv[0] value is lost, so when you run something like `ls`, the ls program should be given argv[0]="ls", but instead it receives argv[0]="/bin/ls".

when the P flag is enabled, the kernel will pass along the original argv[0].  this way the interp can open the file it needs to, and then before calling the program, swap back in the original argv[0].  that means the qemu interp receives both "/bin/ls" and "ls".

unfortunately, qemu doesn't know about this additional thing, so it passes along both to the target program leading to confusing output.  when you run `ls`, you get back:
  $ ls
  /bin/ls: cannot access ls: No such file or directory
that's because it behaves as if you ran it like so:
  $ /bin/ls ls

bug 516454 is not a blocker as it affects few programs and is not a new issue.  we can resolve that w/out making things work now.

the O flag is not needed normally, but it does make execution work when it might otherwise not (per the documentation wrt unreadable files by the user).  the security issues i don't think matter in these cases as the loader is trusted (we emerged it after all).

the C flag falls into the same bucket.  if you're trusting qemu to chroot somewhere, then you're also trusting the capabilities in there.
Comment 8 SpanKY gentoo-dev 2014-08-04 06:45:27 UTC
should be all set now in the tree; thanks for the report!

Commit message: Add aarch64 to the init script, and switch flags from P to OC until qemu itself can understand the extra argv[0]
http://sources.gentoo.org/app-emulation/qemu/files/qemu-binfmt.initd-r1?r1=1.3&r2=1.4
http://sources.gentoo.org/app-emulation/qemu/qemu-2.1.0.ebuild?rev=1.1