Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 544150 - app-emulation/qemu is missing binfmt registration files in /usr/lib/binfmt.d
Summary: app-emulation/qemu is missing binfmt registration files in /usr/lib/binfmt.d
Status: IN_PROGRESS
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Current packages (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: John Helmert III
URL:
Whiteboard:
Keywords: PullRequest
Depends on: 545162
Blocks:
  Show dependency tree
 
Reported: 2015-03-22 21:18 UTC by Łukasz Stelmach
Modified: 2022-09-23 07:44 UTC (History)
9 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
Implement qemu-binfmt for systemd (f1c8e99b8e25619f9fa2e7c800fa1af9953101e8.patch,31.84 KB, patch)
2016-04-04 22:30 UTC, Mike Gilbert
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Łukasz Stelmach 2015-03-22 21:18:48 UTC
The package provides registration of binfmt handlers only in the init script /etc/init.d/qemu-binfmt. There are no files in /etc/binfmt.d to use with systemd-binfmt.

http://www.freedesktop.org/software/systemd/man/systemd-binfmt.html
http://www.freedesktop.org/software/systemd/man/binfmt.d.html

Reproducible: Always
Comment 1 Łukasz Stelmach 2015-03-22 21:19:59 UTC
Actually the package should provide files in /usr/lib/binfmt.d/
Comment 2 Mike Gilbert gentoo-dev 2015-03-26 01:11:09 UTC
Do any other distros do this?

This comment in the init script gives me pause:

# register the interpreter for each cpu except for the native one
Comment 3 Alexander Tsoy 2015-03-26 01:17:20 UTC
(In reply to Mike Gilbert from comment #2)
> Do any other distros do this?

Sure.

http://pkgs.fedoraproject.org/cgit/qemu.git/tree/qemu.spec#n988
Comment 4 Alexander Tsoy 2015-03-26 01:18:57 UTC
(In reply to Alexander Tsoy from comment #3)

And %{SOURCE1} in the spec is the following file:
http://pkgs.fedoraproject.org/cgit/qemu.git/tree/qemu.binfmt
Comment 5 SpanKY gentoo-dev 2015-03-28 04:17:43 UTC
i've specifically not installed binfmt.d.  i'm not sure it's a good idea to automatically register all these formats merely by installing the package.  it's why we have the init script.
Comment 6 Łukasz Stelmach 2015-03-28 09:41:42 UTC
How about installing only files for architectures the package has been configured to support? That is, more or less, moving all these ifs from the initscript to ebuild.
Comment 7 SpanKY gentoo-dev 2015-03-28 17:12:08 UTC
(In reply to Łukasz Stelmach from comment #6)

that would make no difference to my concerns
Comment 8 Łukasz Stelmach 2015-03-28 21:10:25 UTC
Forgive me but I do not exactly follow you. Let me present the situation the way I see it.

Today we've got /etc/init.d/qemu-binfmt.

Which probes cpu

   cpu=`uname -m`
   case "$cpu" in
   [...]
   esac

and then registers handlers depending on what's been installed

   if [ $cpu != "i386" -a -x "/usr/bin/qemu-i386" ] ; then
      echo ':i386:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x03\x00:\xff\xff\
      echo ':i486:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x06\x00:\xff\xff\
   fi
   [...]

One can disable a handler registration by disabling executable bit on the qemu binary. With binfmt.d you move the first two steps from the qemu-binfmt script to ebuild's src_install(). According to binfmt.d(5) disabling a handler isn't any harder than today. In a way it is even better because it does not involve any modification to /usr/* files but rather a creation of a link from /etc/binfmt.d/foo.conf to /dev/null[1]. On OpenRC systems the qemu-binfmt can use the files from binfmt.d the same way systemd-binfmt does.

What is the problem I am missing?

[1] https://www.freedesktop.org/software/systemd/man/binfmt.d.html
Comment 9 SpanKY gentoo-dev 2015-03-29 06:04:21 UTC
(In reply to Łukasz Stelmach from comment #8)

Gentoo, by policy, does not enable things by default.  the reason it's an init script is because the user must *opt in* to having this be setup for their system.

binfmt.d violates that.
Comment 10 Łukasz Stelmach 2015-03-29 10:09:15 UTC
Thank you, now I understand the problem. There is quite an easy solution though. Do not install /usr/lib/systemd/system/sysinit.target.wants/systemd-binfmt.service in the systemd package by default to disable systemd-binfmt altogether. Do you want me to file a bug about it?
Comment 11 Mike Gilbert gentoo-dev 2015-03-29 14:08:19 UTC
(In reply to SpanKY from comment #9)
> (In reply to Łukasz Stelmach from comment #8)
> 
> Gentoo, by policy, does not enable things by default.  the reason it's an
> init script is because the user must *opt in* to having this be setup for
> their system.
> 
> binfmt.d violates that.

Where is this policy documented?

Anyway, I think they should be treated just like tmpfiles.d, logrotate, or cron.d files: small drop-in config files that are used at runtime by optional services.

The user is able to prevent binfmt.d files from being loaded by disabling the relevant systemd service.

That systemd enables this service by default should really have no bearing on whether the qemu package installs these files. That's a separate issue, and if you disagree with upstream's default, we can deal with it in a separate bug.
Comment 12 SpanKY gentoo-dev 2015-03-29 16:05:23 UTC
(In reply to Mike Gilbert from comment #11)

i don't know where it's documented, but it's been that way as long as i've been here.  the act of installing a package does not imply that package should automatically be activated.

if any systemd aspect is automatically turning on all binfmt fragments that are installed, then i'm not moving forward with qemu.  you can argue it's orthogonal, but i'm not going to negatively impact user's systems.
Comment 13 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-03-29 16:16:15 UTC
Vapier, I'd really appreciate if you started seeing Gentoo as a collaborative effort of our community instead of the tiny thing you imagine in your head and bend as you see fit. 'Being that way' in your head doesn't make it a valid policy. If you believe there is a policy in place, find it. If you want a policy in place, follow the proper procedure. Otherwise, stop forcing everyone to think like you do.
Comment 14 SpanKY gentoo-dev 2015-03-29 16:48:49 UTC
(In reply to Michał Górny from comment #13)

i'm just going to ignore your trolling
Comment 15 Mike Gilbert gentoo-dev 2015-03-29 17:57:00 UTC
(In reply to SpanKY from comment #12)
> (In reply to Mike Gilbert from comment #11)
> 
> i don't know where it's documented, but it's been that way as long as i've
> been here.  the act of installing a package does not imply that package
> should automatically be activated.

udev rules have always worked this way.

> if any systemd aspect is automatically turning on all binfmt fragments that
> are installed, then i'm not moving forward with qemu.  you can argue it's
> orthogonal, but i'm not going to negatively impact user's systems.

What is the negative impact? Presumably the user has installed qemu to be able to run binaries for some other CPU, this just makes it easier to do so.

Would a USE flag be an acceptable alternative?
Comment 16 Mike Gilbert gentoo-dev 2015-03-30 22:32:39 UTC
Another alternative to a USE flag would be to install the binfmt.d files in a separate ebuild, like app-emulation/qemu-binfmt. That would let people install it without having to rebuild qemu.

They could also select which binfmt entries they want as a subset of the QEMU_USER_TARGETS setting on app-emulation/qemu.
Comment 17 Mike Gilbert gentoo-dev 2015-03-30 22:49:57 UTC
Hell, we could even just install the binfmt with a different file extension and let the user rename it if they want to use it.
Comment 18 Anthony Basile gentoo-dev 2015-03-30 22:58:56 UTC
(In reply to Mike Gilbert from comment #17)
> Hell, we could even just install the binfmt with a different file extension
> and let the user rename it if they want to use it.

Let's do that and not introduce another ebuild or extra flags.  And a little documentation on how to rename them to enable them.
Comment 19 Mike Gilbert gentoo-dev 2015-03-30 23:12:17 UTC
(In reply to Anthony Basile from comment #18)

Ok. Someone will need to write some ebuild code to generate an appropriate file based on ARCH or CHOST I suppose.
Comment 20 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2015-03-31 04:58:18 UTC
Sounds like you want people to rename them on every upgrade...
Comment 21 Łukasz Stelmach 2015-03-31 06:32:36 UTC
(In reply to Mike Gilbert from comment #17)
> Hell, we could even just install the binfmt with a different file extension
> and let the user rename it if they want to use it.

This is a bad idea. systemd-binfmt, as well as all the other parts the systemd suite adhere to a certain file location scheme, which is described in binfmt.d(5) too.

    Files in /etc/ override files with the same name in /usr/lib/ 
    and /run/. Files in /run/ override files with the same name
    in /usr/lib/.  Packages should install their configuration
    files in /usr/lib/, files in /etc/ are reserved for the
    local administrator, who may use this logic to override the
    configuration files installed from vendor packages.

In general /usr is supposed to be read-only (or at capable of being so) and /etc is the place to put machine-local configuration. Thus: binfmt files in /usr/lib/binfmt.d MUST match *.conf pattern.

If you don't want them to be activated by default disable systemd-binfmt.service (see my comment #10).
Comment 22 SpanKY gentoo-dev 2015-03-31 06:50:48 UTC
(In reply to SpanKY from comment #12)

in case people are misreading this, i am not saying "it's been that way as long as i've been here" as in "since i joined, i created this policy by myself and forced people to follow it".  the policy predates me.

(In reply to Mike Gilbert from comment #15)

negative impacts:

 - increased exposure to security problems: with non-native ELFs, you can now attempt to exploit bugs in qemu and not just the ELF itself.  qemu-the-user is not nearly as secure as qemu-the-system and it's fairly trivial to trigger crashes by accident (try using pthreads) or on purpose (qemu-the-user shares memory spaces with the executable code).  you might argue this is a bit far-fetched, but we had the same exact problem with wine in the past -- malicious EXEs sent via mail were directly executable.  i don't recall the bug (and a quick search doesn't turn it up), but we had to undo some behavior from upstream.  you could say that by users opting in they still expose themselves, but there's only so much we can do, and at that point the user has decided the tradeoffs for themselves.

 - cross-compiling gets weird.  configure scripts/etc... detect cross-compiling basically via `gcc test.c && ./a.out` and that now passes.  you could say that it's what the user wants, but i don't think it's what all users want all the time even: there are weird edge cases where qemu doesn't behave like native code leading to build-time tests getting the wrong answer, and different arches vary greatly in quality.  while some might be pretty solid, others are anything but.  even common syscalls are straight up not supported (signalfd), and trying to use common libs (like pthreads) leads to crashes.  it's how things like scratchbox work, but those are frameworks users are actively enabling and generally understand why things are going wrong.

(In reply to Mike Gilbert from comment #16)

i'm not digging any of those options.  does the binfmt.d systemd service allow you to control exactly which formats are enabled ?  if we changed the default there to not enable any at all and put a big warning/explanation, then we could freely install fragments into the dir from any package.  since users have to turn on the filter, we've satisfied the not-enable-by-default requirement.
Comment 23 Mike Gilbert gentoo-dev 2015-03-31 12:41:44 UTC
(In reply to Michał Górny from comment #20)
> Sounds like you want people to rename them on every upgrade...

A symlink would solve that.
Comment 24 Mike Gilbert gentoo-dev 2015-03-31 12:56:09 UTC
(In reply to SpanKY from comment #22)
> in case people are misreading this, i am not saying "it's been that way as long as i've been here" as in "since i joined, i created this policy by myself and forced people to follow it".  the policy predates me.

The "policy" almost certainly pre-dates binfmt.d as well.

It applies to things like init scripts and daemons, not drop-in configuration files for system services. See my previous examples of drop-in configs.

Now that you have explained the downsides, I understand why this might not be a good thing for everyone. But please don't push this bunk "policy" argument. Security and cross-compile issues are a much more compelling explanation.

> i'm not digging any of those options.

I think we have basically exhausted the options, though I added another one below. Please pick your poison, or come up with something better.

> does the binfmt.d systemd service
> allow you to control exactly which formats are enabled ?  if we changed the
> default there to not enable any at all and put a big warning/explanation,
> then we could freely install fragments into the dir from any package.  since
> users have to turn on the filter, we've satisfied the not-enable-by-default
> requirement.

This is how binfmt.d was designed to operate:

The distro/upstream software puts fragments into /usr/lib/binfmt.d. They are enabled by virtue of existing in that directory with a name like *.conf.

The sysadmin can override the fragments by placing a file of the same name in /etc/binfmt.d. If the override file is empty (or a symlink to /dev/null), the fragment is effectively disabled.

So, options for Gentoo:

We could drop an empty file in /etc/binfmt.d to disable it by default, but that would get replaced on every upgrade.

We could install a commented-out file in /etc/binfmt.d where it would be config protected. That would go against the upstream spec, which says "Packages should install their configuration files in /usr/lib/. Files in /etc/ are reserved for the local administrator". Despite that, I think this might be the best compromise for Gentoo at the moment.
Comment 25 Mike Gilbert gentoo-dev 2015-03-31 12:58:19 UTC
(In reply to Łukasz Stelmach from comment #21)
> If you don't want them to be activated by default disable
> systemd-binfmt.service (see my comment #10).

That disables all binfmt fragments, including those that are installed by other packages. I'm not ready to go there.
Comment 26 Mike Gilbert gentoo-dev 2015-03-31 13:10:33 UTC
(In reply to SpanKY from comment #22)
> does the binfmt.d systemd service allow you to control exactly which formats are enabled ?

To more directly address this: No, systemd-binfmt does not have any method to "filter" the formats beyond the /etc/binfmt.d override mechanic I have previously described.
Comment 27 William Hubbs gentoo-dev 2015-03-31 14:30:44 UTC
Note that as of bug #382723, OpenRC has support for binfnt.d built into
the procfs service, so technically qemu doesn't need an init script to
set this up.
Comment 28 SpanKY gentoo-dev 2015-03-31 16:54:10 UTC
(In reply to Mike Gilbert from comment #25)

... and those packages are already violating things.  the fact that it was noticed for qemu is irrelevant.

(In reply to William Hubbs from comment #27)

should be easy to update openrc to include a filter and have it off by default.
Comment 29 Mike Gilbert gentoo-dev 2015-03-31 17:17:56 UTC
(In reply to SpanKY from comment #28)
> (In reply to Mike Gilbert from comment #25)
> 
> ... and those packages are already violating things.  the fact that it was
> noticed for qemu is irrelevant.

If you want people to follow a policy, it helps to have it written down so that people know it exists.

I think this idea of not enabling anything by default does not make sense in many cases. But there's no way for me to discuss a change to it if I have no written policy to cite.
Comment 30 Mike Gilbert gentoo-dev 2015-04-01 00:43:02 UTC
Another thing that a few people have suggested is an eselect module to manage this.

In that case, I think we would ignore /usr/lib/binfmt.d, and just have the eselect module add/remove links in /etc/binfmt.d.
Comment 31 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2015-04-01 12:53:28 UTC
(In reply to Mike Gilbert from comment #29)
> (In reply to SpanKY from comment #28)
> > (In reply to Mike Gilbert from comment #25)
> > 
> > ... and those packages are already violating things.  the fact that it was
> > noticed for qemu is irrelevant.
> 
> If you want people to follow a policy, it helps to have it written down so
> that people know it exists.
> 
> I think this idea of not enabling anything by default does not make sense in
> many cases. But there's no way for me to discuss a change to it if I have no
> written policy to cite.

This policy *does* make sense.
This is another case where systemd upstream thinks they know better than the system administrators or that any system where systemd is installed is under their control.
I'm sure there are many Gentoo installs around where this "automatic enabling" is anything but desired.
Comment 32 Mike Gilbert gentoo-dev 2015-04-01 13:06:38 UTC
(In reply to Jorge Manuel B. S. Vicetto from comment #31)
> This policy *does* make sense.
> This is another case where systemd upstream thinks they know better than the
> system administrators or that any system where systemd is installed is under
> their control.
> I'm sure there are many Gentoo installs around where this "automatic
> enabling" is anything but desired.

I agree that it probably does not make sense to enable the binfmt.d stuff automatically.

However, there is no *policy* that prevents a maintainer from doing so.
Comment 33 William Hubbs gentoo-dev 2015-04-04 22:16:56 UTC
(In reply to Jorge Manuel B. S. Vicetto from comment #31)
> (In reply to Mike Gilbert from comment #29)
> > (In reply to SpanKY from comment #28)
> > > (In reply to Mike Gilbert from comment #25)
> > > 
> > > ... and those packages are already violating things.  the fact that it was
> > > noticed for qemu is irrelevant.
> > 
> > If you want people to follow a policy, it helps to have it written down so
> > that people know it exists.
> > 
> > I think this idea of not enabling anything by default does not make sense in
> > many cases. But there's no way for me to discuss a change to it if I have no
> > written policy to cite.
> 
> This policy *does* make sense.
> This is another case where systemd upstream thinks they know better than the
> system administrators or that any system where systemd is installed is under
> their control.
> I'm sure there are many Gentoo installs around where this "automatic
> enabling" is anything but desired.

The way you disable these is put links in /etc/binfmt.d with the same names as the files in /usr/lib/binfmt.d, but link them to /dev/null, for example, if /usr/lib/binfmt.d/foo exists, and you want to ignore it, run this command:

ln -s /dev/null /etc/binfmt.d/foo

The files in /etc/binfmt.d/* should be controlled by the sys admin, not the package manager.

Also,  /etc/tmpfiles.d works the same way.

If you want an eselect module, it should add and remove links in /etc/binfmt.d based on what it sees in /usr/lib/binfmt.d.

So, the files should be installed by default in /usr/lib/binfmt.d, then the eselect module should take over from there.
Comment 34 Jorge Manuel B. S. Vicetto Gentoo Infrastructure gentoo-dev 2015-04-05 02:39:40 UTC
(In reply to William Hubbs from comment #33)
> (In reply to Jorge Manuel B. S. Vicetto from comment #31)
> > This policy *does* make sense.

<snip>

> The way you disable these is put links in /etc/binfmt.d with the same names
> as the files in /usr/lib/binfmt.d, but link them to /dev/null, for example,
> if /usr/lib/binfmt.d/foo exists, and you want to ignore it, run this command:
> 
> ln -s /dev/null /etc/binfmt.d/foo
> 
> The files in /etc/binfmt.d/* should be controlled by the sys admin, not the
> package manager.

William, I was agreeing with Mike that this type of functionality should never be enabled by default on a Gentoo system. This should always require intervention by system administrators, meaning it should be opt-in. You're describing how to make it opt-out.
Comment 35 William Hubbs gentoo-dev 2015-04-05 19:06:04 UTC
The binfmt.d files can only be processed if the kernel is configured with CONFIG_BINFMT_MISC=y or CONFIG_BINFMT_MISC=m and the binfmt-misc module is loaded.

If this level of configuration isn't done, nothing happens, even though the files may be installed in /usr/lib/binfmt.d/*.

In other words, users already have to opt into this by configuring their kernel to support it, so I don't know why we want to add a second opt-in layer.
Comment 36 William Hubbs gentoo-dev 2015-04-07 17:03:56 UTC
(In reply to William Hubbs from comment #27)
> Note that as of bug #382723, OpenRC has support for binfnt.d built into
> the procfs service, so technically qemu doesn't need an init script to
> set this up.

I am willing to split the support in OpenRC for binfmt to a separate service from procfs. This would be enabled by default, as the binfmt service is, but it would allow users to disable the service if they wanted.

I was chatting with @floppym about this just now also, and I suggested that he write a system service like the OpenRC service script for qemu.
Comment 37 Mike Gilbert gentoo-dev 2015-04-07 17:23:03 UTC
(In reply to William Hubbs from comment #36)
> I was chatting with @floppym about this just now also, and I suggested that
> he write a system service like the OpenRC service script for qemu.

This would be simple enough to do. I would just need a qemu maintainer to move the logic from the init script to a standalone shell script that I could call like this:

[Service]
Type=oneshot
ExecStart=/path/to/qemu-binfmt.sh
Comment 38 Łukasz Stelmach 2015-04-09 17:11:50 UTC
(In reply to Mike Gilbert from comment #37)

> [Service]
> Type=oneshot
> ExecStart=/path/to/qemu-binfmt.sh

Is this service meant to replace or be started next to systemd-binfmt.service?
Comment 39 Mike Gilbert gentoo-dev 2015-04-09 17:52:13 UTC
(In reply to Łukasz Stelmach from comment #38)
> Is this service meant to replace or be started next to
> systemd-binfmt.service?

It would not replace that service.
Comment 40 Priit Laes (IRC: plaes) 2015-08-09 06:48:51 UTC
I'm in favor of having eselect.

Things worked fine with openrc, but currently with systemd, I have to open up /etc/init.d/qemu-binfmt, copy required entry, become root (because it cannot be done via sudo) and paste the magic.

Also, few other packages that I can think of using the systemd-binfmt.service:

* app-emulation/wine
* dev-lang/mono
Comment 41 SpanKY gentoo-dev 2015-08-10 03:28:31 UTC
(In reply to Mike Gilbert from comment #30)

if someone wanted to write an eselect module to control things, we could install all the fragments into /usr/lib/binfmt.d/available/.  and then the module would symlink them into /etc/binfmt.d/ (since the admin is indirectly managing it).

(In reply to William Hubbs from comment #33)

i don't think tmpfiles.d is a relevant example.  that initializes paths with the correct owners/permissions, and that's it -- it doesn't otherwise impact the running system.  if it messed with permissions in a way that impacted security, then it'd be a problem regardless of the service running.

we could tie tmpfiles fragments to a specific service, but then that'd prevent people from running things by hand or via a different service manager.

(In reply to William Hubbs from comment #35)

the binfmt module is usually enabled by people (either by default, or the fact the help text suggests enabling it is harmless), and the init will automount the fs for them, and we include the binfmt init by default in the boot runlevel.

regardless, making a feature available is not the same thing to turning it on.  it's like saying "i emerged openssh so why isn't it autostarting".
Comment 42 Mike Gilbert gentoo-dev 2016-04-04 22:30:41 UTC
Created attachment 429654 [details, diff]
Implement qemu-binfmt for systemd

I implemented my suggestion from comment 37. It's on github, and I have also attached a patch.

https://github.com/floppym/gentoo/commit/f1c8e99b8e25619f9fa2e7c800fa1af9953101e8

As I suggested, I moved most logic into a script which gets installed in /usr/libexec. Both openrc and systemd call this script.
Comment 43 Matthias Maier gentoo-dev 2017-02-23 20:24:39 UTC
I took the liberty to finalize Mike's latest binfmt related changes to the live ebuild and added additional systemd support.

Beginning with app-emulation/qemu-2.8.0-r4 (and qemu-9999):

As a stopgap solution until a proper eselect-binfmt module is in place there are now two mechanisms for adding binfmt handlers:

 - a runscript /etc/init.d/qemu-binfmt
 - a binfmt configuration file /usr/share/qemu/binfmt.d/qemu.conf

The first one can (and should) be used with openrc, the latter can be easily symlinked to /etc/binfmt.d:

  # ln -s /usr/share/qemu/binfmt.d/qemu.conf /etc/binfmt.d

I have updated README.gentoo accordingly.

Both files are created dynamically and register only interpreters that are actually installed with the current use-flag configuration.