Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 704416 - env-update: Consider transforming /etc/env.d into systemd environment.d files on env-update
Summary: env-update: Consider transforming /etc/env.d into systemd environment.d files...
Status: RESOLVED FIXED
Alias: None
Product: Portage Development
Classification: Unclassified
Component: Core (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: Portage team
URL:
Whiteboard:
Keywords: InVCS
Depends on:
Blocks: 704412 739718
  Show dependency tree
 
Reported: 2020-01-01 10:20 UTC by Florian Schmaus
Modified: 2020-11-22 07:17 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments
0001-Add-gentoo-env-to-systemd.patch (0001-Add-gentoo-env-to-systemd.patch,1.79 KB, patch)
2020-09-01 07:39 UTC, Florian Schmaus
Details | Diff
0001-env-update-create-systemd-env-configuration-if-requi.patch (0001-env-update-create-systemd-env-configuration-if-requi.patch,3.65 KB, patch)
2020-09-02 16:48 UTC, Florian Schmaus
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Florian Schmaus gentoo-dev 2020-01-01 10:20:54 UTC
Currently env-update produces /etc/profile.env which is unfortunately not compatible with systemd's EnvironmentFile (see "man 5 systemd.exec"):
profile.env is shell syntax and hence the variable definitions are prefixed with 'export ', but EnvironmentFile expects just new-line-separated variable assignments.

This means that there is no convenient way on Gentoo for systemd services, especially user services, to run with the same environment as a user. This leads to issues like bug #704412.

On a minor note: Maybe consider not placing that new file under /etc, potentially /var, e.g. /var/lib/portage is better suited.
Comment 1 Florian Schmaus gentoo-dev 2020-01-02 16:06:17 UTC
After reading more into the topic, I think env-update should create /etc/environment.d/*.conf (or maybe /run/environment.d ?) files, which are parsed by systemd and update the environment for systemd user services. See "man 5 environment.d".
Comment 2 Mike Gilbert gentoo-dev 2020-01-02 16:32:01 UTC
We addressed a similar issue with PATH in system services using an environment generator in the gentoo-systemd-integration package (bug 651904).

My main concern with this request is the significant change in behavior if we suddenly start importing all environment settings from profile.env for all system and user services. That requires some thought/review.
Comment 3 Mike Gilbert gentoo-dev 2020-01-02 16:44:06 UTC
(In reply to Florian Schmaus from comment #1)
> After reading more into the topic, I think env-update should create
> /etc/environment.d/*.conf (or maybe /run/environment.d ?) files, which are
> parsed by systemd and update the environment for systemd user services. See
> "man 5 environment.d".

This seems fairly safe since only user services would be affected.
Comment 4 Florian Schmaus gentoo-dev 2020-01-02 16:49:19 UTC
(In reply to Mike Gilbert from comment #2)
> My main concern with this request is the significant change in behavior if
> we suddenly start importing all environment settings from profile.env for
> all system and user services. That requires some thought/review.

If I understand the man page for environment.d correctly, then it only affects the *user* session environment. System-wide systemd services are not affected by this. And since the environment specified by /etc/env.d should be part of the standard user environment, transforming it to environment.d should be safe.
Comment 5 Florian Schmaus gentoo-dev 2020-09-01 07:39:19 UTC
Created attachment 657706 [details, diff]
0001-Add-gentoo-env-to-systemd.patch

Attached patch for sys-apps/gentoo-systemd-integration adds a new script 'gentoo-env-to-systemd' which transforms /etc/profile.env to /usr/lib/environment.d/gentoo-env.conf.

What is missing is an portage env-update hook, which is invoked after /etc/profile.env was (re-)generated, that can be used by sys-apps/gentoo-systemd-integration to ensure that 'gentoo-env-to-systemd' is called every time /etc/profile.env is updated.
Comment 6 Arfrever Frehtes Taifersar Arahesis 2020-09-01 23:01:22 UTC
If any change is to be made, env-update can unconditionally create one additional small file for systemd.
It would be harmless for users of OpenRC.
Better to avoid duplication of parsing code in another tool.
Comment 7 Florian Schmaus gentoo-dev 2020-09-02 16:48:11 UTC
Created attachment 657948 [details, diff]
0001-env-update-create-systemd-env-configuration-if-requi.patch

(In reply to Arfrever Frehtes Taifersar Arahesis from comment #6)
> If any change is to be made, env-update can unconditionally create one
> additional small file for systemd.
> It would be harmless for users of OpenRC.

I considered that, but I personally would go with env-update hooks and having systemd specific code in sys-apps/gentoo-systemd-integration. But that is certainly not a hill for me to die on. Meanwhile I send the attached patch to gentoo-portage-dev@.

It is basically what you suggested, but instead of unconditionally creating the file, I only create it if /usr/lib/environment.d exists.

> Better to avoid duplication of parsing code in another tool.

Not sure where there you see duplication.
Comment 8 Arfrever Frehtes Taifersar Arahesis 2020-09-02 21:07:24 UTC
(In reply to Florian Schmaus from comment #7)
> +		with open(profile_env_path, "r") as profile_env:
> +			lines = profile_env.readlines()

This file is created 15-26 lines earlier, in the same Python function.
All data remain in env_keys dictionary.
There is no point in reading this just created file, you can directly use env_keys elements.
Then you do not need to discard comments, empty lines etc.
Comment 9 Larry the Git Cow gentoo-dev 2020-09-08 00:04:13 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/proj/portage.git/commit/?id=45a5982fe8076066323e91f6b5fe860f3a429f9f

commit 45a5982fe8076066323e91f6b5fe860f3a429f9f
Author:     Florian Schmaus <flo@geekplace.eu>
AuthorDate: 2020-09-05 07:18:17 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2020-09-08 00:01:47 +0000

    env-update: create systemd user-session environment definition
    
    Portage's env-update currently transforms the environment information
    from /etc/env.d into /etc/profile.env, which is typically sourced by
    every user session, setting up its environment.
    
    However, /etc/profile.env is not sourced by systemd user
    services. Instead, for the definition of a systemd user session
    environment, the 'environment.d' machinery exists. Unfortunately, up
    to now, env-update does not produce a profile.env equivalent for this
    machinery, causing issues for systemd user services. For example, an
    emacs daemon run as user systemd service does not have a complete
    PATH (bug #704412 [1]), because some PATH components are injected by
    packages via /etc/env.d. For example, an LLVM ebuild may set
    PATH="/usr/lib/llvm/9/bin".
    
    This commit changes env-update so that a systemd user session
    environment configuration file named
    
    /etc/environment.d/10-gentoo-env.conf
    
    is created.
    
    Thanks to Michael 'veremitz' Everitt, Arfrever Frehtes Taifersar
    Arahesis, Ulrich Müller, Joakim Tjernlund, and Zac Medico for the
    useful feedback.
    
    1: https://bugs.gentoo.org/704412
    
    Bug: https://bugs.gentoo.org/704416
    Signed-off-by: Florian Schmaus <flo@geekplace.eu>
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 lib/portage/util/env_update.py | 42 +++++++++++++++++++++++++++++++++++++++---
 1 file changed, 39 insertions(+), 3 deletions(-)
Comment 10 Larry the Git Cow gentoo-dev 2020-09-08 03:51:43 UTC
The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=fd840baeef963f5fe746f9167976ce0eed83a3b4

commit fd840baeef963f5fe746f9167976ce0eed83a3b4
Author:     Zac Medico <zmedico@gentoo.org>
AuthorDate: 2020-09-08 03:44:51 +0000
Commit:     Zac Medico <zmedico@gentoo.org>
CommitDate: 2020-09-08 03:51:23 +0000

    sys-apps/portage: Bump to version 3.0.6
    
     #668302 binrepos.conf: support fetchcommand customization
     #668334 Add binrepos.conf to replace PORTAGE_BINHOST
     #704416 env-update: create systemd user-session environment
     #737470 egencache --update-pkg-desc: emulate esync --verbose
     #737480 emerge --search: auto-detect regular expressions
     #739908 @change-deps: fix erroneous repeated rebuilds
     #740588 make.conf: Treat __* variables as local
     #740898 PORTAGE_BINHOST urlopen proxy support
     #740904 rsync and gemato proxy support
    
    Bug: https://bugs.gentoo.org/739718
    Bug: https://bugs.gentoo.org/668302
    Bug: https://bugs.gentoo.org/668334
    Bug: https://bugs.gentoo.org/704416
    Bug: https://bugs.gentoo.org/737470
    Bug: https://bugs.gentoo.org/737480
    Bug: https://bugs.gentoo.org/739908
    Bug: https://bugs.gentoo.org/740588
    Bug: https://bugs.gentoo.org/740898
    Bug: https://bugs.gentoo.org/740904
    Package-Manager: Portage-3.0.6, Repoman-3.0.1
    Signed-off-by: Zac Medico <zmedico@gentoo.org>

 sys-apps/portage/Manifest             |   1 +
 sys-apps/portage/portage-3.0.6.ebuild | 265 ++++++++++++++++++++++++++++++++++
 2 files changed, 266 insertions(+)