Gentoo Websites Logo
Go to: Gentoo Home Documentation Forums Lists Bugs Planet Store Wiki Get Gentoo!
Bug 732702 - nvidia-drivers.eclass should allow mirroring of nvidia-drivers package
Summary: nvidia-drivers.eclass should allow mirroring of nvidia-drivers package
Status: RESOLVED FIXED
Alias: None
Product: Gentoo Linux
Classification: Unclassified
Component: Eclasses (show other bugs)
Hardware: All Linux
: Normal normal (vote)
Assignee: David Seifert
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-07-15 07:33 UTC by Zoltan Puskas
Modified: 2021-03-21 15:53 UTC (History)
4 users (show)

See Also:
Package list:
Runtime testing required: ---


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Zoltan Puskas 2020-07-15 07:33:27 UTC
There is an exception in the NVIDIA drivers license that seems to allow redistributing the GPU drivers package as is (i.e. unmodified) on Linux/Unix like systems. The exact language is as follows:

"2.1.2 Linux/FreeBSD Exception. Notwithstanding the foregoing terms of Section 2.1.1, SOFTWARE designed exclusively for use on the Linux or FreeBSD operating systems, or other operating systems derived from the source code to these operating systems, may be copied and redistributed, provided that the binary files thereof are not modified in any way (except for unzipping of compressed files)."

The full license can be found at: https://www.nvidia.com/content/DriverDownload-March2009/licence.php?lang=us&type=TITAN

How did I get to this page:

1. Open https://www.nvidia.com/en-us/drivers/unix/
2. Click on the "Latest Long Lived Branch Version" in the "Linux x86_64/AMD64/EM64T", which is 450.57 at the time of filing this bug, which opens https://www.nvidia.com/Download/driverResults.aspx/162107/en-us
3. Click the download button, that will lead to another download page (https://www.nvidia.com/content/DriverDownload-March2009/confirmation.php?url=/XFree86/Linux-x86_64/450.57/NVIDIA-Linux-x86_64-450.57.run&lang=us&type=TITAN), this time giving you a link to the software agreement and another "Download" button
4. Click on the "NVIDIA End User License Agreement", which will lead to the aforementioned license page.

It would be nice if we could mirror these packages too, but IANAL. Other distros, e.g. Debian, also seem to mirror it in their non-free repos.

Reproducible: Always
Comment 1 Ulrich Müller gentoo-dev 2020-07-15 08:36:20 UTC
The NVIDIA license isn't the problem here. IIRC mirror restriction is in place because some components of the package are licensed under the GPL, and we cannot distribute them without their source code.
Comment 2 Zoltan Puskas 2020-07-19 21:41:07 UTC
How do other Linux distros are distributing it then? Which particular components are GPL and missing the source? If they are GPL-ed code could we not ask nvidia to provide the source code for it?
Comment 3 Ulrich Müller gentoo-dev 2020-07-20 08:21:00 UTC
(In reply to Zoltan Puskas from comment #2)

Bug 460460 says that the issue is with nvidia-{settings,installer}, and as a matter of fact, Nvidia _does_ provide their source code. The problem is that _we_ don't distribute their source code which would violate the GPL if we distributed the binary.

IIUC, the ebuild builds nvidia-settings from source now, so that part of the problem appears to be gone.

For nvidia-installer, the ebuild would have to list https://download.nvidia.com/XFree86/nvidia-installer/nvidia-installer-${PV}.tar.bz2 in SRC_URI, and the mirror restriction could be lifted. It's a small tarball (140 KiB) so downloading it won't be a large burden on users (but of course, it won't be used for anything).
Comment 4 Jeroen Roovers (RETIRED) gentoo-dev 2020-07-22 08:01:59 UTC
(In reply to Ulrich Müller from comment #3)
> (In reply to Zoltan Puskas from comment #2)
> 
> Bug 460460 says that the issue is with nvidia-{settings,installer},

We already compile nvidia-settings from source, which is work done after bug #460460 was resolved.

commit 53461784a9f947e01ea5fb96b93f8e9d6cba3fcc
Author: Jeroen Roovers <jer@gentoo.org>
Date:   Sat Jan 30 12:15:36 2016 +0100

    x11-drivers/nvidia-drivers: Block media-video/nvidia-settings.

    Package-Manager: portage-2.2.27

commit 41cac5acdd0d07eb014e814f96df05aef393cff3
Author: Jeroen Roovers <jer@gentoo.org>
Date:   Sat Jan 30 12:14:25 2016 +0100

    x11-drivers/nvidia-drivers: Build tools from source (bug #562910 by Christian Strahl).

    Package-Manager: portage-2.2.27

> and as a
> matter of fact, Nvidia _does_ provide their source code. The problem is that
> _we_ don't distribute their source code which would violate the GPL if we
> distributed the binary.
> 
> IIUC, the ebuild builds nvidia-settings from source now, so that part of the
> problem appears to be gone.

Yes. As above.

> For nvidia-installer, the ebuild would have to list
> https://download.nvidia.com/XFree86/nvidia-installer/nvidia-installer-${PV}.
> tar.bz2 in SRC_URI, and the mirror restriction could be lifted. It's a small
> tarball (140 KiB) so downloading it won't be a large burden on users (but of
> course, it won't be used for anything).

The whole point of the ebuilds and eclass is to emulate nvidia-installer's purpose in a way that is technically compliant with the Gentoo Linux project's guidelines. That it happens to get shipped inside the nvidia-drivers tarballs but not used is kind of unfortunate, but that is how bug #460460 was resolved, by restricting mirroring.

The same argument that was made in bug #460460. Is it desirable to mirror nvidia-drivers but then having to burden those mirrors with additional source code for nvidia-installer that will not be used? Will this be challenged in the future by the QA team deciding that unused SRC_URI elements should be removed? Should we install the source code to the ROOT filesystem?

Or is this a new argument that nvidia-installer should be built and installed in order to perhaps create binary distribution tools of our own (not involving portage-built tbz2s)? I would hate to see bug reports complaining that the x11-drivers/nvidia-drivers installed "nvidia-installer does not work".
Comment 5 Ulrich Müller gentoo-dev 2020-07-22 08:25:42 UTC
(In reply to Jeroen Roovers from comment #4)
> The whole point of the ebuilds and eclass is to emulate nvidia-installer's
> purpose in a way that is technically compliant with the Gentoo Linux
> project's guidelines. That it happens to get shipped inside the
> nvidia-drivers tarballs but not used is kind of unfortunate, but that is how
> bug #460460 was resolved, by restricting mirroring.
> 
> The same argument that was made in bug #460460. Is it desirable to mirror
> nvidia-drivers but then having to burden those mirrors with additional
> source code for nvidia-installer that will not be used?

That would be an argument if the nvidia-installer distfile had a significant size. It had only 140 KiB which is completely negligible compared to the total download size of the package. IMHO, adding it would be a small price to pay, if it would allow lifting the mirror restriction.

> Will this be challenged in the future by the QA team deciding that unused
> SRC_URI elements should be removed?

I've confirmed with QA that they won't challenge it. They suggest adding a comment to the ebuild explaining why the distfile is listed.
Comment 6 Jeroen Roovers (RETIRED) gentoo-dev 2020-07-22 09:22:33 UTC
I still don't see anyone arguing why "nvidia-drivers.eclass should allow mirroring of nvidia-drivers package". Hint: "It would be nice" doesn't quite cut it.
Comment 7 Jeroen Roovers (RETIRED) gentoo-dev 2020-07-22 09:24:49 UTC
(In reply to Ulrich Müller from comment #5)
> I've confirmed with QA that they won't challenge it.

I've had really bad experiences with QA in recent years, so if we get to that point where I need to distribute pointless code, they should probably publish that promise in writing.
Comment 8 Zoltan Puskas 2020-07-23 05:41:50 UTC
(In reply to Jeroen Roovers from comment #6)
> I still don't see anyone arguing why "nvidia-drivers.eclass should allow
> mirroring of nvidia-drivers package". Hint: "It would be nice" doesn't quite
> cut it.

It's not simply because "it would be nice", but actually has practical implications. In my example I happen to run a local mirror, that serves multiple hosts on the internal LAN. Not mirroring means:

- Every host during an update has to download it over the internet instead of using it from the local mirror. This is a waste of my local limited bandwidth as well as being slower.
- It has privacy implications as now I'm leaking metadata regarding my installs.

Additionally I'm doing some ebuild maintainership, which has recently started involving running tests in containers including builds from scratch (e.g. for stabilization tests, different profiles, etc). Running rsync for my mirror once a day means, that the nvidia drivers are wiped from the cache after install (mirror also has an nvidia card in it), requiring a re-fetching it over and over when needed.
Comment 9 Ulrich Müller gentoo-dev 2020-07-25 08:15:38 UTC
(In reply to Jeroen Roovers from comment #7)
> I've had really bad experiences with QA in recent years, so if we get to
> that point where I need to distribute pointless code, they should probably
> publish that promise in writing.

CCing QA. @soap: Can you comment?
Comment 10 Jeroen Roovers (RETIRED) gentoo-dev 2020-07-25 09:46:12 UTC
(In reply to Zoltan Puskas from comment #8)
> (In reply to Jeroen Roovers from comment #6)
> > I still don't see anyone arguing why "nvidia-drivers.eclass should allow
> > mirroring of nvidia-drivers package". Hint: "It would be nice" doesn't quite
> > cut it.
> 
> It's not simply because "it would be nice", but actually has practical
> implications. In my example I happen to run a local mirror

OK, so you need a local, internal mirror. Have you considered mounting DISTDIR from a local server to solve that problem? Did you look into PORTAGE_RO_DISTDIRS? How about FETCHCOMMAND?
Comment 11 David Seifert gentoo-dev 2020-07-25 14:20:20 UTC
(In reply to Ulrich Müller from comment #9)
> (In reply to Jeroen Roovers from comment #7)
> > I've had really bad experiences with QA in recent years, so if we get to
> > that point where I need to distribute pointless code, they should probably
> > publish that promise in writing.
> 
> CCing QA. @soap: Can you comment?

As discussed, QA is fine with mirroring this, including the small distfile, if only for satisfying the licence.
Comment 12 Michał Górny archtester Gentoo Infrastructure gentoo-dev Security 2020-11-03 21:46:55 UTC
Sorry, misclick.
Comment 13 Larry the Git Cow gentoo-dev 2021-03-21 15:53:35 UTC
The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=26146d1510fd678538b7d02400c1eb8e66e20212

commit 26146d1510fd678538b7d02400c1eb8e66e20212
Author:     Ionen Wolkens <sudinave@gmail.com>
AuthorDate: 2021-03-21 15:52:10 +0000
Commit:     David Seifert <soap@gentoo.org>
CommitDate: 2021-03-21 15:52:10 +0000

    x11-drivers/nvidia-drivers: bump to 460.67 with refactored ebuild
    
    ebuild carries a lot of history and, rather than cleanups, it needed
    something closer to a rewrite.
    
    Bugfixes:
    - Removed all udev rules to solve long standing issues (bug #454740)
    - Install libraries with no X11 dependencies with USE=-X,
      notably for headless OpenCL/CUDA (bug #561706)
    - Install systemd unit for persistenced + nvpd user (bug #591638)
    - Add custom error message for DRM_KMS_HELPER and ensure driver
      doesn't attempt building DRM support without it (bug #603818)
    - Warn about AMD SME if enabled by default (bug #652408)
    - Distribute extra sources to lift RESTRICT="bindist mirror", the
      nvidia-driver.eclass is no longer used (bug #732702)
    - Build modprobe and persistenced from source (bug #747145)
    - Use system locations for vulkan icd/layers (bug #749600)
    
    Others:
    - Dropped IUSE=compat/multilib/kms/uvm/wayland
      > compat: was for non-GLVND variants and currently a no-op
      > multilib: obsolete, abi_x86_32 does all that's needed
      > kms/uvm: modules are loaded by nvidia-modprobe as-needed and
        there's not much sense in skipping installation. Will also save
        OpenCL/CUDA packages from having to depend on [uvm]
      > wayland: library is provided by gui-libs/egl-wayland instead which
        now also provides pkgconfig files and can be a newer version.
        optfeature warning was added for awareness.
    - Dropped REQUIRED_USE, all USE can now be used independently, e.g.
      now possible to get libXNVCtrl.a (static-libs) without the
      deps-heavy USE=tools
    - Dropped locale patch, the offending code it was meant to fix is gone.
    - Dropped linker patch, uses right linker even with -native-symlinks.
    - Added modprobe.d .conf to blacklist nouveau by default.
    - Patched nvidia-modprobe to respect nvidia.conf's permissions when
      creating uvm devices, was previously created as world read-write.
    - No longer installing libOpenCL.so loader (not needed to use OpenCL,
      was used by the no longer available eselect-opencl).
    - nvidia-persistenced init script simplified and updated for nvpd user.
    - nvidia-smi init script removed (all it did was query cards every 300
      seconds), mentioned behavior is no longer observable (fan scales
      normally without X) and it wasn't intended for this purpose.
    - Removed I2C_NVIDIA_GPU check as it caused unnecessary noise for
      gentoo-kernel-bin users (built as module), and being a bad thing
      even if loaded is questionable.
    - Attempt to reduce message noise. The only fatal CONFIG_CHECK is
      fairly rare so there's little reason to check twice with pkg_pretend.
    - ... but added new conditional messages to explain important things
      often seen as common sense but that a new user likely won't know.
    - Replaced the nvidia-driver.eclass legacy test with a compact version
      that reads supported-gpus.json (usable on >450).
    - More strict deps, some may sound strange but nvidia-settings only
      use headers for some of these (dbus/Xrandr/Xv/vdpau).
      > X? libs kept separate as it's the only one needing multilib deps.
      > pax-utils now unconditional for scanelf as libraries are always
        installed. Alternatively could've generated those, but prefer to
        leave it easier to maintain for future generations.
      > virtual/opencl removed, no sense in the drivers depending on this
        and it's instead applications using opencl that should.
      > Added MODULES_OPTIONAL_USE="driver" to handle linux-mod deps
    - Added MIT license for persistenced
    - Added ZLIB license for supported-gpus.json
    - NV_KERNEL_MAX (previously NV_KV_MAX_PLUS) set to be <=5.11 form
      rather than <5.12 given that often confused users thinking it meant
      5.12 support from quick looks.
    - arm64 support "should" work but runtime untested
    - And a long list of cleanups that "hopefully" won't cause new issues.
    
    Closes: https://bugs.gentoo.org/454740
    Closes: https://bugs.gentoo.org/561706
    Closes: https://bugs.gentoo.org/591638
    Closes: https://bugs.gentoo.org/603818
    Closes: https://bugs.gentoo.org/652408
    Closes: https://bugs.gentoo.org/732702
    Closes: https://bugs.gentoo.org/747145
    Closes: https://bugs.gentoo.org/749600
    Signed-off-by: Ionen Wolkens <sudinave@gmail.com>
    Signed-off-by: David Seifert <soap@gentoo.org>

 x11-drivers/nvidia-drivers/Manifest                |   7 +
 .../files/nvidia-blacklist-nouveau.conf            |   3 +
 .../files/nvidia-modprobe-390.141-uvm-perms.patch  |  12 +
 .../nvidia-drivers/files/nvidia-persistenced.confd |   7 +
 .../nvidia-drivers/files/nvidia-persistenced.initd |  12 +
 .../nvidia-drivers/nvidia-drivers-460.67.ebuild    | 391 +++++++++++++++++++++
 6 files changed, 432 insertions(+)