Bug 729928 – x11-drivers/nvidia-drivers-450.51-r1 without CONFIG_MMU_NOTIFIER - .../work/kernel/nvidia-uvm/uvm8_va_space_mm.c:256:16: error: implicit declaration of function '__mmu_notifier_register'; did you mean 'uvm_mmu_notifier_register'?

• Status: RESOLVED FIXED
• Product: Gentoo Linux

[Description Ionen Wolkens 2020-06-27 18:09:06 UTC]

Created attachment 646866 [details]
emerge --info and build.log

This is an issue I've seen reported on IRC and two forum threads:
https://forums.gentoo.org/viewtopic-p-8475544.html
https://forums.gentoo.org/viewtopic-p-8475092.html

While it worked for me, I was able to reproduce by removing CONFIG_MMU_NOTIFIER=y from my kernel. Versions below 450.51 are unaffected.

However, can't just put a check for it given this option doesn't have a menu option for users to select. It commonly gets enabled as a dependency by other things (such as having KVM or IOMMU support).

Seems to happen while building the uvm module but USE=uvm/-uvm will not affect this given the module gets built either way.
[ebuild   R    ] x11-drivers/nvidia-drivers-450.51-r1:0/450::gentoo  USE="X driver gtk3 kms libglvnd multilib static-libs tools -compat -uvm -wayland" ABI_X86="32 (64) (-x32)" 0 KiB

[Comment 1 Vadim Dyadkin 2020-06-28 11:37:32 UTC]

(In reply to Ionen Wolkens from comment #0)
> While it worked for me, I was able to reproduce by removing
> CONFIG_MMU_NOTIFIER=y from my kernel. Versions below 450.51 are unaffected.

Exactly, I did not have this option checked, and could not build nvidia-drivers. After checking it, it builds OK, but kwin-5.19.2 crashes all the time. I had to roll back to nvidia-drivers-440.100-r1.

[Comment 2 Jeroen Roovers (RETIRED) 2020-06-28 12:16:23 UTC]

I could add a kernel check for CONFIG_MMU_NOTIFIER.

[Comment 3 Jeroen Roovers (RETIRED) 2020-06-28 12:32:36 UTC]

Comment on attachment 646866 [details]
emerge --info and build.log

> CONFTEST: mmu_notifier_ops_invalidate_range

This test succeeded and based on that, __mmu_notifier_register is used.

[Comment 4 Jeroen Roovers (RETIRED) 2020-06-28 12:34:39 UTC]

(In reply to Jeroen Roovers from comment #2)
> I could add a kernel check for CONFIG_MMU_NOTIFIER.

This is not a configurable option - it depends on other options which might be hardware specific.

[Comment 5 Ionen Wolkens 2020-06-28 13:22:55 UTC]

(In reply to Jeroen Roovers from comment #4)
> This is not a configurable option - it depends on other options which might
> be hardware specific.

Yeah, this is what I meant by can't just add a check. And telling users to enable something they may not want/need just so the dependency MMU_NOTIFIER is enabled doesn't seem right.

Wish I had some idea to suggest but I don't know. User from the forum just selected amd IOMMU as module which added MMU_NOTIFIER, but well.

[Comment 6 Ionen Wolkens 2020-06-28 13:41:43 UTC]

Well, maybe nvidia's test should be patched instead?

Then again, it is a beta driver which nvidia will likely fix themselves (been fine for me but likely has other issues too), so just masking for now doesn't sound so bad either.

[Comment 7 Larry the Git Cow 2020-06-28 14:11:46 UTC]

The bug has been referenced in the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=bfe74ecceb0d42a900759a0307e0173beed48f95

commit bfe74ecceb0d42a900759a0307e0173beed48f95
Author:     Jeroen Roovers <[e-mail]>
AuthorDate: 2020-06-28 14:09:29 +0000
Commit:     Jeroen Roovers <[e-mail]>
CommitDate: 2020-06-28 14:11:42 +0000

    x11-drivers/nvidia-drivers: Drop ZONE_DMA kernel configuration check
    
    Package-Manager: Portage-2.3.103, Repoman-2.3.23
    Bug: https://bugs.gentoo.org/729928
    Signed-off-by: Jeroen Roovers <[e-mail]>

 x11-drivers/nvidia-drivers/nvidia-drivers-390.138.ebuild    | 8 +++++++-
 x11-drivers/nvidia-drivers/nvidia-drivers-430.64-r5.ebuild  | 8 +++++++-
 x11-drivers/nvidia-drivers/nvidia-drivers-435.21-r5.ebuild  | 8 +++++++-
 x11-drivers/nvidia-drivers/nvidia-drivers-440.100-r1.ebuild | 8 +++++++-
 x11-drivers/nvidia-drivers/nvidia-drivers-450.51-r1.ebuild  | 8 +++++++-
 5 files changed, 35 insertions(+), 5 deletions(-)

[Comment 8 Jeroen Roovers (RETIRED) 2020-06-28 14:29:33 UTC]

(In reply to Ionen Wolkens from comment #6)
> Well, maybe nvidia's test should be patched instead?

Yes.

[Comment 9 Larry the Git Cow 2020-07-09 17:19:01 UTC]

The bug has been closed via the following commit(s):

https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=50dc6c4c1b89062ff7c06f2626900a76e4f72653

commit 50dc6c4c1b89062ff7c06f2626900a76e4f72653
Author:     Jeroen Roovers <[e-mail]>
AuthorDate: 2020-07-09 17:17:45 +0000
Commit:     Jeroen Roovers <[e-mail]>
CommitDate: 2020-07-09 17:18:58 +0000

    x11-drivers/nvidia-drivers: Version 450.57
    
    - Install nvidia_icd.json unconditionally (bug #730738)
    
    Package-Manager: Portage-2.3.103, Repoman-2.3.23
    Closes: https://bugs.gentoo.org/729928
    Closes: https://bugs.gentoo.org/730738
    Signed-off-by: Jeroen Roovers <[e-mail]>

 x11-drivers/nvidia-drivers/Manifest                |   3 +
 .../nvidia-drivers/nvidia-drivers-450.57.ebuild    | 593 +++++++++++++++++++++
 2 files changed, 596 insertions(+)

[Comment 10 Ionen Wolkens 2020-07-12 11:18:37 UTC]

Seen a user report that this was still happening with 450.57, gave it a try as well and could reproduce. Re-opening given I don't think this is fixed.

[Comment 11 Stefan Wimmer 2020-07-19 09:45:33 UTC]

It still happens to me with CONFIG_MMU_NOTIFIER disabled. I've attached the logfile ...

[Comment 12 Stefan Wimmer 2020-07-19 09:47:23 UTC]

Created attachment 649822 [details]
build.log

It still happens to me with CONFIG_MMU_NOTIFIER disabled ...

[Comment 13 Jeroen Roovers (RETIRED) 2020-07-19 14:22:02 UTC]

*** Bug 733094 has been marked as a duplicate of this bug. ***

[Comment 14 FR 2020-08-06 17:32:31 UTC]

I have the same problem with kernel-5.8.0 and nvidia-drivers-450.57.

As a consequence, I can't use the new kernel-5.8 until this issue is resolved.

With kernel-5.8, nvidia-drivers-440.100 fails to build with a different error.

However, I do not know about the state of CONFIG_MMU_NOTIFIER.  That particular
configuration does not appear in my kernel config file.

[Comment 15 Jeroen Roovers (RETIRED) 2020-08-10 15:32:30 UTC]

*** Bug 736641 has been marked as a duplicate of this bug. ***

[Comment 16 Phil Stracchino (Unix Ronin) 2020-08-10 15:50:03 UTC]

I haven't updated to kernel 5.7 yet, I'm still on 5.6.19.  The following are ALL occurences of 'MMU' in my kernel .config:


CONFIG_IRQ_MSI_IOMMU=y
CONFIG_MMU=y
# CONFIG_GART_IOMMU is not set
CONFIG_IOMMU_IOVA=y
CONFIG_IOMMU_API=y
CONFIG_IOMMU_SUPPORT=y
# Generic IOMMU Pagetable Support
# end of Generic IOMMU Pagetable Support
# CONFIG_IOMMU_DEBUGFS is not set
# CONFIG_IOMMU_DEFAULT_PASSTHROUGH is not set
CONFIG_IOMMU_DMA=y
CONFIG_AMD_IOMMU=y
# CONFIG_AMD_IOMMU_V2 is not set
# CONFIG_INTEL_IOMMU is not set


CONFIG_MMU_NOTIFIER is not present at all.  I'm about to try compiling against kernel 5.7.14 and see whether that works.

[Comment 17 Jeroen Roovers (RETIRED) 2020-08-10 15:56:27 UTC]

(In reply to FR from comment #14)
> With kernel-5.8, nvidia-drivers-440.100 fails to build with a different
> error.

You should definitely file a new bug report about that.

[Comment 18 Jeroen Roovers (RETIRED) 2020-08-10 15:59:35 UTC]

(In reply to Phil Stracchino (Unix Ronin) from comment #16)
> I haven't updated to kernel 5.7 yet, I'm still on 5.6.19.

It doesn't matter what version you're on. If you see the same compiler error in the same file, then you have the same problem:

.../work/kernel/nvidia-uvm/uvm8_va_space_mm.c:256:16: error: implicit declaration of function '__mmu_notifier_register'; did you mean 'uvm_mmu_notifier_register'?

[Comment 19 Phil Stracchino (Unix Ronin) 2020-08-10 16:03:03 UTC]

(In reply to Phil Stracchino (Unix Ronin) from comment #16)
> CONFIG_MMU_NOTIFIER is not present at all.  I'm about to try compiling
> against kernel 5.7.14 and see whether that works.

And nope, still no joy with 5.7.14.  No CONFIG_MMU_NOTIFIER flag present in the .config at all, and nvidia-drivers-450.57-r1 fails to compile.

This is a Heisenberg flag.  You don't know whether you need it or not, but it doesn't matter, because you can't control whether it's set or not (or even exists).

[Comment 20 Jeroen Roovers (RETIRED) 2020-08-10 16:11:51 UTC]

(In reply to Phil Stracchino (Unix Ronin) from comment #19)
> (In reply to Phil Stracchino (Unix Ronin) from comment #16)
> > CONFIG_MMU_NOTIFIER is not present at all.  I'm about to try compiling
> > against kernel 5.7.14 and see whether that works.
> 
> And nope, still no joy with 5.7.14.

You can stop trying new versions of kernels. I'm running 5.8.0-gentoo-r1 with CONFIG_MMU_NOTIFIER set and nvidia-drivers-450.57-r1 just fine.

> This is a Heisenberg flag.

You've just made that up, haven't you? No relation to anything the famous Heisenberg did or indeed the term "heisenbug".

> You don't know whether you need it or not

No, you absolutely do know you need it. Says right there in the Description of this bug report.

> , but it doesn't matter, because you can't control whether it's set or not (or even exists).

No, you can absolutely enable CONFIG_MMU_NOTIFIER in your kernel configuration; you just cannot do it in a direct, intuitive and convenient way.

[Comment 21 Phil Stracchino (Unix Ronin) 2020-08-10 16:51:31 UTC]

(In reply to Jeroen Roovers from comment #20)
> No, you absolutely do know you need it. Says right there in the Description
> of this bug report.

Granted, I misread.

> > , but it doesn't matter, because you can't control whether it's set or not (or even exists).
> 
> No, you can absolutely enable CONFIG_MMU_NOTIFIER in your kernel
> configuration; you just cannot do it in a direct, intuitive and convenient
> way.

If there is even an indirect and non-intuitive way, I do not see it described in this ticket.  You said yourself,
> This is not a configurable option - it depends on other options which might be hardware specific.

So we need to enable a kernel config option that we can't enable directly, we don't know what options turn it on, and the options needed may be hardware specific.

I went through my kernel config again, and I THINK I found it in my case:  On my hardware (AMD PhenomII), it appears the crucial config option is CONFIG_AMD_IOMMU_V2.  I'm building with that right now and hoping that PhenomII is actually supported by CONFIG_AMD_IOMMU_V2.  The notes on CONFIG_AMD_IOMMU_V2 don't tell me.  CONFIG_AMD_IOMMU is NOT sufficient.

That enabled me to build nvidia-drivers-450.57.  I'll report back in a few minutes on whether the kernel with that config option actually works on a PhenomII.

[Comment 22 Phil Stracchino (Unix Ronin) 2020-08-10 18:43:13 UTC]

(In reply to Phil Stracchino (Unix Ronin) from comment #21)
> On my hardware (AMD PhenomII), it appears the crucial config option is
> CONFIG_AMD_IOMMU_V2.
> CONFIG_AMD_IOMMU is NOT sufficient.
> 
> That enabled me to build nvidia-drivers-450.57.  I'll report back in a few
> minutes on whether the kernel with that config option actually works on a
> PhenomII.

And that appears to be working.  So if you're on reasonably recent AMD hardware, the key option to enable is CONFIG_AMD_IOMMU_V2.  You'll find it under Device Drivers/IOMMU Support.  Presumably the Intel equivalent of that option if you're on recent Intel hardware.

If those two cases don't cover you, sorry, I have no further insights.

[Comment 23 FR 2020-08-11 14:22:25 UTC]

(In reply to Phil Stracchino (Unix Ronin) from comment #22)
>
>  You'll find it
> under Device Drivers/IOMMU Support.  Presumably the Intel equivalent of that
> option if you're on recent Intel hardware.
> 

On my Intel system, I enabled "IOMMU Hardware Support" and "Support for Intel IOMMU using DMA Remapping Devices" and then re-compiled kernel-5.8.

Nvidia-drivers now fails with a NEW error:

 In function 'uvm_mmu_notifier_unregister':
 error: implicit declaration of function 'mmu_notifier_unregister'


At this point I am stuck with kernel-5.7 and nvidia-440.82.

This needs to be reported to Nvidia.  They have a tendency to suddenly incorporate all sorts of irrelevant stuff.  Several versions back they had a dependency on NUMA which is not relevant to a desktop workstation.  After someone pointed it out they removed it.

[Comment 24 FR 2020-08-23 20:55:38 UTC]

SOLVED

After posting a description of this issue on the Nvidia forum, a patch has been provided.

With this patch, both nvidia-450.57 and nvidia-450.66 build and run without the "implicit declaration of function '__mmu_notifier_register'" error.

Furthermore, CONFIG_MMU_NOTIFIER does *not* have to be enabled in the kernel configuration.

The patch is found here:

https://forums.developer.nvidia.com/t/build-failure-450-57-and-450-66-with-kernel-5-8-implicit-declaration-mmu-notifier-register/147977/3


Thanks to Nvidia for their quick response.

[Comment 25 Christophe PEREZ 2020-10-21 13:45:12 UTC]

(In reply to FR from comment #24)
> SOLVED
> 
> After posting a description of this issue on the Nvidia forum, a patch has
> been provided.
> 
> With this patch, both nvidia-450.57 and nvidia-450.66 build and run without
> the "implicit declaration of function '__mmu_notifier_register'" error.
> 
> Furthermore, CONFIG_MMU_NOTIFIER does *not* have to be enabled in the kernel
> configuration.
> 
> The patch is found here:
> 
> https://forums.developer.nvidia.com/t/build-failure-450-57-and-450-66-with-
> kernel-5-8-implicit-declaration-mmu-notifier-register/147977/3
> 
> 
> Thanks to Nvidia for their quick response.

I can confirm this patch works with 
x11-drivers/nvidia-drivers-455.28
sys-kernel/gentoo-sources-5.8.16

thanks !

[Comment 26 Sam James 2020-10-23 01:49:20 UTC]

*** Bug 750485 has been marked as a duplicate of this bug. ***

[Comment 27 Ionen Wolkens 2021-02-14 19:34:54 UTC]

I'm hearing reports that this now works properly in 460.39 (or at least with 5.10.16) even with CONFIG_MMU_NOTIFIER unset.

[Comment 28 Ionen Wolkens 2021-03-02 21:37:24 UTC]

As aforementioned this looks fixed, so I'll just close my own bug.