I'm having trouble mounting a kerberized NFS (version 3) share on Gentoo that works OK with CentOS on the same client. I can mount other NFS shares (with NFS option sec=sys), and I can get kerberos tickets (using kinit) for my own username. But, when I try to put it all together (with sec=krb5), it won't mount. The problem seems to be a missing krb5.keytab file. CentOS complains about it with warnings, but continues anyway, as the man pages say it should. Faced with the same situation, Gentoo throws errors in place of warnings and refuses to even try. Both have the same krb5.conf and relevant fstab lines. kernel: 2.6.32-gentoo-r7 net-fs/nfs-utils: 1.1.4-r1 app-crypt/mit-krb5: 1.6.3-r6 verbose rpc.gssd output On CentOS: Using keytab file '/etc/krb5.keytab' ERROR: No such file or directory while beginning keytab scan for keytab '/etc/krb5.keytab' ERROR: No usable keytab entries found in keytab '/etc/krb5.keytab' Do you have a valid keytab entry for nfs/<your.host>@<YOUR.REALM> in keytab file /etc/krb5.keytab ? Continuing without (machine) credentials - nfs4 mounts with Kerberos will fail destroying client clnt5 handling krb5 upcall Using keytab file '/etc/krb5.keytab' WARNING: Failed to obtain machine credentials for connection to server netapp.example.com doing error downcall handling krb5 upcall Using keytab file '/etc/krb5.keytab' WARNING: Failed to obtain machine credentials for connection to server netapp.example.com doing error downcall verbose rpc.gssd output on Gentoo: beginning poll destroying client clnt11 handling krb5 upcall Full hostname for 'netapp.example.com' is 'netapp.example.com' Full hostname for 'localhost' is 'localhost' No such file or directory while getting keytab entry for 'root/localhost@AD.EXAMPLE.COM' No such file or directory while getting keytab entry for 'nfs/localhost@AD.EXAMPLE.COM' No such file or directory while getting keytab entry for 'host/localhost@AD.EXAMPLE.COM' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' No such file or directory while getting keytab entry for 'root/localhost@example.com' No such file or directory while getting keytab entry for 'nfs/localhost@example.com' No such file or directory while getting keytab entry for 'host/localhost@example.com' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: gssd_refresh_krb5_machine_credential: no usable keytab entry found in keytab /etc/krb5.keytab for connection with host netapp.example.com ERROR: No credentials found for connection to server netapp.example.com doing error downcall handling krb5 upcall Full hostname for 'netapp.example.com' is 'netapp.example.com' Full hostname for 'localhost' is 'localhost' No such file or directory while getting keytab entry for 'root/localhost@AD.EXAMPLE.COM' No such file or directory while getting keytab entry for 'nfs/localhost@AD.EXAMPLE.COM' No such file or directory while getting keytab entry for 'host/localhost@AD.EXAMPLE.COM' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' No such file or directory while getting keytab entry for 'root/localhost@example.com' No such file or directory while getting keytab entry for 'nfs/localhost@example.com' No such file or directory while getting keytab entry for 'host/localhost@example.com' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: gssd_refresh_krb5_machine_credential: no usable keytab entry found in keytab /etc/krb5.keytab for connection with host netapp.example.com ERROR: No credentials found for connection to server netapp.example.com doing error downcall handling krb5 upcall Full hostname for 'netapp.example.com' is 'netapp.example.com' Full hostname for 'localhost' is 'localhost' No such file or directory while getting keytab entry for 'root/localhost@AD.EXAMPLE.COM' No such file or directory while getting keytab entry for 'nfs/localhost@AD.EXAMPLE.COM' No such file or directory while getting keytab entry for 'host/localhost@AD.EXAMPLE.COM' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' No such file or directory while getting keytab entry for 'root/localhost@example.com' No such file or directory while getting keytab entry for 'nfs/localhost@example.com' No such file or directory while getting keytab entry for 'host/localhost@example.com' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: No such file or directory while beginning keytab scan for keytab 'FILE:/etc/krb5.keytab' ERROR: gssd_refresh_krb5_machine_credential: no usable keytab entry found in keytab /etc/krb5.keytab for connection with host netapp.example.com ERROR: No credentials found for connection to server netapp.example.com doing error downcall destroying client clnt10 destroying client clntf exiting on signal 2
What options are you using for mount ? Can it be something like the problems reported in bug 295598 and bug 295552 ?
Quite possibly! I had seen bug 295552, though not 295598. I had already tried using option "vers=3", which didn't help, but I wasn't aware of "nfsvers"-- I will try that also. The relevant line from fstab: netapp.example.com:/vol/users/ /ad/eng/users nfs sec=krb5,hard,nolock,intr,tcp,rsize=8192,wsize=8192 0 0 That's copied over directly from CentOS, though; the only option that I believe is truly essential is "sec=krb5". Again, I tried appending vers=3 to the options, which had no effect. It might help to mention that I do have support for NFS version 4 in my kernel, though I'm not attempting to use it. I understand that nfs-utils may default to v4 on this kernel (2.6.32), but I don't see why specifying v3 explicitly doesn't seem to work. Is it possible to get it to report what version it's attempting to use when it mounts, to double-check?
Oh, and for the sake of completeness: it behaves the same whether I use "vers" or "nfsvers" (which, consulting the nfs man page again, isn't surprising). (In reply to comment #2) > Quite possibly! I had seen bug 295552, though not bug 295598. I had already tried > using option "vers=3", which didn't help, but I wasn't aware of "nfsvers"-- I > will try that also. > > The relevant line from fstab: > > netapp.example.com:/vol/users/ /ad/eng/users nfs > sec=krb5,hard,nolock,intr,tcp,rsize=8192,wsize=8192 0 0 > > That's copied over directly from CentOS, though; the only option that I believe > is truly essential is "sec=krb5". Again, I tried appending vers=3 to the > options, which had no effect. It might help to mention that I do have support > for NFS version 4 in my kernel, though I'm not attempting to use it. > > I understand that nfs-utils may default to v4 on this kernel (2.6.32), but I > don't see why specifying v3 explicitly doesn't seem to work. Is it possible to > get it to report what version it's attempting to use when it mounts, to > double-check? >
You can always trace the network communication (tcpdump/wireshark), the packets should tell the NFS version to be used. You can disable NFSv4 on the server. This might be a take some time to debug, since not many users have NFSv4 _and_ Kerberos. I'm planning to do so, but not now. Michael
Have you met ... http://bugs.gentoo.org/show_bug.cgi?id=293593 (same buzz words).
Thanks, I'll take a look at the network traffic, and then I should know what version I'm actually using. For the record, my goal is to use version 3, and not do anything with v4 at all-- from what I understand, what I'm trying actually *should* fail on version 4. (If the traffic shows that it's using v4, my next question would be to ask why it's ignoring the "vers" option, but I guess I'll cross that bridge if I get to it.) bug 293593 looks like they're specifically trying to use v4, so it most likely isn't the same issue (but again, we'll see what my traffic shows). Thanks for the suggestions. (In reply to comment #4) > You can always trace the network communication (tcpdump/wireshark), the packets > should tell the NFS version to be used. You can disable NFSv4 on the server. > > This might be a take some time to debug, since not many users have NFSv4 _and_ > Kerberos. I'm planning to do so, but not now. > > Michael >
I've tried tcpdump on both systems, and the traffic looks similar. (Is there anything in particular I should expect to see if I'm on one version of NFS versus the other?) I also tried removing version 4 support in the kernel, just for good measure, with no effect. (Oh, and just to clarify, I can't disable it server-side, because, well, it's not my server :) This has me thinking that it's using version 3 as it should be, but refusing to continue without the keytab. I'm pretty much out of ideas of what to try, though. Should I be talking with the NFS people at this point? (In reply to comment #6) > Thanks, I'll take a look at the network traffic, and then I should know what > version I'm actually using. For the record, my goal is to use version 3, and > not do anything with v4 at all-- from what I understand, what I'm trying > actually *should* fail on version 4. (If the traffic shows that it's using v4, > my next question would be to ask why it's ignoring the "vers" option, but I > guess I'll cross that bridge if I get to it.) bug 293593 looks like they're > specifically trying to use v4, so it most likely isn't the same issue (but > again, we'll see what my traffic shows). Thanks for the suggestions. > > (In reply to comment #4) > > You can always trace the network communication (tcpdump/wireshark), the packets > > should tell the NFS version to be used. You can disable NFSv4 on the server. > > > > This might be a take some time to debug, since not many users have NFSv4 _and_ > > Kerberos. I'm planning to do so, but not now. > > > > Michael > > >
Did a bit more digging, and this definitely isn't a gentoo issue. What I'm trying works on CentOS 5.5 and Fedora Core 6 (similar systems package-wise), and fails on Fedora 13 and gentoo. I tried compiling nfs-utils-1.0.10 from source on gentoo and retracing my steps, and it finally works. The rpc.gssd output is also identical to that in "verbose rpc.gssd output On CentOS" originally posted. So, it seems to be an issue with the behavior of recent versions of nfs-utils. I'll try to find out exactly *why* this is, and then post what I find. (In reply to comment #7) > I've tried tcpdump on both systems, and the traffic looks similar. (Is there > anything in particular I should expect to see if I'm on one version of NFS > versus the other?) I also tried removing version 4 support in the kernel, just > for good measure, with no effect. (Oh, and just to clarify, I can't disable it > server-side, because, well, it's not my server :) > > This has me thinking that it's using version 3 as it should be, but refusing to > continue without the keytab. I'm pretty much out of ideas of what to try, > though. Should I be talking with the NFS people at this point? > > (In reply to comment #6) > > Thanks, I'll take a look at the network traffic, and then I should know what > > version I'm actually using. For the record, my goal is to use version 3, and > > not do anything with v4 at all-- from what I understand, what I'm trying > > actually *should* fail on version 4. (If the traffic shows that it's using v4, > > my next question would be to ask why it's ignoring the "vers" option, but I > > guess I'll cross that bridge if I get to it.) bug 293593 looks like they're > > specifically trying to use v4, so it most likely isn't the same issue (but > > again, we'll see what my traffic shows). Thanks for the suggestions. > > > > (In reply to comment #4) > > > You can always trace the network communication (tcpdump/wireshark), the packets > > > should tell the NFS version to be used. You can disable NFSv4 on the server. > > > > > > This might be a take some time to debug, since not many users have NFSv4 _and_ > > > Kerberos. I'm planning to do so, but not now. > > > > > > Michael > > > > > >
