Kerberos ticket

[kandolfp]

Would it be possible to not go for username and password but to use a Kerberos ticket for the mount instead?

It would be the equivalent of a mount on the host as:
//server/share /mnt cifs multiuser,sec=krb5,user 0 0

Users will have access to it after a kinit to get a kerberos ticket. Equivalently the pod would need a kerberos ticket but that could be provided by an init script.

[andyzhangx]

so what's the mount options in this case?

[kandolfp]

the options above are multiuser,sec=krb5,user this way the mount uses kerberos and it separate per user. So if the drive is mounted but your users has no kerberos ticket you will not be able to access it.

get a ticket with kinit and you can access it.

It just exchanges user, passwd with a kerberos ticket. the ticket would be generated in the pod.

[avishefi]

Working with Kerberos also solves working under FIPS since NTLM-based SMB requires hmac-md5.

[shaunrampersad]

@kandolfp did you manage to get this to work with Kerberos? If so, how did you do it?

My issue is as @avishefi states, the cluster is installed with FIPS enabled so NTLM wont work.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[yrro]

the options above are multiuser,sec=krb5,user this way the mount uses kerberos and it separate per user. So if the drive is mounted but your users has no kerberos ticket you will not be able to access it.

Not sure what user means here. Isn't it the option used to specify username? It's probably just redundant when multiuser is specified.

In this mode, when a process tries to access a share, the kernel makes a cifs.spnego upcall to request encryption keys.

It's not clear how this can work when the process triggering the upcall is running in a container. I found a linux-security-module mailing list thread proposing changes to make it work smoothly but I don't know if it was adopted.

[andyzhangx]

for Kerberos ticket support, it requires agent node domain join first, here is one example:
https://learn.microsoft.com/en-us/azure/azure-netapp-files/configure-nfs-clients#ubuntu-configuration

[andyzhangx]

since this feature is already implemented by this driver, when the kerberos ticket is expired, how does this driver handle this issue? do we need to unmount the pv and mount again?

[yrro]

I think the existing Kerberos support is a little bit incomplete. As a user I would like to put a keytab into a secret and have that be used to authenticate the mount; I don't want to have to manage a directory of credentials cache files on every node...

[avishefi]

@yrro take a look at: https://github.com/kubernetes-csi/csi-driver-smb/blob/master/docs/driver-parameters.md#pass-kerberos-ticket-in-kubernetes-secret

[yrro]

"To pass a ticket through secret, it needs to be acquired" - there's the problem. The tickets in a credential cache expire within a few hours.

What I'd like as a user is to have csi-driver-smb handle obtaining the ticket automatically. To do this, a long term secret (in the form of the user's Kerberos keys stored in a keytab file) are required. I'd like to provide the keytab file to csi-driver-smb via a secret object. The mount performed by csi-driver-smb would use the sec=krb5i option.

The difficultly comes when the kernel makes the cifs.spengo upcall in order to obtain Kerberos keys. csi-driver-smb would have to service this upcall, and at that point it would need to use the keytab to obtain a TGT, and then use the TGT to obtain a ST and return that back to the kernel (I'm handwaving here, but hopefully that makes sense to someone who knows how kerberos-authenticated SMB mounts on Linux is implemented).

[cccsss01]

I'd like this feature as well

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[yrro]

/remove-lifecycle rotten

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle rotten
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[yrro]

/remove-lifecycle rotten

[RustyF]

I would like this feature too. At the moment, I am faced with trying to create a sidecar container to refresh the tickets and then update the secret.

[andyzhangx]

I would like this feature too. At the moment, I am faced with trying to create a sidecar container to refresh the tickets and then update the secret.

@RustyF how do you do this? would you mind sharing your config here? thanks. and when the secret is updated, do you make remount with the new ticket?

[RustyF]

I've now done the work to create a sidecar that re-builds the ticket cache every 8 hours and updates the secret. Let me know if you want me to expand on this. It's probably an addition to the documentation.

I'm not sure if I've solved the PV remount issue yet. I've just installed https://github.com/stakater/Reloader to restart the deployment but not sure if this will have the required effect.

[avishefi]

@RustyF Secrets are updated on projected keys

[RustyF]

@avishefi The secret in this case is on the PV - not the pod - so I'm not sure how this would help? I'm a k8s newbie so maybe I've missed your point ;)

[avishefi]

I though you meant it is a Kubernetes secret. If it is a file on a PV then usually applications can refresh their ticket from the credentials cache and you don't need to restart the pod. You probably want to use an emptyDir instead of a PV if it is ephemeral.