How to Add a User to Kubernetes?

calendar Jun 10, 2023 · 8 min read · k8s auth token x.509  ·
Share on: linkedin copy

Let's look at the authentication options in k8s and try to understand their pros and cons

All Kubernetes clusters have two categories of users: Kubernetes-managed service accounts and regular users. In other words, there's no way to add a user to Kubernetes via kubectl or an API call. Below we'll work out how to solve this problem.

Unlike users, a service account can be added to k8s as a separate resource, and its access credentials are stored in Secrets

You can enable several authentication methods at once. It's usually recommended to use at least two methods:

  • service account tokens for service accounts;
  • at least one other method for authenticating users.

When several authentication modules are enabled, the first module that successfully authenticates the request halts further execution. The API server does not guarantee the order in which the authenticators run.

x.509

Even though a regular user cannot be added via an API call, any user presenting a valid certificate signed by the cluster's certificate authority (CA) is considered authenticated. In this configuration, Kubernetes determines the username from the Common name field in the certificate's "subject" (for example, "/CN=bob"). The role-based access control (RBAC) subsystem then determines whether the user has the rights to perform a specific operation on a resource.

To begin with, you need to create a private key and a certificate signing request (CSR):

1openssl genrsa -out devopstrain.pem
2openssl req -new -key devopstrain.pem -out devopstrain.csr -subj "/CN=devopstrain"

You can also specify the user's groups:

1openssl req -new -key devopstrain.pem -out devopstrain.csr -subj "/CN=devopstrain/O=devgroup"

In this example the username will be devopstrain. The RBAC rules will be tied to it.

Let's output the contents of the CSR in base64 format:

1cat devopstrain.csr | base64 | tr -d '\n'

Let's create a yaml with a CertificateSigningRequest. Note the request field — you need to fill it with the content obtained in the previous command.

 1cat <<'EOF'> devopstrain-csr.yaml
 2apiVersion: certificates.k8s.io/v1
 3kind: CertificateSigningRequest
 4metadata:
 5  name: user-request-devopstrain
 6spec:
 7  groups:
 8  - system:authenticated
 9  request: LS0tLS1CRUdJTi...
10  signerName: kubernetes.io/kube-apiserver-client
11  expirationSeconds: 315569260
12  usages:
13  - digital signature
14  - key encipherment
15  - client auth
16EOF

The expirationSeconds field sets the certificate's validity period. In this case, 10 years.

Let's create the resource and check its status:

1kubectl create -f devopstrain-csr.yaml
2kubectl certificate approve user-request-devopstrain

The certificate should be signed. Now let's download it

1kubectl get csr user-request-devopstrain -o jsonpath='{.status.certificate}' | base64 -d > devopstrain-user.crt

From the already active (current working kubeconfig) you need to extract the certificate authority:

1kubectl config view --raw -o go-template='{{index ((index (index .clusters 0) "cluster")) "certificate-authority-data"|base64decode}}' > ca.crt

And let's create a kubeconfig for the new user:

1kubectl --kubeconfig ~/.kube/config-devopstrain config set-cluster preprod --server=https://KUBERNETES-API-ADDRESS
2kubectl --kubeconfig ~/.kube/config-devopstrain config set-cluster preprod --certificate-authority=ca.crt
3kubectl --kubeconfig ~/.kube/config-devopstrain config set-credentials devopstrain --client-certificate=devopstrain-user.crt --client-key=devopstrain.pem --embed-certs=true
4kubectl --kubeconfig ~/.kube/config-devopstrain config set-context default --cluster=preprod --user=devopstrain
5kubectl --kubeconfig ~/.kube/config-devopstrain config use-context default

Replace KUBERNETES-API-ADDRESS with the host or IP address of the k8s API server

Add RBAC rules for the new user:

 1cat <<'EOF'> devopstrain-rbac.yaml
 2apiVersion: v1
 3kind: Namespace
 4metadata:
 5  name: devopstrain-ns
 6spec: {}
 7status: {}
 8---
 9kind: Role
10apiVersion: rbac.authorization.k8s.io/v1
11metadata:
12  name: devopstrain
13  namespace: devopstrain-ns
14rules:
15- apiGroups: ["", "extensions", "apps"]
16  resources: ["*"]
17  verbs: ["*"]
18- apiGroups: ["batch"]
19  resources:
20  - jobs
21  - cronjobs
22  verbs: ["*"]
23---
24kind: RoleBinding
25apiVersion: rbac.authorization.k8s.io/v1
26metadata:
27  name: devopstrain
28  namespace: devopstrain-ns
29subjects:
30- kind: User
31  name: devopstrain
32  apiGroup: rbac.authorization.k8s.io
33roleRef:
34  apiGroup: rbac.authorization.k8s.io
35  kind: Role
36  name: devopstrain
37EOF
38
39kubectl apply -f devopstrain-rbac.yaml

Let's test access through the kubeconfig we created:

1kubectl --kubeconfig ~/.kube/config-devopstrain get pods -n devopstrain-ns

Downsides of this approach:

  • external certificate management is required (there's no built-in user database)
  • a user's certificate cannot be revoked (that is, access can only be restricted at the IP or RBAC level)

Static token

The API server reads tokens from a file when the --token-auth-file=SOMEFILE parameter is specified on the command line. Currently the tokens are valid indefinitely, and the list of tokens cannot be changed without restarting the API server.

The token file is a csv file with at least 3 columns: token, username, user uid, followed by optional group names.

Note:

If you have more than one group, the column must be in double quotes, for example:

1token,user,uid,"group1,group2,group3"

Adding a bearer token to a request

When using bearer token authentication from an http client, the API server expects an Authorization header with the value Bearer .

1Authorization: Bearer 31ada4fd-adec-460c-809a-9e56ceb75269

Bootstrap token

To enable simplified initialization of new clusters, Kubernetes includes a dynamically managed token type called a Bootstrap Token. These tokens are stored as Secrets in the kube-system namespace, where they can be dynamically managed and created. The Controller Manager contains a TokenCleaner controller that removes Bootstrap tokens as they expire.

Service account token

This is the authentication method enabled by default, which uses signed tokens to validate requests. Let's create the necessary resources:

 1apiVersion: v1
 2kind: ServiceAccount
 3metadata:
 4  name: devopstrain
 5  namespace: default
 6secrets:
 7- name: devopstrain-token
 8---
 9apiVersion: v1
10kind: Secret
11metadata:
12  name: devopstrain-token
13  annotations:
14    kubernetes.io/service-account.name: devopstrain
15type: kubernetes.io/service-account-token

After this, you can create a kubeconfig file in a similar way to the x.509 certificate case. The difference is that you need to pass in the token:

1kubectl config set-credentials USER --token=TOKEN-FROM-SECRET

The downside of this approach is that the access credentials live in secrets, and anyone who can read them can obtain the same level of access as this service account. The upside is that access can be revoked.

OpenID Connect token

OIDC stands for "OpenID Connect" and is an authentication protocol built on top of the OAuth 2.0 protocol. OIDC provides a standardized way to authenticate users and pass claims about a user between web applications and identity providers.

OIDC was designed to simplify the development of applications that use multiple identity providers. It allows an application to authorize a user via any provider that supports the OIDC protocol and to obtain information about the user, such as their name, email, and other information that the identity provider supplies.

To configure OIDC support in the Kubernetes API server, you need to pass the following parameters:

--oidc-issuer-url - the URL of the OpenID provider you want to use for authentication.

--oidc-client-id - the client identifier that was registered on the OpenID provider's side, identifying your Kubernetes cluster.

--oidc-username-claim - the name of the field that contains the username in the OpenID token. This may differ depending on the OpenID provider.

--oidc-groups-claim - the name of the field that contains the user's list of groups in the OpenID token. This may also differ depending on the OpenID provider.

--oidc-ca-file - the path to the root certificate authority's certificate file

There are other parameters besides these. Refer to the documentation.

Webhooks

In this scenario, each user is issued a token by a third-party identity provider (often abbreviated to IdP). As with other authentication flows, the user includes this when sending an API request to the cluster. However, instead of trying to parse this token itself, the cluster instead triggers a webhook upon receiving the request and passes the token to the third-party provider for authentication. This is essentially similar to the static token flow, except that instead of comparing the token against a file on the API server, the cluster reaches out to the identity provider for a "yes" or "no" answer.

Authenticating proxies

The last method of authenticating third-party users that we'll cover in this article is the use of an Authenticating Proxy (often abbreviated to AP). This approach differs from all the others in this article in that it doesn't require the use of access tokens. You can require users to authenticate for each request to your cluster separately and avoid the problems of token revocation altogether.

This method works such that requests are not routed directly to your cluster's API server, but are instead sent to a third-party service: the Authenticating Proxy. It functions as the Identity Provider for this authentication method, which means it's the place where your users must prove their identity. This can be done in any way you like — a username and password, a hardware token such as a Yubikey, or SSH keys configured in your terminal profile.

Once the user has authenticated with the Authenticating Proxy, the proxy forwards the request to the cluster along with all the necessary information about the user. Communication between the AP and your cluster happens over HTTP secured by client certificates. The AP must present a valid client certificate signed by a certificate authority recognized by the cluster before any other information in its HTTP requests to the cluster will be read.

This authentication method is very flexible, allowing you to use any user authentication method you like, and it fully decouples the authentication layer from your cluster. If you need to revoke access for a particular user, you can do so at the AP level without changing your cluster's settings and risking downtime.