How to add a user to Kubernetes?

calendar Jun 10, 2023 · 8 min read · k8s auth token x.509  ·
Share on: linkedin copy

Let's consider authentication options in Kubernetes (k8s) and try to understand their pros and cons.

All Kubernetes clusters have two categories of users: managed Kubernetes service accounts and regular users. In other words, there is no way to add a user to Kubernetes through kubectl or an API call. We will explore how to solve this problem below.

Unlike users, a service account can be added to k8s as a separate resource, and its access data is stored in Secrets.

You can enable multiple authentication methods simultaneously. It is usually recommended to use at least two methods:

  • Service account tokens for service accounts;
  • At least one other method for user authentication.

When multiple authentication modules are enabled, the first module that successfully authenticates the request stops further execution. The API server does not guarantee the order of authenticator execution.

x.509

Although a regular user cannot be added through an API call, any user presenting a valid certificate signed by the cluster's Certificate Authority (CA) is considered authenticated. In this configuration, Kubernetes determines the username from the Common name field in the certificate's "subject" (e.g., "/CN=bob"). Then, the Role-Based Access Control (RBAC) subsystem will determine if the user has the rights to perform a specific operation on a resource.

To start, you need to create a private key and a certificate signing request (CSR):

1openssl genrsa -out devopstrain.pem
2openssl req -new -key devopstrain.pem -out devopstrain.csr -subj "/CN=devopstrain"

You can also specify user groups:

1openssl req -new -key devopstrain.pem -out devopstrain.csr -subj "/CN=devopstrain/O=devgroup"

In this example, the username will be devopstrain. The RBAC will be tied to this name.

Let's output the content of the CSR in base64 format:

1cat devopstrain.csr | base64 | tr -d '\n'

Create a YAML file for the CertificateSigningRequest. Note the request field, which needs to be filled with the content obtained from the previous command.

 1cat <<'EOF'> devopstrain-csr.yaml
 2apiVersion: certificates.k8s.io/v1
 3kind: CertificateSigningRequest
 4metadata:
 5  name: user-request-devopstrain
 6spec:
 7  groups:
 8  - system:authenticated
 9  request: LS0tLS1CRUdJTi...
10  signerName: kubernetes.io/kube-apiserver-client
11  expirationSeconds: 315569260
12  usages:
13  - digital signature
14  - key encipherment
15  - client auth
16EOF

The expirationSeconds field sets the certificate's validity period. In this case, it is 10 years.

Create the resource and check its status:

1kubectl create -f devopstrain-csr.yaml
2kubectl certificate approve user-request-devopstrain

The certificate should be signed. Now, download it:

1kubectl get csr user-request-devopstrain -o jsonpath='{.status.certificate}' | base64 -d > devopstrain-user.crt

Let's break down the steps to create a kubeconfig for the new user devopstrain and set up RBAC rules for them:

Extract the Certificate Authority

First, extract the certificate authority (CA) from the current active kubeconfig:

1kubectl config view --raw -o go-template='{{index ((index (index .clusters 0) "cluster")) "certificate-authority-data"|base64decode}}' > ca.crt

Create a kubeconfig for the New User

Next, create a kubeconfig file for the new user devopstrain. Replace KUBERNETES-API-ADDRESS with the actual host or IP address of your Kubernetes API server:

1kubectl --kubeconfig ~/.kube/config-devopstrain config set-cluster preprod --server=https://KUBERNETES-API-ADDRESS
2kubectl --kubeconfig ~/.kube/config-devopstrain config set-cluster preprod --certificate-authority=ca.crt
3kubectl --kubeconfig ~/.kube/config-devopstrain config set-credentials devopstrain --client-certificate=devopstrain-user.crt --client-key=devopstrain.pem --embed-certs=true
4kubectl --kubeconfig ~/.kube/config-devopstrain config set-context default --cluster=preprod --user=devopstrain
5kubectl --kubeconfig ~/.kube/config-devopstrain config use-context default

Add RBAC Rules for the New User

Create a YAML file to define the RBAC rules for the new user:

 1cat <<'EOF'> devopstrain-rbac.yaml
 2apiVersion: v1
 3kind: Namespace
 4metadata:
 5  name: devopstrain-ns
 6spec: {}
 7status: {}
 8---
 9kind: Role
10apiVersion: rbac.authorization.k8s.io/v1
11metadata:
12  name: devopstrain
13  namespace: devopstrain-ns
14rules:
15- apiGroups: ["", "extensions", "apps"]
16  resources: ["*"]
17  verbs: ["*"]
18- apiGroups: ["batch"]
19  resources:
20  - jobs
21  - cronjobs
22  verbs: ["*"]
23---
24kind: RoleBinding
25apiVersion: rbac.authorization.k8s.io/v1
26metadata:
27  name: devopstrain
28  namespace: devopstrain-ns
29subjects:
30- kind: User
31  name: devopstrain
32  apiGroup: rbac.authorization.k8s.io
33roleRef:
34  apiGroup: rbac.authorization.k8s.io
35  kind: Role
36  name: devopstrain
37EOF

Apply the RBAC rules:

1kubectl apply -f devopstrain-rbac.yaml

Test Access with the Created kubeconfig

Finally, test access using the newly created kubeconfig:

1kubectl --kubeconfig ~/.kube/config-devopstrain get pods -n devopstrain-ns

Drawbacks of This Approach

  1. External Certificate Management: This approach requires external management of certificates as Kubernetes does not have a built-in user database.
  2. Revoking Certificates: It is not possible to revoke a certificate for a user directly. Access can only be restricted at the IP level or through RBAC rules.

Static token

The API server reads tokens from a file when the --token-auth-file=SOMEFILE parameter is specified on the command line. Currently, tokens are valid indefinitely, and the list of tokens cannot be changed without restarting the API server.

The token file is a CSV file with at least 3 columns: token, username, user UID, followed by optional group names.

Note:

If you have more than one group, the column should be in double quotes, for example:

1token,user,uid,"group1,group2,group3"

Adding a Bearer Token to a Request

When using bearer token authentication from an HTTP client, the API server expects an Authorization header with the value Bearer <token>.

1Authorization: Bearer 31ada4fd-adec-460c-809a-9e56ceb75269

Bootstrap token

To facilitate the simplified initialization of new clusters, Kubernetes includes a dynamically managed token type called the Bootstrap Token. These tokens are stored as Secrets in the kube-system namespace, where they can be dynamically managed and created. The Controller Manager includes a TokenCleaner controller that removes Bootstrap tokens as they expire.

Service account token

This is the default authentication method that uses signed tokens to validate requests. Let's create the necessary resources:

 1apiVersion: v1
 2kind: ServiceAccount
 3metadata:
 4  name: devopstrain
 5  namespace: default
 6secrets:
 7- name: devopstrain-token
 8---
 9apiVersion: v1
10kind: Secret
11metadata:
12  name: devopstrain-token
13  annotations:
14    kubernetes.io/service-account.name: devopstrain
15type: kubernetes.io/service-account-token

After this, you can create a kubeconfig file similarly to how you would with x.509 certificates. The difference is that you need to pass the token:

1kubectl config set-credentials USER --token=TOKEN-FROM-SECRET

The downside of this approach is that access data is stored in secrets, so anyone who can read them can gain the same level of access as the service account. The advantage is that access can be revoked.

OpenID Connect token

OIDC stands for "OpenID Connect" and is an authentication protocol built on top of the OAuth 2.0 protocol. OIDC provides a standardized way to authenticate users and convey claims about the user between web applications and identity providers.

OIDC was designed to simplify the development of applications that use multiple identity providers. It allows an application to authenticate a user with any provider that supports the OIDC protocol and obtain information about the user, such as name, email, and other information provided by the identity provider.

To configure OIDC support in the Kubernetes API server, you need to pass the following parameters:

--oidc-issuer-url - The URL of the OpenID provider you want to use for authentication.

--oidc-client-id - The client ID that was registered with the OpenID provider, identifying your Kubernetes cluster.

--oidc-username-claim - The name of the field that contains the username in the OpenID token. This may vary depending on the OpenID provider.

--oidc-groups-claim - The name of the field that contains the list of user groups in the OpenID token. This may also vary depending on the OpenID provider.

--oidc-ca-file - The path to the root certificate authority file.

Additionally, there are other parameters. Refer to the documentation for more details.

Webhooks

In this scenario, each user is issued a token by a third-party identity provider (often abbreviated as IdP). Similar to other authentication flows, the user includes this token when making an API request to the cluster. However, instead of attempting to parse this token itself, the cluster instead triggers a webhook upon receiving the request and passes the token to the third-party provider for authentication. This is essentially similar to the static token flow, except that instead of comparing the token to a file in the API server, the cluster contacts the identity provider for a "yes" or "no" response.

Authenticating proxies

The last method of third-party user authentication we will discuss is the use of an Authenticating Proxy (often abbreviated as AP). This approach differs from all others in this article in that it does not require the use of access tokens. You can require users to authenticate for each request to your cluster separately and avoid the issues of token revocation altogether.

This method works such that requests are not sent directly to your cluster's API server but to a third-party service: the Authenticating Proxy. It functions as the Identity Provider for this authentication method, meaning it is where your users must prove their identity. This can be done in any way you like - username and password, a hardware token like a Yubikey, or SSH keys configured in the terminal profile.

Once the user has authenticated with the Authenticating Proxy, the proxy forwards the request to the cluster along with all the necessary user information. Communication between the AP and your cluster is via HTTP, secured with client certificates. The AP must present a valid client certificate, signed by a certificate authority recognized by the cluster, before any other information in its HTTP requests to the cluster will be read.

This authentication method is very flexible, allowing you to use any user authentication method you like and completely separate the authentication layer from your cluster. If you need to revoke access for a specific user, you can do so at the AP level without changing your cluster's settings and risking downtime.