Certificate-based Kubernetes authentication

This topic describes how to set up a Kubernetes Authenticator for certificate-based authentication of Kubernetes environments.

The Kubernetes Authenticator enables the following components to authenticate to Conjur using certificate-based authentication:

  • Conjur Kubernetes Follower

  • Conjur Follower deployed inside Kubernetes

  • Kubernetes workloads

 
    • Unless specifically noted otherwise, all references to Kubernetes apply to Self-hosted Kubernetes as well as Red Hat OpenShift and other supported Kubernetes-based implementations.
    • All references to Kubernetes namespaces intentionally include the OpenShift concept of project.

Prerequisites

  • Conjur cluster: This configuration assumes you have access to an up-and-running Conjur cluster. For details, see Setup.

  • This configuration assumes you are working from a Linux Shell or macOS.

  • To perform this task, you need:

    • Conjur admin permissions

    • Kubernetes cluster admin permissions OR the help of an available Kubernetes cluster admin

  • Make sure you have the Conjur CLI (v7.x+) installed and that you are logged in. For details, see Set up the Conjur CLI.

  • Make sure you have access to the Helm charts or raw manifests for preparing Kubernetes clusters and namespaces. These are available from the following locations:

Configure and enable a Kubernetes Authenticator

This section describes how to configure and enable a Kubernetes Authenticator.

 

  • This procedure involves tasks for both the Conjur admin and the Kubernetes cluster admin.

  • In the examples in this procedure, we use dev-cluster for the service ID. You can replace this service ID with any unique name for the webservice that describes the Kubernetes cluster that it's scoped for, for example, dev-cluster, test1, qa, prod.

  1. Kubernetes cluster admin: In this step, you create the following Kubernetes resources for the Kubernetes Authenticator inside your Kubernetes cluster:

    • A namespace, called cyberark-conjur

    • A service account for the Kubernetes Authenticator, authn-k8s-sa

    • A cluster role with the necessary RBAC permissions, conjur-clusterrole

    • A Golden ConfigMap, conjur-configmap, containing Conjur connection and configuration information

    To create the Kubernetes resources in your cluster:

    1. Before you begin, collect the following information from the Conjur admin:

      Conjur account

      The account name used when deploying Conjur

      Example: myorg

      Conjur URL

      • If your Follower is outside of Kubernetes, this is the Follower load balancer URL.

        Example: https://follower-lb.example.com

      • If you are deploying the Follower inside Kubernetes, this is the Conjur service URL within the Kubernetes cluster, in the following format: https://<follower-service-account>.<follower-namespace>.svc.cluster.local

        Example: https://conjur-follower.cyberark-conjur.svc.cluster.local

        If you have not yet deployed the Follower inside Kubernetes, provide the intended URL.

      Conjur certificate file

      The path to the Conjur certificate file.

      The path can be full or relative.

      For Helm, this must be a relative path starting from the Helm chart directory.

      Example: path/to/conjur.pem

      Kubernetes Authenticator service ID

      The service ID that will be used for Kubernetes Authenticator.

      Example: dev-cluster

      The Conjur admin will define Kubernetes Authenticator in the next steps, They should, however, be able to provide you with the intended authenticator service ID.

    2. Create the Kubernetes resources using the Kubernetes cluster preparation Helm chart, or using the raw Kubernetes manifest file:

       

      We recommend using this method to create the resources in your cluster. However, if the Helm repo is not available, use the raw Kubernetes manifest (see the Raw Kubernetes manifest tab).

      1. Add the Helm chart repository to your local list of Helm chart repositories:

          
        $ helm repo add cyberark https://cyberark.github.io/helm-charts

      2. Run the helm install command. The following helm install command creates: 

        • the cyberark-conjur namespace

        • the authn-k8s-sa service account

        • the Golden ConfigMap, conjur-configmap, using --set commands for the Conjur connection details

          
        helm install cluster-prep cyberark/conjur-config-cluster-prep \
  --namespace cyberark-conjur \
  --create-namespace \
  --set conjur.account="myorg" \
  --set conjur.applianceUrl="https://conjur-follower.cyberark-conjur.svc.cluster.local" \
  --set conjur.certificateFilePath="path/to/conjur.pem" \
  --set authnK8s.authenticatorID="dev-cluster" \
  --set authnK8s.serviceAccount.name="authn-k8s-sa"

        For more information about configurable values for the Kubernetes cluster preparation Helm chart, see Helm Chart for Conjur Config Cluster Preparation > Configuration.

        Kubernetes v1.24 and later do not generate secrets automatically for service accounts. Use Conjur Config cluster preparation Helm chart v0.2.1 or later, which creates these service account secrets automatically.

        If you use Helm chart v0.1.x with Kubernetes v1.24, you must create the secret manually, such as in the following example:

        apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
  name: conjur-oss-service-account-token
  annotations:
    kubernetes.io/service-account.name: conjur-oss
       

      If you are able to use Helm charts, we recommend doing so. For details, click the Helm chart tab.

      1. Create the cyberark-conjur namespace for Kubernetes Authenticator (for OpenShift used oc in place of kubectl):

          
        kubectl create namespace cyberark-conjur

      2. Download the raw Kubernetes manifest: In the Kubernetes Authenticator Client GitHub repository, from the version's Assets list, download the compressed TAR file (.tar.gz) which contains the raw Kubernetes manifests.

      3. Open the manifest in an editor and fill in the following values:

        Placeholder

        Description

        <Insert-Conjur-Account-Here>

        Conjur account to be used by the Kubernetes Authenticator, for example, myorg

        <Insert-Authenticator-ID-Here>

        Kubernetes Authenticator service ID, for example dev-cluster

        <Insert-Conjur-SSL-Certificate-Here>

        Conjur certificate file, for example, path/to/conjur.pem

      4. Apply the configuration to the cyberark-conjur namespace (for OpenShift used oc in place of kubectl):

          
        kubectl apply -f conjur-config-namespace-prep.yaml --namespace cyberark-conjur

  2. Conjur admin: In this step you define and load a policy for the Kubernetes Authenticator, dev-cluster.

    Conjur uses policy to enable (allowlist) the workloads that have access to the Kubernetes Authenticator and store the values necessary to create client certificates for mutual TLS.

    1. Using the Conjur CLI, log in to Conjur as admin. For details, see login.

    2. Save the following policy as k8s-authenticator-webservice.yml:

        
      # k8s-authenticator-webservice.yml
---
# =================================================
# == Enroll a Kubernetes authentication service
# =================================================
- !policy
  id: conjur/authn-k8s/dev-cluster
  annotations:
    description: K8s Authenticator policy definitions

  body:
  # vars for ocp/k8s api url & access creds
  - !variable kubernetes/service-account-token
  - !variable kubernetes/ca-cert
  - !variable kubernetes/api-url

  # vars for CA for this service ID
  - !variable ca/cert
  - !variable ca/key

  - !webservice
    annotations:
      description: Authenticator service for K8s cluster dev-cluster

  # Create 'consumers' group
  - !group consumers

  # Grant consumers group role authentication privileges
  - !permit
    role: !group consumers
    privilege: [ read, authenticate ]
    resource: !webservice

      The policy:

      • Defines variables to store details of the Kubernetes Authenticator service account used by the webservice to log in to the Kubernetes API as part of the authentication workflow

      • Defines variables to store a CA certificate and key used to create a CA for the authentication workflow

      • Creates a Kubernetes Authenticator webservice

      • Creates a consumers group that is authorized to use the Kubernetes Authenticator

    3. Load the policy into root:

        
      $ conjur policy load -f k8s-authenticator-webservice.yml -b root

  3. Conjur admin: In this step you create a private key (ca.key) and a root certificate (ca.cert). The content of these files is stored in the conjur/authn-k8s/<SERVICE_ID>/ca/key and conjur/authn-k8s/<SERVICE_ID>/ca/cert Conjur variables.

    1. Create environment variables:

      1. Create an environment variable for the service ID used when you created the policy above:

          
        $ SERVICE_ID=dev-cluster

      2. Create an environment variable for the Conjur account. This is the same account used when deploying Conjur (see account-name).

          
        $ CONJUR_ACCOUNT=myorg

      3. Create a CONFIG environment variable:

          
        $ CONFIG="
[ req ]
distinguished_name = dn
x509_extensions = v3_ca
[ dn ]
[ v3_ca ]
basicConstraints = critical,CA:TRUE
subjectKeyIdentifier   = hash
authorityKeyIdentifier = keyid:always,issuer:always
"

    2. Generate the OpenSSL private key:

        
      $ openssl genrsa -out ca.key 2048

    3. Generate the root CA certificate:

        
      $ openssl req -x509 -new -nodes -key ca.key -sha1 -days 3650 -set_serial 0x0 -out ca.cert \
  -subj "/CN=conjur.authn-k8s.$SERVICE_ID/OU=Conjur Kubernetes CA/O=$CONJUR_ACCOUNT" \
  -config <(echo "$CONFIG")

    4. Verify the certificate:

        
      $ openssl x509 -in ca.cert -text -noout

    5. Load the variable values:

        
      $ conjur variable set -i conjur/authn-k8s/$SERVICE_ID/ca/key -v "$(cat ca.key)"
        
      $ conjur variable set -i conjur/authn-k8s/$SERVICE_ID/ca/cert -v "$(cat ca.cert)"

  4. Conjur admin and Kubernetes cluster admin: In this step you collect information about the Kubernetes cluster so that you can configure Conjur to access the Kubernetes API.

    1. Kubernetes cluster admin: Obtain the Kubernetes cluster details listed below. For convenience, we have provided example commands (for macOS/Linux) for retrieving these details from the cluster, as well as sample output for each command. Retain this information as you'll need it when you create environment variables below.

       

      • For OpenShift, replace kubectl with oc

      • based64 -D is macOS notation, and might differ for Linux.

      • The Kubernetes client in Conjur must reach the Kubernetes API server. If there is a proxy defined between the Conjur Client and Kubernetes API server, define the environment variables https_proxy and/or http_proxy on each Host/Node. Pod picks up these environment settings and automatically redirects traffic through proxy to the Kubernetes API server, and allows configurations in the Kubernetes API server that is hosted in Cloud or remote network. These can also be set as ENV variables in Pods, and you can specify a username and password to authenticate to a proxy host.

        For example:

        http_proxy=http://conjur.org:8888/

        https_proxy=https://conjur.org:8888/

      • Get the Kubernetes service account token:

        Run:

          
        $ TOKEN_SECRET_NAME="$(kubectl get secrets -n cyberark-conjur | grep 'authn-k8s-sa.*service-account-token' | head -n1 | awk '{print $1}')"

        Then run:

          
        $ kubectl get secret $TOKEN_SECRET_NAME -n cyberark-conjur --output='go-template={{ .data.token }}' | base64 -D

        Sample output:

          
        eyJhbGciOiJSUzI1NiIsImtpZCI6IkQ5YXJsczlMRkU3ajB3Y19qLWhuWWF4eUhy
RldKdzdFOHZDOW5SalBDVWcifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2N
vdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJje
WJlcmFyay1jb25qdXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlY3J
ldC5uYW1lIjoiYXV0aG4tazhzLXNhLXRva2VuLTl2Y25uIiwia3ViZXJuZXRlcy5pb
y9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImF1dGhuLWs4cy1
zYSIsImt1YmVybmV0ZXMuaW8vc2VydmljZ

      • Get the Kubernetes cluster API URL:

          
        $ kubectl config view --raw --minify --flatten --output='jsonpath={.clusters[].cluster.server}'

        Sample output:

          
        https://mykubecluster.domain:6443

      • Create the certificate for the Kubernetes cluster:

        1. Run the OpenSSL s_client -connect command to get the first certificate chain, for example:

          openssl s_client -showcerts -connect k8s-lb.example.com:6443 < /dev/null 2> /dev/null | sed -ne '/-BEGIN CERTIFICATE/,/END CERTIFICATE/p' >

        2. Run the following to get the Kubernetes cluster API CA certificate:

          $ kubectl config view --raw --minify --flatten --output='jsonpath={.clusters[].cluster.certificate-authority-data}' | base64 -D
           

          This command requires users to authenticate with the cluster using a verified TLS connection.

          Sample output:

            
          -----BEGIN CERTIFICATE-----
MIIDdTCCAl2gAwIBAgIIRxsiSRkjFpwwDQYJKoZIhvcNAQELBQAwJjEkMCIGA1UE
AwwbaW5ncmVzcy1vcGVyYXRvckAxNjE5NDM0OTgzMB4XDTIxMDQyNjExMDMwNFoX
DTIzMDQyNjExMDMwNVowKjEoMCYGA1UEAwwfKi5hcHBzLmd1eXRlc3QuY3liZXJh
cmtkZW1vLmNvbTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAOVDkm+N
+6FQEGkJheRaKLQBwWnDfGEsMo8KBjU9JCFDGNs4aG54sXNfPiXx/MoOwB7y8dQ4
i4a+mRBkoOqRzmXqbMXAfDxON7ipzPbp/5fl8qDI6rKI001j5JAMs3ziGTXU87Hc
IiXQvvsi4A2ed10XGRGdxwMk4V54LnZTiG/bNIhPb6erMmTVaVfJjUAMCNWY/waN
VH0U51TnKpS4oGzbLnxnkgcp5rE6vZd2pquc8li6YrgOsv0Dl0cDeR6QDJ2zdTPg
dl5OjFS70MVGYeKdIwjeO6nTMd6r+OI85mgOsrBBW7Qlow+lFKFc/VNzs8ALunH0
ULb15BmVBjPu3SsCAwEAAaOBojCBnzAOBgNVHQ8BAf8EBAMCBaAwEwYDVR0lBAww
CgYIKwYBBQUHAwEwDAYDVR0TAQH/BAIwADAdBgNVHQ4EFgQU1OFLcHSg5pu+tvcO
8ZbE0Xs8Nc4wHwYDVR0jBBgwFoAUcLNcrEAE4dWcKnMhRedPjBmVkEgwKgYDVR0R
BCMwIYIfKi5hcHBzLmd1eXRlc3QuY3liZXJhcmtkZW1vLmNvbTANBgkqhkiG9w0B
AQsFAAOCAQEAZ53ohghLUtw9kf3z65p9zbpe4tqxQ5kzTNn0ghG+664/DwJz75Yo
CkfN5gvoxD2tVss1L8wzLhddr1tH9U1nOGa4vf5L7EoKz6+798XoKjQTTxHFh78U
VukSZyvwJV1Wmp1b+8wrayg4DhKyX13AwtXwuLYVAdBEjy14uYJ5aYEFWCE5+1pA
icEjaZGzEw18vc+IGhHfBascvefwKSQsjdIHBLYJyvg+W2cO076/G0EazL9Ko9hx
5lDlbv+VjeiH19RFMRd1qFYZSxLsiSM9XvyUgEoh7qpzfZeU4VDf6gcFS8bbrqmj
/Jb12ykw3kbSX6bPX3ImBJtY71JMsocY5g==
-----END CERTIFICATE-----
        3. Concatenate the two certificates and save as k8s_SNI_chain.pem.

        4. Verify the concatenated certificate from the Follower server against the Kubernetes load balancer:

          curl --cacert "k8s_SNI_chain.pem" https://k8s-lb.example.com:6443

          Expected response: "401: Unauthorized”. If you receive an SSL error, the certificate is not valid.

    2. Conjur admin: When you have the Kubernetes resource details:

      1. In your browser, log in to the Conjur UI as Conjur admin (see Login).

      2. Under Secrets, update the relevant conjur/authn-k8s/dev-cluster/kubernetes secrets (/api-url, /ca-cert, and /service-account-token) with the corresponding values.

      1. Create the following environment variables, replacing the <> with the Kubernetes details:

          
        $ SA_TOKEN=<Kubernetes service account token> https_proxy=<https proxy url>
          
        $ K8S_API_URL=<Kubernetes cluster API URL>

      2. Using the certificate that you created above, create the following environment variable:

          
        $ K8S_CA_CERT="$(cat k8s_SNI_chain.pem)"

      3. Store this information as Conjur variables:

        1. Kubernetes service account:

            
          $ conjur variable set -i conjur/authn-k8s/dev-cluster/kubernetes/service-account-token -v $SA_TOKEN

        2. Kubernetes cluster CA certificate:

            
          $ conjur variable set -i conjur/authn-k8s/dev-cluster/kubernetes/ca-cert -v $K8S_CA_CERT

        3. Kubernetes cluster API server URL:

            
          $ conjur variable set -i conjur/authn-k8s/dev-cluster/kubernetes/api-url -v $K8S_API_URL

  5. Conjur admin: In this step, you allowlist the Kubernetes Authenticator in Conjur.

    You need to allowlist the Kubernetes Authenticator on every Conjur node (Leader/Standby/Follower) that Kubernetes needs to authenticate to. For more details, see Configure authenticators.

     

    The value for this variable should be identical to the name given to the policy ID above, excluding the conjur/ prefix.

    For example, to allowlist conjur/authn-k8s/dev-cluster, allowlist authn-k8s/dev-cluster.

  6. Conjur admin:

    From the Leader run the following:

      
    $ curl -k https://localhost/info

    The output shows that the Kubernetes Authenticator is configured and enabled:

      
    "authenticators": {
    "installed": [
      "authn",
      "authn-azure",
      ..
      ..
      "authn-k8s",
      ..
      ..
    ],
    "configured": [
      "authn",
      "authn-k8s/dev-cluster"
    ],
    "enabled": [
      "authn",
      "authn-k8s/dev-cluster"
    ]
  }

Authentication steps

The following describes the security-related flow:

1

Establish mutual TLS

Before retrieving secrets, the authentication client establishes mutual TLS with the Kubernetes Authenticator in Conjur to ensure that both parties can verify their intended recipients. The mutual TLS protocol requires the client to have a client certificate that has been signed by a trusted CA (in this case, Conjur itself).

To retrieve this certificate, the client sends a Certificate Signing Request (CSR) to Conjur. The CSR metadata includes the host identity that the pod would like to use to authenticate, as well as information about the pod making the request.

Conjur checks the pod information against Kubernetes to verify that the pod is legitimate, and verifies that the pod should be allowed to authenticate as the host identity from the CSR.

If this verification succeeds, the client certificate is injected into the application Pod using the Kubernetes API. By using the API to perform this injection out-of-band (instead of returning it in the request response), the Kubernetes Authenticator ensures that the client certificate is delivered to the pod that matches the metadata in the CSR.

The certificate expires and is renewed automatically on a regular basis to reduce the chances of a malicious third party being able to use a compromised certificate to assume the application Pod's identity.

2

Get an access token

After receiving a client certificate, the authentication client uses it to establish a mutual TLS connection with Conjur and requests a short-lived access token. The access token is written to shared pod memory and can be used with one of the secret retrieval methods to retrieve secrets from Conjur. The token is short-lived to reduce the possibility that it can be used by a malicious third party if it is somehow compromised.

3

Retrieve secrets

The workflow can now use the access token to retrieve secrets with one of the secret retrieval methods.

If you have not yet deployed a Follower, see Set up Follower.

If you have already deployed a Follower, the integration setup is complete. App owners can now use this cert-based authentication endpoint when setting up their workloads to authenticate to and retrieve secrets from Conjur. For details, see Set up workloads (cert-based authentication).