Demo: Bank-Vaults secret injection

Based on this blog post.

What's this about?

Bank-Vaults is able to inject secrets directly into your applications running on Kubernetes without using Kubernetes secrets.

This repo is a quick and easy demonstration of the feature.

Prerequisites

• Ability to setup a Kubernetes cluster (eg. using KinD)
• kubectl
• kustomize
• Helm
• vault CLI
• kubectl view-secret plugin (optional)

Preparations

Set up a new Kubernetes cluster using the tools of your choice.

This guide uses KinD:

kind create cluster

The rest of the instructions assume your current context is set to your demo cluster.

Install the Vault operator:

helm upgrade --install --wait --namespace vault-system --create-namespace vault-operator oci://ghcr.io/bank-vaults/helm-charts/vault-operator

Install the mutating webhook:

helm upgrade --install --wait --namespace vault-system --create-namespace vault-secrets-webhook oci://ghcr.io/bank-vaults/helm-charts/vault-secrets-webhook

Install a new Vault instance:

kustomize build deploy/vault | kubectl apply -f -

sleep 2
kubectl -n vault wait pods vault-0 --for condition=Ready --timeout=120s # wait for Vault to become ready

Set the Vault token from the Kubernetes secret:

export VAULT_TOKEN=$(kubectl -n vault get secrets vault-unseal-keys -o jsonpath={.data.vault-root} | base64 --decode)

Tell the CLI where Vault is listening (optional: this should be the default):

export VAULT_ADDR=http://127.0.0.1:8200

Port forward to the Vault service:

kubectl -n vault port-forward service/vault 8200 1>/dev/null &

Check access to Vault:

vault kv get secret/accounts/aws

Alternatively, open the UI (and login with the root token):

open $VAULT_ADDR

Demo

Deploy the demo application:

kustomize build deploy/demo | kubectl apply -f -

kubectl wait deploy http-echo --for condition=Available=true --timeout=60s # wait for the application to become ready

Port forward to the http-echo service:

kubectl port-forward service/http-echo 8080 1>/dev/null &

Look at the Pod (and notice that no mutation happened):

kubectl get pods -o yaml

Look at the environment variable values:

curl http://127.0.0.1:8080/env 2>/dev/null | grep -e AWS -e MYSQL

Expected output:

MYSQL_PASSWORD=vault:secret/data/mysql#MYSQL_PASSWORD
AWS_SECRET_ACCESS_KEY=vault:secret/data/accounts/aws#AWS_SECRET_ACCESS_KEY
AWS_ACCESS_KEY_ID=vault:secret/data/accounts/aws#AWS_ACCESS_KEY_ID

Enable mutation to inject secret values:

kubectl patch deploy http-echo --type=json -p='[{"op":"remove","path":"/spec/template/metadata/annotations/vault.security.banzaicloud.io~1mutate"}]'

kubectl rollout status deploy http-echo --timeout=60s # wait for the rollout to finish

(You have to restart the port forward at this point):

kill %2
wait %2
kubectl port-forward service/http-echo 8080 1>/dev/null &

Look at the Pod (and notice a number of mutations: init container, volumes and mounts, entrypoint (command) changed):

kubectl get pods -o yaml

Look at the environment variable values again:

curl http://127.0.0.1:8080/env 2>/dev/null | grep -e AWS -e MYSQL

Expected output:

MYSQL_PASSWORD=3xtr3ms3cr3t
AWS_SECRET_ACCESS_KEY=s3cr3t
AWS_ACCESS_KEY_ID=secretId

Cleanup

Kill background jobs:

kill %2 # demo app port-forward
kill %1 # vault port-forward

Tear down the Kubernetes cluster:

kind delete cluster