Kubernetes Data Protection: Getting Started with Kasten (K10)
In a recent Kubernetes project I was involved in, our team had to conduct an in-depth proof of concept for several Kubernetes data protection solutions. The main highlights of the PoC covered data protection for stateful applications and databases, disaster recovery, and application mobility, including relocating applications across Kubernetes clusters and even different types of Kubernetes clusters (for example, from TKG on-premise to AWS EKS, etc.).
One of the solutions we evaluated was Kasten (K10), a data management platform for Kubernetes, which is now a part of Veeam. The implementation of Kasten was one of the smoothest we have ever experienced in terms of ease of use, stability, and general clarity around getting things done, as everything is very well documented, which certainly cannot be taken for granted these days. :)
Another thing worth mentioning is that Kasten seems highly extensible and provides out-of-the-box integrations for industry standards, such as Kyverno, Open Policy Agent (OPA), HashiCorp Vault, Kanister, and more. It also seems to be one of the few Kubernetes data protection solutions to offer an extremely intuitive user interface, and I have to admit - I prefer the declarative approach (using Kubernetes YAML manifests) than UI whenever possible. Still, Kasten gives you the freedom of choice! You can do almost anything via the UI or declaratively using YAML manifests.
After gaining some experience and building my confidence around Kasten, I composed this post to provide a step-by-step installation guide for anyone looking to get started with the platform. Note: This guide assumes an enterprise-grade/production-ready deployment, leveraging ingress and LDAPS-based authentication for Kasten UI, the least privilege approach for Kasten permissions, etc. However, you can customize the configuration for your environment for simplicity (for example, if ingress, LDAPS, and least privilege are not necessary).
Prerequisites
To follow along, you will need the following:
- A Kubernetes cluster - any flavor of Kubernetes. I use TKG, but any flavor is fine. Your Kubernetes cluster also needs to have an ingress controller (I use Contour, but any is fine) and a CSI (I use vSphere CSI) installed.
- A backup location for storing the backups. Kasten supports many storage backends (refer to the documentation for specific ones). I used NFS for this tutorial.
- Helm CLI
To compose this guide, I used a TKG cluster running on vSphere. If your Kubernetes cluster is also running on vSphere and you want to create a least privileged role for Kasten, you can use my Ansible Playbook, which will also bind your Active Directory service account to the role in vSphere. Take the following steps to apply the Playbook:
Clone my Kasten repository from GitHub and cd into the installation/ansible/vsphere-permissions directory.
git clone https://github.com/itaytalmi/kasten-k10.git
cd kasten-k10/installation/ansible/vsphere-permissions
Modify the k10-vars.yml file, specify your vSphere credentials, and set your domain name and service account under the vsphere_roles section (e.g., terasky.demo\k10-admin).
Install the Playbook requirements.
pip3 install -r requirements.txt
ansible-galaxy install -r requirements.yml
Run the Playbook.
ansible-playbook playbook.yml -e action=configure -e=@k10-vars.yml
You can now check the new role and role binding in vSphere.
Deploy Kasten
Clone my Kasten repository from GitHub if you haven’t already, and cd into the installation directory.
git clone https://github.com/itaytalmi/kasten-k10.git
cd kasten-k10
Create the kasten-io namespace on your Kubernetes cluster.
kubectl apply -f k10-namespace.yaml
Deploy the pre-check tool and ensure it completes successfully.
curl https://docs.kasten.io/tools/k10_primer.sh | bash
Example output:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Namespace option not provided, using default namespace
100 7113 100 7113 0 0 29611 0 --:--:-- --:--:-- --:--:-- 29514
Checking for tools
--> Found kubectl
--> Found helm
Checking if the Kasten Helm repo is present
--> The Kasten Helm repo was found
Checking for required Helm version (>= v3.0.0)
--> No Tiller needed with Helm v3.9.2
K10Primer image
--> Using Image (gcr.io/kasten-images/k10tools:5.0.5) to run test
Checking access to the Kubernetes context it-tkg-shared-svc-cls-admin@it-tkg-shared-svc-cls
--> Able to access the default Kubernetes namespace
K10 Kanister tools image
--> Using Kanister tools image (ghcr.io/kanisterio/kanister-tools:0.81.0) to run test
Running K10Primer Job in cluster with command-
./k10tools primer
serviceaccount/k10-primer created
clusterrolebinding.rbac.authorization.k8s.io/k10-primer created
job.batch/k10primer created
Waiting for pod k10primer--1-nf86r to be ready - ContainerCreating
Waiting for pod k10primer--1-nf86r to be ready - ContainerCreating
Pod Ready!
Kubernetes Version Check:
Valid kubernetes version (v1.22.9+vmware.1) - OK
RBAC Check:
Kubernetes RBAC is enabled - OK
Aggregated Layer Check:
The Kubernetes Aggregated Layer is enabled - OK
CSI Capabilities Check:
VolumeSnapshot CRD-based APIs are not installed - OK
Validating Provisioners:
csi.vsphere.vmware.com:
Storage Classes:
default
K10 supports the vSphere CSI driver natively. Creation of a K10 infrastucture profile is required.
Valid Storage Class - OK
k8s-storage-policy-vsan
K10 supports the vSphere CSI driver natively. Creation of a K10 infrastucture profile is required.
Valid Storage Class - OK
Validate Generic Volume Snapshot:
Pod created successfully - OK
GVS Backup command executed successfully - OK
Pod deleted successfully - OK
serviceaccount "k10-primer" deleted
clusterrolebinding.rbac.authorization.k8s.io "k10-primer" deleted
job.batch "k10primer" deleted
Remove the temporary file created by the pre-check tool.
rm -f k10primer.yaml
Modify the k10-ingress-tls-secret.yaml and specify using your TLS certificate and private key for Kasten UI. The certificate should be valid for Kasten hostname (e.g., it-k10.terasky.demo). Also, unless you have External DNS installed on your cluster, you will have to manually create the DNS record for Kasten.
Apply the secret.
kubectl apply -f k10-ingress-tls-secret.yaml
For LDAPS authentication, you must specify either your LDAPS certificate or your CA certificates/chain in the k10-ca-cert.yaml manifest, then apply the secret.
kubectl apply -f k10-ca-cert.yaml
Modify the values.yaml file using your environment configuration. Parameters you must update:
clusterName: the name of your Kubernetes cluster. This parameter is optional but recommended for better log visibility. For example:it-k10-clsingress.host: the FQDN of your Kasten instance. For example:it-k10.terasky.demo- In the
authsection, specify your LDAP/LDAPS configuration. Note: specify the FQDN of your Kasten instance for theauth.ldap.dashboardURLparameter. - You can specify LDAP groups under
auth.ldap.k10AdminGroupsandauth.ldap.k10AdminUsers. In my example, an Active Directory group namedk10-adminscontains user accounts with administrative access to Kasten. - Under the
eulasection, you must accept the EULA by setting it totrue, as well as your company name and email address. If you already have a Kasten license, you can specify it in thelicenseparameter. Otherwise, leave it blank for an evaluation license.
You can further customize your values.yaml file if necessary. Refer to the Helm chart documentation for more information.
Note: basic authentication is disabled by default. To enable it, add the following under the auth section:
auth:
basicAuth:
enabled: true
Add the kasten Helm repository.
helm repo add kasten https://charts.kasten.io
helm repo update
Deploy the Kasten Helm chart.
# Version-pinning to avoid breaking changes
CHART_VERSION=5.0.6
helm upgrade -i k10 kasten/k10 -n kasten-io --version "$CHART_VERSION" -f values.yaml
Example output:
Release "k10" does not exist. Installing it now.
NAME: k10
LAST DEPLOYED: Wed Aug 17 08:14:30 2022
NAMESPACE: kasten-io
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
Thank you for installing Kasten’s K10 Data Management Platform!
Documentation can be found at https://docs.kasten.io/.
How to access the K10 Dashboard:
You are using the system's default ingress controller. Please ask your
administrator for instructions on how to access the cluster.
WebUI location: https://it-k10.terasky.demo/k10
...
Wait for all Pods to initialize. It may take a few minutes. You can also view the ingress resource created by the Helm chart.
kubectl get pods,ingress -n kasten-io
Example output:
NAME READY STATUS RESTARTS AGE
pod/aggregatedapis-svc-7bb8498ccb-tf72l 1/1 Running 0 7m8s
pod/auth-svc-7489dd878b-qg2pn 2/2 Running 0 7m6s
pod/catalog-svc-7ff7988d9f-b985b 2/2 Running 0 7m7s
pod/controllermanager-svc-789784f76d-kprl5 1/1 Running 0 7m8s
pod/crypto-svc-8578899cdf-8n8fk 3/3 Running 0 7m6s
pod/dashboardbff-svc-f46b9cc98-zj8cr 1/1 Running 0 7m7s
pod/executor-svc-5bfbf987b8-bjc4c 2/2 Running 0 7m7s
pod/executor-svc-5bfbf987b8-f6sxj 2/2 Running 0 7m7s
pod/executor-svc-5bfbf987b8-wskcj 2/2 Running 0 7m7s
pod/frontend-svc-5695d9474f-x99jh 1/1 Running 0 7m7s
pod/gateway-786dd79479-g9xgg 1/1 Running 0 7m8s
pod/jobs-svc-9c6f4ddc-9kfzv 1/1 Running 0 7m7s
pod/k10-grafana-5ff69c8d54-ztw22 1/1 Running 0 7m7s
pod/kanister-svc-bc8d9499c-phg95 1/1 Running 0 7m7s
pod/logging-svc-5874577c74-n656g 1/1 Running 0 7m8s
pod/metering-svc-79d4b9447f-dzfz4 1/1 Running 0 7m7s
pod/prometheus-server-cf868d45d-zfgmq 2/2 Running 0 7m7s
pod/state-svc-cbcfc7dff-5bqhv 2/2 Running 0 7m8s
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.networking.k8s.io/k10-ingress <none> it-k10.terasky.demo 10.100.154.9 80, 443 7m8s
You should now be able to access the Kasten UI by browsing to https://your-kasten-fqdn/k10/. Note: do not omit the URL’s trailing slash (/). The ingress configuration requires it.
You should be able to log in using your LDAP user. For example, I use an Active Directory service account named k10-admin in a group named k10-admins, specified in the values.yaml file.
Configure Kasten
Once Kasten is deployed, the infrastructure must be configured before you can test your backup and restore.
Since I used NFS for this guide, I had to create a PV and a PVC pointing to my NFS export. If you are following along and using NFS, you can modify the k10-nfs.yaml file and set your NFS server information. Then, apply the manifest.
kubectl apply -f k10-nfs.yaml
Then, you have to create a location profile on Kasten, pointing to the PVC you created in the previous step. You can do so from the UI under Settings > Locations or by applying the k10-nfs-location-profile.yaml manifest.
kubectl apply -f k10-nfs-location-profile.yaml
After applying the manifest, you can also view the location profile from the UI.
Modify the k10-infra-vsphere.yaml manifest using your environment configuration. For the secret, you have to specify the Kasten service account. For the profile configuration, enter the vCenter server address. Then, apply the manifest. If you prefer to configure the infrastructure from the UI, you can do so under Settings > Infrastructure.
kubectl apply -f k10-infra-vsphere.yaml
After applying the manifest, you can also view the infrastructure profile from the UI.
Deploy a Sample Application
For this guide, I used a sample WordPress application since it is easy to deploy and contains a MySQL database and persistent volumes, which is nice to use for demonstrating backup and restore.
Add the Bitnami Helm repository.
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
Deploy the WordPress Helm chart.
helm upgrade -i demo-k10-wordpress -n demo-k10-wordpress --create-namespace bitnami/wordpress \
--set wordpressUsername="demo-k10" \
--set wordpressPassword="Kasten1!" \
--set wordpressBlogName="K10 Demo Blog!"
You can list the WordPress pods and PVCs. Wait for the Pods to initialize.
kubectl get pods,pvc -n demo-k10-wordpress
Example output:
NAME READY STATUS RESTARTS AGE
pod/demo-k10-wordpress-68ff5d89c9-jknzg 1/1 Running 0 114s
pod/demo-k10-wordpress-mariadb-0 1/1 Running 0 114s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
persistentvolumeclaim/data-demo-k10-wordpress-mariadb-0 Bound pvc-8359f4a3-31d1-459e-a2ee-d08054d0b6eb 8Gi RWO k8s-storage-policy-vsan 114s
persistentvolumeclaim/demo-k10-wordpress Bound pvc-6b171519-d178-4668-b83d-b37774ddc9a3 10Gi RWO k8s-storage-policy-vsan 114s
Get the WordPress load balancer IP address.
kubectl get svc -n demo-k10-wordpress demo-k10-wordpress -o jsonpath='{.status.loadBalancer.ingress[].ip}'
If you browse to WordPress, you should see the default “Hello world” page.
Let’s create a new blog post. Navigate to http://your-wordpress-LB-IP/admin and log in using the credentials we specified for the Helm chart. Username: demo-k10, password: Kasten1!
To create a new post, click the New button at the top.
Set a title, and some example content, then click Publish.
Now browse to your blog, and you should see the new post.
Backup and Restore the Application
Now that we have some data, we can run a backup.
From Kasten UI, navigate to Applications.
Set the filter to Unmanaged.
Locate the demo-k10-wordpress application, then select Create a policy.
Select the Enable Backups via Snapshot Exports option, then click Create Policy.
Side note: notice the YAML button at the bottom? Clicking it will give you the policy manifest, so you can create it declaratively by simply applying it using kubectl without interacting with the Kasten UI!
After creating the policy, click Run Once to trigger the backup.
Click Run Policy.
Go to the main dashboard and click the running task.
Wait for the backup to complete.
Let’s destroy the WordPress application before we run the restore.
helm uninstall demo-k10-wordpress -n demo-k10-wordpress
kubectl delete ns demo-k10-wordpress
Now that the application and its data are gone, we can attempt our restore.
From Kasten UI, navigate to Applications, click Filter by Status and select Removed.
Locate the demo-k10-wordpress application and select Restore.
Select the restore point.
Click Restore.
Go to the main dashboard and wait for the restore task to complete.
You should see the restored resources on your Kubernetes cluster.
Wait for the Pods to initialize.
kubectl get pods,pvc -n demo-k10-wordpress
Get the WordPress load balancer IP address (it might have changed) and access your blog from a web browser. You should see your Kasten Demo post on the front page. Your data has been successfully restored. :)
Another thing worth mentioning is that the Kasten Helm chart also deploys a Grafana instance by default. You can browse to https://your-kasten-fqdn/k10/grafana/ to access Grafana. It contains a pre-built K10 Dashboard you can use to monitor the Kasten platform. You can also further customize this Grafana instance and build your dashboards.
Hopefully, this gives you an idea of Kasten’s simplicity and straightforwardness. In future posts, I will dive deeper into additional features Kasten has to offer.