HashiCorp Consul Service Mesh on Kubernetes Series - Part 1 - Introduction and Setup

Modern cloud-native architectures rely heavily on microservices, and Kubernetes has become the go-to platform for deploying, managing, and scaling these distributed applications. As the number of microservices grows, ensuring secure, reliable, and observable service-to-service communication becomes increasingly complex. This is where service mesh solutions, such as HashiCorp Consul, step in to provide a seamless approach to managing these challenges. In this blog post, we will delve into the integration of HashiCorp Consul Service Mesh with Kubernetes, exploring its architecture, features, and step-by-step deployment guide.

HashiCorp Vault Enterprise - Performance Replication on Kubernetes

This blog post dives into the technical implementation of Vault Enterprise replication within a Kubernetes environment. We’ll explore how to set up performance and disaster recovery replication, overcome common challenges, and ensure smooth synchronization between clusters. Whether you’re aiming for redundancy or better data locality, this guide will equip you with the insights and tools needed to leverage Vault’s enterprise-grade features in Kubernetes effectively.

Architecture

Prerequisites

• 2 Kubernetes clusters. *Note: for simulation purposes, you can also use a single Kubernetes cluster with multiple namespaces to host both Vault clusters.
• Helm installed
• kubectl installed
• Vault CLI installed
• jq installed
• Vault Enterprise license

Note: for this implementation LoadBalancer services are used on Kubernetes to expose the Vault services (the API/UI and the cluster address for replication). It is highly recommended to use a LoadBalancer rather than ingress to expose the cluster address for replication. Vault itself performs the TLS termination as the TLS certificates are mounted to the Vault pods from Kubernetes. Additionally, note that when enabling the replication, the primary cluster points to the secondary cluster address (port 8201) and not the API/UI address (port 8200). When the secondary cluster applies the replication token, however, it points to the API/UI address (port 8200) to unwrap it and compelete the setup of the replication. We will see this in more detail in the implementation section.

Harbor Registry - Automating LDAP/S Configuration - Part 1

The Harbor Registry is involved in many of my Kubernetes implementations in the field, and in almost every implementation I am asked about the options to configure LDAP/S authentication for the registry. Unfortuntely, neither the community Helm chart nor the Tanzu Harbor package provides native inputs for this setup. Fortunately, the Harbor REST API enables LDAP configuration programmatically. Automating this process ensures consistency across environments, faster deployments, and reduced chances of human error.

MinIO on vSphere - Automated Deployment and Onboarding

In the world of Kubernetes, reliable S3-compliant object storage is essential for tasks like storing backups. However, not everyone has access to a native S3-compatible solution, and setting one up can feel like a daunting task. MinIO, an open-source object storage solution, is a popular choice to fill this gap. Its lightweight, high-performance architecture makes it an excellent option for Kubernetes users seeking quick and reliable storage.

MinIO is also one of the most widely adopted open-source object storage solutions, thanks to its simplicity and S3 compatibility. It’s perfect for Kubernetes environments that need a reliable and scalable storage layer for backups, logs, or other data.

Fixing Missing TKRs in Existing TKGS Deployments

I regularly check the Tanzu Kubernetes Releases (TKR) release notes page for new updates. Yesterday, a new TKR was released with support for Kubernetes 1.28.8, and while attempting to test this new version in my TKGS environment, I realized that the TKR was not present in my environment and I started wondering why, as normally, when new TKRs are released, they immediately become available for deployment, since the vCenter is subscribed to the VMware public content library where all the TKRs are hosted. This time, that was not the case, so I started investigating.

HashiCorp Vault Intermediate CA Setup with Cert-Manager and Microsoft Root CA

In this post, we’ll explore how to set up HashiCorp Vault as an Intermediate Certificate Authority (CA) on a Kubernetes cluster, using a Microsoft CA as the Root CA. We’ll then integrate this setup with cert-manager, a powerful Kubernetes add-on for automating the management and issuance of TLS certificates.

The following is an architecture diagram for the use case I’ve built.

• A Microsoft Windows server is used as the Root CA of the environment.
• A Kubernetes cluster hosting shared/common services, including HashiCorp Vault. This is a cluster that can serve many other purposes/solutions, consumed by other clusters. The Vault server is deployed on this cluster and serves as an intermediate CA server, under the Microsoft Root CA server.
• A second Kubernetes cluster hosting the application(s). Cert-Manager is deployed on this cluster, integrated with Vault, and handles the management and issuance of TLS certificates against Vault using the ClusterIssuer resource. A web application, exposed via ingress, is running on this cluster. The ingress resource consumes its TLS certificate from Vault.

Prerequisites

• Atleast one running Kubernetes cluster. To follow along, you will need two Kubernetes clusters, one serving as the shared services cluster and the other as the workload/application cluster.
• Access to a Microsoft Root Certificate Authority (CA).
• The Helm CLI installed.
• Clone my GitHub repository. This repository contains all involved manifests, files and configurations needed.

Setting Up HashiCorp Vault as Intermediate CA

Deploy Initialize and Configure Vault

Install the Vault CLI. In the following example, Linux Ubuntu is used. If you are using a different operating system, refer to these instructions.

CAPV: Addressing Node Provisioning Issues Due to an Invalid State of ETCD

I recently ran into a strange scenario on a Kubernetes cluster after a sudden and unexpected crash it had experienced due to an issue in the underlying vSphere environment. In this case, the cluster was a TKG cluster (in fact, it happened to be the TKG management cluster), however, the same situation could have occurred on any cluster managed by Cluster API Provider vSphere (CAPV).

I have seen clusters unexpectedly crash many times before and most of the time, they successfully went back online when all nodes were up and running. In this case, however, some of the nodes could not boot properly, and Cluster API started attempting their reconciliation.