Platform Engineering |

Stage I: Architecting a Bare-Metal KVM & K3s Foundation

Thu, 14 May 2026 00:00:00 +0000

Building Kubernetes in the cloud is easy—cloud providers abstract all the hard networking and compute layers away from you. To truly understand Cloud Native architecture, I decided to build a zero-trust environment from the ground up.

This post covers Stage I of my 10-Phase Engineering Roadmap: establishing the virtualized compute layer, deploying a zero-trust mesh VPN, bootstrapping the K3s cluster, and configuring edge routing.

1. The Base Compute Environment & Zero-Trust Access

To simulate a true enterprise environment, I am running a bare-metal Kali Linux host. However, I am not running the cluster directly on the host OS. Instead, I am using KVM (Kernel-based Virtual Machine) to spin up an isolated Ubuntu server guest.

Furthermore, to enforce zero-trust from day one, SSH port 22 is completely blocked from the public internet. Access to the Ubuntu KVM is handled exclusively via Tailscale, creating an encrypted wireguard mesh tunnel that allows me to SSH into the root environment securely from anywhere.

To confirm the virtualized compute layer was running smoothly, here is the output verifying the KVM guest status on the Kali host:

2. Declarative Infrastructure with OpenTofu

To eliminate manual configuration drift right from the start, I used OpenTofu with the libvirt provider to manage the underlying state of the Ubuntu virtual machine.

Here is a snippet of how I structured the infrastructure provisioning:

terraform {
 required_providers {
 libvirt = {
 source = "dmacvicar/libvirt"
 version = "0.7.6"
 }
 }
}

provider "libvirt" {
 uri = "qemu:///system"
}

# Download the official Ubuntu Cloud Image
# 1. The Base Image (Downloads and stores the raw Ubuntu image)
resource "libvirt_volume" "ubuntu_base" {
 name = "ubuntu-base.qcow2"
 pool = "default"
 source = "${path.module}/noble-server-cloudimg-amd64.img"
 format = "qcow2"
}

# 2. The Actual VM Drive (Clones the base and stretches it to 30GB)
resource "libvirt_volume" "ubuntu_image" {
 name = "ubuntu-24.04.qcow2"
 pool = "default"
 base_volume_id = libvirt_volume.ubuntu_base.id
 size = 32212254720 # 30GB - SSD
 format = "qcow2"
}

# Inject the cloud-init file (which now contains Tailscale)
resource "libvirt_cloudinit_disk" "commoninit" {
 name = "commoninit.iso"
 user_data = file("${path.module}/cloud_init.cfg")
 pool = "default"
}

# Define the Virtual Machine
resource "libvirt_domain" "k3s_node" {
 name = "k3s-server"
 memory = "6144" # 6GB RAM
 vcpu = 2 # 2 CPU Cores

 cloudinit = libvirt_cloudinit_disk.commoninit.id

 network_interface {
 network_name = "default"
 wait_for_lease = true
 }

 disk {
 volume_id = libvirt_volume.ubuntu_image.id
 }

 console {
 type = "pty"
 target_port = "0"
 target_type = "serial"
 }
}

The Challenge: The most frustrating part of this setup was dealing with libvirt socket permissions. By default, the OpenTofu provider running on the Kali host was getting permission denied when trying to connect to the qemu:///system socket. I had to properly configure the libvirt group policies and Polkit rules on the host to allow the declarative pipeline to automatically provision the VM without requiring interactive root prompts.

3. Bootstrapping Bare-Metal K3s

For container orchestration, I chose K3s. It is lightweight enough to run highly efficiently on the Ubuntu guest, but fully conformant for enterprise-grade deployments.

To install and bootstrap the cluster securely, I explicitly passed arguments to bind to my Tailscale interface and disabled the default local storage path to maintain strict control:

curl -sfL [https://get.k3s.io](https://get.k3s.io) | INSTALL_K3S_EXEC="--node-ip <TAILSCALE_IP_ADDRESS> --bind-address <TAILSCALE_IP_ADDRESS> --tls-san <TAILSCALE_IP_ADDRESS>" sh -

Once the script executed, the node successfully registered itself using the secure Tailscale interface, as shown in the cluster node status:

4. Edge Routing & Traefik Challenges

Networking on local KVM Kubernetes is drastically different than the cloud. You don’t have AWS or GCP to automatically provision an Elastic Load Balancer (ELB) for you.

I utilized Traefik as the primary Ingress Controller, deploying it via Helm to manage routing for my internal services.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
 name: n8n-ingress
 namespace: whatsapp-bot 
spec:
 ingressClassName: traefik
 rules:
 - host: <Magic-DNS-FROM-TAILSCALE>  # MagicDNS name
 http:
 paths:
 - path: /
 pathType: Prefix
 backend:
 service:
 name: n8n-service  # n8n service
 port:
 number: 5678 # 5678 default n8n port

The Fix: Originally, exposing services from an isolated KVM guest required complex iptables rules, sysctl IP forwarding, and dealing with port 80/443 conflicts on the Kali host. To completely bypass this networking headache, I utilized Tailscale MagicDNS. By binding K3s and Traefik directly to the Tailscale interface inside the VM, I created a secure overlay network. Now, Traefik routes traffic seamlessly via MagicDNS, completely bypassing the Kali host’s physical network bridge and eliminating the need for complex port-forwarding.

With the local DNS override in place and Traefik configured, I could securely access the internal dashboard. Here is the active HTTP routing rule proving the configuration works:

5. Stage I Outcomes

By the end of Phase 4, the foundation was set:

Infrastructure is codified: The Ubuntu VM environment can be reliably rebuilt using OpenTofu.
Orchestration is live: K3s is running natively and efficiently.
Access is secured: The host and guest are protected behind a Tailscale zero-trust perimeter.

With the base compute and networking established, the cluster was ready for application workloads.

Next up: In , I dive into how I layered GitHub Actions and ArgoCD on top of this foundation to enforce GitOps compliance, and how I secured the cluster credentials using HashiCorp Vault.

Cloud Native Architecture: Zero-Trust Bare Metal Kubernetes

Wed, 28 Jan 2026 00:00:00 +0000

An ongoing, 10-phase infrastructure build demonstrating modern DevSecOps principles. This active laboratory project is transforming bare-metal hardware into a highly available, self-healing, and secure Kubernetes environment using GitOps methodologies.

Overview

I wanted to move beyond simple cloud provider tutorials and understand how the underlying compute layers actually work. I engineered this cluster to enforce zero-trust security and eliminate manual configuration drift, proving that enterprise-grade automation can be built from bare metal using virtualization, secure tunneling, and GitOps.

Infrastructure Capabilities

1. Virtualized Compute & Zero-Trust Access

KVM Hypervisor - Ubuntu guest virtual machine running efficiently on a bare-metal Kali Linux host.
Tailscale Mesh VPN - Hardened, zero-trust SSH access tunnel completely isolating the host from the public internet.
Declarative Provisioning - Utilizing OpenTofu to dynamically provision and manage the infrastructure state.

2. Container Orchestration & Networking

K3s Orchestration - Lightweight, highly available Kubernetes deployment.
Dynamic Ingress - Traefik configured as the primary ingress controller to manage robust routing and load balancing.

3. CI/CD & GitOps

Continuous Integration - GitHub Actions automates the building and pushing of Docker images for immutable rollbacks.
ArgoCD Synchronization - Cluster state is bound directly to the Git repository using Helm and Kustomize.
Zero Manual Drift - ArgoCD automatically detects and overwrites any manual changes, enforcing strict GitOps compliance.

4. Secret Management

HashiCorp Vault - Centralized, encrypted storage for all sensitive credentials and API keys.
External Secrets Operator (ESO) - Dynamically injects Vault secrets directly into Kubernetes pods.

System Architecture

┌─────────────────┐ (Builds Docker Image) ┌───────────────────────┐
│ GitHub Actions │────────────────────────────▶│ Container Registry │
│ (CI Pipeline) │ │ (GHCR / Hub) │
└─────────────────┘ └───────────────────────┘
│ │
│ (Updates Manifests) │ (Pulls Image)
▼ ▼
┌──────────────┐ ┌───────────────┐ ┌───────────────────────┐
│ │ │ │ │ Kubernetes (K3s) │
│ Git Repo │────▶│ ArgoCD │────▶│ ┌───────────────────┐ │
│ (Manifests) │ │ (Controller) │ │ │ Traefik Ingress │ │
│ │ │ │ │ └───────────────────┘ │
└──────────────┘ └───────────────┘ │ ┌───────────────────┐ │
│ │ k3s-whatsapp-bot │ │
┌──────────────┐ ┌───────────────┐ │ │ (n8n + Postgres) │ │
│ │ │ External │ │ └───────────────────┘ │
│ HashiCorp │◀────│ Secrets │◀────│ ┌───────────────────┐ │
│ Vault │ │ Operator │ │ │ ESO Injector │ │
│ │ │ │ │ └───────────────────┘ │
└──────────────┘ └───────────────┘ └───────────────────────┘
[ Infrastructure: KVM Ubuntu Guest on Kali Metal | Secured via Tailscale ]

Engineering Outcomes

🚀 Full CI/CD: 100% automated pipeline from code push (GitHub Actions) to cluster synchronization (ArgoCD).
🔒 Security: Host isolated via Tailscale; zero hardcoded secrets via Vault and ESO.
📉 Configuration Drift: Reduced to 0% through strict ArgoCD reconciliation loops.

Technical Deep Dives (Architecture Series)

To explore the raw code, YAML manifests, and how I solved specific architectural challenges across the lifecycle, read my detailed engineering write-ups:

📝 - Deep dive into Phases 1-4: Virtualizing Ubuntu on Kali via KVM, Tailscale SSH tunnels, and OpenTofu provisioning.
📝 - Deep dive into Phases 5-6: Docker builds via GitHub Actions, Helm/ArgoCD drift elimination, and Vault/ESO injection.
📝 (Coming Soon) - Deep dive into Phases 7-8: Cloudflare Tunnels and Prometheus/Grafana telemetry.
📝 (Coming Soon) - Deep dive into Phases 9-10: Executing automated attack paths against the cluster and Building a SIEM alerting pipeline.

The 10-Phase Engineering Roadmap

This cluster is designed as a living DevSecOps laboratory. I am currently executing Phase 7 of a comprehensive, capability-driven lifecycle.

Stage I: The Compute Foundation (✅ Completed)

✅ Phase 1: Base Hypervisor - Bare-metal Kali Linux hosting KVM.
✅ Phase 2: Virtualization & Access - OpenTofu provisioning of the Ubuntu guest and zero-trust Tailscale SSH tunneling.
✅ Phase 3: Container Orchestration - High-availability K3s cluster bootstrapping.
✅ Phase 4: Edge Routing - Dynamic ingress and load balancing via Traefik.

Stage II: Automation & Zero-Trust (✅ Completed)

✅ Phase 5: CI/CD Pipeline - GitHub Actions building Docker images and ArgoCD/Helm synchronizing the GitOps state.
✅ Phase 6: Ephemeral Secrets - Zero-trust credential injection via HashiCorp Vault and ESO.

Stage III: Perimeter Defense & Observability (⏳ In Progress)

⏳ Phase 7: Zero-Trust Perimeter - Integrating Cloudflare Tunnels for unexposed, secure ingress.
⏳ Phase 8: Full-Stack Telemetry - Deploying Prometheus & Grafana for cluster observability.

Stage IV: Purple Teaming Laboratory (📅 Planned)

📅 Phase 9: Offensive Simulation (Red) - Executing automated attack paths against the cluster to validate resilience.
📅 Phase 10: Threat Detection (Blue) - Building a SIEM alerting pipeline to capture the offensive testing telemetry.