My Kubernetes Setup with OPNsense and RKE2

Summary

This post documents my current Kubernetes architecture running in my homelab. The goal was not just “make it work”, but build something that is structured, secure, reproducible, and close to how I would design it in a small production environment.

The stack is built around:

  • OPNsense as firewall and entry point
  • HAProxy for load balancing
  • RKE2 as the Kubernetes distribution
  • Proxmox as virtualization layer
  • Dedicated database VMs (PostgreSQL and MariaDB)
  • WireGuard for secure access
  • Restic for encrypted S3 backups
  • Traefik + CrowdSec for ingress and security
  • MetalLB for static service IPs

Network Design

The internal cluster network runs on:

10.0.0.0/24

I wanted full control over routing and segmentation instead of relying on consumer router defaults. OPNsense sits at the edge and handles:

  • WAN connectivity
  • Internal routing
  • Firewall rules
  • HAProxy load balancing
  • WireGuard VPN termination

This gives me a clean separation between internet, VPN access, and internal cluster traffic.

Virtualization Layer (Proxmox)

The cluster runs on Proxmox.

Control Plane

  • 3x RKE2 master nodes (VMs)

I chose three masters for etcd quorum and high availability. Even in a homelab, I prefer testing HA designs instead of single-node control planes.

Workers

  • 1x worker VM on Proxmox
  • 2x worker nodes on dedicated Mini PCs

The Mini PCs give me hardware separation from the hypervisor. If Proxmox needs maintenance or reboots, I still have compute capacity outside the main host.

Why RKE2?

RKE2 was chosen because:

  • It is lightweight
  • It follows upstream Kubernetes closely
  • It includes sensible defaults
  • Installation is straightforward

The installation was done following the official RKE2 documentation. Bootstrap was simple:

  • Install first server node
  • Join remaining masters
  • Join workers with token

No complex external etcd setup, no heavy platform layer.

I wanted Kubernetes, not a management platform around it.

Entry Point Architecture

One of the most important design decisions was how traffic enters the cluster.

OPNsense + HAProxy

Instead of exposing nodes directly, I used OPNsense as the single entry point.

HAProxy has:

  • Backend “k8s-api” → all 3 master nodes
  • Backend “k8s-https” → all worker nodes

This provides:

  • Load balancing
  • Failover
  • Single public endpoint
  • Clean firewall control

If a node goes down, HAProxy removes it automatically from rotation.

WireGuard for Secure Access

Administrative access to the cluster is only possible via WireGuard.

I do not expose the Kubernetes API publicly.

WireGuard terminates on OPNsense and gives me secure internal access to:

  • Kubernetes API
  • Proxmox
  • Database VMs
  • Internal services

This reduces attack surface significantly.

Cloudflare as HTTP Gatekeeper

All HTTP/HTTPS traffic is proxied through Cloudflare.

Firewall rules only allow Cloudflare IP ranges to access the cluster.

This gives:

  • DDoS protection
  • TLS termination at edge
  • IP masking
  • Simple certificate handling

The cluster never talks directly to the open internet for web traffic.

Ingress and Security

Inside the cluster:

  • Traefik handles ingress routing
  • CrowdSec analyzes logs and blocks malicious traffic

Traefik manages:

  • Host-based routing
  • Path-based routing
  • TLS handling (when needed internally)

CrowdSec adds behavioral detection and automated blocking.

This combination gives both structured routing and dynamic protection.

MetalLB for Static Service IPs

For certain services like game servers, I need predictable IPs.

MetalLB provides static IP assignment inside the 10.0.0.0/24 network.

This is useful for:

  • Game servers
  • Services that require fixed endpoints
  • Clean separation of service exposure

Without MetalLB, NodePorts or hostPorts would become messy over time.

Game Server Architecture

Game servers are exposed through:

  • A VPS at Hetzner
  • WireGuard tunnel into the cluster

The VPS acts as the public-facing node.

This design allows:

  • Public IP exposure without exposing the home network
  • Flexible routing
  • Isolation between web and game traffic

Database Layer

Instead of running databases inside Kubernetes, I created dedicated VMs:

  • PostgreSQL VM
  • MariaDB VM

Reasons:

  • Clear separation of concerns
  • Easier performance tuning
  • Simpler backup strategy
  • Avoid storage coupling with cluster lifecycle

Stateful data remains outside the Kubernetes scheduling layer.

Backup Strategy

Backups are handled using Restic.

  • Encrypted repositories
  • Regular snapshots
  • Upload to S3-compatible storage

I back up:

  • Website data
  • Database dumps
  • Critical application data

Encryption happens before upload, so storage providers never see plaintext data.

Why This Architecture?

The main goals were:

  • High availability for control plane
  • Clean entry point design
  • Minimal public exposure
  • Clear separation of stateful and stateless workloads
  • Reproducible setup

OPNsense gives me full control over traffic. RKE2 gives me a stable Kubernetes base. Proxmox gives hardware flexibility. WireGuard reduces exposed services. Cloudflare filters external HTTP traffic. MetalLB simplifies internal service IP management. Restic protects data.

Each component has a single responsibility.

Final Thoughts

This setup is not enterprise scale. But it follows production principles:

  • Redundant control plane
  • Layered security
  • Encrypted backups
  • Segmented network
  • Controlled ingress

It allows me to experiment, host services, run game servers, and test HA behavior in a realistic environment.

And most importantly: everything is documented and reproducible.