I made a project. It was inspired by my experiences running a workshop on Kubernetes at the 2024 Developer’s Conference in Mauritius. For workshops, it is important to have something on the cloud as attendees may have any kind of device or any kind of operating system, and may even bring their work laptops with restrictions. The cloud provides a consistent working environment, but at what cost?

KRaft is a “cloud provider” built around K3k, a Rancher technology for running virtual private clusters on top of a host Kubernetes cluster. Other such projects exist, such as VCluster from Loft Labs, but I went with K3k as it leverages K3s, which I am already familiar with. I say “cloud provider” as it is pretty much that - users are able to login, create virtual clusters, and download the kubeconfig file. They can then deploy their normal Kubernetes workloads on their cluster and expose it through ingresses. It is something you can use for running workshops or to simply share your homelab with friends among the ongoing RAM price crisis.

What is KRaft made out of?

KRaft is made up of a handful of services which run alongside each other on the host cluster:

• Authentication: manages everything for registration, login, and password resets.
• Resource management: collects metrics from across the cluster to get an idea of how much of your CPU/RAM/Storage are in use.
• Cluster management: the core of the platform, which handles creation of virtual clusters, the ingresses to expose them, cluster deletion, retrieving kubeconfigs, fetching logs, and more.
• Frontend: a very simple static site written in plain HTML and TailwindCSS for interacting with the backend, the way things should be.

And of course a database to store user and cluster info. While the resource management service is written in Python, both the authentication and cluster management parts are written entirely in Rust. Rust has allowed for incredible memory savings, using less than 10MB each, while the Python container sits at 300MB.

How KRaft Works

At the base, KRaft relies on K3k to run the virtual clusters. You can read more about what K3k is and how it works in their docs.

KRaft runs K3k in shared mode, giving virtual clusters access to resources on the host cluster, such as the storageclass or ingressclass. This means the user does not have to install Longhorn or Rook+Ceph on their cluster, or Ingress Nginx or Traefik, and resources they create will just be passed through to the host. I made this choice to save on resources and for convenience. Plus, I only have one public IP address for my homelab, which would complicate exposing ‘virtual’ K3k clusters which are more isolated. You can literally create a cluster and start deploying your services in a minute or less.

When you create a virtual cluster, KRaft will create a K3k cluster, apply the virtualclusterpolicy with resource quotas, and create an ingress so that you can reach the virtual cluster’s API server. Under the hood, K3k will spawn virtual kubelet pods, a coredns pod, and a virtual Kubernetes API server. The latter is one of many factors which provide isolation when several virtual clusters exist at the same time. Another is that K3k automatically creates network policies, preventing the virtual cluster from reaching other workloads on the same host cluster. You can the access the server or kubelet logs, and download the kubeconfig file which you simply plug into kubectl to access your cluster. And when you’re done, you can delete the cluster which wipes everything from the host cluster to the workload pods, including all the PVCs.

k3k-rs

If you’re familiar with Rust and the CNCF landscape, you might know of kube-rs which is a crate for interacting with Kubernetes resources from Rust code. With K3k implementing two CRDs, the proper way of doing things meant reimplementing them in structs which kube-rs could then use. Just for this purpose, I started another project which itself is a Rust crate. It implements the CRDs and offers functions to easily create a cluster, virtualclusterpolicies, and even retrieve the cluster’s Kubeconfig.

Splitting this into a crate means that anyone else can then use its functionality to interact with K3k clusters from Rust.

Misc

KRaft is open source, and has the manifests ready for anyone to run it on their own clusters with minimal tweaking. The repository is accessible preferably from my Forgejo instance, and also on Github, and all code is published under the GPLv3 license.

KRaft is currently hosted in a “beta release” on my homelab, under kraftcloud.dev or kraft.alexbissessur.dev. If you want to give it a try, please ping me on Mastodon or Telegram for a beta code and a rundown on how to use KRaft.

Finally, there is a much more comprehensive explanation of how KRaft works and the hardware it runs on at the website.

So, this is the project which has occupied my thoughts for a few months, hoping to make cloud native technologies more accessible to Mauritians, as well as to people of the larger cloud native community. Contributions are always more than welcome, and the list of issues keeps growing like a hydra - mark one as completed and two take its place. I would love to hear your thoughts, which you can leave on my main development thread on Mastodon!