Last week at Kubecon EU 2024 Fermyon Technologies announced their new open-source project SpinKube that allows running WASM workloads (originally built for their custom orchestrator Spin) on Kubernetes. This is a clear "if you can't fight them - join them" decision. WASM with all its glorious promises still has a very long adoption curve ahead of it. One of the main issues many folks were trying to wrap their head around - is how to fit WASM into their current Kubernetes-based stack. And SpinKube provides yet another exciting answer to this conundrum.
WASM and Kubernetes
The idea of running WASM on K8s isn't new. It was first introduced by the Krustlet project built at Microsoft and then found a more generic support in the runwasi initiative. Runwasi provides a library that lies at the foundation of containerd-shim-wasm that allows to run generic WASM workloads on top of containerd using one of the leading WebAssembly runtimes (wasmtime,wasmer or wasmedge).
SpinKube provides a higher-level containerd shim (built on top of the wasm shim) for running Spin application bundles on container hosts. It also gives us spin-operator and a CRD that allows us to manage Spin apps in Kubernetes as if they were regular container deployments - with support for scheduling constraints, resource allocations and HPA.
The selling points of WebAssembly are tiny binary sizes, lightning fast initialization and low resource footprint. All great for maximum efficiency and cost optimization. Or, as we like to put it at PerfectScale by DoiT - for keeping your clusters lean.
So quite naturally I was curious to take SpinKube for a spin. I also like to go into a bit more detail in order to understand how things work. So here's my own walkthrough that goes beyond the basic "getting started".
Things we're gonna need:
- A k3d cluster
- A SpinKube-enabled node
- A Spin application
As a side note - if running WASM on Kubernetes excites you as much as it does me - join our upcoming webinar with Saiyam Pathak of Civo, where we'll dive deeper into WASM and how it fits into the cloud native landscape.
A k3d cluster with SpinKube
Create the Cluster
If you're a cloud native aficionado like myself - high chance you already have a k3d cluster running on your machine. If you're ok with using it for this experiment - skip to the next section. If not - install k3d and then create the cluster with the following command:
k3d cluster create spincluster \ --api-port 6550 \ -p "8081:80@loadbalancer" \ --agents 1 \ --registry-create spincluster-registry:12345This creates a cluster with:
- an API server exposed on port 6550
- ingress load balancer exposed on local port 8081
- just one agent node - because I want to have one 'regular' node for running Linux containers and one Spin-enabled node for running Spin applications (which we'll add in the next section).
- internal OCI registry - it will be available at
localhost:12345from our docker host and atspincluster-registry:5000from within k3d nodes.
Add a Spin-enabled Node
The SpinKube project is providing a container image which includes the containerd-shim-spin already installed. Let's create a k3d node from it:
k3d node create spin-node -c spincluster \--k3s-node-label spin=enabled \--image ghcr.io/spinkube/containerd-shim-spin/k3d:v0.13.1Verify the all nodes are up and running:
docker exec k3d-spin-node-0 sh -c "ls bin/containerd*"And for another reality check - let's verify that the containerd shim for Spin is installed in the node we've added:
docker exec k3d-spin-node-0 sh -c "ls bin/containerd*"Yup, the shim is there. Now let's configure the cluster.
Configure the Cluster for SpinKube:
First we'll install:
- the cert-manager which is used by spin-operator to sign certificates
- the RuntimeClass for spin:
wasmtime-spin-v2 - the spin-operator CRDs:
SpinAppandSpinAppExecutor - the
SpinAppExecutorcalledcontainerd-shim-spinto connect the apps to theRunTimeClass - and finally - we'll patch the
RuntimeClassso it only schedules Spin apps to our Spin-enabled node
# install cert-managerkubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.14.3/cert-manager.yaml# apply runtime classkubectl apply -f https://github.com/spinkube/spin-operator/releases/download/v0.1.0/spin-operator.runtime-class.yaml#install CRDskubectl apply -f https://github.com/spinkube/spin-operator/releases/download/v0.1.0/spin-operator.crds.yaml# patch the runtime classkubectl patch runtimeclasses.node.k8s.io wasmtime-spin-v2 \ -p '{"scheduling" : { "nodeSelector" : {"spin" : "enabled"}}}'#install the SpinAppExecutorkubectl apply -f https://github.com/spinkube/spin-operator/releases/download/v0.1.0/spin-operator.shim-executor.yamlFinally, install the spin-operator:
helm install spin-operator \ --namespace spin-operator \ --create-namespace \ --version 0.1.0 \ --wait \ oci://ghcr.io/spinkube/charts/spin-operatorAnd now - let's build the app.
Build a Spin app
Install Rust
The application I'm testing this with is written in Rust. So yes, if you haven't done this already - it's time to install Rust. After all - it's been the most admired programming language for 8 years in a row.
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | shAnd while at it - install the wasm32-wasi target for Rust:
rustup target add wasm32-wasiInstall Spin
Now let's install Spin:
curl -fsSL https://developer.fermyon.com/downloads/install.sh | bashsudo mv ./spin /usr/local/bin/spinCreate or get the app:
If you want to build your own Spin app from scratch - generate the code with:
# Create a new Spin application named 'lean-cluster' based on the Rust http template, accepting all defaultsspin new --accept-defaults -t http-rust lean-clusterAlternatively - you can clone my app code:
git clone https://github.com/antweiss/spinkube-lean-cluster.gitChange into the app directory, build and push the application image:
spin registry push -k --build 127.0.0.1:12345/lean-cluster:0.1.0Note the -k here - we're using an insecure registry so this is needed in order to push to it.
Now - let's generate the SpinApp spec to deploy our app to the cluster:
spin kube scaffold --from spincluster-registry:5000/lean-cluster:0.1.0 -o app.yamlThis will give us the following in the app.yaml file:
apiVersion: core.spinoperator.dev/v1alpha1kind: SpinAppmetadata: name: lean-clusterspec: image: "spincluster-registry:5000/lean-cluster:0.1.0" executor: containerd-shim-spin replicas: 2This can now be deployed with:
kubectl apply -f app.yamlLet's see what this results in:
kubectl get all -l core.spinoperator.dev/app-nameThe output looks something like:
NAME READY STATUS RESTARTS AGEpod/lean-cluster-647cb497f5-ml444 1/1 Running 0 4m48spod/lean-cluster-647cb497f5-t62ht 1/1 Running 0 4m48s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEservice/lean-cluster ClusterIP 10.43.214.210 80/TCP 4m49s
NAME READY UP-TO-DATE AVAILABLE AGEdeployment.apps/lean-cluster 2/2 2 2 4m49s
NAME DESIRED CURRENT READY AGEreplicaset.apps/lean-cluster-647cb497f5 2 2 2 4m49sSo basically - what spin-operator does is create a Deployment and a Service, whereas the pods are instantiated using the containerd-shim-spin-v2 on the spin-enabled node.
Now let's deploy an in Ingress to access that service with a yaml I've prepared:
kubectl apply -f https://raw.githubusercontent.com/antweiss/spinkube-lean-cluster/main/resources/ingress.yamlAnd finally - verify your app is working and returning responses by:
$ curl localhost:80801/helloWe're keeping our clusters lean.And taking SpinKube for a spin.Voila! We now have a Spin app running in our cluster!
Get the Spin app code here: https://github.com/antweiss/spinkube-lean-cluster
For more SpinKube tutorials - visit the official website.
And for a deeper dive into the whats and whys of WASM on Kubernetes - make sure to sign up for our webinar on April 24th.
May your clusters be lean!