🐎 K3D

February 3, 2026 4 min read Devops Docs Kubernetes Infrastructure

On this page

K3D equal k3s in a container. a tools to create single- and multi-node k3s clusters. Our favorite use case, is with podman and rootless. So there is some customization upstream to do.

One downside I’ve found with k3d is that the Kubernetes version it uses is behind the current k3s release.

Note for ARM PC:

1sudo apt install qemu-user-static
2podman run --rm --privileged multiarch/qemu-user-static --reset -p yes

Install

1# Manual way
2curl -s https://raw.githubusercontent.com/k3d-io/k3d/main/install.sh | bash
3
4# or with arkade:
5arkade get k3d
6
7# Auto-completion
8k3d completion zsh > "$ZSH/completions/_k3d"

Tweaks for podman and rootless

The issue:

1k3d cluster create test
2
3ERRO[0000] Failed to get nodes for cluster 'test': docker failed to get containers with labels 'map[k3d.cluster:test]': failed to list containers: permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get "http://%2Fvar%2Frun%2Fdocker.sock/v1.46/containers/json?all=1&filters=%7B%22label%22%3A%7B%22app%3Dk3d%22%3Atrue%2C%22k3d.cluster%3Dtest%22%3Atrue%7D%7D": dial unix /var/run/docker.sock: connect: permission denied

The solution:

 1# TODO 
 2loginctl enable-linger $(whoami)
 3
 4# Either reload terminal or do below: 
 5export XDG_RUNTIME_DIR=/tmp/run-$(id -u)
 6mkdir -p $XDG_RUNTIME_DIR
 7chmod 700 $XDG_RUNTIME_DIR
 8
 9sudo mkdir -p /etc/containers/containers.conf.d
10sudo sh -c "echo 'service_timeout=0' > /etc/containers/containers.conf.d/timeout.conf"
11
12sudo ln -s /run/podman/podman.sock /var/run/docker.sock
13
14XDG_RUNTIME_DIR=${XDG_RUNTIME_DIR:-/run/user/$(id -u)}
15export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock
16export DOCKER_SOCK=$XDG_RUNTIME_DIR/podman/podman.sock
17
18systemctl --user enable --now podman.socket

If /sys/fs/cgroup/cgroup.controllers is present on your system, you are using v2, otherwise you are using v1.
in rootless, to run properly we need to enable CPU, CPUSET, and I/O delegation

1sudo mkdir -p /etc/systemd/system/user@.service.d
2
3echo -e "[Service]\nDelegate=cpu cpuset io memory pids" | \
4sudo tee /etc/systemd/system/user@.service.d/delegate.conf > /dev/null
5
6sudo systemctl daemon-reload

Create Podman Network

The default podman network has dns disabled. To allow k3d cluster nodes to communicate with dns, so a new network must be created.

1podman network create k3d
2podman network inspect k3d -f '{{ .DNSEnabled }}'

Create local registry

Create a local registry using the podman network

1k3d registry create mycluster-registry --default-network k3d --port 5000 --delete-enabled

podman does not appreciate http… but let do an execption: sudo vi ~/.config/containers/registries.conf.
Add and restart with systemctl --user restart podman

1[[registry]]
2location = "localhost:5000"
3insecure = true

Push some image and Check registry at :

http://localhost:5000/v2/_catalog
http://k3d-mycluster-registry.localhost:5000/v2/_catalog

Quickstart k3d cluster

For a Quick cluster use the registry and network created before:

1# args REQUIRED for rootless Podman — SERVER and AGENT
2k3d cluster create mycluster \
3  --registry-use k3d-mycluster-registry:5000 \
4  --network k3d \
5  --k3s-arg "--kubelet-arg=feature-gates=KubeletInUserNamespace=true@server:*" \
6  --k3s-arg "--kubelet-arg=feature-gates=KubeletInUserNamespace=true@agent:*" \
7  --k3s-arg "--kubelet-arg=fail-swap-on=false@server:*" \
8  --k3s-arg "--kubelet-arg=fail-swap-on=false@agent:*"

Important
Pay attention to always export those variables if you use k3d with podman the rootless way. Can be added to ~/.zshrc
XDG_RUNTIME_DIR=${XDG_RUNTIME_DIR:-/run/user/$(id -u)}
export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock
export DOCKER_SOCK=$XDG_RUNTIME_DIR/podman/podman.sock

K3D Verify and List

1k3d cluster list
2k3d node list
3k3d registry list
4podman ps --format "table {{.Names}}\t{{.Image}}\t{{.Ports}}"

K3D use config.yaml

Create a config.yaml

 1apiVersion: k3d.io/v1alpha5
 2kind: Simple
 3metadata:
 4  name: MyCluster
 5
 6network: k3d
 7
 8servers: 1
 9agents: 1
10
11options:
12  k3s:
13    extraArgs:
14      - arg: "--disable=traefik"
15        nodeFilters:
16          - server:*
17      - arg: "--disable=servicelb"
18        nodeFilters:
19          - server:*
20      # REQUIRED for rootless Podman — SERVER and AGENT
21      - arg: "--kubelet-arg=feature-gates=KubeletInUserNamespace=true"
22        nodeFilters:
23          - server:*
24          - agent:*
25      - arg: "--kubelet-arg=fail-swap-on=false"
26        nodeFilters:
27          - server:*
28          - agent:*
29  k3d:
30    wait: true
31    timeout: "60s"
32    disableLoadbalancer: true
33
34registries:
35  use:
36    - k3d-mycluster-registry:5000

Create a registry.yaml:

1mirrors:
2  "localhost:5000":
3    endpoint:
4      - http://k3d-mycluster-registry:5000

Launch it

1k3d cluster create --config config.yaml --registry-config registry.yaml

Restart K3D

1# Restart registry
2podman start k3d-mycluster-registry
3podman ps -f name=k3d-mycluster-registry
4
5# Restart K3d
6k3d cluster start mycluster

Manage Registry

Check registry:

1# Not super usefull but ...
2arkade get regctl
3regctl completion zsh > "$ZSH/completions/_regctl"
4regctl registry set --tls disabled localhost:5000
5regctl repo ls localhost:5000
6
7# check if REGISTRY_STORAGE_DELETE_ENABLED=true
8podman inspect k3d-mycluster-registry --format '{{range .Config.Env}}{{println .}}{{end}}'

List images and tags

1curl -s http://localhost:5000/v2/_catalog \
2| jq -r '.repositories[]' \
3| while read repo; do
4    echo "Repository: $repo"
5    curl -s http://localhost:5000/v2/$repo/tags/list | jq
6  done

Delete an image and clean garbage collector

 1repo="backstage-backend"; tag="local"; \
 2digest=$(curl -sI \
 3  -H "Accept: application/vnd.oci.image.manifest.v1+json" \
 4  http://localhost:5000/v2/$repo/manifests/$tag \
 5  | awk -F': ' '/Docker-Content-Digest/ {print $2}' | tr -d '\r'); \
 6echo "Deleting $digest"; \
 7curl -v -X DELETE http://localhost:5000/v2/$repo/manifests/$digest
 8podman exec -it k3d-mycluster-registry registry garbage-collect /etc/docker/registry/config.yml
 9# then restart registry to make it effective
10podman stop k3d-mycluster-registry && podman start k3d-mycluster-registry

Cleanup

1k3d cluster delete --config config.yaml
2k3d registry delete k3d-mycluster-registry
3
4# Should be clean
5podman ps -a
6podman network ls
7podman volume ls