🦾 Talos

Talos Linux is a minimal, immutable operating system built only for Kubernetes. It has no SSH server, interactive shell, or package manager. Nodes are configured declaratively and managed through the authenticated Talos API with talosctl.

Why Talos?

K3s or RKE2 on Ubuntu is familiar and flexible, but it leaves two platforms to operate: the Linux servers and Kubernetes. That includes SSH access, users, packages, repositories, systemd services, patching, hardening, and configuration drift.

Talos narrows the node to one purpose:

This reduces the node attack surface and configuration drift. It does not replace Kubernetes RBAC, admission policies, NetworkPolicy, secrets management, backups, monitoring, or workload security.

Talos is a good fit when nodes run only Kubernetes, supported hardware is used, and replacement is preferred over manual repair. A general-purpose distribution may remain simpler for unusual drivers, host-installed vendor agents, or software that expects systemd and a conventional filesystem.

Homelab Architecture

Hardware and Cluster Requirements

A practical highly available homelab uses three similar x86-64 mini PCs, each connected by wired Ethernet. All three can run the control plane, etcd, and workloads.

Before installation:

• enable UEFI boot and hardware virtualization;
• create DHCP reservations for all node MAC addresses;
• reserve one unused address for the Kubernetes API virtual IP (VIP);
• exclude the VIP and any LoadBalancer range from the DHCP pool;
• install compatible talosctl and kubectl versions on the workstation;
• identify every target disk and network interface from Talos maintenance mode.

Example address plan:

Three-Node High Availability

Three etcd members provide quorum while one member is unavailable. The API VIP moves to a healthy control-plane node, so kubectl does not depend on one machine. talosctl, however, normally targets the individual node addresses.

Networking, API VIP, and etcd Quorum

Keep these paths distinct:

• talosctl → each node on TCP 50000;
• kubectl → API VIP on TCP 6443;
• etcd → three members, requiring two for quorum;
• applications → Cilium Gateway or LoadBalancer addresses, if configured.

The API VIP must be on the same layer-2 network as the control-plane interfaces. For routed or cloud environments, use a supported load balancer instead.

Deploying Kubernetes with Talos

Installing Talos Linux

Boot each machine from the Talos metal ISO or USB image. In maintenance mode, record the node address, wired interface, and installation disk:

1export NODE1=192.168.1.111
2export NODE2=192.168.1.112
3export NODE3=192.168.1.113
4export VIP=192.168.1.200
5
6for node in "$NODE1" "$NODE2" "$NODE3"; do
7  talosctl --nodes "$node" get links --insecure
8  talosctl --nodes "$node" get disks --insecure
9done

Generating Machine Configurations

Generate one cluster configuration locally and protect the generated credentials:

1umask 077
2mkdir -p talos/patches talos/generated/nodes
3
4talosctl gen config homelab "https://${VIP}:6443" \
5  --output-dir talos/generated

Create one patch per node. Change the disk, interface, and hostname:

 1machine:
 2  install:
 3    disk: /dev/nvme0n1
 4  network:
 5    interfaces:
 6      - interface: enp1s0
 7        dhcp: true
 8        vip:
 9          ip: 192.168.1.200
10---
11apiVersion: v1alpha1
12kind: HostnameConfig
13hostname: node-1
14auto: off

Patch and validate each machine configuration before applying it:

1for node in node-1 node-2 node-3; do
2  talosctl machineconfig patch talos/generated/controlplane.yaml \
3    --patch "@talos/patches/${node}.yaml" \
4    --output "talos/generated/nodes/${node}.yaml" || break
5  talosctl validate \
6    --config "talos/generated/nodes/${node}.yaml" \
7    --mode metal || break
8done

Do not commit controlplane.yaml, worker.yaml, talosconfig, kubeconfig, or rendered node configurations. They contain cluster credentials.

Apply each configuration while its node is still in maintenance mode:

1talosctl apply-config --insecure --nodes "$NODE1" \
2  --file talos/generated/nodes/node-1.yaml
3talosctl apply-config --insecure --nodes "$NODE2" \
4  --file talos/generated/nodes/node-2.yaml
5talosctl apply-config --insecure --nodes "$NODE3" \
6  --file talos/generated/nodes/node-3.yaml

Remove the installation media and boot from the internal disks.

Bootstrapping the Kubernetes Cluster

Configure several Talos API endpoints, then bootstrap one node once:

1export TALOSCONFIG="$PWD/talos/generated/talosconfig"
2
3talosctl config endpoint "$NODE1" "$NODE2" "$NODE3"
4talosctl config node "$NODE1"
5talosctl version
6
7talosctl bootstrap --nodes "$NODE1"
8talosctl health

Export the Kubernetes client configuration and inspect the cluster:

1talosctl kubeconfig ./talos/generated/kubeconfig --nodes "$NODE1"
2export KUBECONFIG="$PWD/talos/generated/kubeconfig"
3
4kubectl cluster-info
5kubectl get nodes -o wide
6kubectl get pods --all-namespaces

For a small lab where control-plane nodes also run applications, apply this machine patch and include it whenever configurations are regenerated:

1cluster:
2  allowSchedulingOnControlPlanes: true

1talosctl --nodes "$NODE1,$NODE2,$NODE3" patch machineconfig \
2  --patch @talos/patches/allow-workloads.yaml

Validating Cluster Health and Failover

Check Talos, Kubernetes, etcd, DNS, and workload scheduling:

 1talosctl health
 2talosctl --nodes "$NODE1" etcd members
 3kubectl get nodes -o wide
 4kubectl get pods --all-namespaces
 5
 6kubectl create deployment web-test --image=nginx --replicas=3
 7kubectl rollout status deployment/web-test
 8kubectl expose deployment web-test --port=80
 9kubectl run curl-test --rm -it --restart=Never --image=curlimages/curl \
10  -- curl --fail http://web-test
11kubectl delete service web-test
12kubectl delete deployment web-test

To test high availability, begin with all nodes healthy and stop only one:

1talosctl health
2talosctl --nodes "$NODE1" shutdown
3kubectl get nodes
4kubectl cluster-info

The API VIP should remain available through either surviving node. Power the stopped node back on, wait until all three nodes are Ready, and run talosctl health again before any further failure test.

Talos Debug

Start from the management layer that is still available. The Talos API is separate from the Kubernetes API, so it can often diagnose a failed control plane even when kubectl no longer works.

 1# Overview
 2talosctl --nodes "$NODE1" dashboard
 3talosctl health
 4talosctl --nodes "$NODE1" service
 5
 6# Service and kernel logs
 7talosctl --nodes "$NODE1" service kubelet
 8talosctl --nodes "$NODE1" logs kubelet --tail 100
 9talosctl --nodes "$NODE1" dmesg
10
11# Talos resources
12talosctl --nodes "$NODE1" get links
13talosctl --nodes "$NODE1" get addresses
14talosctl --nodes "$NODE1" get routes
15talosctl --nodes "$NODE1" get disks
16talosctl --nodes "$NODE1" get mounts
17
18# Runtime and capacity
19talosctl --nodes "$NODE1" processes
20talosctl --nodes "$NODE1" memory
21talosctl --nodes "$NODE1" stats
22talosctl --nodes "$NODE1" usage

When the Kubernetes API is unavailable, inspect its underlying services and containers through Talos:

1talosctl --nodes "$NODE1" service etcd
2talosctl --nodes "$NODE1" logs etcd --tail 100
3talosctl --nodes "$NODE1" etcd status
4talosctl --nodes "$NODE1" etcd alarm list
5talosctl --nodes "$NODE1" service containerd
6talosctl --nodes "$NODE1" containers --kubernetes

For network incidents, inspect state first and capture packets to the local workstation instead of installing tools on the node:

1talosctl --nodes "$NODE1" netstat --all
2talosctl --nodes "$NODE1" pcap --interface enp1s0 \
3  --output talos-node-1.pcap

Collect a support bundle before rebooting or restarting services, because a recovery action can erase useful evidence:

1talosctl support --nodes "$NODE1,$NODE2,$NODE3" \
2  --output talos-support.zip

Treat support bundles as sensitive operational data. If the Talos API itself is unreachable, use the local video console, hypervisor console, or out-of-band KVM. Keep tested copies of the machine configurations, talosconfig, etcd snapshots, and node replacement procedure outside the cluster.

Networking and Secure Remote Access

Cilium and Kubernetes Networking

Cilium can replace the default CNI and kube-proxy, enforce NetworkPolicy, and provide LoadBalancer IPAM, layer-2 announcements, Gateway API, and Hubble flow visibility. Plan the following before bootstrap or migration:

• reserve a LoadBalancer range outside DHCP;
• include the required Cilium machine configuration patch when generating nodes;
• validate pod and service CIDRs;
• test DNS, Services, LoadBalancers, and NetworkPolicy after installation;
• preserve quorum by changing one control-plane node at a time.

If Tailscale traffic enters through tailscale0, include that interface in Cilium’s device list. Pin the intended Cilium MTU rather than allowing the Tailscale interface’s smaller MTU to resize the pod network unexpectedly.

Configuring Tailscale on Talos

Tailscale runs as a Talos system extension, not as a package installed on the immutable host. Build a Talos Image Factory schematic containing the official extension:

1customization:
2  systemExtensions:
3    officialExtensions:
4      - siderolabs/tailscale

1curl --silent --request POST --data-binary @tailscale/schematic.yaml \
2  https://factory.talos.dev/schematics

Use the returned schematic ID to construct the installer image for the exact Talos version being deployed:

1factory.talos.dev/metal-installer/<schematic-id>:<talos-version>

Configure the extension before upgrading a running node to that image. Store the rendered configuration outside Git because it contains the auth key:

1apiVersion: v1alpha1
2kind: ExtensionServiceConfig
3name: tailscale
4environment:
5  - TS_AUTHKEY=<tailscale-auth-key>
6  - TS_HOSTNAME=node-1
7  - TS_ROUTES=192.168.1.111/32,192.168.1.112/32,192.168.1.113/32,192.168.1.200/32,192.168.1.201/32
8  - TS_ACCEPT_DNS=false
9  - TS_AUTH_ONCE=true

Also constrain Kubernetes and etcd to the LAN. Otherwise a node may advertise its Tailscale 100.x address as its Kubernetes internal IP or etcd peer address:

1machine:
2  kubelet:
3    nodeIP:
4      validSubnets:
5        - 192.168.1.0/24
6cluster:
7  etcd:
8    advertisedSubnets:
9      - 192.168.1.0/24

Apply the node-specific extension configuration and LAN-address patch without a reboot, then upgrade one node at a time with the custom installer image:

1talosctl --nodes "$NODE1" patch machineconfig \
2  --patch @tailscale/node-ip.yaml \
3  --patch @tailscale/node-1.yaml \
4  --mode no-reboot
5
6talosctl --nodes "$NODE1" upgrade \
7  --image factory.talos.dev/metal-installer/<schematic-id>:<talos-version>
8kubectl wait --for=condition=Ready node/node-1 --timeout=10m

Verify that the extension runs while Kubernetes and etcd keep their LAN addresses:

1talosctl --nodes "$NODE1" get extensions
2talosctl --nodes "$NODE1" services
3kubectl get nodes -o wide
4talosctl --nodes "$NODE1" etcd members

Every later Talos upgrade must continue using an installer image containing the required extensions. Revoke the reusable Tailscale auth key after all nodes have joined; the nodes retain their state.

Secure Remote Access Without Exposing Ports

Advertise only the routes remote operators need, preferably individual /32 addresses rather than the entire home LAN:

• three node addresses for the Talos API on TCP 50000;
• the Kubernetes API VIP on TCP 6443;
• the internal Gateway on TCP 80 and 443.

Approve only those routes in the Tailscale policy and restrict tailnet members to the matching destination ports. Deny access to the router and unrelated LAN devices. Linux clients must enable route acceptance:

1sudo tailscale up --accept-routes
2tailscale status

All three nodes may advertise the same routes so Tailscale can fail over between subnet routers. Test from outside the home network, then reboot the active router and confirm that API and application traffic recover through another node.

Sources

• Talos homelab repository
• Building a 3-node highly available Kubernetes homelab with Talos Linux
• Tailscale on Talos Linux
• From hardened Ubuntu nodes to Talos Linux
• Debugging Talos Linux without SSH
• Official Talos documentation