Creating a Kubernetes cluster on Fedora

Update system with DNF. Reboot if necessary, although a reboot can be deferred until after the next step.

sudo dnf update

Disable swap. The kubeadm installation process will generate a warning if swap is detected (see this ticket for details). For a learning and lab environment it may be easiest to disable swap. Swap can be left enabled if desired and the kubeadm is configured to not stop if swap is detected. Modern Fedora systems use zram by default. Reboot after disabling swap.

sudo systemctl stop swap-create@zram0
sudo dnf remove zram-generator-defaults
sudo reboot now

SELinux. Most guides to installing Kubernetes on Fedora recommend that SELinux be disabled. Kubernetes will work well with SELinux enabled and many containers will work as intended. If problems are encountered then disabling SELinux might be one option to try. See the Quick Doc SELinux guide to changing SELinux states for more information.

Disable the firewall. Kubeadm will generate an installation warning if the firewall is running. Disabling the firewall removes one source of complexity in a learning environment. Modern Fedora systems use firewalld.

sudo systemctl disable --now firewalld

Install iptables and iproute-tc. Newer Kubernetes rpms include these packages by default.

sudo dnf install iptables iproute-tc

Configure IPv4 forwarding and bridge filters. Below copied from https://kubernetes.io/docs/setup/production-environment/container-runtimes/

sudo cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF

Load the overlay and bridge filter modules.

sudo modprobe overlay
sudo modprobe br_netfilter

Add required sysctl parameters and persist.

# sysctl params required by setup, params persist across reboots
sudo cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF

Apply sysctl parameters without a reboot.

sudo sysctl --system

Verify br_filter and overlay modules are loaded.

lsmod | grep br_netfilter
lsmod | grep overlay

Verify that the net.bridge.bridge-nf-call-iptables, net.bridge.bridge-nf-call-ip6tables, and net.ipv4.ip_forward system variables are set to 1 in your sysctl configuration by running the following command:

sysctl net.bridge.bridge-nf-call-iptables net.bridge.bridge-nf-call-ip6tables net.ipv4.ip_forward

Install a container runtime. CRI-O is installed in this example. Containerd is also an option. Note: If using cri-o, verify that the major:minor version of cri-o is the same as the version of Kubernetes (installed below).

sudo dnf install cri-o containernetworking-plugins

Install Kubernetes. In this example, all three Kubernetes applications (kubectl, kubelet, and kubeadm) are installed on this single node machine. Please see the notes above on recommended packages for control plane or worker nodes if the cluster will have both types of machines.

# Fedora 39 and earlier use:
sudo dnf install kubernetes-client kubernetes-node kubernetes-kubeadm

# Fedora 40 and later use:
sudo dnf install kubernetes kubernetes-kubeadm kubernetes-client

Start and enable cri-o.

sudo systemctl enable --now crio

Pull needed system container images for Kubernetes. This is strictly optional. The kubeadm init command below will pull images, if needed.

sudo kubeadm config images pull

Start and enable kubelet. Kubelet will be in a crash loop until the cluster is initialized in the next step.

sudo systemctl enable --now kubelet

Initialize the cluster.

sudo kubeadm init --pod-network-cidr=10.244.0.0/16

kubeadm will generate output to the terminal tracking initialization steps. If successful, the output below is displayed. At this point there is a cluster running on this single machine. After kubeadm finishes you should see:

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

The steps listed above allow a non-root user to use kubectl, the Kubernetes command line tool. Run these commands now.

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Allow the control plane machine to also run pods for applications. Otherwise more than one machine is needed in the cluster.

kubectl taint nodes --all node-role.kubernetes.io/control-plane-

Install flannel into the cluster to provide cluster networking. There are many other networking solutions besides flannel. Flannel is straightforward and suitable for this guide.

kubectl apply -f https://github.com/coreos/flannel/raw/master/Documentation/kube-flannel.yml

Display list of running pods in the cluster. All pods should display a status of Running. A status of CrashLoopBackOff may show up for the coredns pod. This happens commonly when installing Kubernetes on a virtual machine and the DNS service in the cluster may not select the proper network. Use your favorite internet search engine to find possible solutions. See also the troubleshooting section below for two possible solutions.

kubectl get pods --all-namespaces