Resilient kubelet configuration

The kubelet is the Kubernetes agent that runs on every node in a cluster. kubelet is installed using the kubernetes rpm (e.g. kubernetes1.30 is a versioned rpm for Kubernetes v1.30). The kubelet runs as a systemd service on Fedora. In early implementations, the kubelet was configured via flags that were set in a systemd unit file and passed to the kubelet as command line parameters.

In more recent versions of the kubelet these flags are deprecated in favor of a configuration file that uses either JSON or YAML for the configuration syntax.

The legacy non-versioned rpms use, by default, flags to configure the kubelet. Versioned rpms use the configuration file method.

With both versioned and non-versioned rpms, all files, including systemd related files, can be erased during version updates (e.g. kubernetes1.29 to kubernetes1.30 - minor version updates). If these files are modified by the user then there is risk that useful or important changes can be lost. Systemd provides options that help safeguard against loss of node-specific configurations.

Flags for the kubelet running on a node are set in a systemd unit file with the relevant file dependent on which rpms are installed.

The kubernetes rpm (e.g kubernetes1.30 for version 1.30) installs the default kubelet systemd file at:

The kubernetes-kubeadm rpm installs an overriding kubelet unit file at:

We strongly recommend to not modify either file as any changes could be lost during an update.

As documented by the Kubernetes team (https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/kubelet-integration/#the-kubelet-drop-in-file-for-systemd), create the following directory for user managed, system-level systemd kubelet overrides:

Then create a unit file (.conf extension required) and copy the file to the directory listed above. Settings in this file will override settings from either or both of the default systemd files.

This file is not managed by the system package manager and will be unchanged by kubernetes version updates. Be sure to have software version control and/or a backup plan in place to avoid loss during a Fedora system upgrade or crash.

All versioned kubernetes rpms use a kubelet configuration file by default. If this file does not exist it will be created during the cluster instantiation process. The default configuration file location is:

This file is not managed by rpm so will persist across kubernetes upgrades.