Testando atualizações do Fedora CoreOS

Certifique-se de ter concluído as etapas descritas em página de configuração inicial antes de iniciar este tutorial.

In this tutorial, we will not focus on provisioning but on what happens during updates and the options that are available in case of failures.

Baixando uma release mais antiga do Fedora CoreOS

One of the defining features of Fedora CoreOS is automatic updates. To see them in action, we have to download an older Fedora CoreOS release. In this case we’ll boot the N-1 release of Fedora CoreOS, which can be seen from the Fedora CoreOS release page or by using the releases.json metadata:

RELEASE=$(curl https://builds.coreos.fedoraproject.org/prod/streams/stable/releases.json | jq -r '.releases[-2].version')
curl -O https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/$RELEASE/x86_64/fedora-coreos-$RELEASE-qemu.x86_64.qcow2.xz
curl -O https://builds.coreos.fedoraproject.org/prod/streams/stable/builds/$RELEASE/x86_64/fedora-coreos-$RELEASE-qemu.x86_64.qcow2.xz.sig

Uma vez que o arquivo fora baixado, vamos verificar sua integridade:

curl https://fedoraproject.org/fedora.gpg | gpg --import
gpg --verify fedora-coreos-$RELEASE-qemu.x86_64.qcow2.xz.sig
Procure por "Boa assinatura de" na saída.

Uma vez que você verificou o arquivo, você pode extraí-lo com:

unxz fedora-coreos-$RELEASE-qemu.x86_64.qcow2.xz

Para fazer o tutorial ser mais simples, você deve renomear essa imagem para um nome mais curto:

mv fedora-coreos-$RELEASE-qemu.x86_64.qcow2 fedora-coreos-older.qcow2

Writing the Butane config and converting to Ignition

We will create a Butane config that will:

  • Set up console autologin.

  • Add an SSH Key for the core user from the local ssh-key.pub file.

Let’s write this Butane config to a file called updates.bu:

variant: fcos
version: 1.5.0
passwd:
  users:
    - name: core
      ssh_authorized_keys_local:
        - ssh-key.pub
systemd:
  units:
    - name: serial-getty@ttyS0.service
      dropins:
      - name: autologin-core.conf
        contents: |
          [Service]
          # Override Execstart in main unit
          ExecStart=
          # Add new Execstart with `-` prefix to ignore failure
          ExecStart=-/usr/sbin/agetty --autologin core --noclear %I $TERM
          TTYVTDisallocate=no
storage:
  files:
    - path: /etc/profile.d/systemd-pager.sh
      mode: 0644
      contents:
        inline: |
          # Tell systemd to not use a pager when printing information
          export SYSTEMD_PAGER=cat
Opcionalmente, você pode substituir a chave pública SSH no arquivo yaml com sua própria chave pública para que você possa logar na instância inicializada. Se você escolher não fazer isso, você ainda será automaticamente logado no console serial.

Run Butane to convert that to an Ignition config:

butane --pretty --strict --files-dir=./ updates.bu --output updates.ign

Inicialização e atualização inicial

Agora vamos provisionar isso. Tenha certeza que você está começando de uma imagem mais antiga do Fedora CoreOS nesse passo:

# Configura o rótulo correto do SELinux para permitir acesso à configuração
chcon --verbose --type svirt_home_t containers.ign

# Inicia uma máquina virtual do Fedora CoreOS
virt-install --name=fcos --vcpus=2 --ram=2048 --os-variant=fedora-coreos-stable \
    --import --network=bridge=virbr0 --graphics=none \
    --qemu-commandline="-fw_cfg name=opt/com.coreos/config,file=${PWD}/containers.ign" \
    --disk=size=20,backing_store=${PWD}/fedora-coreos.qcow2

Disconnect from the serial console by pressing CTRL + ] and then use the reported IP address for the NIC from the serial console to log in using the core user via SSH:

As the system is not up to date, Zincati will notice this and will start updating the system. You should see the update process happening right away:

All necessary network services may not be up and running during the initial check. In such case Zincati will check for updates again in about 5 minutes.
[core@localhost ~]$ systemctl status --full zincati.service
● zincati.service - Zincati Update Agent
     Loaded: loaded (/usr/lib/systemd/system/zincati.service; enabled; preset: enabled)
    Drop-In: /usr/lib/systemd/system/service.d
             └─10-timeout-abort.conf
     Active: active (running) since Thu 2023-08-03 19:52:41 UTC; 10s ago
       Docs: https://github.com/coreos/zincati
   Main PID: 1816 (zincati)
     Status: "found update on remote: 38.20230709.3.0"
      Tasks: 12 (limit: 2239)
     Memory: 6.8M
        CPU: 273ms
     CGroup: /system.slice/zincati.service
             ├─1816 /usr/libexec/zincati agent -v
             └─1873 rpm-ostree deploy --lock-finalization --skip-branch-check revision=552de26fe0fe6a5e491f7a4163db125e3d44b144ae53a8f5f488e3f8481c46f9 --disallow-downgrade

Aug 03 19:52:41 localhost.localdomain zincati[1816]: [INFO  zincati::cli::agent] starting update agent (zincati 0.0.25)
Aug 03 19:52:41 localhost.localdomain zincati[1816]: [INFO  zincati::cincinnati] Cincinnati service: https://updates.coreos.fedoraproject.org
Aug 03 19:52:41 localhost.localdomain zincati[1816]: [INFO  zincati::cli::agent] agent running on node '87c9ec3e0a4045a19b74b54ae5ed986a', in update group 'default'
Aug 03 19:52:41 localhost.localdomain zincati[1816]: [INFO  zincati::update_agent::actor] registering as the update driver for rpm-ostree
Aug 03 19:52:41 localhost.localdomain zincati[1816]: [INFO  zincati::update_agent::actor] initialization complete, auto-updates logic enabled
Aug 03 19:52:41 localhost.localdomain zincati[1816]: [INFO  zincati::strategy] update strategy: immediate
Aug 03 19:52:41 localhost.localdomain systemd[1]: Started zincati.service - Zincati Update Agent.
Aug 03 19:52:41 localhost.localdomain zincati[1816]: [INFO  zincati::update_agent::actor] reached steady state, periodically polling for updates
Aug 03 19:52:42 localhost.localdomain zincati[1816]: [INFO  zincati::cincinnati] current release detected as not a dead-end
Aug 03 19:52:43 localhost.localdomain zincati[1816]: [INFO  zincati::update_agent::actor] target release '38.20230709.3.0' selected, proceeding to stage it

[core@localhost ~]$ rpm-ostree status
State: idle
AutomaticUpdatesDriver: Zincati
  DriverState: active; update staged: 38.20230709.3.0; reboot delayed due to active user sessions
Deployments:
  fedora:fedora/x86_64/coreos/stable
                  Version: 38.20230709.3.0 (2023-07-24T12:25:01Z)
                   Commit: 552de26fe0fe6a5e491f7a4163db125e3d44b144ae53a8f5f488e3f8481c46f9
             GPGSignature: Valid signature by 6A51BBABBA3D5467B6171221809A8D7CEB10B464
                     Diff: 61 upgraded

● fedora:fedora/x86_64/coreos/stable
                  Version: 38.20230625.3.0 (2023-07-11T11:57:53Z)
                   Commit: e841d77aadb875bb801ac845a0d9b8a70b4224bdeb15e7d6c5bff1da932c0301
             GPGSignature: Valid signature by 6A51BBABBA3D5467B6171221809A8D7CEB10B464

[core@localhost ~]$ systemctl status --full zincati.service
● zincati.service - Zincati Update Agent
     Loaded: loaded (/usr/lib/systemd/system/zincati.service; enabled; preset: enabled)
    Drop-In: /usr/lib/systemd/system/service.d
             └─10-timeout-abort.conf
     Active: active (running) since Thu 2023-08-03 19:52:41 UTC; 1min 21s ago
       Docs: https://github.com/coreos/zincati
   Main PID: 1816 (zincati)
     Status: "update staged: 38.20230709.3.0; reboot delayed due to active user sessions"
      Tasks: 6 (limit: 2239)
     Memory: 3.2M
        CPU: 362ms
     CGroup: /system.slice/zincati.service
             └─1816 /usr/libexec/zincati agent -v

Aug 03 19:52:41 localhost.localdomain zincati[1816]: [INFO  zincati::cli::agent] agent running on node '87c9ec3e0a4045a19b74b54ae5ed986a', in update group 'default'
Aug 03 19:52:41 localhost.localdomain zincati[1816]: [INFO  zincati::update_agent::actor] registering as the update driver for rpm-ostree
Aug 03 19:52:41 localhost.localdomain zincati[1816]: [INFO  zincati::update_agent::actor] initialization complete, auto-updates logic enabled
Aug 03 19:52:41 localhost.localdomain zincati[1816]: [INFO  zincati::strategy] update strategy: immediate
Aug 03 19:52:41 localhost.localdomain systemd[1]: Started zincati.service - Zincati Update Agent.
Aug 03 19:52:41 localhost.localdomain zincati[1816]: [INFO  zincati::update_agent::actor] reached steady state, periodically polling for updates
Aug 03 19:52:42 localhost.localdomain zincati[1816]: [INFO  zincati::cincinnati] current release detected as not a dead-end
Aug 03 19:52:43 localhost.localdomain zincati[1816]: [INFO  zincati::update_agent::actor] target release '38.20230709.3.0' selected, proceeding to stage it
Aug 03 19:52:55 localhost.localdomain zincati[1816]: [INFO  zincati::update_agent::actor] update staged: 38.20230709.3.0
Aug 03 19:52:55 localhost.localdomain zincati[1816]: [WARN  zincati::update_agent] interactive sessions detected, entering grace period (maximum 10 minutes)

Shortly after the update has been staged the system will reboot by default. However if it detects active user sessions it will wait and show a message to the user:

Broadcast message from Zincati at Thu 2023-08-03 19:52:55 UTC:
New update 38.20230709.3.0 is available and has been deployed.
If permitted by the update strategy, Zincati will reboot into this update when
all interactive users have logged out, or in 10 minutes, whichever comes
earlier. Please log out of all active sessions in order to let the auto-update
process continue.

Once the user sessions have ceased the reboot will continue. In this case we need to stop the autologin on ttyS0 service and log out of SSH as well:

[core@localhost ~]$ sudo systemctl stop serial-getty@ttyS0.service
[core@localhost ~]$ exit
logout
Connection to 192.168.124.222 closed.
You can monitor the serial console to see when the machine has performed the reboot via virsh console fcos.

When we log back in we can view the current version of Fedora CoreOS is now the latest one. The rpm-ostree status output will also show the older version, which still exists in case we need to rollback:

[core@localhost ~]$ rpm-ostree status
State: idle
AutomaticUpdatesDriver: Zincati
  DriverState: active; periodically polling for updates (last checked Thu 2023-08-03 19:59:16 UTC)
Deployments:
● fedora:fedora/x86_64/coreos/stable
                  Version: 38.20230709.3.0 (2023-07-24T12:25:01Z)
                   Commit: 552de26fe0fe6a5e491f7a4163db125e3d44b144ae53a8f5f488e3f8481c46f9
             GPGSignature: Valid signature by 6A51BBABBA3D5467B6171221809A8D7CEB10B464

  fedora:fedora/x86_64/coreos/stable
                  Version: 38.20230625.3.0 (2023-07-11T11:57:53Z)
                   Commit: e841d77aadb875bb801ac845a0d9b8a70b4224bdeb15e7d6c5bff1da932c0301
             GPGSignature: Valid signature by 6A51BBABBA3D5467B6171221809A8D7CEB10B464
NOTA: O deployment atualmente inicializado é denotado pelo carácter .

Você também pode ver diferenças entre duas versões rodando o comando rpm-ostree db diff:

[core@localhost ~]$ rpm-ostree db diff
ostree diff commit from: rollback deployment (e841d77aadb875bb801ac845a0d9b8a70b4224bdeb15e7d6c5bff1da932c0301)
ostree diff commit to:   booted deployment (552de26fe0fe6a5e491f7a4163db125e3d44b144ae53a8f5f488e3f8481c46f9)
Upgraded:
  NetworkManager 1:1.42.6-1.fc38 -> 1:1.42.8-1.fc38
  NetworkManager-cloud-setup 1:1.42.6-1.fc38 -> 1:1.42.8-1.fc38
  NetworkManager-libnm 1:1.42.6-1.fc38 -> 1:1.42.8-1.fc38
  NetworkManager-team 1:1.42.6-1.fc38 -> 1:1.42.8-1.fc38
  NetworkManager-tui 1:1.42.6-1.fc38 -> 1:1.42.8-1.fc38
  aardvark-dns 1.6.0-1.fc38 -> 1.7.0-1.fc38
  ...

Revertendo para a versão anterior

Se o sistema não está funcionando totalmente por qualquer razão nós podemos voltar para a versão prévia:

sudo rpm-ostree rollback --reboot

After logging back in after reboot we can see we are now booted back into the old deployment from before the upgrade occurred:

[core@localhost ~]$ rpm-ostree status
State: idle
AutomaticUpdatesDriver: Zincati
  DriverState: active; periodically polling for updates (last checked Thu 2023-08-03 20:05:05 UTC)
Deployments:
● fedora:fedora/x86_64/coreos/stable
                  Version: 38.20230625.3.0 (2023-07-11T11:57:53Z)
                   Commit: e841d77aadb875bb801ac845a0d9b8a70b4224bdeb15e7d6c5bff1da932c0301
             GPGSignature: Valid signature by 6A51BBABBA3D5467B6171221809A8D7CEB10B464

  fedora:fedora/x86_64/coreos/stable
                  Version: 38.20230709.3.0 (2023-07-24T12:25:01Z)
                   Commit: 552de26fe0fe6a5e491f7a4163db125e3d44b144ae53a8f5f488e3f8481c46f9
             GPGSignature: Valid signature by 6A51BBABBA3D5467B6171221809A8D7CEB10B464

E vocẽ também pode verificar que Zincati não tentará atualizar para a nova versão de onde realizamos o rollback:

[core@localhost ~]$ systemctl status --full zincati.service
● zincati.service - Zincati Update Agent
     Loaded: loaded (/usr/lib/systemd/system/zincati.service; enabled; preset: enabled)
    Drop-In: /usr/lib/systemd/system/service.d
             └─10-timeout-abort.conf
     Active: active (running) since Thu 2023-08-03 20:05:05 UTC; 45s ago
       Docs: https://github.com/coreos/zincati
   Main PID: 813 (zincati)
     Status: "periodically polling for updates (last checked Thu 2023-08-03 20:05:05 UTC)"
      Tasks: 6 (limit: 2239)
     Memory: 22.0M
        CPU: 238ms
     CGroup: /system.slice/zincati.service
             └─813 /usr/libexec/zincati agent -v

Aug 03 20:05:05 localhost.localdomain zincati[813]: [INFO  zincati::cincinnati] Cincinnati service: https://updates.coreos.fedoraproject.org
Aug 03 20:05:05 localhost.localdomain zincati[813]: [INFO  zincati::cli::agent] agent running on node '87c9ec3e0a4045a19b74b54ae5ed986a', in update group 'default'
Aug 03 20:05:05 localhost.localdomain zincati[813]: [INFO  zincati::update_agent::actor] registering as the update driver for rpm-ostree
Aug 03 20:05:05 localhost.localdomain zincati[813]: [INFO  zincati::update_agent::actor] found 1 other finalized deployment
Aug 03 20:05:05 localhost.localdomain zincati[813]: [INFO  zincati::update_agent::actor] deployment 38.20230709.3.0 (552de26fe0fe6a5e491f7a4163db125e3d44b144ae53a8f5f488e3f8481c46f9) will be excluded from being a future update target
Aug 03 20:05:05 localhost.localdomain zincati[813]: [INFO  zincati::update_agent::actor] initialization complete, auto-updates logic enabled
Aug 03 20:05:05 localhost.localdomain zincati[813]: [INFO  zincati::strategy] update strategy: immediate
Aug 03 20:05:05 localhost.localdomain systemd[1]: Started zincati.service - Zincati Update Agent.
Aug 03 20:05:05 localhost.localdomain zincati[813]: [INFO  zincati::update_agent::actor] reached steady state, periodically polling for updates
Aug 03 20:05:06 localhost.localdomain zincati[813]: [INFO  zincati::cincinnati] current release detected as not a dead-end

Limpeza

Now let’s clean up the instance. Disconnect from the machine and then destroy it:

virsh destroy fcos
virsh undefine --remove-all-storage fcos

Conclusão

Nesses tutoriais nós aprendemos um pouco sobre o Fedora CoreOS. Nós aprendemos como é entregue como uma imagem de disco pré-criada, como é provisionada em um forma automática via Ignition, e também como atualizações automatizadas que são configurados e arquivos via zincati e RPM-OSTree. O próximo passo é tentar o Fedora CoreOS para seu próprio caso e se juntar à comunidade!