feat(dependencies): upgrade supported cluster components (#681)

* fix(calico): support split CRDs for current releases

- Download the v1_crd_projectcalico_org.yaml bundle before the operator
- Apply both files with server-side apply and force-conflicts per the
  upstream upgrade procedure
- Wait for the operator Deployment and for the managed CRDs to be
  Established after the operator starts
- Replace the create/rescue/replace flow with an idempotent apply that
  no longer conceals partial failures
- Verify TigeraStatus for calico and apiserver is Available, not just
  that Pods exist

* feat(dependencies): upgrade supported cluster components

- Bump K3s to v1.36.2+k3s1, Calico to v3.32.1, Cilium to v1.20.0,
  kube-vip to v1.2.2, kube-vip cloud provider to v0.0.12, and MetalLB to
  v0.16.0 across sample inventory, role defaults, and argument specs
- Pin the Cilium CLI with a new cilium_cli_tag (v0.19.7) instead of the
  floating stable.txt lookup
- Replace the CiliumBGPPeeringPolicy v2alpha1 BGP template with the
  v2 CiliumBGPClusterConfig, CiliumBGPPeerConfig, CiliumBGPAdvertisement,
  and CiliumLoadBalancerIPPool resource set
- Move Cilium load balancer Helm keys from bpf.loadBalancer to the valid
  top-level loadBalancer path
- Add preflight schema validation and remove the deprecated policy after
  the v2 objects are accepted
- Wait for cilium status after installation
- Pin kube-vip RBAC in a repository template instead of fetching a
  mutable URL, and include EndpointSlice permissions
- Fix the kube-vip bgppeers format to address:ASN comma-separated peers
- Fail clearly when the MetalLB speaker tag replacement does not apply
- Drop the obsolete MetalLB webhook service name version branch

* test(molecule): verify upgraded cluster components

- Assert every node reports the expected K3s kubelet version
- Verify the active CNI (Flannel / Calico / Cilium) is Ready and runs
  the expected image tag, including Calico TigeraStatus Available
- Verify the active load balancer (MetalLB / kube-vip) runs the expected
  image tags and that MetalLB is absent when kube-vip is active
- Assert no Flannel DaemonSet remains when Calico or Cilium is enabled
- Assert the example LoadBalancer address falls inside the configured
  pool range
- Add a manifest-only Cilium BGP regression test that renders the v2
  template with zero, one, and multiple neighbors and rejects any v2alpha1
  or CiliumBGPPeeringPolicy output

* fix(dependencies): correct dependency version pins

- Set the sample kube-vip image to v1.2.2 and repair the damaged comment
- Pin the kube-vip cloud provider default to v0.0.12 in the task URL
- Set the MetalLB controller argument-spec default to v0.16.0
- Restore the MetalLB available timeout default to 240s

* docs(dependencies): document current cluster versions

- Update kube-vip, kube-vip cloud provider, and MetalLB defaults
- Add cilium_tag and cilium_cli_tag rows
- Explain that MetalLB v0.16.0 is the application image target even though
  a newer chart-only tag (metallb-chart-0.16.1) exists
- Add an existing-cluster upgrade warning covering the K3s etcd 3.5.26
  bridge and one-minor-at-a-time rule, consecutive Cilium minor upgrades,
  Calico v3 resource UID handling, and MetalLB app vs chart tags

* fix(dependencies): address PR review findings

- Read the MetalLB speaker tag check from the managed host with slurp
  instead of a controller-side file lookup, and match the full image
  reference
- Restore the tigera-operator namespace on the Calico operator Deployment
  wait while keeping the managed CRD waits cluster-scoped
- Make Molecule verify inputs durable and scenario-specific via a
  per-scenario verify-vars.yml, driven by explicit verify_cni/verify_lb
  values instead of non-persisted converge facts
- Rename the kube-vip multi-peer BGP env var from bgppeers to bgp_peers
  and vip_cidr to vip_subnet so v1.2.2 actually reads them
- Map the legacy Cilium routed mode to tunnel and stop passing the alias
  directly to the chart
- Use return-code based failed_when on apply and preflight commands so
  non-error failures are no longer treated as success
- Clarify the sequential K3s upgrade path and backups in the README
- Add kube-vip and MetalLB regression tests and a Cilium mode mapping unit

* fix(dependencies): resolve re-review findings

- correct the Calico TigeraStatus resource kind\n- document tunnel as the supported Cilium routing mode\n- validate load balancer addresses across range and CIDR pools

* fix(molecule): verify embedded flannel instead of a flannel DaemonSet

- K3s 1.36 runs flannel embedded in the k3s agent rather than as a
  kube-flannel-ds DaemonSet, so the flannel verifier queried a workload
  that no longer exists and failed the verify step
- For the flannel scenarios, assert every node is Ready and that neither
  the Calico nor the Cilium namespace exists
- Drop the now-invalid kube-flannel-ds DaemonSet assertion

* fix(molecule): wait for the LoadBalancer address before asserting reachability

- The nginx LoadBalancer service had no ingress address when the
  reachability assertion ran, so status.loadBalancer.ingress[0].ip was
  undefined and the ipwrap filter failed during verify
- Poll the service until MetalLB or kube-vip assigns an external IP
- Record the assigned address once and reuse it for the reachability probe
  and the pool membership checks

* fix(ci): harden calico apiserver wait and extend molecule job timeout

- Bump calico system resources wait retries 30->60 and delay 7->10 so the
  slow-to-reconcile calico-apiserver deployment has enough time under nested-virt
- Raise the molecule step timeout-minutes from 90 to 150 to accommodate
  contended 5-node scenarios (cilium, kube-vip) that were hitting the 90-min cap

* fix(calico): treat optional API server as best-effort on converge

- The Calico API server (calico-apiserver) is an optional add-on for managing
  Calico policy through the projectcalico.org/v3 Kubernetes API; it is not
  required for Calico CNI data plane operation
- With Calico v3.32.1 on K3s 1.36 the tigera-operator never provisions the
  calico-apiserver namespace, causing the converge wait to fail deterministically
- Keep the strict wait for core Calico components (typha, kube-controllers,
  calico-node, csi-node-driver) and make the API server wait tolerate failure
- Restrict the TigeraStatus Available check to the calico status, matching the
  upstream v3.32.1 K3s quickstart which validates without the API server

.ansible-lint

@@ -1,17 +1,31 @@
 ---
+profile: production
 exclude_paths:
   # default paths
-  - '.cache/'
+  - .cache/
-  - '.github/'
+  - .ansible/
-  - 'test/fixtures/formatting-before/'
+  - .github/
-  - 'test/fixtures/formatting-prettier/'
+  - test/fixtures/formatting-before/
+  - test/fixtures/formatting-prettier/
 
   # The "converge" and "reset" playbooks use import_playbook in
   # conjunction with the "env" lookup plugin, which lets the
   # syntax check of ansible-lint fail.
-  - 'molecule/**/converge.yml'
+  - molecule/**/converge.yml
-  - 'molecule/**/prepare.yml'
+  - molecule/**/prepare.yml
-  - 'molecule/**/reset.yml'
+  - molecule/**/reset.yml
+
+  # Scenario verify inputs are plain variable files, not playbooks. They are
+  # loaded as vars, not executed, so ansible-lint must not treat them as plays.
+  - molecule/**/verify-vars.yml
+
+  # The file was generated by galaxy ansible - don't mess with it.
+  - galaxy.yml
 
 skip_list:
-  - 'fqcn-builtins'
+  - var-naming[no-role-prefix]
+
+# The Molecule Vagrant driver injects this module at runtime. The custom create
+# playbook is syntax-checked separately against the exact pinned plugin module.
+mock_modules:
+  - vagrant

.github/ISSUE_TEMPLATE.md

@@ -1,5 +1,5 @@
 
-<!-- It's a good idea to check this post first for general troubleshooting https://github.com/techno-tim/k3s-ansible/discussions/19   -->
+<!-- It's a good idea to check this post first for general troubleshooting https://github.com/timothystewart6/k3s-ansible/discussions/19   -->
 
 <!--- Provide a general summary of the issue in the Title above -->
 
@@ -37,6 +37,11 @@ systemd_dir: ""
 
 flannel_iface: ""
 
+#calico_iface: ""
+calico_ebpf: ""
+calico_cidr: ""
+calico_tag: ""
+
 apiserver_endpoint: ""
 
 k3s_token: "NA"
@@ -46,6 +51,9 @@ extra_agent_args: ""
 
 kube_vip_tag_version: ""
 
+kube_vip_cloud_provider_tag_version: ""
+kube_vip_lb_ip_range: ""
+
 metal_lb_speaker_tag_version: ""
 metal_lb_controller_tag_version: ""
 
@@ -74,4 +82,4 @@ node
 ## Possible Solution
 <!--- Not obligatory, but suggest a fix/reason for the bug, -->
 
-- [ ] I've checked the [General Troubleshooting Guide](https://github.com/techno-tim/k3s-ansible/discussions/20)
+- [ ] I've checked the [General Troubleshooting Guide](https://github.com/timothystewart6/k3s-ansible/discussions/20)

.github/actionlint.yaml

@@ -0,0 +1,6 @@
+---
+self-hosted-runner:
+  labels:
+    - k3s-ci
+    - virtualbox
+    - nested-virt

.github/dependabot.yml

@@ -9,3 +9,18 @@ updates:
     ignore:
       - dependency-name: "*"
         update-types: ["version-update:semver-major"]
+
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "daily"
+    rebase-strategy: "auto"
+
+  - package-ecosystem: "docker"
+    directory: "/"
+    schedule:
+      interval: "daily"
+    rebase-strategy: "auto"
+    ignore:
+      - dependency-name: "*"
+        update-types: ["version-update:semver-major"]

.github/download-boxes.sh

@@ -1,37 +1,91 @@
 #!/bin/bash
 
 # download-boxes.sh
-# Check all molecule.yml files for required Vagrant boxes and download the ones that are not
+# Validate the pinned Vagrant box set and download exact versions that are not
-# already present on the system.
+# already present in VAGRANT_HOME.
 
 set -euo pipefail
 
 GIT_ROOT=$(git rev-parse --show-toplevel)
 PROVIDER=virtualbox
+LOCK_FILE="${VAGRANT_BOX_LOCK_FILE:-${GIT_ROOT}/.github/vagrant-boxes.lock}"
 
-# Read all boxes for all platforms from the "molecule.yml" files
+MOLECULE_YML_PATH=("${GIT_ROOT}"/molecule/*/molecule.yml)
-all_boxes=$(cat "${GIT_ROOT}"/molecule/*/molecule.yml |
-    yq -r '.platforms[].box' |         # Read the "box" property of each node under "platforms"
-    grep --invert-match --regexp=--- | # Filter out file separators
-    sort |
-    uniq)
 
-# Read the boxes that are currently present on the system (for the current provider)
+# Extract the unique boxes referenced by the scenarios.
+declared_boxes=$(for file in "${MOLECULE_YML_PATH[@]}"; do
+    yq -r '.platforms[].box' "$file"
+done | sort -u)
+
+if [[ ! -r "$LOCK_FILE" ]]; then
+    printf 'Vagrant box lock file is missing or unreadable: %s\n' "$LOCK_FILE" >&2
+    exit 1
+fi
+
+lock_entries=$(awk '
+    /^[[:space:]]*#/ || NF == 0 { next }
+    NF != 3 {
+        printf "Invalid lock entry on line %d: expected box, version, architecture\n", NR > "/dev/stderr"
+        invalid = 1
+        next
+    }
+    { print $1 " " $2 " " $3 }
+    END { exit invalid }
+' "$LOCK_FILE")
+
+duplicate_boxes=$(printf '%s\n' "$lock_entries" | awk '{ print $1 }' | sort | uniq -d)
+if [[ -n "$duplicate_boxes" ]]; then
+    printf 'Duplicate Vagrant box lock entries:\n%s\n' "$duplicate_boxes" >&2
+    exit 1
+fi
+
+locked_boxes=$(printf '%s\n' "$lock_entries" | sort)
+locked_names=$(printf '%s\n' "$locked_boxes" | awk '{ print $1 }')
+missing_locks=$(comm -23 <(printf '%s\n' "$declared_boxes") <(printf '%s\n' "$locked_names"))
+unused_locks=$(comm -13 <(printf '%s\n' "$declared_boxes") <(printf '%s\n' "$locked_names"))
+
+if [[ -n "$missing_locks" || -n "$unused_locks" ]]; then
+    if [[ -n "$missing_locks" ]]; then
+        printf 'Scenario boxes missing from the lock file:\n%s\n' "$missing_locks" >&2
+    fi
+    if [[ -n "$unused_locks" ]]; then
+        printf 'Lock entries not referenced by a scenario:\n%s\n' "$unused_locks" >&2
+    fi
+    exit 1
+fi
+
+printf 'Pinned Vagrant boxes:\n%s\n' "$locked_boxes"
+
+# Read exact box, provider, version, and architecture tuples already present.
 present_boxes=$(
-    (vagrant box list |
+    vagrant box list --machine-readable |
-        grep "${PROVIDER}" | # Filter by boxes available for the current provider
+        awk -F, -v expected_provider="$PROVIDER" '
-        awk '{print $1;}' |  # The box name is the first word in each line
+            $3 == "box-name" { name = $4; next }
-        sort |
+            $3 == "box-provider" { provider = $4; next }
-        uniq) ||
+            $3 == "box-version" { version = $4; next }
-        echo "" # In case any of these commands fails, just use an empty list
+            $3 == "box-architecture" {
+                architecture = $4
+                if (provider == expected_provider) {
+                    print name " " version " " architecture
+                }
+                name = provider = version = architecture = ""
+            }
+        ' |
+        sort -u
 )
 
-# The boxes that we need to download are the ones present in $all_boxes, but not $present_boxes.
+download_boxes=$(comm -23 \
-download_boxes=$(comm -2 -3 <(echo "${all_boxes}") <(echo "${present_boxes}"))
+    <(printf '%s\n' "$locked_boxes") \
+    <(printf '%s\n' "$present_boxes"))
 
-# Actually download the necessary boxes
+if [[ -n "$download_boxes" ]]; then
-if [ -n "${download_boxes}" ]; then
+    printf '%s\n' "$download_boxes" | while read -r box version architecture; do
-    echo "${download_boxes}" | while IFS= read -r box; do
+        vagrant box add \
-        vagrant box add --provider "${PROVIDER}" "${box}"
+            --provider "$PROVIDER" \
+            --box-version "$version" \
+            --architecture "$architecture" \
+            "$box"
     done
+else
+    printf 'All pinned Vagrant boxes are already present.\n'
 fi

.github/scripts/cleanup-runner-resources.sh

@@ -0,0 +1,249 @@
+#!/usr/bin/env bash
+
+set -Eeuo pipefail
+
+usage() {
+  printf '%s\n' \
+    'Usage: cleanup-runner-resources.sh [--snapshot|--dry-run|--apply]' \
+    '' \
+    'Discover and, with --apply, remove only VirtualBox resources referenced by' \
+    'repository-owned Molecule Vagrant state. The default is --dry-run.'
+}
+
+mode="dry-run"
+case "${1:-}" in
+  "") ;;
+  --snapshot) mode="snapshot" ;;
+  --dry-run) mode="dry-run" ;;
+  --apply) mode="apply" ;;
+  --help|-h) usage; exit 0 ;;
+  *) usage >&2; exit 2 ;;
+esac
+
+home_dir="${HOME:?HOME must be set}"
+molecule_root="${K3S_CI_MOLECULE_ROOT:-${home_dir}/.cache/molecule}"
+repository_name="${K3S_CI_MOLECULE_PROJECT:-k3s-ansible}"
+virtualbox_root="${K3S_CI_VIRTUALBOX_ROOT:-${home_dir}/VirtualBox VMs}"
+hostonly_marker="${K3S_CI_HOSTONLY_MARKER:-${home_dir}/.cache/k3s-ci/hostonly-interfaces}"
+
+record_hostonly() {
+  local marker_dir="${hostonly_marker%/*}"
+  local marker_tmp="${hostonly_marker}.tmp"
+  local hostonly_inventory
+  if ! hostonly_inventory="$(VBoxManage list hostonlyifs)"; then
+    fail_closed 'unable to inventory VirtualBox host-only interfaces'
+  fi
+  mkdir -p -- "$marker_dir"
+  awk -F': ' '
+    /^Name:/ { name=$2 }
+    /^IPAddress:/ { print name "|" $2 }
+  ' <<< "$hostonly_inventory" > "$marker_tmp"
+  mv -- "$marker_tmp" "$hostonly_marker"
+  chmod 600 "$hostonly_marker"
+  printf 'Recorded host-only interface baseline: %s\n' "$hostonly_marker"
+}
+
+cleanup_hostonly() {
+  local hostonly_inventory
+  if [[ ! -f "$hostonly_marker" ]]; then
+    printf 'No host-only interface baseline found; leaving interfaces unchanged.\n'
+    return 0
+  fi
+
+  if ! hostonly_inventory="$(VBoxManage list hostonlyifs)"; then
+    fail_closed 'unable to inventory VirtualBox host-only interfaces'
+  fi
+
+  while IFS='|' read -r interface_name interface_ip; do
+    [[ "$interface_name" == vboxnet* ]] || continue
+    [[ "$interface_ip" == 192.168.30.* || "$interface_ip" == fdad:bad:ba55:* ]] || continue
+    if grep -Fqx "${interface_name}|${interface_ip}" "$hostonly_marker"; then
+      continue
+    fi
+    if [[ "$mode" == apply ]]; then
+      VBoxManage hostonlyif remove "$interface_name"
+      printf 'Removed host-only interface %s (%s)\n' "$interface_name" "$interface_ip"
+    else
+      printf 'Would remove host-only interface %s (%s)\n' "$interface_name" "$interface_ip"
+    fi
+  done < <(awk -F': ' '
+    /^Name:/ { name=$2 }
+    /^IPAddress:/ { print name "|" $2 }
+  ' <<< "$hostonly_inventory")
+}
+
+if [[ "$mode" == snapshot ]]; then
+  record_hostonly
+  exit 0
+fi
+
+resolve_existing_dir() {
+  local candidate="$1"
+  if [[ ! -d "$candidate" ]]; then
+    return 1
+  fi
+  readlink -f -- "$candidate"
+}
+
+root_contains() {
+  local root="$1"
+  local path="$2"
+  [[ "$path" == "$root"/* ]]
+}
+
+is_supported_scenario() {
+  case "$1" in
+    default|single_node|calico|cilium|kube-vip|ipv6) return 0 ;;
+    *) return 1 ;;
+  esac
+}
+
+is_unregistered_vm_error() {
+  grep -Eq 'Could not find a registered machine|VBOX_E_OBJECT_NOT_FOUND'
+}
+
+fail_closed() {
+  printf 'cleanup refused: %s\n' "$1" >&2
+  exit 3
+}
+
+if [[ ! "$repository_name" =~ ^[A-Za-z0-9._-]+$ ]]; then
+  fail_closed 'invalid Molecule repository name'
+fi
+
+print_inventory() {
+  local phase="$1"
+  printf '%s VirtualBox inventory:\n' "$phase"
+  VBoxManage list vms || true
+  VBoxManage list hdds || true
+  VBoxManage list hostonlyifs || true
+}
+
+molecule_root_real="$(resolve_existing_dir "$molecule_root" || true)"
+if [[ -z "$molecule_root_real" ]]; then
+  printf 'No Molecule root exists: %s\n' "$molecule_root"
+  cleanup_hostonly
+  exit 0
+fi
+
+print_inventory before
+
+repository_root_real="$(resolve_existing_dir "$molecule_root_real/$repository_name" || true)"
+if [[ -z "$repository_root_real" ]]; then
+  printf 'No repository Molecule state root exists: %s\n' "$molecule_root_real/$repository_name"
+  cleanup_hostonly
+  print_inventory after
+  exit 0
+fi
+if ! root_contains "$molecule_root_real" "$repository_root_real"; then
+  fail_closed "repository Molecule state root is outside Molecule root: $repository_root_real"
+fi
+
+declare -a state_files=()
+while IFS= read -r -d '' state_file; do
+  state_files+=("$state_file")
+done < <(find "$repository_root_real" -mindepth 6 -maxdepth 6 -type f \
+  -path '*/.vagrant/machines/*/virtualbox/id' -print0 2>/dev/null)
+
+if ((${#state_files[@]} == 0)); then
+  printf 'No repository-owned Molecule Vagrant state found under %s\n' "$repository_root_real"
+  cleanup_hostonly
+  print_inventory after
+  exit 0
+fi
+
+virtualbox_root_real="$(resolve_existing_dir "$virtualbox_root" || true)"
+
+declare -a vm_records=()
+for state_file in "${state_files[@]}"; do
+  if [[ ! -f "$state_file" ]]; then
+    printf 'Skipping Vagrant state removed with its stale scenario directory: %s\n' "$state_file"
+    continue
+  fi
+  state_file_real="$(readlink -f -- "$state_file")"
+  state_dir="${state_file_real%/.vagrant/machines/*/virtualbox/id}"
+  machine_dir="${state_file_real%/virtualbox/id}"
+  machine_name="${machine_dir##*/}"
+  scenario_name="${state_dir##*/}"
+
+  if ! root_contains "$repository_root_real" "$state_dir"; then
+    fail_closed "state path is outside the repository Molecule root: $state_file_real"
+  fi
+  if ! is_supported_scenario "$scenario_name"; then
+    fail_closed "unexpected Molecule scenario: $scenario_name"
+  fi
+  if [[ "$machine_name" != control* && "$machine_name" != node* ]]; then
+    fail_closed "unexpected Molecule machine name: $machine_name"
+  fi
+
+  vm_uuid="$(tr -d '[:space:]' < "$state_file_real")"
+  if [[ ! "$vm_uuid" =~ ^[0-9a-fA-F-]{36}$ ]]; then
+    fail_closed "invalid VirtualBox UUID in $state_file_real"
+  fi
+
+  if ! vm_info="$(VBoxManage showvminfo "$vm_uuid" --machinereadable 2>&1)"; then
+    if is_unregistered_vm_error <<< "$vm_info"; then
+      printf 'Stale Vagrant state without a registered VM: %s (%s)\n' "$machine_name" "$vm_uuid"
+      if [[ "$mode" == apply ]]; then
+        rm -rf -- "${state_dir}/.vagrant"
+        printf 'Removed stale Vagrant state: %s\n' "${state_dir}/.vagrant"
+      fi
+      continue
+    fi
+    fail_closed "unable to inspect VirtualBox VM $vm_uuid: $vm_info"
+  fi
+
+  if [[ -z "$virtualbox_root_real" ]]; then
+    fail_closed "VirtualBox VM root does not exist: $virtualbox_root"
+  fi
+
+  cfg_file="$(awk -F= '$1 == "CfgFile" {gsub(/"/, "", $2); print $2; exit}' <<< "$vm_info")"
+  if [[ -z "$cfg_file" ]]; then
+    fail_closed "VirtualBox configuration path missing for $vm_uuid"
+  fi
+  cfg_file_real="$(readlink -f -- "$cfg_file")"
+  if ! root_contains "$virtualbox_root_real" "$cfg_file_real"; then
+    fail_closed "VM configuration is outside VirtualBox root: $cfg_file_real"
+  fi
+
+  while IFS= read -r disk_path; do
+    [[ -z "$disk_path" ]] && continue
+    disk_path_real="$(readlink -f -- "$disk_path" 2>/dev/null || true)"
+    if [[ -z "$disk_path_real" ]] || ! root_contains "$virtualbox_root_real" "$disk_path_real"; then
+      fail_closed "attached disk is outside VirtualBox root: $disk_path"
+    fi
+  done < <(awk -F= '$1 ~ /^(SATA|IDE|SCSI|SAS|VirtioSCSI|NVMe)-[0-9]+-[0-9]+$/ {gsub(/"/, "", $2); print $2}' <<< "$vm_info")
+
+  vm_records+=("$vm_uuid|$machine_name|$cfg_file_real")
+done
+
+if ((${#vm_records[@]} == 0)); then
+  printf 'No live repository-owned VirtualBox resources found\n'
+  cleanup_hostonly
+  print_inventory after
+  exit 0
+fi
+
+for record in "${vm_records[@]}"; do
+  IFS='|' read -r vm_uuid machine_name cfg_file_real <<< "$record"
+  if [[ "$mode" == dry-run ]]; then
+    printf 'Would remove VM %s (%s) config=%s\n' "$machine_name" "$vm_uuid" "$cfg_file_real"
+    continue
+  fi
+
+  vm_state="$(VBoxManage showvminfo "$vm_uuid" --machinereadable | awk -F= '$1 == "VMState" {gsub(/"/, "", $2); print $2; exit}')"
+  if [[ "$vm_state" != poweroff && "$vm_state" != saved ]]; then
+    VBoxManage controlvm "$vm_uuid" poweroff
+  fi
+  VBoxManage unregistervm "$vm_uuid" --delete
+  printf 'Removed VM %s (%s)\n' "$machine_name" "$vm_uuid"
+done
+
+cleanup_hostonly
+print_inventory after
+
+if [[ "$mode" == apply ]]; then
+  printf 'Repository-owned VM and host-only interface cleanup complete.\n'
+else
+  printf 'Dry run complete. No resources were modified.\n'
+fi

.github/scripts/collect-runner-diagnostics.sh

@@ -0,0 +1,47 @@
+#!/usr/bin/env bash
+
+set -Eeuo pipefail
+
+output_dir="${1:-${RUNNER_TEMP:-/tmp}/k3s-ci-diagnostics}"
+mkdir -p -- "$output_dir"
+umask 077
+
+run_capture() {
+  local output_file="$1"
+  shift
+  {
+    printf '$'
+    printf ' %q' "$@"
+    printf '\n'
+    "$@"
+  } > "$output_dir/$output_file" 2>&1 || true
+}
+
+run_capture system.txt uname -a
+run_capture runner-user.txt id
+run_capture memory.txt free -h
+run_capture disk.txt df -h
+run_capture virtualbox-version VBoxManage --version
+run_capture virtualbox-vms VBoxManage list vms
+run_capture virtualbox-running-vms VBoxManage list runningvms
+run_capture virtualbox-disks VBoxManage list hdds
+run_capture virtualbox-hostonlyifs VBoxManage list hostonlyifs
+run_capture virtualbox-groups VBoxManage list groups
+run_capture vagrant-status vagrant global-status
+run_capture molecule-state find "${HOME}/.cache/molecule" -maxdepth 6 -type f -path '*/.vagrant/machines/*/virtualbox/id' -print
+
+scenario_name="${K3S_CI_SCENARIO_NAME:-}"
+if [[ "$scenario_name" =~ ^[A-Za-z0-9_-]+$ ]]; then
+  molecule_state_dir="${HOME}/.cache/molecule/k3s-ansible/${scenario_name}"
+  for log_name in vagrant.out vagrant.err; do
+    if [[ -r "${molecule_state_dir}/${log_name}" ]]; then
+      cp -- "${molecule_state_dir}/${log_name}" "$output_dir/${scenario_name}-${log_name}"
+    fi
+  done
+fi
+
+if [[ -r /etc/vbox/networks.conf ]]; then
+  cp -- /etc/vbox/networks.conf "$output_dir/virtualbox-networks.conf"
+fi
+
+printf 'Diagnostics written to %s\n' "$output_dir"

.github/scripts/monitor-runner-resources.sh

@@ -0,0 +1,44 @@
+#!/usr/bin/env bash
+
+set -Eeuo pipefail
+
+output_dir="${1:?output directory is required}"
+interval="${2:-10}"
+[[ "$interval" =~ ^[1-9][0-9]*$ ]] || {
+  printf 'monitor interval must be a positive integer\n' >&2
+  exit 2
+}
+
+mkdir -p -- "$output_dir"
+umask 077
+
+free -h > "$output_dir/memory-before.txt"
+df -h > "$output_dir/disk-before.txt"
+vmstat -w "$interval" > "$output_dir/vmstat.txt" &
+vmstat_pid=$!
+
+iostat_pid=""
+if command -v iostat >/dev/null 2>&1; then
+  iostat -dx "$interval" > "$output_dir/iostat.txt" &
+  iostat_pid=$!
+else
+  printf 'iostat is not installed on this runner\n' > "$output_dir/iostat-unavailable.txt"
+fi
+
+cleanup() {
+  local rc=$?
+  trap - EXIT INT TERM
+  kill "$vmstat_pid" 2>/dev/null || true
+  [[ -z "$iostat_pid" ]] || kill "$iostat_pid" 2>/dev/null || true
+  wait "$vmstat_pid" 2>/dev/null || true
+  [[ -z "$iostat_pid" ]] || wait "$iostat_pid" 2>/dev/null || true
+  free -h > "$output_dir/memory-after.txt"
+  df -h > "$output_dir/disk-after.txt"
+  exit "$rc"
+}
+trap cleanup EXIT INT TERM
+
+while :; do
+  sleep 3600 &
+  wait $!
+done

.github/scripts/prepare-vagrant-box-masters.sh

@@ -0,0 +1,211 @@
+#!/usr/bin/env bash
+
+set -Eeuo pipefail
+
+fail() {
+  printf 'Vagrant box master preparation refused: %s\n' "$1" >&2
+  exit 3
+}
+
+root_contains() {
+  local root="$1"
+  local path="$2"
+  [[ "$path" == "$root"/* ]]
+}
+
+read_machine_value() {
+  local machine_info="$1"
+  local key="$2"
+  awk -F= -v key="$key" '$1 == key {gsub(/"/, "", $2); print $2; exit}' <<< "$machine_info"
+}
+
+read_extra_data() {
+  local uuid="$1"
+  local key="$2"
+  local value
+  value="$(VBoxManage getextradata "$uuid" "$key" 2>/dev/null || true)"
+  [[ "$value" == 'Value: '* ]] || return 1
+  printf '%s\n' "${value#Value: }"
+}
+
+validate_owned_master() {
+  local uuid="$1"
+  local box="$2"
+  local version="$3"
+  local architecture="$4"
+  local machine_info cfg_file cfg_file_real vm_state groups disk_path disk_path_real
+
+  [[ "$uuid" =~ ^[0-9a-fA-F-]{36}$ ]] || return 1
+  machine_info="$(VBoxManage showvminfo "$uuid" --machinereadable 2>/dev/null)" || return 1
+  vm_state="$(read_machine_value "$machine_info" VMState)"
+  groups="$(read_machine_value "$machine_info" groups)"
+  cfg_file="$(read_machine_value "$machine_info" CfgFile)"
+  [[ "$vm_state" == poweroff ]] || return 1
+  [[ ",$groups," == *,/k3s-ansible/box-masters,* ]] || return 1
+  [[ -n "$cfg_file" ]] || return 1
+  cfg_file_real="$(readlink -f -- "$cfg_file" 2>/dev/null || true)"
+  [[ -n "$cfg_file_real" ]] || return 1
+  root_contains "$virtualbox_root_real" "$cfg_file_real" || return 1
+
+  [[ "$(read_extra_data "$uuid" k3s-ansible/owner || true)" == box-master ]] || return 1
+  [[ "$(read_extra_data "$uuid" k3s-ansible/box || true)" == "$box" ]] || return 1
+  [[ "$(read_extra_data "$uuid" k3s-ansible/version || true)" == "$version" ]] || return 1
+  [[ "$(read_extra_data "$uuid" k3s-ansible/architecture || true)" == "$architecture" ]] || return 1
+
+  while IFS= read -r disk_path; do
+    [[ -z "$disk_path" || "$disk_path" == none ]] && continue
+    disk_path_real="$(readlink -f -- "$disk_path" 2>/dev/null || true)"
+    [[ -n "$disk_path_real" ]] || return 1
+    root_contains "$virtualbox_root_real" "$disk_path_real" || return 1
+  done < <(awk -F= '$1 ~ /^(SATA|IDE|SCSI|SAS|VirtioSCSI|NVMe)-[0-9]+-[0-9]+$/ {
+    gsub(/"/, "", $2); print $2
+  }' <<< "$machine_info")
+}
+
+write_prewarm_vagrantfile() {
+  local destination="$1"
+  local box="$2"
+  local version="$3"
+  {
+    printf '%s\n' "Vagrant.configure('2') do |config|"
+    printf '  config.vm.box = "%s"\n' "$box"
+    printf '  config.vm.box_version = "%s"\n' "$version"
+    printf '%s\n' \
+      '  config.vm.synced_folder ".", "/vagrant", disabled: true' \
+      '  config.vm.hostname = "k3s-ansible-box-prewarm"' \
+      '  config.vm.boot_timeout = 600' \
+      '  config.vm.provider "virtualbox" do |virtualbox|' \
+      '    virtualbox.linked_clone = true' \
+      '    virtualbox.memory = 1024' \
+      '    virtualbox.cpus = 2' \
+      '  end' \
+      'end'
+  } > "$destination"
+}
+
+cleanup_prewarm() {
+  local rc=$?
+  trap - EXIT
+  if [[ -n "${prewarm_dir:-}" && -d "$prewarm_dir" ]]; then
+    VAGRANT_CWD="$prewarm_dir" vagrant destroy --force >/dev/null 2>&1 || true
+    rm -rf -- "$prewarm_dir"
+  fi
+  exit "$rc"
+}
+
+create_owned_master() {
+  local box="$1"
+  local version="$2"
+  local architecture="$3"
+  local master_id_file="$4"
+  local mapping_file="$5"
+  local uuid machine_info cfg_file cfg_file_real vm_state mapping_tmp
+
+  # A master_id restored from an immutable cache is only a hint. Without the
+  # runner-local ownership record and matching VirtualBox metadata it is not
+  # trusted, adopted, modified, or deleted.
+  rm -f -- "$master_id_file"
+
+  prewarm_dir="$(mktemp -d "${master_root}/prewarm.XXXXXX")"
+  trap cleanup_prewarm EXIT
+  write_prewarm_vagrantfile "$prewarm_dir/Vagrantfile" "$box" "$version"
+
+  printf 'Creating runner-owned linked-clone master for %s %s %s\n' \
+    "$box" "$version" "$architecture"
+  VAGRANT_CWD="$prewarm_dir" vagrant up --provider virtualbox --no-provision
+
+  [[ -r "$master_id_file" ]] || fail "Vagrant did not record a master UUID for $box"
+  uuid="$(tr -d '[:space:]' < "$master_id_file")"
+  [[ "$uuid" =~ ^[0-9a-fA-F-]{36}$ ]] || fail "Vagrant recorded an invalid master UUID for $box"
+
+  machine_info="$(VBoxManage showvminfo "$uuid" --machinereadable 2>/dev/null)" || \
+    fail "Vagrant master $uuid for $box is not registered"
+  vm_state="$(read_machine_value "$machine_info" VMState)"
+  cfg_file="$(read_machine_value "$machine_info" CfgFile)"
+  cfg_file_real="$(readlink -f -- "$cfg_file" 2>/dev/null || true)"
+  [[ "$vm_state" == poweroff ]] || fail "new Vagrant master $uuid is not powered off"
+  if [[ -z "$cfg_file_real" ]] || ! root_contains "$virtualbox_root_real" "$cfg_file_real"; then
+    fail "new Vagrant master $uuid is outside the runner VirtualBox root"
+  fi
+
+  VAGRANT_CWD="$prewarm_dir" vagrant destroy --force
+  VBoxManage modifyvm "$uuid" --groups /k3s-ansible/box-masters
+  VBoxManage setextradata "$uuid" k3s-ansible/owner box-master
+  VBoxManage setextradata "$uuid" k3s-ansible/box "$box"
+  VBoxManage setextradata "$uuid" k3s-ansible/version "$version"
+  VBoxManage setextradata "$uuid" k3s-ansible/architecture "$architecture"
+  validate_owned_master "$uuid" "$box" "$version" "$architecture" || \
+    fail "new Vagrant master $uuid failed ownership validation"
+
+  rm -rf -- "$prewarm_dir"
+  prewarm_dir=""
+  trap - EXIT
+
+  mapping_tmp="${mapping_file}.tmp"
+  printf '%s\n' "$uuid" > "$mapping_tmp"
+  chmod 600 "$mapping_tmp"
+  mv -- "$mapping_tmp" "$mapping_file"
+  printf '%s\n' "$uuid" > "$master_id_file"
+  chmod 600 "$master_id_file"
+  printf 'Created and recorded owned master %s for %s\n' "$uuid" "$box"
+}
+
+repository_root="${K3S_CI_REPOSITORY_ROOT:-$(git rev-parse --show-toplevel)}"
+lock_file="${VAGRANT_BOX_LOCK_FILE:-${repository_root}/.github/vagrant-boxes.lock}"
+vagrant_home="${VAGRANT_HOME:?VAGRANT_HOME must be set}"
+master_root="${K3S_CI_VAGRANT_MASTER_ROOT:-${HOME:?HOME must be set}/.cache/k3s-ci/vagrant-masters}"
+virtualbox_root="${K3S_CI_VIRTUALBOX_ROOT:-${HOME}/VirtualBox VMs}"
+
+[[ -r "$lock_file" ]] || fail "box lock file is missing or unreadable: $lock_file"
+[[ -d "$vagrant_home/boxes" ]] || fail "Vagrant box directory is missing: $vagrant_home/boxes"
+[[ -d "$virtualbox_root" ]] || fail "VirtualBox root is missing: $virtualbox_root"
+
+box_root_real="$(readlink -f -- "$vagrant_home/boxes")"
+virtualbox_root_real="$(readlink -f -- "$virtualbox_root")"
+mkdir -p -- "$master_root"
+chmod 700 "$master_root"
+
+exec 9> "${master_root}/prepare.lock"
+flock 9
+
+lock_entries="$(awk '
+  /^[[:space:]]*#/ || NF == 0 { next }
+  NF != 3 { invalid = 1; next }
+  { print $1 " " $2 " " $3 }
+  END { exit invalid }
+' "$lock_file")" || fail 'invalid Vagrant box lock entry'
+[[ -n "$lock_entries" ]] || fail 'Vagrant box lock is empty'
+
+while read -r box version architecture; do
+  [[ "$box" =~ ^[A-Za-z0-9._-]+/[A-Za-z0-9._-]+$ ]] || fail "invalid box name: $box"
+  [[ "$version" =~ ^[A-Za-z0-9._-]+$ ]] || fail "invalid box version: $version"
+  [[ "$architecture" =~ ^[A-Za-z0-9._-]+$ ]] || fail "invalid box architecture: $architecture"
+
+  box_slug="${box//\//-VAGRANTSLASH-}"
+  record_slug="${box//\//_}-${version}-${architecture}"
+  box_dir="${vagrant_home}/boxes/${box_slug}/${version}/${architecture}/virtualbox"
+  [[ -d "$box_dir" ]] || fail "pinned box is not installed: $box $version $architecture"
+  box_dir_real="$(readlink -f -- "$box_dir")"
+  root_contains "$box_root_real" "$box_dir_real" || fail "box directory is outside VAGRANT_HOME: $box_dir_real"
+
+  master_id_file="${box_dir_real}/master_id"
+  mapping_file="${master_root}/${record_slug}.uuid"
+  uuid=""
+  if [[ -r "$mapping_file" ]]; then
+    uuid="$(tr -d '[:space:]' < "$mapping_file")"
+  fi
+
+  if [[ -n "$uuid" ]] && validate_owned_master "$uuid" "$box" "$version" "$architecture"; then
+    printf '%s\n' "$uuid" > "$master_id_file"
+    chmod 600 "$master_id_file"
+    printf 'Reusing owned master %s for %s %s %s\n' "$uuid" "$box" "$version" "$architecture"
+    continue
+  fi
+
+  if [[ -e "$mapping_file" ]]; then
+    printf 'Owned master record is stale for %s; rebuilding without deleting any VM or disk.\n' "$box"
+  fi
+  create_owned_master "$box" "$version" "$architecture" "$master_id_file" "$mapping_file"
+done <<< "$lock_entries"
+
+printf 'All pinned Vagrant box masters are ready.\n'

.github/scripts/test-cilium-bgp-manifest.py

@@ -0,0 +1,134 @@
+#!/usr/bin/env python3
+"""Render the Cilium BGP CRD template and assert it uses the v2 API.
+
+This is a manifest-only regression test used where no real BGP peer is
+available. It renders roles/k3s_server_post/templates/cilium.crs.j2 with
+zero, one, and multiple neighbors, then checks that the output:
+  - never contains CiliumBGPPeeringPolicy or cilium.io/v2alpha1
+  - emits the Cilium v2 BGP resources
+  - emits deterministic DNS-safe peer and instance names
+  - advertises Pod CIDRs only when cilium_exportPodCIDR is true
+"""
+
+from __future__ import print_function
+
+import os
+import subprocess
+
+from jinja2 import Environment, FileSystemLoader, StrictUndefined
+
+
+def repo_root():
+    return subprocess.check_output(
+        ["git", "rev-parse", "--show-toplevel"], text=True
+    ).strip()
+
+
+def fail(message):
+    raise SystemExit("Cilium BGP manifest test failed: " + message)
+
+
+def render(env, extra_vars):
+    base_vars = {
+        "cilium_bgp_my_asn": "64513",
+        "cilium_bgp_peer_asn": "64512",
+        "cilium_bgp_peer_address": "192.168.30.1",
+        "cilium_exportPodCIDR": True,
+        "cilium_bgp_lb_cidr": "192.168.31.0/24",
+    }
+    base_vars.update(extra_vars)
+    template = env.get_template("cilium.crs.j2")
+    return template.render(**base_vars)
+
+
+def check_common(output):
+    if "cilium.io/v2alpha1" in output:
+        fail("rendered output still contains cilium.io/v2alpha1")
+    if "kind: CiliumBGPPeeringPolicy" in output:
+        fail("rendered output still contains CiliumBGPPeeringPolicy")
+    for kind in (
+        "CiliumBGPPeerConfig",
+        "CiliumBGPClusterConfig",
+        "CiliumBGPAdvertisement",
+        "CiliumLoadBalancerIPPool",
+    ):
+        if ("kind: " + kind) not in output:
+            fail("rendered output is missing kind: " + kind)
+
+
+def main():
+    root = repo_root()
+    template_dir = os.path.join(
+        root, "roles", "k3s_server_post", "templates"
+    )
+    env = Environment(
+        loader=FileSystemLoader(template_dir), undefined=StrictUndefined
+    )
+
+    # Zero neighbors -> fall back to the single default peer.
+    output = render(env, {"_cilium_bgp_neighbors": []})
+    check_common(output)
+    if "peer-64512-1" not in output:
+        fail("default single peer name was not rendered")
+    if "peerAddress: 192.168.30.1" not in output:
+        fail("default peer address was not rendered")
+    if 'advertisementType: "PodCIDR"' not in output:
+        fail("PodCIDR advertisement missing when exportPodCIDR is true")
+
+    # One neighbor via the merged list.
+    output = render(
+        env,
+        {"_cilium_bgp_neighbors": [{"peer_address": "10.0.0.1", "peer_asn": "65001"}]},
+    )
+    check_common(output)
+    if "peer-65001-1" not in output:
+        fail("single merged peer name was not rendered")
+    if "peerAddress: 10.0.0.1" not in output:
+        fail("single merged peer address was not rendered")
+
+    # Multiple neighbors.
+    output = render(
+        env,
+        {
+            "_cilium_bgp_neighbors": [
+                {"peer_address": "10.0.0.1", "peer_asn": "65001"},
+                {"peer_address": "10.0.0.2", "peer_asn": "65002"},
+            ]
+        },
+    )
+    check_common(output)
+    if "peer-65001-1" not in output or "peer-65002-2" not in output:
+        fail("multiple merged peer names were not rendered")
+    if "peerAddress: 10.0.0.2" not in output:
+        fail("second merged peer address was not rendered")
+
+    # exportPodCIDR false -> no PodCIDR advertisement, service remains.
+    output = render(
+        env, {"_cilium_bgp_neighbors": [], "cilium_exportPodCIDR": False}
+    )
+    check_common(output)
+    if 'advertisementType: "PodCIDR"' in output:
+        fail("PodCIDR advertisement present when exportPodCIDR is false")
+    if 'advertisementType: "Service"' not in output:
+        fail("Service advertisement missing when exportPodCIDR is false")
+
+    # Load balancer pools: CIDR and start/stop forms.
+    output = render(env, {"_cilium_bgp_neighbors": []})
+    if "cidr: 192.168.31.0/24" not in output:
+        fail("CIDR load balancer pool was not rendered")
+    output = render(
+        env,
+        {
+            "_cilium_bgp_neighbors": [],
+            "cilium_bgp_lb_cidr": "192.168.31.80-192.168.31.90",
+        },
+    )
+    check_common(output)
+    if "start: 192.168.31.80" not in output or "stop: 192.168.31.90" not in output:
+        fail("start/stop load balancer pool was not rendered")
+
+    print("Cilium BGP manifest regression test passed")
+
+
+if __name__ == "__main__":
+    main()

.github/scripts/test-cleanup-runner-resources.sh

@@ -0,0 +1,129 @@
+#!/usr/bin/env bash
+
+# shellcheck disable=SC2016
+
+set -Eeuo pipefail
+
+repo_root="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)"
+test_root="$(mktemp -d "${TMPDIR:-/tmp}/k3s-ci-cleanup-test.XXXXXX")"
+trap 'rm -rf -- "$test_root"' EXIT
+
+molecule_root="$test_root/molecule"
+virtualbox_root="$test_root/VirtualBox VMs"
+fake_bin="$test_root/bin"
+mkdir -p -- "$molecule_root/k3s-ansible/single_node/.vagrant/machines/control1/virtualbox" \
+  "$molecule_root/k3s-ansible/single_node/.vagrant/machines/control2/virtualbox" \
+  "$virtualbox_root/control1" "$virtualbox_root/unmarked" "$fake_bin"
+printf '%s\n' '11111111-1111-1111-1111-111111111111' \
+  > "$molecule_root/k3s-ansible/single_node/.vagrant/machines/control1/virtualbox/id"
+printf '%s\n' '22222222-2222-2222-2222-222222222222' \
+  > "$molecule_root/k3s-ansible/single_node/.vagrant/machines/control2/virtualbox/id"
+touch "$virtualbox_root/control1/control1.vbox" "$virtualbox_root/control1/disk.vdi" \
+  "$virtualbox_root/unmarked/unmarked.vbox"
+
+printf '%s\n' \
+  '#!/usr/bin/env bash' \
+  'set -Eeuo pipefail' \
+  'case "${1:-}" in' \
+  '  list)' \
+  '    if [[ "${2:-}" == hostonlyifs && "${FAKE_HOSTONLY_FAIL:-false}" == true ]]; then exit 1; fi' \
+  '    exit 0' \
+  '    ;;' \
+  '  showvminfo)' \
+  '    if [[ "${FAKE_VM_MODE:-normal}" == missing ]]; then' \
+  '      printf '\''VBoxManage: error: Could not find a registered machine named "missing"\n'\'' >&2' \
+  '      exit 1' \
+  '    fi' \
+  '    if [[ "${FAKE_VM_MODE:-normal}" == fault ]]; then' \
+  '      printf '\''VBoxManage: error: VirtualBox service is unavailable\n'\'' >&2' \
+  '      exit 1' \
+  '    fi' \
+  '    printf '\''CfgFile="%s"\n'\'' "${FAKE_VBOX_ROOT}/control1/control1.vbox"' \
+  '    printf '\''SATA-0-0="%s"\n'\'' "${FAKE_VBOX_ROOT}/control1/disk.vdi"' \
+  '    printf '\''VMState="running"\n'\''' \
+  '    ;;' \
+  '  controlvm) printf '\''controlvm %s\n'\'' "$*" >> "${FAKE_LOG}" ;;' \
+  '  unregistervm)' \
+  '    printf '\''unregistervm %s\n'\'' "$*" >> "${FAKE_LOG}"' \
+  '    rm -f -- "${FAKE_VBOX_ROOT}/control1/control1.vbox" "${FAKE_VBOX_ROOT}/control1/disk.vdi"' \
+  '    ;;' \
+  '  *) : ;;' \
+  'esac' > "$fake_bin/VBoxManage"
+chmod 700 "$fake_bin/VBoxManage"
+
+output="$test_root/output.txt"
+if PATH="$fake_bin:$PATH" \
+  HOME="$test_root/home" \
+  K3S_CI_MOLECULE_ROOT="$molecule_root" \
+  K3S_CI_MOLECULE_PROJECT=k3s-ansible \
+  K3S_CI_VIRTUALBOX_ROOT="$virtualbox_root" \
+  K3S_CI_HOSTONLY_MARKER="$test_root/hostonly-baseline" \
+  FAKE_VM_MODE=fault \
+  FAKE_VBOX_ROOT="$virtualbox_root" \
+  FAKE_LOG="$test_root/vbox.log" \
+  bash "$repo_root/.github/scripts/cleanup-runner-resources.sh" --apply > "$output" 2>&1; then
+  printf '%s\n' 'cleanup unexpectedly accepted a VirtualBox inspection failure' >&2
+  exit 1
+fi
+grep -Fq 'cleanup refused: unable to inspect VirtualBox VM' "$output"
+[[ -f "$molecule_root/k3s-ansible/single_node/.vagrant/machines/control1/virtualbox/id" ]]
+
+printf '%s\n' 'vboxnet0|192.168.30.1' > "$test_root/hostonly-baseline"
+if PATH="$fake_bin:$PATH" \
+  HOME="$test_root/home" \
+  K3S_CI_MOLECULE_ROOT="$molecule_root" \
+  K3S_CI_MOLECULE_PROJECT=k3s-ansible \
+  K3S_CI_VIRTUALBOX_ROOT="$virtualbox_root" \
+  K3S_CI_HOSTONLY_MARKER="$test_root/hostonly-baseline" \
+  FAKE_HOSTONLY_FAIL=true \
+  FAKE_VBOX_ROOT="$virtualbox_root" \
+  FAKE_LOG="$test_root/vbox.log" \
+  bash "$repo_root/.github/scripts/cleanup-runner-resources.sh" --dry-run > "$output" 2>&1; then
+  printf '%s\n' 'cleanup unexpectedly accepted a host-only inventory failure' >&2
+  exit 1
+fi
+grep -Fq 'cleanup refused: unable to inventory VirtualBox host-only interfaces' "$output"
+
+PATH="$fake_bin:$PATH" \
+  HOME="$test_root/home" \
+  K3S_CI_MOLECULE_ROOT="$molecule_root" \
+  K3S_CI_MOLECULE_PROJECT=k3s-ansible \
+  K3S_CI_VIRTUALBOX_ROOT="$virtualbox_root" \
+  K3S_CI_HOSTONLY_MARKER="$test_root/hostonly-baseline" \
+  FAKE_VBOX_ROOT="$virtualbox_root" \
+  FAKE_LOG="$test_root/vbox.log" \
+  bash "$repo_root/.github/scripts/cleanup-runner-resources.sh" --dry-run > "$output"
+
+grep -Fq 'Would remove VM control1 (11111111-1111-1111-1111-111111111111)' "$output"
+[[ ! -e "$test_root/vbox.log" ]]
+
+PATH="$fake_bin:$PATH" \
+  HOME="$test_root/home" \
+  K3S_CI_MOLECULE_ROOT="$molecule_root" \
+  K3S_CI_MOLECULE_PROJECT=k3s-ansible \
+  K3S_CI_VIRTUALBOX_ROOT="$virtualbox_root" \
+  K3S_CI_HOSTONLY_MARKER="$test_root/hostonly-baseline" \
+  FAKE_VBOX_ROOT="$virtualbox_root" \
+  FAKE_LOG="$test_root/vbox.log" \
+  bash "$repo_root/.github/scripts/cleanup-runner-resources.sh" --apply > "$output"
+
+grep -Fq 'controlvm 11111111-1111-1111-1111-111111111111 poweroff' "$test_root/vbox.log"
+grep -Fq 'unregistervm unregistervm 11111111-1111-1111-1111-111111111111 --delete' "$test_root/vbox.log"
+[[ ! -e "$virtualbox_root/control1/control1.vbox" ]]
+[[ -e "$virtualbox_root/unmarked/unmarked.vbox" ]]
+
+PATH="$fake_bin:$PATH" \
+  HOME="$test_root/home" \
+  K3S_CI_MOLECULE_ROOT="$molecule_root" \
+  K3S_CI_MOLECULE_PROJECT=k3s-ansible \
+  K3S_CI_VIRTUALBOX_ROOT="$virtualbox_root" \
+  K3S_CI_HOSTONLY_MARKER="$test_root/hostonly-baseline" \
+  FAKE_VM_MODE=missing \
+  FAKE_VBOX_ROOT="$virtualbox_root" \
+  FAKE_LOG="$test_root/vbox.log" \
+  bash "$repo_root/.github/scripts/cleanup-runner-resources.sh" --apply > "$output"
+
+grep -Fq 'Stale Vagrant state without a registered VM' "$output"
+[[ ! -d "$molecule_root/k3s-ansible/single_node/.vagrant" ]]
+
+printf 'cleanup-runner-resources fixture test passed\n'

.github/scripts/test-fixtures/mock-flock

@@ -0,0 +1,4 @@
+#!/usr/bin/env bash
+
+set -Eeuo pipefail
+[[ "${1:-}" =~ ^[0-9]+$ ]]

.github/scripts/test-fixtures/mock-vagrant

@@ -0,0 +1,30 @@
+#!/usr/bin/env bash
+
+set -Eeuo pipefail
+
+printf '%s\n' "$*" >> "$MOCK_VAGRANT_LOG"
+case "${1:-}" in
+  up)
+    if [[ -n "${MOCK_VAGRANTFILE_CAPTURE:-}" ]]; then
+      cp -- "$VAGRANT_CWD/Vagrantfile" "$MOCK_VAGRANTFILE_CAPTURE"
+    fi
+    counter_file="$MOCK_VBOX_STATE/counter"
+    counter=0
+    [[ ! -r "$counter_file" ]] || counter="$(cat "$counter_file")"
+    counter=$((counter + 1))
+    printf '%s\n' "$counter" > "$counter_file"
+    uuid="00000000-0000-4000-8000-$(printf '%012d' "$counter")"
+    vm_dir="$MOCK_VBOX_ROOT/master-$counter"
+    mkdir -p -- "$vm_dir"
+    : > "$vm_dir/master.vbox"
+    : > "$vm_dir/master.vdi"
+    printf '%s\n' \
+      'VMState="poweroff"' \
+      'groups="/"' \
+      "CfgFile=\"$vm_dir/master.vbox\"" \
+      "SATA-0-0=\"$vm_dir/master.vdi\"" > "$MOCK_VBOX_STATE/vm-$uuid"
+    printf '%s\n' "$uuid" > "$MOCK_BOX_DIR/master_id"
+    ;;
+  destroy) ;;
+  *) exit 2 ;;
+esac

.github/scripts/test-fixtures/mock-vboxmanage

@@ -0,0 +1,40 @@
+#!/usr/bin/env bash
+
+set -Eeuo pipefail
+
+printf '%s\n' "$*" >> "$MOCK_VBOX_LOG"
+command_name="${1:-}"
+uuid="${2:-}"
+
+case "$command_name" in
+  showvminfo)
+    [[ -r "$MOCK_VBOX_STATE/vm-$uuid" ]] || exit 1
+    cat "$MOCK_VBOX_STATE/vm-$uuid"
+    ;;
+  getextradata)
+    key_slug="${3//\//_}"
+    if [[ ! -r "$MOCK_VBOX_STATE/extra-$uuid-$key_slug" ]]; then
+      printf '%s\n' 'No value set!'
+      exit 0
+    fi
+    printf 'Value: '
+    cat "$MOCK_VBOX_STATE/extra-$uuid-$key_slug"
+    ;;
+  modifyvm)
+    [[ "${3:-}" == --groups ]]
+    awk -v groups="${4:-}" '
+      $1 !~ /^groups=/ { print }
+      END { printf "groups=\"%s\"\n", groups }
+    ' "$MOCK_VBOX_STATE/vm-$uuid" > "$MOCK_VBOX_STATE/vm-$uuid.tmp"
+    mv "$MOCK_VBOX_STATE/vm-$uuid.tmp" "$MOCK_VBOX_STATE/vm-$uuid"
+    ;;
+  setextradata)
+    key_slug="${3//\//_}"
+    printf '%s\n' "${4:-}" > "$MOCK_VBOX_STATE/extra-$uuid-$key_slug"
+    ;;
+  unregistervm|closemedium)
+    printf '%s\n' 'destructive VirtualBox command invoked' >&2
+    exit 99
+    ;;
+  *) exit 2 ;;
+esac

.github/scripts/test-k3s-server-bootstrap.sh

@@ -0,0 +1,35 @@
+#!/usr/bin/env bash
+
+set -Eeuo pipefail
+
+repo_root="$(git rev-parse --show-toplevel)"
+main_tasks="$repo_root/roles/k3s_server/tasks/main.yml"
+join_tasks="$repo_root/roles/k3s_server/tasks/join_master.yml"
+
+for task_file in "$main_tasks" "$join_tasks"; do
+  for property in \
+    'Delegate=yes' \
+    'TasksMax=infinity' \
+    'KillMode=process' \
+    'LimitNOFILE=1048576' \
+    'LimitNPROC=infinity' \
+    'LimitCORE=infinity'; do
+    grep -Fq -- "$property" "$task_file" || {
+      printf '%s is missing transient K3s property %s\n' "$task_file" "$property" >&2
+      exit 1
+    }
+  done
+done
+
+if grep -Fq -- "node-role.kubernetes.io/master=true' -o=jsonpath" "$main_tasks"; then
+  printf 'control-plane registration still depends on the optional legacy master role key\n' >&2
+  exit 1
+fi
+
+grep -Fq -- "map('extract', hostvars, 'ansible_hostname')" "$main_tasks"
+grep -Fq -- 'difference(nodes.stdout.split())' "$main_tasks"
+grep -Fq -- 'crd/addons.k3s.cattle.io' "$main_tasks"
+grep -Fq -- 'crd/helmcharts.helm.cattle.io' "$main_tasks"
+grep -Fq -- 'crd/helmchartconfigs.helm.cattle.io' "$main_tasks"
+
+printf 'K3s transient bootstrap regression test passed\n'

.github/scripts/test-kube-vip-manifest.py

@@ -0,0 +1,130 @@
+#!/usr/bin/env python3
+"""Render the kube-vip DaemonSet template and assert env key correctness.
+
+kube-vip v1.2.2 reads `bgp_peers` and `vip_subnet`; it ignores the older
+`bgppeers` and `vip_cidr` names. This test proves the rendered manifest uses
+the keys the target image actually parses.
+"""
+
+from __future__ import print_function
+
+import os
+import subprocess
+
+from jinja2 import Environment, FileSystemLoader, StrictUndefined
+
+
+def repo_root():
+    return subprocess.check_output(
+        ["git", "rev-parse", "--show-toplevel"], text=True
+    ).strip()
+
+
+def fail(message):
+    raise SystemExit("kube-vip manifest test failed: " + message)
+
+
+def fake_ipsubnet(value):
+    # ansible.utils.ipsubnet -> network of the address as x.y.z.0/24
+    parts = value.split(".")
+    return ".".join(parts[:3]) + ".0/24"
+
+
+def fake_ipaddr(_value, expr=None):
+    # ansible.utils.ipaddr('prefix') -> prefix length
+    return "24"
+
+
+def fake_bool(value):
+    # Minimal stand-in for Ansible's truthiness filter used by the template.
+    if isinstance(value, bool):
+        return value
+    return str(value).lower() in ("1", "true", "yes", "on")
+
+
+def fake_map(seq, *args, **kwargs):
+    # Minimal stand-in for Ansible's map() filter in the two forms used by the
+    # template: map(attribute='x') on a list of dicts, and map('join', sep) on
+    # a list of sequences.
+    if "attribute" in kwargs:
+        return [item[kwargs["attribute"]] for item in seq]
+    if kwargs:
+        # e.g. map(default='x') not used here; ignore unknown kwargs.
+        return list(seq)
+    if args:
+        filter_name = args[0]
+        sep = args[1] if len(args) > 1 else ""
+        if filter_name == "join":
+            return [sep.join(str(x) for x in item) for item in seq]
+    return list(seq)
+
+
+def fake_zip(*seqs):
+    return list(zip(*seqs))
+
+
+def render(env, extra_vars):
+    base_vars = {
+        "apiserver_endpoint": "192.168.30.222",
+        "kube_vip_iface": "",
+        "kube_vip_arp": True,
+        "kube_vip_bgp": True,
+        "kube_vip_bgp_routerid": "127.0.0.1",
+        "_kube_vip_bgp_peers": [
+            {"peer_address": "192.168.30.1", "peer_asn": "64512"},
+            {"peer_address": "192.168.30.2", "peer_asn": "64513"},
+        ],
+        "kube_vip_tag_version": "v1.2.2",
+    }
+    base_vars.update(extra_vars)
+    template = env.get_template("vip.yaml.j2")
+    return template.render(**base_vars)
+
+
+def main():
+    root = repo_root()
+    template_dir = os.path.join(root, "roles", "k3s_server", "templates")
+    env = Environment(
+        loader=FileSystemLoader(template_dir), undefined=StrictUndefined
+    )
+    env.filters["ansible.utils.ipsubnet"] = fake_ipsubnet
+    env.filters["ansible.utils.ipaddr"] = fake_ipaddr
+    env.filters["bool"] = fake_bool
+    env.filters["map"] = fake_map
+    env.filters["zip"] = fake_zip
+
+    # Multi-peer BGP armed: must emit bgp_peers, never bgppeers.
+    output = render(env, {})
+    if "name: bgp_peers" not in output:
+        fail("rendered manifest is missing bgp_peers")
+    if "name: bgppeers" in output:
+        fail("rendered manifest still uses the ignored bgppeers key")
+    if "name: vip_subnet" not in output:
+        fail("rendered manifest is missing vip_subnet")
+    if "name: vip_cidr" in output:
+        fail("rendered manifest still uses the ignored vip_cidr key")
+    if "192.168.30.1:64512,192.168.30.2:64513" not in output:
+        fail("bgp_peers value is not comma-separated address:ASN entries")
+    if "ghcr.io/kube-vip/kube-vip:v1.2.2" not in output:
+        fail("kube-vip image tag is not v1.2.2")
+
+    # BGP enabled with no merged peers: single-peer fallback vars, no bgp_peers.
+    output = render(
+        env,
+        {
+            "_kube_vip_bgp_peers": [],
+            "kube_vip_bgp_as": "64513",
+            "kube_vip_bgp_peeraddress": "192.168.30.1",
+            "kube_vip_bgp_peeras": "64512",
+        },
+    )
+    if "name: bgp_as" not in output:
+        fail("single-peer bgp_as was not rendered")
+    if "name: bgp_peers" in output:
+        fail("bgp_peers present even though the peer list is empty")
+
+    print("kube-vip manifest regression test passed")
+
+
+if __name__ == "__main__":
+    main()

.github/scripts/test-metallb-remote-read.sh

@@ -0,0 +1,29 @@
+#!/usr/bin/env bash
+
+set -Eeuo pipefail
+
+repo_root="$(git rev-parse --show-toplevel)"
+metallb_task="$repo_root/roles/k3s_server/tasks/metallb.yml"
+
+# The speaker tag verification must read the rendered manifest on the managed
+# host with slurp. A controller-side lookup('ansible.builtin.file', ...) would
+# read from the Ansible control node, which does not have the file, and would
+# fail on every MetalLB scenario.
+grep -Fq -- 'ansible.builtin.slurp' "$metallb_task" || {
+  printf 'MetalLB speaker tag check does not use slurp on the managed host\n' >&2
+  exit 1
+}
+
+grep -Eq -- 'lookup\(.?ansible\.builtin\.file' "$metallb_task" && {
+  printf 'MetalLB speaker tag check uses a controller-side file lookup\n' >&2
+  exit 1
+}
+
+# The check must reference the full image reference, not just a bare version
+# string that could appear anywhere in the manifest.
+grep -Fq -- 'quay.io/metallb/speaker:' "$metallb_task" || {
+  printf 'MetalLB speaker tag check does not match the full image reference\n' >&2
+  exit 1
+}
+
+printf 'MetalLB remote manifest read regression test passed\n'

.github/scripts/test-prepare-vagrant-box-masters.sh

@@ -0,0 +1,89 @@
+#!/usr/bin/env bash
+
+set -Eeuo pipefail
+
+repo_root="$(git rev-parse --show-toplevel)"
+fixture="$(mktemp -d)"
+trap 'rm -rf -- "$fixture"' EXIT
+
+mock_bin="$fixture/bin"
+mock_state="$fixture/state"
+mock_home="$fixture/home"
+mock_vagrant_home="$fixture/vagrant-home"
+mock_box_dir="$mock_vagrant_home/boxes/bento-VAGRANTSLASH-ubuntu-26.04/202606.01.0/amd64/virtualbox"
+mock_vbox_root="$mock_home/VirtualBox VMs"
+mock_master_root="$mock_home/.cache/k3s-ci/vagrant-masters"
+lock_file="$fixture/vagrant-boxes.lock"
+mkdir -p -- "$mock_bin" "$mock_state" "$mock_box_dir" "$mock_vbox_root"
+printf '%s\n' 'bento/ubuntu-26.04 202606.01.0 amd64' > "$lock_file"
+ln -s "$repo_root/.github/scripts/test-fixtures/mock-vboxmanage" "$mock_bin/VBoxManage"
+ln -s "$repo_root/.github/scripts/test-fixtures/mock-vagrant" "$mock_bin/vagrant"
+ln -s "$repo_root/.github/scripts/test-fixtures/mock-flock" "$mock_bin/flock"
+
+export PATH="$mock_bin:$PATH"
+export HOME="$mock_home"
+export VAGRANT_HOME="$mock_vagrant_home"
+export VAGRANT_BOX_LOCK_FILE="$lock_file"
+export K3S_CI_REPOSITORY_ROOT="$repo_root"
+export K3S_CI_VAGRANT_MASTER_ROOT="$mock_master_root"
+export K3S_CI_VIRTUALBOX_ROOT="$mock_vbox_root"
+export MOCK_VBOX_STATE="$mock_state"
+export MOCK_VBOX_ROOT="$mock_vbox_root"
+export MOCK_BOX_DIR="$mock_box_dir"
+export MOCK_VBOX_LOG="$fixture/vbox.log"
+export MOCK_VAGRANT_LOG="$fixture/vagrant.log"
+export MOCK_VAGRANTFILE_CAPTURE="$fixture/prewarm-Vagrantfile"
+: > "$MOCK_VBOX_LOG"
+: > "$MOCK_VAGRANT_LOG"
+
+unowned_uuid='99999999-9999-4999-8999-999999999999'
+printf '%s\n' "$unowned_uuid" > "$mock_box_dir/master_id"
+
+script="$repo_root/.github/scripts/prepare-vagrant-box-masters.sh"
+first_output="$fixture/first-output"
+second_output="$fixture/second-output"
+third_output="$fixture/third-output"
+
+"$script" > "$first_output"
+if grep -Fq 'config.ssh.insert_key' "$MOCK_VAGRANTFILE_CAPTURE"; then
+  printf 'prewarm Vagrantfile unexpectedly overrides Vagrant SSH key insertion\n' >&2
+  exit 1
+fi
+grep -Fq 'virtualbox.memory = 1024' "$MOCK_VAGRANTFILE_CAPTURE"
+grep -Fq 'virtualbox.cpus = 2' "$MOCK_VAGRANTFILE_CAPTURE"
+grep -Fq 'config.vm.boot_timeout = 600' "$MOCK_VAGRANTFILE_CAPTURE"
+mapping_file="$mock_master_root/bento_ubuntu-26.04-202606.01.0-amd64.uuid"
+test -s "$mapping_file"
+cmp -s "$mapping_file" "$mock_box_dir/master_id"
+grep -Fq 'Created and recorded owned master' "$first_output"
+grep -Fq 'modifyvm' "$MOCK_VBOX_LOG"
+grep -Fq 'setextradata' "$MOCK_VBOX_LOG"
+if grep -Fq "$unowned_uuid" "$MOCK_VBOX_LOG"; then
+  printf 'unowned cached master UUID was unexpectedly inspected or modified\n' >&2
+  exit 1
+fi
+if grep -Eq 'unregistervm|closemedium' "$MOCK_VBOX_LOG"; then
+  printf 'master preparation invoked a destructive VirtualBox command\n' >&2
+  exit 1
+fi
+
+: > "$MOCK_VAGRANT_LOG"
+"$script" > "$second_output"
+grep -Fq 'Reusing owned master' "$second_output"
+if grep -Fq 'up ' "$MOCK_VAGRANT_LOG"; then
+  printf 'valid owned master was unexpectedly rebuilt\n' >&2
+  exit 1
+fi
+
+stale_uuid="$(tr -d '[:space:]' < "$mapping_file")"
+rm -f -- "$mock_state/vm-$stale_uuid"
+: > "$MOCK_VAGRANT_LOG"
+"$script" > "$third_output"
+grep -Fq 'rebuilding without deleting any VM or disk' "$third_output"
+grep -Fq 'up ' "$MOCK_VAGRANT_LOG"
+if grep -Eq 'unregistervm|closemedium' "$MOCK_VBOX_LOG"; then
+  printf 'stale master recovery invoked a destructive VirtualBox command\n' >&2
+  exit 1
+fi
+
+printf 'Vagrant box master preparation fixture test passed\n'

.github/scripts/vagrant-up-timed.sh

@@ -0,0 +1,24 @@
+#!/usr/bin/env bash
+
+set -Eeuo pipefail
+
+if (($# < 2)); then
+  printf 'Usage: vagrant-up-timed.sh WORKDIR MACHINE [MACHINE ...]\n' >&2
+  exit 2
+fi
+
+workdir="$1"
+shift
+timing_log="${K3S_CI_CREATE_TIMING_LOG:-${RUNNER_TEMP:-/tmp}/k3s-ci-create-timing.log}"
+mkdir -p -- "${timing_log%/*}"
+
+printf '%s batch-start machines=%s\n' "$(date --iso-8601=ns)" "$*" | tee -a "$timing_log"
+set +e
+VAGRANT_CWD="$workdir" vagrant up "$@" --provider virtualbox --no-provision 2>&1 |
+  while IFS= read -r line; do
+    printf '%s %s\n' "$(date --iso-8601=ns)" "$line"
+  done | tee -a "$timing_log"
+rc=${PIPESTATUS[0]}
+set -e
+printf '%s batch-end rc=%d machines=%s\n' "$(date --iso-8601=ns)" "$rc" "$*" | tee -a "$timing_log"
+exit "$rc"

.github/test-download-boxes.sh

@@ -0,0 +1,98 @@
+#!/usr/bin/env bash
+
+# The single-quoted expressions below are written into fake executables and
+# intentionally expand only when those executables run.
+# shellcheck disable=SC2016
+
+set -Eeuo pipefail
+
+repo_root=$(git rev-parse --show-toplevel)
+test_root=$(mktemp -d)
+fake_bin="$test_root/bin"
+fake_log="$test_root/vagrant.log"
+output="$test_root/output.txt"
+mkdir -p "$fake_bin"
+trap 'rm -rf "$test_root"' EXIT
+
+printf '%s\n' \
+    '#!/usr/bin/env bash' \
+    'set -Eeuo pipefail' \
+    'printf "%s\n" bento/debian-13 bento/rockylinux-10.1 bento/ubuntu-26.04' \
+    >"$fake_bin/yq"
+
+printf '%s\n' \
+    '#!/usr/bin/env bash' \
+    'set -Eeuo pipefail' \
+    'emit_box() {' \
+    '    case "$1" in' \
+    '        bento/debian-13) version=202510.26.0 ;;' \
+    '        bento/rockylinux-10.1) version=202512.01.0 ;;' \
+    '        bento/ubuntu-26.04) version=202606.01.0 ;;' \
+    '        *) printf "Unexpected box: %s\n" "$1" >&2; exit 1 ;;' \
+    '    esac' \
+    '    printf "0,,box-name,%s\n" "$1"' \
+    '    printf "0,,box-provider,virtualbox\n"' \
+    '    printf "0,,box-version,%s\n" "$version"' \
+    '    printf "0,,box-architecture,amd64\n"' \
+    '}' \
+    'if [[ "${1:-}" == box && "${2:-}" == list ]]; then' \
+    '    emit_box bento/debian-13' \
+    '    emit_box bento/rockylinux-10.1' \
+    '    if [[ "${FAKE_PRESENT_MODE:-all}" == all ]]; then' \
+    '        emit_box bento/ubuntu-26.04' \
+    '    fi' \
+    'elif [[ "${1:-}" == box && "${2:-}" == add ]]; then' \
+    '    printf "%s\n" "$*" >>"${FAKE_VAGRANT_LOG:?}"' \
+    'else' \
+    '    printf "Unexpected vagrant arguments: %s\n" "$*" >&2' \
+    '    exit 1' \
+    'fi' \
+    >"$fake_bin/vagrant"
+chmod +x "$fake_bin/yq" "$fake_bin/vagrant"
+
+PATH="$fake_bin:$PATH" \
+    FAKE_VAGRANT_LOG="$fake_log" \
+    "$repo_root/.github/download-boxes.sh" >"$output"
+grep -Fq 'All pinned Vagrant boxes are already present.' "$output"
+[[ ! -e "$fake_log" ]]
+
+PATH="$fake_bin:$PATH" \
+    FAKE_PRESENT_MODE=partial \
+    FAKE_VAGRANT_LOG="$fake_log" \
+    "$repo_root/.github/download-boxes.sh" >"$output"
+grep -Fxq \
+    'box add --provider virtualbox --box-version 202606.01.0 --architecture amd64 bento/ubuntu-26.04' \
+    "$fake_log"
+
+incomplete_lock="$test_root/incomplete.lock"
+printf '%s\n' \
+    'bento/debian-13 202510.26.0 amd64' \
+    'bento/rockylinux-10.1 202512.01.0 amd64' \
+    >"$incomplete_lock"
+if PATH="$fake_bin:$PATH" \
+    VAGRANT_BOX_LOCK_FILE="$incomplete_lock" \
+    FAKE_VAGRANT_LOG="$fake_log" \
+    "$repo_root/.github/download-boxes.sh" >"$output" 2>&1; then
+    printf 'Download script accepted a lock missing a scenario box.\n' >&2
+    exit 1
+fi
+grep -Fq 'Scenario boxes missing from the lock file:' "$output"
+grep -Fq 'bento/ubuntu-26.04' "$output"
+
+duplicate_lock="$test_root/duplicate.lock"
+printf '%s\n' \
+    'bento/debian-13 202510.26.0 amd64' \
+    'bento/debian-13 202508.10.0 amd64' \
+    'bento/rockylinux-10.1 202512.01.0 amd64' \
+    'bento/ubuntu-26.04 202606.01.0 amd64' \
+    >"$duplicate_lock"
+if PATH="$fake_bin:$PATH" \
+    VAGRANT_BOX_LOCK_FILE="$duplicate_lock" \
+    FAKE_VAGRANT_LOG="$fake_log" \
+    "$repo_root/.github/download-boxes.sh" >"$output" 2>&1; then
+    printf 'Download script accepted duplicate box lock entries.\n' >&2
+    exit 1
+fi
+grep -Fq 'Duplicate Vagrant box lock entries:' "$output"
+
+printf 'Vagrant box download tests passed.\n'

.github/vagrant-boxes.lock

@@ -0,0 +1,4 @@
+# box version architecture
+bento/debian-13 202510.26.0 amd64
+bento/rockylinux-10.1 202512.01.0 amd64
+bento/ubuntu-26.04 202606.01.0 amd64

.github/workflows/cache.yml

@@ -0,0 +1,57 @@
+---
+name: "Cache"
+on:
+  workflow_call:
+jobs:
+  molecule:
+    name: cache
+    runs-on: [self-hosted, linux, x64, k3s-ci, virtualbox, nested-virt]
+    env:
+      PYTHON_VERSION: "3.11"
+      VAGRANT_DEFAULT_PROVIDER: virtualbox
+      VAGRANT_HOME: ${{ github.workspace }}/.vagrant-home
+
+    steps:
+      - name: Check out the codebase
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # 7.0.1
+        with:
+          ref: ${{ github.event.pull_request.head.sha || github.sha }}
+
+      - name: Check nested VirtualBox platform
+        run: |
+          set -Eeuo pipefail
+          grep -Eq 'vmx|svm' /proc/cpuinfo
+          test -c /dev/kvm
+          test -c /dev/vboxdrv
+          VBoxManage --version
+          vagrant --version
+          test -r /etc/vbox/networks.conf
+          test "$(stat -c '%u' /etc/vbox/networks.conf)" -eq 0
+          free -h
+          df -Pk "${RUNNER_TEMP}"
+
+      - name: Set up Python ${{ env.PYTHON_VERSION }}
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # 7.0.0
+        with:
+          python-version: ${{ env.PYTHON_VERSION }}
+          cache: 'pip' # caching pip dependencies
+
+      - name: Cache Vagrant boxes
+        id: cache-vagrant
+        uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # 6.1.0
+        with:
+          # This producer only needs to know whether the immutable cache exists.
+          # Molecule jobs restore it after this job completes.
+          lookup-only: true
+          path: |
+            .vagrant-home/boxes
+          key: vagrant-boxes-${{ runner.name }}-${{ runner.os }}-${{ runner.arch }}-virtualbox-7.2-vagrant-2.4-${{ hashFiles('.github/vagrant-boxes.lock') }} # yamllint disable-line rule:line-length
+
+      - name: Download Vagrant boxes for all scenarios
+        # An exact hit skips both cache restoration and upstream downloads.
+        # A lock change builds and saves one clean, version-pinned cache.
+        if: steps.cache-vagrant.outputs.cache-hit != 'true'
+        run: |
+          ./.github/download-boxes.sh
+          ./.github/scripts/prepare-vagrant-box-masters.sh
+          vagrant box list

.github/workflows/ci.yml

@@ -2,14 +2,39 @@
 name: "CI"
 on:
   pull_request:
-  push:
     branches:
       - master
+    types:
+      - opened
+      - synchronize
+      - reopened
+      - ready_for_review
     paths-ignore:
-      - '**/README.md'
+      - '**/.gitignore'
+      - '**/FUNDING.yml'
+      - '**/host.ini'
+      - '**/*.md'
+      - '**/.editorconfig'
+      - '**/ansible.example.cfg'
+      - '**/deploy.sh'
+      - '**/LICENSE'
+      - '**/reboot.sh'
+      - '**/reset.sh'
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ci-${{ github.event.pull_request.number || github.run_id }}
+  cancel-in-progress: true
+
 jobs:
+  pre:
+    uses: ./.github/workflows/cache.yml
+    needs: [lint]
   lint:
     uses: ./.github/workflows/lint.yml
   test:
     uses: ./.github/workflows/test.yml
-    needs: [lint]
+    needs: [pre, lint]

.github/workflows/lint.yml

@@ -7,35 +7,26 @@ jobs:
     name: Pre-Commit
     runs-on: ubuntu-latest
     env:
-      PYTHON_VERSION: "3.11"
+      PYTHON_VERSION: "3.12"
 
     steps:
       - name: Check out the codebase
-        uses: actions/checkout@e2f20e631ae6d7dd3b768f56a5d2af784dd54791 # v3 2.5.0
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # 7.0.1
         with:
-          ref: ${{ github.event.pull_request.head.sha }}
+          ref: ${{ github.event.pull_request.head.sha || github.sha }}
 
       - name: Set up Python ${{ env.PYTHON_VERSION }}
-        uses: actions/setup-python@75f3110429a8c05be0e1bf360334e4cced2b63fa # 2.3.3
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # 7.0.0
         with:
           python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip' # caching pip dependencies
 
-      - name: Cache pip
+      - name: Restore Ansible cache
-        uses: actions/cache@9b0c1fce7a93df8e3bb8926b0d6e9d89e92f20a7 # 3.0.11
+        id: cache-ansible
-        with:
+        uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # 6.1.0
-          path: ~/.cache/pip
-          key: ${{ runner.os }}-pip-${{ hashFiles('./requirements.txt') }}
-          restore-keys: |
-            ${{ runner.os }}-pip-
-
-      - name: Cache Ansible
-        uses: actions/cache@9b0c1fce7a93df8e3bb8926b0d6e9d89e92f20a7 # 3.0.11
         with:
           path: ~/.ansible/collections
-          key: ${{ runner.os }}-ansible-${{ hashFiles('collections/requirements.txt') }}
+          key: ansible-${{ hashFiles('collections/requirements.yml') }}
-          restore-keys: |
-            ${{ runner.os }}-ansible-
 
       - name: Install dependencies
         run: |
@@ -47,21 +38,37 @@ jobs:
           python3 -m pip install -r requirements.txt
           echo "::endgroup::"
 
-          echo "::group::Install Ansible role requirements from collections/requirements.yml"
+      - name: Install Ansible collections with retries
-          ansible-galaxy install -r collections/requirements.yml
+        if: steps.cache-ansible.outputs.cache-hit != 'true'
-          echo "::endgroup::"
+        run: |
+          set -Eeuo pipefail
+          for attempt in 1 2 3 4 5; do
+            if ansible-galaxy collection install -r collections/requirements.yml; then
+              exit 0
+            fi
+            echo "Ansible Galaxy attempt ${attempt} failed; retrying."
+            sleep $((attempt * 10))
+          done
+          exit 1
+
+      - name: Save Ansible collection cache
+        if: steps.cache-ansible.outputs.cache-hit != 'true'
+        uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # 6.1.0
+        with:
+          path: ~/.ansible/collections
+          key: ansible-${{ hashFiles('collections/requirements.yml') }}
 
       - name: Run pre-commit
-        uses: pre-commit/action@646c83fcd040023954eafda54b4db0192ce70507 # 3.0.0
+        uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # 3.0.1
 
   ensure-pinned-actions:
     name: Ensure SHA Pinned Actions
     runs-on: ubuntu-latest
     steps:
       - name: Checkout code
-        uses: actions/checkout@e2f20e631ae6d7dd3b768f56a5d2af784dd54791 # v3 2.5.0
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # 7.0.1
       - name: Ensure SHA pinned actions
-        uses: zgosalvez/github-actions-ensure-sha-pinned-actions@af2eb3226618e2494e3d9084f515ad6dcf16e229 # 2.0.1
+        uses: zgosalvez/github-actions-ensure-sha-pinned-actions@3db98c0363e2fa5df3e1c4c471777a7c10b24cc9 # 5.0.5
         with:
           allowlist: |
             aws-actions/

.github/workflows/test.yml

@@ -5,60 +5,64 @@ on:
 jobs:
   molecule:
     name: Molecule
-    runs-on: macos-12
+    runs-on: [self-hosted, linux, x64, k3s-ci, virtualbox, nested-virt]
     strategy:
       matrix:
         scenario:
           - default
-          - ipv6
           - single_node
-      fail-fast: false
+          - calico
+          - cilium
+          - kube-vip
+          # - ipv6
+      fail-fast: true
+      max-parallel: 1
     env:
       PYTHON_VERSION: "3.11"
+      VAGRANT_DEFAULT_PROVIDER: virtualbox
+      VAGRANT_HOME: ${{ github.workspace }}/.vagrant-home
 
     steps:
       - name: Check out the codebase
-        uses: actions/checkout@e2f20e631ae6d7dd3b768f56a5d2af784dd54791 # v3 2.5.0
+        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # 7.0.1
         with:
-          ref: ${{ github.event.pull_request.head.sha }}
+          ref: ${{ github.event.pull_request.head.sha || github.sha }}
 
-      - name: Configure VirtualBox
+      - name: Clean repository-owned resources before testing
-        run: |-
+        run: ./.github/scripts/cleanup-runner-resources.sh --apply
-          sudo mkdir -p /etc/vbox
-          cat <<EOF | sudo tee -a /etc/vbox/networks.conf > /dev/null
-          * 192.168.30.0/24
-          * fdad:bad:ba55::/64
-          EOF
 
-      - name: Cache pip
+      - name: Record host-only network baseline
-        uses: actions/cache@9b0c1fce7a93df8e3bb8926b0d6e9d89e92f20a7 # 3.0.11
+        run: ./.github/scripts/cleanup-runner-resources.sh --snapshot
-        with:
-          path: ~/.cache/pip
-          key: ${{ runner.os }}-pip-${{ hashFiles('./requirements.txt') }}
-          restore-keys: |
-            ${{ runner.os }}-pip-
 
-      - name: Cache Vagrant boxes
+      - name: Check nested VirtualBox platform
-        uses: actions/cache@9b0c1fce7a93df8e3bb8926b0d6e9d89e92f20a7 # 3.0.11
+        run: |
-        with:
+          set -Eeuo pipefail
-          path: |
+          grep -Eq 'vmx|svm' /proc/cpuinfo
-            ~/.vagrant.d/boxes
+          test -c /dev/kvm
-          key: vagrant-boxes-${{ hashFiles('**/molecule.yml') }}
+          test -c /dev/vboxdrv
-          restore-keys: |
+          VBoxManage --version
-            vagrant-boxes
+          vagrant --version
-
+          test -r /etc/vbox/networks.conf
-      - name: Download Vagrant boxes for all scenarios
+          test "$(stat -c '%u' /etc/vbox/networks.conf)" -eq 0
-        # To save some cache space, all scenarios share the same cache key.
+          free -h
-        # On the other hand, this means that the cache contents should be
+          df -Pk "${RUNNER_TEMP}"
-        # the same across all scenarios. This step ensures that.
-        run: ./.github/download-boxes.sh
 
       - name: Set up Python ${{ env.PYTHON_VERSION }}
-        uses: actions/setup-python@75f3110429a8c05be0e1bf360334e4cced2b63fa # 2.3.3
+        uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # 7.0.0
         with:
           python-version: ${{ env.PYTHON_VERSION }}
           cache: 'pip' # caching pip dependencies
 
+      - name: Restore vagrant Boxes cache
+        uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # 6.1.0
+        with:
+          path: .vagrant-home/boxes
+          key: vagrant-boxes-${{ runner.name }}-${{ runner.os }}-${{ runner.arch }}-virtualbox-7.2-vagrant-2.4-${{ hashFiles('.github/vagrant-boxes.lock') }} # yamllint disable-line rule:line-length
+          fail-on-cache-miss: true
+
+      - name: Prepare runner-owned Vagrant box masters
+        run: ./.github/scripts/prepare-vagrant-box-masters.sh
+
       - name: Install dependencies
         run: |
           echo "::group::Upgrade pip"
@@ -70,23 +74,45 @@ jobs:
           echo "::endgroup::"
 
       - name: Test with molecule
-        run: molecule test --scenario-name ${{ matrix.scenario }}
+        run: |
-        timeout-minutes: 90
+          set -Eeuo pipefail
+          resource_dir="${RUNNER_TEMP}/logs/resources/${{ matrix.scenario }}"
+          timing_file="${RUNNER_TEMP}/logs/timing/${{ matrix.scenario }}.txt"
+          mkdir -p -- "${timing_file%/*}"
+          ./.github/scripts/monitor-runner-resources.sh "$resource_dir" 10 &
+          monitor_pid=$!
+          stop_monitor() {
+            kill -TERM "$monitor_pid" 2>/dev/null || true
+            wait "$monitor_pid" 2>/dev/null || true
+          }
+          trap stop_monitor EXIT
+          /usr/bin/time -v -o "$timing_file" \
+            molecule test --scenario-name ${{ matrix.scenario }}
+        timeout-minutes: 150
         env:
           ANSIBLE_K3S_LOG_DIR: ${{ runner.temp }}/logs/k3s-ansible/${{ matrix.scenario }}
           ANSIBLE_SSH_RETRIES: 4
-          ANSIBLE_TIMEOUT: 60
+          ANSIBLE_TIMEOUT: 120
           PY_COLORS: 1
           ANSIBLE_FORCE_COLOR: 1
+          K3S_CI_CREATE_TIMING_LOG: ${{ runner.temp }}/logs/timing/${{ matrix.scenario }}-create.log
+
+      - name: Collect runner diagnostics
+        if: always()
+        run: ./.github/scripts/collect-runner-diagnostics.sh "${RUNNER_TEMP}/logs/runner"
+        env:
+          K3S_CI_SCENARIO_NAME: ${{ matrix.scenario }}
+
+      - name: Clean repository-owned resources after testing
+        if: always()
+        run: ./.github/scripts/cleanup-runner-resources.sh --apply
 
       - name: Upload log files
         if: always() # do this even if a step before has failed
-        uses: actions/upload-artifact@83fd05a356d7e2593de66fc9913b3002723633cb # 3.1.1
+        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # 7.0.1
         with:
-          name: logs
+          name: logs-${{ matrix.scenario }}-${{ github.run_id }}-${{ github.run_attempt }}
           path: |
             ${{ runner.temp }}/logs
-
+          if-no-files-found: warn
-      - name: Delete old box versions
+          retention-days: 14
-        if: always() # do this even if a step before has failed
-        run: vagrant box prune --force

.gitignore

@@ -1,3 +1,6 @@
 .env/
 *.log
 ansible.cfg
+.ansible/
+kubeconfig
+zIgnore/

.pre-commit-config.yaml

@@ -1,7 +1,7 @@
 ---
 repos:
   - repo: https://github.com/pre-commit/pre-commit-hooks
-    rev: f71fa2c1f9cf5cb705f73dffe4b21f7c61470ba9  # frozen: v4.4.0
+    rev: v4.5.0
     hooks:
       - id: requirements-txt-fixer
       - id: sort-simple-yaml
@@ -12,24 +12,75 @@ repos:
       - id: trailing-whitespace
         args: [--markdown-linebreak-ext=md]
   - repo: https://github.com/adrienverge/yamllint.git
-    rev: b05e028c5881819161d11cb543fd96a30c06cceb  # frozen: v1.32.0
+    rev: v1.33.0
     hooks:
       - id: yamllint
         args: [-c=.yamllint]
   - repo: https://github.com/ansible-community/ansible-lint.git
-    rev: 3293b64b939c0de16ef8cb81dd49255e475bf89a  # frozen: v6.17.2
+    rev: v6.22.2
     hooks:
       - id: ansible-lint
+        additional_dependencies: [ansible-core==2.18.0]
+        language_version: python3.12
+        args: [--offline]
   - repo: https://github.com/shellcheck-py/shellcheck-py
-    rev: 375289a39f5708101b1f916eb729e8d6da96993f  # frozen: v0.9.0.5
+    rev: v0.9.0.6
     hooks:
       - id: shellcheck
   - repo: https://github.com/Lucas-C/pre-commit-hooks
-    rev: 12885e376b93dc4536ad68d156065601e4433665  # frozen: v1.5.1
+    rev: v1.5.4
     hooks:
       - id: remove-crlf
       - id: remove-tabs
   - repo: https://github.com/sirosen/texthooks
-    rev: c4ffd3e31669dd4fa4d31a23436cc13839730084  # frozen: 0.5.0
+    rev: 0.6.4
     hooks:
       - id: fix-smartquotes
+  - repo: local
+    hooks:
+      - id: cleanup-runner-resources-test
+        name: cleanup runner resources test
+        entry: .github/scripts/test-cleanup-runner-resources.sh
+        language: system
+        pass_filenames: false
+        files: ^\.github/scripts/(cleanup-runner-resources|test-cleanup-runner-resources)\.sh$
+      - id: download-vagrant-boxes-test
+        name: Vagrant box download test
+        entry: .github/test-download-boxes.sh
+        language: system
+        pass_filenames: false
+        files: ^\.github/(download-boxes|test-download-boxes)\.sh$|^\.github/vagrant-boxes\.lock$
+      - id: prepare-vagrant-box-masters-test
+        name: Vagrant box master preparation test
+        entry: .github/scripts/test-prepare-vagrant-box-masters.sh
+        language: system
+        pass_filenames: false
+        files: ^\.github/scripts/(prepare-vagrant-box-masters|test-prepare-vagrant-box-masters)\.sh$
+      - id: k3s-server-bootstrap-test
+        name: K3s transient bootstrap test
+        entry: .github/scripts/test-k3s-server-bootstrap.sh
+        language: system
+        pass_filenames: false
+        files: ^roles/k3s_server/tasks/(main|join_master)\.yml$|^\.github/scripts/test-k3s-server-bootstrap\.sh$
+      - id: cilium-bgp-manifest-test
+        name: Cilium BGP manifest test
+        entry: python3 .github/scripts/test-cilium-bgp-manifest.py
+        language: python
+        additional_dependencies:
+          - Jinja2>=3.1
+        pass_filenames: false
+        files: ^roles/k3s_server_post/templates/cilium\.crs\.j2$|^\.github/scripts/test-cilium-bgp-manifest\.py$
+      - id: kube-vip-manifest-test
+        name: kube-vip manifest test
+        entry: python3 .github/scripts/test-kube-vip-manifest.py
+        language: python
+        additional_dependencies:
+          - Jinja2>=3.1
+        pass_filenames: false
+        files: ^roles/k3s_server/templates/vip\.yaml\.j2$|^\.github/scripts/test-kube-vip-manifest\.py$
+      - id: metallb-remote-read-test
+        name: MetalLB remote read test
+        entry: .github/scripts/test-metallb-remote-read.sh
+        language: system
+        pass_filenames: false
+        files: ^roles/k3s_server/tasks/metallb\.yml$|^\.github/scripts/test-metallb-remote-read\.sh$

.yamllint

@@ -2,8 +2,19 @@
 extends: default
 
 rules:
+  comments:
+    min-spaces-from-content: 1
+  comments-indentation: false
+  braces:
+    max-spaces-inside: 1
+  octal-values:
+    forbid-implicit-octal: true
+    forbid-explicit-octal: true
   line-length:
     max: 120
     level: warning
   truthy:
-    allowed-values: ['true', 'false', 'yes', 'no']
+    allowed-values: ["true", "false"]
+
+ignore:
+  - galaxy.yml

README.md

@@ -8,7 +8,7 @@ This is based on the work from [this fork](https://github.com/212850a/k3s-ansibl
 
 If you want more context on how this works, see:
 
-📄 [Documentation](https://technotim.live/posts/k3s-etcd-ansible/) (including example commands)
+📄 [Documentation](https://technotim.com/posts/k3s-etcd-ansible/) (including example commands)
 
 📺 [Watch the Video](https://www.youtube.com/watch?v=CbkEWcUZ7zM)
 
@@ -16,9 +16,9 @@ If you want more context on how this works, see:
 
 Build a Kubernetes cluster using Ansible with k3s. The goal is easily install a HA Kubernetes cluster on machines running:
 
-- [x] Debian (tested on version 11)
+- [x] Debian (tested on version 13)
-- [x] Ubuntu (tested on version 22.04)
+- [x] Ubuntu (tested on version 26.04 LTS)
-- [x] Rocky (tested on version 9)
+- [x] Rocky (tested on version 10)
 
 on processor architecture:
 
@@ -28,7 +28,7 @@ on processor architecture:
 
 ## ✅ System requirements
 
-- Control Node (the machine you are running `ansible` commands) must have Ansible 2.11+ If you need a quick primer on Ansible [you can check out my docs and setting up Ansible](https://technotim.live/posts/ansible-automation/).
+- Control Node (the machine you are running `ansible` commands) must have Ansible 2.11+ If you need a quick primer on Ansible [you can check out my docs and setting up Ansible](https://technotim.com/posts/ansible-automation/).
 
 - You will also need to install collections that this playbook uses by running `ansible-galaxy collection install -r ./collections/requirements.yml` (important❗)
 
@@ -91,21 +91,134 @@ ansible-playbook reset.yml -i inventory/my-cluster/hosts.ini
 
 >You should also reboot these nodes due to the VIP not being destroyed
 
+## 🔁 Upgrading an existing cluster
+
+These version variables select the components used for a **fresh** installation.
+They are not a supported direct in-place upgrade path for an existing cluster.
+K3s, Calico, and Cilium each require staged upgrades for long-lived clusters.
+
+- **K3s**: do not jump an embedded-etcd cluster straight to Kubernetes 1.36.
+  Upgrade one Kubernetes minor version at a time. From the sample default
+  (`v1.30.2+k3s2`) the sequence is: the latest supported 1.30 patch, then 1.31,
+  1.32, a 1.33 patch that contains etcd 3.5.26 (for example `v1.33.7+k3s3`),
+  then 1.34, 1.35, and finally 1.36. Upgrade servers one at a time before
+  agents. Take backups and confirm cluster health at each step; this playbook
+  does not automate the upgrade, so those remain manual operational steps. See
+  [K3s manual upgrades](https://docs.k3s.io/upgrades/manual) and the
+  [v1.34 release notes](https://docs.k3s.io/release-notes/v1.34.X).
+- **Cilium**: upstream supports only consecutive minor upgrades. Update to the
+  latest patch of the current minor, then upgrade 1.17, 1.18, 1.19, and 1.20 in
+  order, reading each version's upgrade notes and running preflight checks.
+  Do not attempt a direct upgrade from an old Cilium to 1.20.
+- **Calico**: starting with 3.28 the v3 resource UID behavior changed. If you
+  have operators with OwnerReferences pointing to `projectcalico.org/v3`
+  resources, remove and recreate those references around an in-place upgrade.
+- **MetalLB**: this project installs application tag `v0.16.0`. A newer
+  chart-only tag such as `metallb-chart-0.16.1` is not an application or image
+  release and must not be used as the controller or speaker image tag.
+
 ## ⚙️ Kube Config
 
 To copy your `kube config` locally so that you can access your **Kubernetes** cluster run:
 
 ```bash
-scp debian@master_ip:~/.kube/config ~/.kube/config
+scp debian@master_ip:/etc/rancher/k3s/k3s.yaml ~/.kube/config
 ```
+If you get file Permission denied, go into the node and temporarly run:
+```bash
+sudo chmod 777 /etc/rancher/k3s/k3s.yaml
+```
+Then copy with the scp command and reset the permissions back to:
+```bash
+sudo chmod 600 /etc/rancher/k3s/k3s.yaml
+```
+
+You'll then want to modify the config to point to master IP by running:
+```bash
+sudo nano ~/.kube/config
+```
+Then change `server: https://127.0.0.1:6443` to match your master IP: `server: https://192.168.1.222:6443`
 
 ### 🔨 Testing your cluster
 
-See the commands [here](https://technotim.live/posts/k3s-etcd-ansible/#testing-your-cluster).
+See the commands [here](https://technotim.com/posts/k3s-etcd-ansible/#testing-your-cluster).
+
+### Variables
+
+| Role(s) | Variable | Type | Default | Required | Description |
+|---|---|---|---|---|---|
+| `download` | `k3s_version` | string | ❌ | Required | K3s binaries version |
+| `k3s_agent`, `k3s_server`, `k3s_server_post` | `apiserver_endpoint` | string | ❌ | Required | Virtual ip-address configured on each master |
+| `k3s_agent` | `extra_agent_args` | string | `null` | Not required | Extra arguments for agents nodes |
+| `k3s_agent`, `k3s_server` | `group_name_master` | string | `null` | Not required | Name othe master group |
+| `k3s_agent` | `k3s_token` | string | `null` | Not required | Token used to communicate between masters |
+| `k3s_agent`, `k3s_server` | `proxy_env` | dict | `null` | Not required | Internet proxy configurations |
+| `k3s_agent`, `k3s_server` | `proxy_env.HTTP_PROXY` | string | ❌ | Required | HTTP internet proxy |
+| `k3s_agent`, `k3s_server` | `proxy_env.HTTPS_PROXY` | string | ❌ | Required | HTTP internet proxy |
+| `k3s_agent`, `k3s_server` | `proxy_env.NO_PROXY` | string | ❌ | Required | Addresses that will not use the proxies |
+| `k3s_agent`, `k3s_server`, `reset` | `systemd_dir` | string | `/etc/systemd/system` | Not required | Path to systemd services |
+| `k3s_custom_registries` | `custom_registries_yaml` | string | ❌ | Required | YAML block defining custom registries. The following is an example that pulls all images used in this playbook through your private registries. It also allows you to pull your own images from your private registry, without having to use imagePullSecrets in your deployments. If all you need is your own images and you don't care about caching the docker/quay/ghcr.io images, you can just remove those from the mirrors: section. |
+| `k3s_server`, `k3s_server_post` | `cilium_bgp` | bool | `~` | Not required | Enable cilium BGP control plane for LB services and pod cidrs. Disables the use of MetalLB. |
+| `k3s_server`, `k3s_server_post` | `cilium_iface` | string | ❌ | Not required | The network interface used for when Cilium is enabled |
+| `k3s_server` | `extra_server_args` | string | `""` | Not required | Extra arguments for server nodes |
+| `k3s_server` | `k3s_create_kubectl_symlink` | bool | `false` | Not required | Create the kubectl -> k3s symlink |
+| `k3s_server` | `k3s_create_crictl_symlink` | bool | `true` | Not required | Create the crictl -> k3s symlink |
+| `k3s_server` | `kube_vip_arp` | bool | `true` | Not required | Enables kube-vip ARP broadcasts |
+| `k3s_server` | `kube_vip_bgp` | bool | `false` | Not required | Enables kube-vip BGP peering |
+| `k3s_server` | `kube_vip_bgp_routerid` | string | `"127.0.0.1"` | Not required | Defines the router ID for the kube-vip BGP server |
+| `k3s_server` | `kube_vip_bgp_as` | string | `"64513"` | Not required | Defines the AS for the kube-vip BGP server |
+| `k3s_server` | `kube_vip_bgp_peeraddress` | string | `"192.168.30.1"` | Not required | Defines the address for the kube-vip BGP peer |
+| `k3s_server` | `kube_vip_bgp_peeras` | string | `"64512"` | Not required | Defines the AS for the kube-vip BGP peer |
+| `k3s_server` | `kube_vip_bgp_peers` | list | `[]` | Not required | List of BGP peer ASN & address pairs |
+| `k3s_server` | `kube_vip_bgp_peers_groups` | list | `['k3s_master']` | Not required | Inventory group in which to search for additional `kube_vip_bgp_peers` parameters to merge. |
+| `k3s_server` | `kube_vip_iface` | string | `~` | Not required | Explicitly define an interface that ALL control nodes should use to propagate the VIP, define it here. Otherwise, kube-vip will determine the right interface automatically at runtime. |
+| `k3s_server` | `kube_vip_tag_version` | string | `v1.2.2` | Not required | Image tag for kube-vip |
+| `k3s_server` | `kube_vip_cloud_provider_tag_version` | string | `v0.0.12` | Not required | Tag for kube-vip-cloud-provider manifest when enable |
+| `k3s_server`, `k3_server_post` | `kube_vip_lb_ip_range` | string | `~` | Not required | IP range for kube-vip load balancer |
+| `k3s_server`, `k3s_server_post` | `metal_lb_controller_tag_version` | string | `v0.16.0` | Not required | Image tag for MetalLB |
+| `k3s_server` | `metal_lb_speaker_tag_version` | string | `v0.16.0` | Not required | Image tag for MetalLB |
+| `k3s_server` | `metal_lb_type` | string | `native` | Not required | Use FRR mode or native. Valid values are `frr` and `native` |
+| `k3s_server` | `retry_count` | int | `20` | Not required | Amount of retries when verifying that nodes joined |
+| `k3s_server` | `server_init_args` | string | ❌ | Not required | Arguments for server nodes |
+| `k3s_server_post` | `bpf_lb_algorithm` | string | `maglev` | Not required | BPF lb algorithm |
+| `k3s_server_post` | `bpf_lb_mode` | string | `hybrid` | Not required | BPF lb mode |
+| `k3s_server_post` | `calico_blocksize` | int | `26` | Not required | IP pool block size |
+| `k3s_server_post` | `calico_ebpf` | bool | `false` | Not required | Use eBPF dataplane instead of iptables |
+| `k3s_server_post` | `calico_encapsulation` | string | `VXLANCrossSubnet` | Not required | IP pool encapsulation |
+| `k3s_server_post` | `calico_natOutgoing` | string | `Enabled` | Not required | IP pool NAT outgoing |
+| `k3s_server_post` | `calico_nodeSelector` | string | `all()` | Not required | IP pool node selector |
+| `k3s_server_post` | `calico_iface` | string | `~` | Not required | The network interface used for when Calico is enabled |
+| `k3s_server_post` | `calico_tag` | string | `v3.32.1` | Not required | Calico version tag |
+| `k3s_server_post` | `cilium_bgp_my_asn` | int | `64513` | Not required | Local ASN for BGP peer |
+| `k3s_server_post` | `cilium_bgp_peer_asn` | int | `64512` | Not required | BGP peer ASN |
+| `k3s_server_post` | `cilium_bgp_peer_address` | string | `~` | Not required | BGP peer address |
+| `k3s_server_post` | `cilium_bgp_neighbors` | list | `[]` | Not required | List of BGP peer ASN & address pairs |
+| `k3s_server_post` | `cilium_bgp_neighbors_groups` | list | `['k3s_all']` | Not required | Inventory group in which to search for additional `cilium_bgp_neighbors` parameters to merge. |
+| `k3s_server_post` | `cilium_bgp_lb_cidr` | string | `192.168.31.0/24` | Not required | BGP load balancer IP range |
+| `k3s_server_post` | `cilium_exportPodCIDR` | bool | `true` | Not required | Export pod CIDR |
+| `k3s_server_post` | `cilium_hubble` | bool | `true` | Not required | Enable Cilium Hubble |
+| `k3s_server_post` | `cilium_mode` | string | `native` | Not required | Inner-node communication mode (choices are `native` and `tunnel`; `routed` is a deprecated alias for `tunnel`) |
+| `k3s_server_post` | `cilium_tag` | string | `v1.20.0` | Not required | Cilium version tag |
+| `k3s_server_post` | `cilium_cli_tag` | string | `v0.19.7` | Not required | Cilium CLI version tag |
+| `k3s_server_post` | `cluster_cidr` | string | `10.52.0.0/16` | Not required | Inner-cluster IP range |
+| `k3s_server_post` | `enable_bpf_masquerade` | bool | `true` | Not required | Use IP masquerading |
+| `k3s_server_post` | `kube_proxy_replacement` | bool | `true` | Not required | Replace the native kube-proxy with Cilium |
+| `k3s_server_post` | `metal_lb_available_timeout` | string | `240s` | Not required | Wait for MetalLB resources |
+| `k3s_server_post` | `metal_lb_ip_range` | string | `192.168.30.80-192.168.30.90` | Not required | MetalLB ip range for load balancer |
+| `k3s_server_post` | `metal_lb_controller_tag_version` | string | `v0.16.0` | Not required | Image tag for MetalLB |
+| `k3s_server_post` | `metal_lb_mode` | string | `layer2` | Not required | Metallb mode (choices are `bgp` and `layer2`) |
+| `k3s_server_post` | `metal_lb_bgp_my_asn` | string | `~` | Not required | BGP ASN configurations |
+| `k3s_server_post` | `metal_lb_bgp_peer_asn` | string | `~` | Not required | BGP peer ASN configurations |
+| `k3s_server_post` | `metal_lb_bgp_peer_address` | string | `~` | Not required | BGP peer address |
+| `lxc` | `custom_reboot_command` | string | `~` | Not required | Command to run on reboot |
+| `prereq` | `system_timezone` | string | `null` | Not required | Timezone to be set on all nodes |
+| `proxmox_lxc`, `reset_proxmox_lxc` | `proxmox_lxc_ct_ids` | list | ❌ | Required | Proxmox container ID list |
+| `raspberrypi` | `state` | string | `present` | Not required | Indicates whether the k3s prerequisites for Raspberry Pi should be set up (possible values are `present` and `absent`) |
+
 
 ### Troubleshooting
 
-Be sure to see [this post](https://github.com/techno-tim/k3s-ansible/discussions/20) on how to troubleshoot common problems
+Be sure to see [this post](https://github.com/timothystewart6/k3s-ansible/discussions/20) on how to troubleshoot common problems
 
 ### Testing the playbook using molecule
 
@@ -118,6 +231,28 @@ You can find more information about it [here](molecule/README.md).
 
 This repo uses `pre-commit` and `pre-commit-hooks` to lint and fix common style and syntax errors.  Be sure to install python packages and then run `pre-commit install`.  For more information, see [pre-commit](https://pre-commit.com/)
 
+## 🌌 Ansible Galaxy
+
+This collection can now be used in larger ansible projects.
+
+Instructions:
+
+- create or modify a file `collections/requirements.yml` in your project
+
+```yml
+collections:
+  - name: ansible.utils
+  - name: community.general
+  - name: ansible.posix
+  - name: kubernetes.core
+  - name: https://github.com/timothystewart6/k3s-ansible.git
+    type: git
+    version: master
+```
+
+- install via `ansible-galaxy collection install -r ./collections/requirements.yml`
+- every role is now available via the prefix `techno_tim.k3s_ansible.` e.g. `techno_tim.k3s_ansible.lxc`
+
 ## Thanks 🤝
 
 This repo is really standing on the shoulders of giants. Thank you to all those who have contributed and thanks to these repos for code and ideas:

galaxy.yml

@@ -0,0 +1,81 @@
+### REQUIRED
+# The namespace of the collection. This can be a company/brand/organization or product namespace under which all
+# content lives. May only contain alphanumeric lowercase characters and underscores. Namespaces cannot start with
+# underscores or numbers and cannot contain consecutive underscores
+namespace: techno_tim
+
+# The name of the collection. Has the same character restrictions as 'namespace'
+name: k3s_ansible
+
+# The version of the collection. Must be compatible with semantic versioning
+version: 1.0.0
+
+# The path to the Markdown (.md) readme file. This path is relative to the root of the collection
+readme: README.md
+
+# A list of the collection's content authors. Can be just the name or in the format 'Full Name <email> (url)
+# @nicks:irc/im.site#channel'
+authors:
+- your name <example@domain.com>
+
+
+### OPTIONAL but strongly recommended
+# A short summary description of the collection
+description: >
+  The easiest way to bootstrap a self-hosted High Availability Kubernetes
+  cluster. A fully automated HA k3s etcd install with kube-vip, MetalLB,
+  and more.
+
+# Either a single license or a list of licenses for content inside of a collection. Ansible Galaxy currently only
+# accepts L(SPDX,https://spdx.org/licenses/) licenses. This key is mutually exclusive with 'license_file'
+license:
+- Apache-2.0
+
+
+# A list of tags you want to associate with the collection for indexing/searching. A tag name has the same character
+# requirements as 'namespace' and 'name'
+tags:
+  - etcd
+  - high-availability
+  - k8s
+  - k3s
+  - k3s-cluster
+  - kube-vip
+  - kubernetes
+  - metallb
+  - rancher
+
+# Collections that this collection requires to be installed for it to be usable. The key of the dict is the
+# collection label 'namespace.name'. The value is a version range
+# L(specifiers,https://python-semanticversion.readthedocs.io/en/latest/#requirement-specification). Multiple version
+# range specifiers can be set and are separated by ','
+dependencies:
+  ansible.utils: '*'
+  ansible.posix: '*'
+  community.general: '*'
+  kubernetes.core: '*'
+
+# The URL of the originating SCM repository
+repository: https://github.com/timothystewart6/k3s-ansible
+
+# The URL to any online docs
+documentation: https://github.com/timothystewart6/k3s-ansible
+
+# The URL to the homepage of the collection/project
+homepage: https://www.youtube.com/watch?v=CbkEWcUZ7zM
+
+# The URL to the collection issue tracker
+issues: https://github.com/timothystewart6/k3s-ansible/issues
+
+# A list of file glob-like patterns used to filter any files or directories that should not be included in the build
+# artifact. A pattern is matched from the relative path of the file or directory of the collection directory. This
+# uses 'fnmatch' to match the files or directories. Some directories and files like 'galaxy.yml', '*.pyc', '*.retry',
+# and '.git' are always filtered. Mutually exclusive with 'manifest'
+build_ignore: []
+
+# A dict controlling use of manifest directives used in building the collection artifact. The key 'directives' is a
+# list of MANIFEST.in style
+# L(directives,https://packaging.python.org/en/latest/guides/using-manifest-in/#manifest-in-commands). The key
+# 'omit_default_directives' is a boolean that controls whether the default directives are used. Mutually exclusive
+# with 'build_ignore'
+# manifest: null

inventory/sample/group_vars/all.yml

@@ -1,53 +1,104 @@
 ---
-k3s_version: v1.26.11+k3s2
+k3s_version: v1.36.2+k3s1
 # this is the user that has ssh access to these machines
 ansible_user: ansibleuser
 systemd_dir: /etc/systemd/system
 
 # Set your timezone
-system_timezone: "Your/Timezone"
+system_timezone: Your/Timezone
 
 # interface which will be used for flannel
-flannel_iface: "eth0"
+flannel_iface: eth0
+
+# uncomment calico_iface to use tigera operator/calico cni instead of flannel https://docs.tigera.io/calico/latest/about
+# calico_iface: "eth0"
+calico_ebpf: false # use eBPF dataplane instead of iptables
+calico_tag: v3.32.1 # calico version tag
+
+# uncomment cilium_iface to use cilium cni instead of flannel or calico
+# ensure v4.19.57, v5.1.16, v5.2.0 or more recent kernel
+# cilium_iface: "eth0"
+cilium_mode: native # native when nodes are on the same subnet or use BGP, otherwise set tunnel
+cilium_tag: v1.20.0 # cilium version tag
+cilium_cli_tag: v0.19.7 # cilium cli version tag
+cilium_hubble: true # enable hubble observability relay and ui
+
+# if using calico or cilium, you may specify the cluster pod cidr pool
+cluster_cidr: 10.52.0.0/16
+
+# enable cilium bgp control plane for lb services and pod cidrs. disables metallb.
+cilium_bgp: false
+
+# bgp parameters for cilium cni. only active when cilium_iface is defined and cilium_bgp is true.
+cilium_bgp_my_asn: "64513"
+cilium_bgp_peer_asn: "64512"
+cilium_bgp_peer_address: 192.168.30.1
+cilium_bgp_lb_cidr: 192.168.31.0/24 # cidr for cilium loadbalancer ipam
+
+# enable kube-vip ARP broadcasts
+kube_vip_arp: true
+
+# enable kube-vip BGP peering
+kube_vip_bgp: false
+
+# bgp parameters for kube-vip
+kube_vip_bgp_routerid: "127.0.0.1"  # Defines the router ID for the BGP server
+kube_vip_bgp_as: "64513"  # Defines the AS for the BGP server
+kube_vip_bgp_peeraddress: "192.168.30.1"  # Defines the address for the BGP peer
+kube_vip_bgp_peeras: "64512"  # Defines the AS for the BGP peer
 
 # apiserver_endpoint is virtual ip-address which will be configured on each master
-apiserver_endpoint: "192.168.30.222"
+apiserver_endpoint: 192.168.30.222
 
 # k3s_token is required  masters can talk together securely
 # this token should be alpha numeric only
-k3s_token: "some-SUPER-DEDEUPER-secret-password"
+k3s_token: some-SUPER-DEDEUPER-secret-password
 
 # The IP on which the node is reachable in the cluster.
 # Here, a sensible default is provided, you can still override
 # it for each of your hosts, though.
-k3s_node_ip: '{{ ansible_facts[flannel_iface]["ipv4"]["address"] }}'
+k3s_node_ip: "{{ ansible_facts[(cilium_iface | default(calico_iface | default(flannel_iface)))]['ipv4']['address'] }}"
 
 # Disable the taint manually by setting: k3s_master_taint = false
 k3s_master_taint: "{{ true if groups['node'] | default([]) | length >= 1 else false }}"
 
 # these arguments are recommended for servers as well as agents:
 extra_args: >-
-  --flannel-iface={{ flannel_iface }}
+  {{ '--flannel-iface=' + flannel_iface if calico_iface is not defined and cilium_iface is not defined else '' }}
   --node-ip={{ k3s_node_ip }}
 
 # change these to your liking, the only required are: --disable servicelb, --tls-san {{ apiserver_endpoint }}
+# the contents of the if block is also required if using calico or cilium
 extra_server_args: >-
   {{ extra_args }}
   {{ '--node-taint node-role.kubernetes.io/master=true:NoSchedule' if k3s_master_taint else '' }}
+  {% if calico_iface is defined or cilium_iface is defined %}
+  --flannel-backend=none
+  --disable-network-policy
+  --cluster-cidr={{ cluster_cidr | default('10.52.0.0/16') }}
+  {% endif %}
   --tls-san {{ apiserver_endpoint }}
   --disable servicelb
   --disable traefik
+
 extra_agent_args: >-
   {{ extra_args }}
 
 # image tag for kube-vip
-kube_vip_tag_version: "v0.5.12"
+kube_vip_tag_version: v1.2.2
+
+# tag for kube-vip-cloud-provider manifest
+# kube_vip_cloud_provider_tag_version: "v0.0.12"
+
+# kube-vip ip range for load balancer
+# (uncomment to use kube-vip for services instead of MetalLB)
+# kube_vip_lb_ip_range: "192.168.30.80-192.168.30.90"
 
 # metallb type frr or native
-metal_lb_type: "native"
+metal_lb_type: native
 
 # metallb mode layer2 or bgp
-metal_lb_mode: "layer2"
+metal_lb_mode: layer2
 
 # bgp options
 # metal_lb_bgp_my_asn: "64513"
@@ -55,20 +106,20 @@ metal_lb_mode: "layer2"
 # metal_lb_bgp_peer_address: "192.168.30.1"
 
 # image tag for metal lb
-metal_lb_speaker_tag_version: "v0.13.9"
+metal_lb_speaker_tag_version: v0.16.0
-metal_lb_controller_tag_version: "v0.13.9"
+metal_lb_controller_tag_version: v0.16.0
 
 # metallb ip range for load balancer
-metal_lb_ip_range: "192.168.30.80-192.168.30.90"
+metal_lb_ip_range: 192.168.30.80-192.168.30.90
 
 # Only enable if your nodes are proxmox LXC nodes, make sure to configure your proxmox nodes
 # in your hosts.ini file.
 # Please read https://gist.github.com/triangletodd/02f595cd4c0dc9aac5f7763ca2264185 before using this.
 # Most notably, your containers must be privileged, and must not have nesting set to true.
 # Please note this script disables most of the security of lxc containers, with the trade off being that lxc
-# containers are significantly more resource efficent compared to full VMs.
+# containers are significantly more resource efficient compared to full VMs.
 # Mixing and matching VMs and lxc containers is not supported, ymmv if you want to do this.
-# I would only really recommend using this if you have partiularly low powered proxmox nodes where the overhead of
+# I would only really recommend using this if you have particularly low powered proxmox nodes where the overhead of
 # VMs would use a significant portion of your available resources.
 proxmox_lxc_configure: false
 # the user that you would use to ssh into the host, for example if you run ssh some-user@my-proxmox-host,
@@ -122,6 +173,10 @@ custom_registries_yaml: |
         username: yourusername
         password: yourpassword
 
+# On some distros like Diet Pi, there is no dbus installed. dbus required by the default reboot command.
+# Uncomment if you need a custom reboot command
+# custom_reboot_command: /usr/sbin/shutdown -r now
+
 # Only enable and configure these if you access the internet through a proxy
 # proxy_env:
 #   HTTP_PROXY: "http://proxy.domain.local:3128"

inventory/sample/group_vars/proxmox.yml

@@ -1,2 +1,2 @@
 ---
-ansible_user: '{{ proxmox_lxc_ssh_user }}'
+ansible_user: "{{ proxmox_lxc_ssh_user }}"

molecule/README.md

@@ -13,6 +13,12 @@ We have these scenarios:
   To save a bit of test time, this cluster is _not_ highly available, it consists of only one control and one worker node.
 - **single_node**:
   Very similar to the default scenario, but uses only a single node for all cluster functionality.
+- **calico**:
+  The same as single node, but uses calico cni instead of flannel.
+- **cilium**:
+  The same as single node, but uses cilium cni instead of flannel.
+- **kube-vip**
+  The same as single node, but uses kube-vip as service loadbalancer instead of MetalLB
 
 ## How to execute
 

molecule/calico/molecule.yml

@@ -0,0 +1,44 @@
+---
+dependency:
+  name: galaxy
+driver:
+  name: vagrant
+platforms:
+  - name: control1
+    box: bento/ubuntu-26.04
+    memory: 4096
+    cpus: 4
+    groups:
+      - k3s_cluster
+      - master
+    interfaces:
+      - network_name: private_network
+        ip: 192.168.30.62
+provisioner:
+  name: ansible
+  env:
+    ANSIBLE_VERBOSITY: 1
+  playbooks:
+    converge: ../resources/converge.yml
+    side_effect: ../resources/reset.yml
+    verify: ../resources/verify.yml
+  inventory:
+    links:
+      group_vars: ../../inventory/sample/group_vars
+scenario:
+  test_sequence:
+    - dependency
+    - cleanup
+    - destroy
+    - syntax
+    - create
+    - prepare
+    - converge
+    # idempotence is not possible with the playbook in its current form.
+    - verify
+    # We are repurposing side_effect here to test the reset playbook.
+    # This is why we do not run it before verify (which tests the cluster),
+    # but after the verify step.
+    - side_effect
+    - cleanup
+    - destroy

molecule/calico/overrides.yml

@@ -0,0 +1,18 @@
+---
+- name: Apply overrides
+  hosts: all
+  serial: 1
+  tasks:
+    - name: Override host variables
+      ansible.builtin.set_fact:
+        # See:
+        # https://github.com/flannel-io/flannel/blob/67d603aaf45ef80f5dd39f43714fc5e6f8a637eb/Documentation/troubleshooting.md#Vagrant
+        calico_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
+        kube_vip_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
+
+        # The test VMs might be a bit slow, so we give them more time to join the cluster:
+        retry_count: 45
+
+        # Make sure that our IP ranges do not collide with those of the other scenarios
+        apiserver_endpoint: 192.168.30.224
+        metal_lb_ip_range: 192.168.30.100-192.168.30.109

molecule/calico/verify-vars.yml

@@ -0,0 +1,6 @@
+---
+# Durable verify inputs for the calico (Calico CNI + MetalLB) scenario.
+verify_cni: calico
+verify_lb: metallb
+verify_lb_ip_range:
+  - 192.168.30.100-192.168.30.109

molecule/cilium/molecule.yml

@@ -0,0 +1,44 @@
+---
+dependency:
+  name: galaxy
+driver:
+  name: vagrant
+platforms:
+  - name: control1
+    box: bento/ubuntu-26.04
+    memory: 4096
+    cpus: 4
+    groups:
+      - k3s_cluster
+      - master
+    interfaces:
+      - network_name: private_network
+        ip: 192.168.30.63
+provisioner:
+  name: ansible
+  env:
+    ANSIBLE_VERBOSITY: 1
+  playbooks:
+    converge: ../resources/converge.yml
+    side_effect: ../resources/reset.yml
+    verify: ../resources/verify.yml
+  inventory:
+    links:
+      group_vars: ../../inventory/sample/group_vars
+scenario:
+  test_sequence:
+    - dependency
+    - cleanup
+    - destroy
+    - syntax
+    - create
+    - prepare
+    - converge
+    # idempotence is not possible with the playbook in its current form.
+    - verify
+    # We are repurposing side_effect here to test the reset playbook.
+    # This is why we do not run it before verify (which tests the cluster),
+    # but after the verify step.
+    - side_effect
+    - cleanup
+    - destroy

molecule/cilium/overrides.yml

@@ -0,0 +1,18 @@
+---
+- name: Apply overrides
+  hosts: all
+  serial: 1
+  tasks:
+    - name: Override host variables
+      ansible.builtin.set_fact:
+        # See:
+        # https://github.com/flannel-io/flannel/blob/67d603aaf45ef80f5dd39f43714fc5e6f8a637eb/Documentation/troubleshooting.md#Vagrant
+        cilium_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
+        kube_vip_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
+
+        # The test VMs might be a bit slow, so we give them more time to join the cluster:
+        retry_count: 45
+
+        # Make sure that our IP ranges do not collide with those of the other scenarios
+        apiserver_endpoint: 192.168.30.225
+        metal_lb_ip_range: 192.168.30.110-192.168.30.119

molecule/cilium/verify-vars.yml

@@ -0,0 +1,6 @@
+---
+# Durable verify inputs for the cilium (Cilium CNI + MetalLB) scenario.
+verify_cni: cilium
+verify_lb: metallb
+verify_lb_ip_range:
+  - 192.168.30.110-192.168.30.119

molecule/default/create.yml

@@ -0,0 +1,80 @@
+---
+- name: Create
+  hosts: localhost
+  connection: local
+  gather_facts: false
+  no_log: "{{ molecule_no_log }}"
+  vars:
+    create_batches:
+      - [control1, control2]
+      - [control3, node1]
+      - [node2]
+  tasks:
+    - name: Verify that bounded batches cover the configured platforms exactly once
+      ansible.builtin.assert:
+        that:
+          - create_batches | flatten | sort == molecule_yml.platforms | map(attribute='name') | sort
+          - create_batches | flatten | length == create_batches | flatten | unique | length
+          - create_batches | map('length') | max <= 2
+        fail_msg: Bounded create batches do not match the configured default platforms.
+
+    - name: Generate the complete Vagrant configuration # noqa fqcn[action]
+      vagrant:
+        instances: "{{ molecule_yml.platforms }}"
+        default_box: "{{ molecule_yml.driver.default_box | default('generic/alpine316') }}"
+        provider_name: "{{ molecule_yml.driver.provider.name | default(omit, true) }}"
+        provision: "{{ molecule_yml.driver.provision | default(omit) }}"
+        cachier: "{{ molecule_yml.driver.cachier | default(omit) }}"
+        parallel: false
+        state: halt
+      changed_when: false
+
+    - name: Start clean Vagrant guests in bounded batches
+      ansible.builtin.command:
+        argv: >-
+          {{
+            [playbook_dir + '/../../.github/scripts/vagrant-up-timed.sh',
+             molecule_ephemeral_directory] + item
+          }}
+      loop: "{{ create_batches }}"
+      loop_control:
+        label: "{{ item | join(', ') }}"
+      changed_when: true
+
+    - name: Reconcile all instances and collect their connection configuration # noqa fqcn[action]
+      vagrant:
+        instances: "{{ molecule_yml.platforms }}"
+        default_box: "{{ molecule_yml.driver.default_box | default('generic/alpine316') }}"
+        provider_name: "{{ molecule_yml.driver.provider.name | default(omit, true) }}"
+        provision: "{{ molecule_yml.driver.provision | default(omit) }}"
+        cachier: "{{ molecule_yml.driver.cachier | default(omit) }}"
+        parallel: false
+        state: up
+      register: server
+      no_log: false
+
+    - name: Populate instance configuration dictionaries
+      ansible.builtin.set_fact:
+        instance_conf_dict:
+          instance: "{{ item.Host }}"
+          address: "{{ item.HostName }}"
+          user: "{{ item.User }}"
+          port: "{{ item.Port }}"
+          identity_file: "{{ item.IdentityFile }}"
+      loop: "{{ server.results }}"
+      register: instance_config_dict
+
+    - name: Convert instance configuration dictionaries to a list
+      ansible.builtin.set_fact:
+        instance_conf: >-
+          {{
+            instance_config_dict.results
+            | map(attribute='ansible_facts.instance_conf_dict')
+            | list
+          }}
+
+    - name: Write Molecule instance configuration
+      ansible.builtin.copy:
+        content: "{{ instance_conf | to_json | from_json | to_yaml }}"
+        dest: "{{ molecule_instance_config }}"
+        mode: "0600"

molecule/default/molecule.yml

@@ -3,76 +3,83 @@ dependency:
   name: galaxy
 driver:
   name: vagrant
+  # The Vagrant driver warns that parallel VirtualBox creation can cause
+  # platform issues. Keep this five-node, mixed-distribution scenario serial.
+  parallel: false
 platforms:
-
   - name: control1
-    box: generic/ubuntu2204
+    box: bento/ubuntu-26.04
-    memory: 2048
+    memory: 1024
     cpus: 2
     groups:
       - k3s_cluster
       - master
+    # Keep adapter 2 stable across linked-clone rebuilds so stale host-only
+    # neighbor state still identifies the current scenario guest.
+    provider_raw_config_args:
+      - "customize ['modifyvm', :id, '--mac-address2', '080027A13038']"
     interfaces:
       - network_name: private_network
         ip: 192.168.30.38
-    config_options:
-      # We currently can not use public-key based authentication on Ubuntu 22.04,
-      # see: https://github.com/chef/bento/issues/1405
-      ssh.username: "vagrant"
-      ssh.password: "vagrant"
 
   - name: control2
-    box: generic/debian11
+    box: bento/debian-13
-    memory: 2048
+    memory: 1024
     cpus: 2
     groups:
       - k3s_cluster
       - master
+    provider_raw_config_args:
+      - "customize ['modifyvm', :id, '--mac-address2', '080027A13039']"
     interfaces:
       - network_name: private_network
         ip: 192.168.30.39
 
   - name: control3
-    box: generic/rocky9
+    box: bento/rockylinux-10.1
-    memory: 2048
+    memory: 1024
     cpus: 2
     groups:
       - k3s_cluster
       - master
+    provider_raw_config_args:
+      - "customize ['modifyvm', :id, '--mac-address2', '080027A13040']"
     interfaces:
       - network_name: private_network
         ip: 192.168.30.40
 
   - name: node1
-    box: generic/ubuntu2204
+    box: bento/ubuntu-26.04
-    memory: 2048
+    memory: 1024
     cpus: 2
     groups:
       - k3s_cluster
       - node
+    provider_raw_config_args:
+      - "customize ['modifyvm', :id, '--mac-address2', '080027A13041']"
     interfaces:
       - network_name: private_network
         ip: 192.168.30.41
-    config_options:
-      # We currently can not use public-key based authentication on Ubuntu 22.04,
-      # see: https://github.com/chef/bento/issues/1405
-      ssh.username: "vagrant"
-      ssh.password: "vagrant"
 
   - name: node2
-    box: generic/rocky9
+    box: bento/rockylinux-10.1
-    memory: 2048
+    memory: 1024
     cpus: 2
     groups:
       - k3s_cluster
       - node
+    provider_raw_config_args:
+      - "customize ['modifyvm', :id, '--mac-address2', '080027A13042']"
     interfaces:
       - network_name: private_network
         ip: 192.168.30.42
 
 provisioner:
   name: ansible
+  env:
+    ANSIBLE_VERBOSITY: 1
   playbooks:
+    create: create.yml
     converge: ../resources/converge.yml
     side_effect: ../resources/reset.yml
     verify: ../resources/verify.yml
@@ -82,7 +89,6 @@ provisioner:
 scenario:
   test_sequence:
     - dependency
-    - lint
     - cleanup
     - destroy
     - syntax

molecule/default/overrides.yml

@@ -1,12 +1,16 @@
 ---
 - name: Apply overrides
   hosts: all
+  serial: 1
   tasks:
     - name: Override host variables
       ansible.builtin.set_fact:
         # See:
         # https://github.com/flannel-io/flannel/blob/67d603aaf45ef80f5dd39f43714fc5e6f8a637eb/Documentation/troubleshooting.md#Vagrant
-        flannel_iface: eth1
+        flannel_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
+        # kube-vip cannot infer the cluster interface in these multi-NIC
+        # Vagrant guests because the default route is on eth0.
+        kube_vip_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
 
         # The test VMs might be a bit slow, so we give them more time to join the cluster:
         retry_count: 45

molecule/default/prepare.yml

@@ -5,18 +5,137 @@
 
 - name: Network setup
   hosts: all
+  vars:
+    primary_master: "{{ groups[group_name_master | default('master')][0] }}"
+    primary_cluster_ip: >-
+      {{ hostvars[primary_master].k3s_node_ip | split(',') | first }}
+    cluster_interface: >-
+      {{ cilium_iface | default(calico_iface | default(flannel_iface)) }}
+    primary_cluster_interface: >-
+      {{ hostvars[primary_master].cilium_iface
+      | default(hostvars[primary_master].calico_iface
+      | default(hostvars[primary_master].flannel_iface)) }}
+    primary_cluster_mac: >-
+      {{ hostvars[primary_master].ansible_facts[primary_cluster_interface].macaddress }}
   tasks:
-    - name: Disable firewalld
+    - name: Gather service facts
-      when: ansible_distribution == "Rocky"
+      ansible.builtin.service_facts:
-      # Rocky Linux comes with firewalld enabled. It blocks some of the network
+
-      # connections needed for our k3s cluster. For our test setup, we just disable
+    - name: Disable guest firewall services
-      # it since the VM host's firewall is still active for connections to and from
+      # The disposable test guests use an isolated VirtualBox network. A distro
-      # the Internet.
+      # firewall can allow ICMP while silently blocking the inter-node Kubernetes
+      # API connection, so disable the known guest firewalls consistently.
       # When building your own cluster, please DO NOT blindly copy this. Instead,
       # please create a custom firewall configuration that fits your network design
       # and security needs.
       ansible.builtin.systemd:
-        name: firewalld
+        name: "{{ item }}"
-        enabled: no
+        enabled: false
         state: stopped
       become: true
+      loop:
+        - firewalld.service
+        - nftables.service
+        - ufw.service
+      when: item in ansible_facts.services
+
+    - name: Verify the private cluster interface
+      ansible.builtin.assert:
+        that:
+          - flannel_iface in ansible_facts
+          - ansible_facts[flannel_iface].ipv4 is defined
+          - ansible_facts[flannel_iface].ipv4.address is defined
+        fail_msg: >-
+          The Vagrant private interface {{ flannel_iface }} does not have an
+          IPv4 address on {{ inventory_hostname }}.
+
+    - name: Pin disposable cluster peer neighbor entries
+      ansible.builtin.command:
+        argv:
+          - ip
+          - neigh
+          - replace
+          - "{{ peer_cluster_ip }}"
+          - lladdr
+          - "{{ peer_cluster_mac }}"
+          - nud
+          - permanent
+          - dev
+          - "{{ cluster_interface }}"
+      become: true
+      changed_when: false
+      loop: "{{ groups['k3s_cluster'] }}"
+      loop_control:
+        label: "{{ inventory_hostname }} -> {{ item }}"
+      vars:
+        peer_cluster_interface: >-
+          {{ hostvars[item].cilium_iface
+          | default(hostvars[item].calico_iface
+          | default(hostvars[item].flannel_iface)) }}
+        peer_cluster_ip: >-
+          {{ hostvars[item].k3s_node_ip | split(',') | first }}
+        peer_cluster_mac: >-
+          {{ hostvars[item].ansible_facts[peer_cluster_interface].macaddress }}
+      when: item != inventory_hostname
+
+    - name: Verify guest-to-guest cluster network reachability
+      ansible.builtin.command:
+        argv:
+          - ping
+          - -c
+          - "1"
+          - -W
+          - "1"
+          - "{{ primary_cluster_ip }}"
+      register: primary_cluster_ping
+      until: primary_cluster_ping.rc == 0
+      retries: 6
+      delay: 2
+      changed_when: false
+
+    - name: Read the primary neighbor entry
+      ansible.builtin.command:
+        argv:
+          - ip
+          - neigh
+          - show
+          - to
+          - "{{ primary_cluster_ip }}"
+          - dev
+          - "{{ cluster_interface }}"
+      register: primary_cluster_neighbor
+      changed_when: false
+      when: inventory_hostname != primary_master
+
+    - name: Verify the primary neighbor identity
+      ansible.builtin.assert:
+        that:
+          - (primary_cluster_mac | lower) in (primary_cluster_neighbor.stdout | lower)
+        fail_msg: >-
+          {{ inventory_hostname }} resolved primary {{ primary_cluster_ip }} to
+          an unexpected MAC on {{ cluster_interface }}. Expected
+          {{ primary_cluster_mac }}, got: {{ primary_cluster_neighbor.stdout }}
+      when: inventory_hostname != primary_master
+
+    - name: Verify GitHub release host DNS
+      ansible.builtin.getent:
+        database: hosts
+        key: github.com
+      register: github_dns
+      retries: 6
+      delay: 5
+      until: github_dns is succeeded
+
+    - name: Verify k3s checksum URL is reachable
+      ansible.builtin.uri:
+        url: >-
+          https://github.com/k3s-io/k3s/releases/download/{{ k3s_version
+          }}/sha256sum-amd64.txt
+        method: HEAD
+        follow_redirects: safe
+        status_code: [200, 302]
+        timeout: 15
+      register: k3s_checksum_request
+      retries: 3
+      delay: 5
+      until: k3s_checksum_request.status in [200, 302]

molecule/default/verify-vars.yml

@@ -0,0 +1,9 @@
+---
+# Durable verify inputs for the default (flannel + MetalLB) scenario.
+# These are plain inventory vars linked into the shared Molecule inventory so
+# the verify play can see them even though the converge play's set_fact values
+# are not persisted between the two Ansible processes.
+verify_cni: flannel
+verify_lb: metallb
+verify_lb_ip_range:
+  - 192.168.30.80-192.168.30.90

molecule/ipv6/molecule.yml

@@ -5,8 +5,8 @@ driver:
   name: vagrant
 platforms:
   - name: control1
-    box: generic/ubuntu2204
+    box: bento/ubuntu-26.04
-    memory: 2048
+    memory: 1024
     cpus: 2
     groups:
       - k3s_cluster
@@ -14,15 +14,10 @@ platforms:
     interfaces:
       - network_name: private_network
         ip: fdad:bad:ba55::de:11
-    config_options:
-      # We currently can not use public-key based authentication on Ubuntu 22.04,
-      # see: https://github.com/chef/bento/issues/1405
-      ssh.username: "vagrant"
-      ssh.password: "vagrant"
 
   - name: control2
-    box: generic/ubuntu2204
+    box: bento/ubuntu-26.04
-    memory: 2048
+    memory: 1024
     cpus: 2
     groups:
       - k3s_cluster
@@ -30,15 +25,10 @@ platforms:
     interfaces:
       - network_name: private_network
         ip: fdad:bad:ba55::de:12
-    config_options:
-      # We currently can not use public-key based authentication on Ubuntu 22.04,
-      # see: https://github.com/chef/bento/issues/1405
-      ssh.username: "vagrant"
-      ssh.password: "vagrant"
 
   - name: node1
-    box: generic/ubuntu2204
+    box: bento/ubuntu-26.04
-    memory: 2048
+    memory: 1024
     cpus: 2
     groups:
       - k3s_cluster
@@ -46,13 +36,10 @@ platforms:
     interfaces:
       - network_name: private_network
         ip: fdad:bad:ba55::de:21
-    config_options:
-      # We currently can not use public-key based authentication on Ubuntu 22.04,
-      # see: https://github.com/chef/bento/issues/1405
-      ssh.username: "vagrant"
-      ssh.password: "vagrant"
 provisioner:
   name: ansible
+  env:
+    ANSIBLE_VERBOSITY: 1
   playbooks:
     converge: ../resources/converge.yml
     side_effect: ../resources/reset.yml
@@ -63,7 +50,6 @@ provisioner:
 scenario:
   test_sequence:
     - dependency
-    - lint
     - cleanup
     - destroy
     - syntax

molecule/ipv6/overrides.yml

@@ -1,17 +1,18 @@
 ---
 - name: Apply overrides
   hosts: all
+  serial: 1
   tasks:
     - name: Override host variables (1/2)
       ansible.builtin.set_fact:
         # See:
         # https://github.com/flannel-io/flannel/blob/67d603aaf45ef80f5dd39f43714fc5e6f8a637eb/Documentation/troubleshooting.md#Vagrant
-        flannel_iface: eth1
+        flannel_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
 
         # In this scenario, we have multiple interfaces that the VIP could be
         # broadcasted on. Since we have assigned a dedicated private network
         # here, let's make sure that it is used.
-        kube_vip_iface: eth1
+        kube_vip_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
 
         # The test VMs might be a bit slow, so we give them more time to join the cluster:
         retry_count: 45

molecule/ipv6/prepare.yml

@@ -38,7 +38,7 @@
         dest: /etc/netplan/55-flannel-ipv4.yaml
         owner: root
         group: root
-        mode: 0644
+        mode: "0644"
       register: netplan_template
 
     - name: Apply netplan configuration

molecule/ipv6/verify-vars.yml

@@ -0,0 +1,7 @@
+---
+# Durable verify inputs for the ipv6 (flannel CNI + MetalLB) scenario.
+verify_cni: flannel
+verify_lb: metallb
+verify_lb_ip_range:
+  - fdad:bad:ba55::1b:0/112
+  - 192.168.123.80-192.168.123.90

molecule/kube-vip/molecule.yml

@@ -0,0 +1,44 @@
+---
+dependency:
+  name: galaxy
+driver:
+  name: vagrant
+platforms:
+  - name: control1
+    box: bento/ubuntu-26.04
+    memory: 4096
+    cpus: 4
+    groups:
+      - k3s_cluster
+      - master
+    interfaces:
+      - network_name: private_network
+        ip: 192.168.30.62
+provisioner:
+  name: ansible
+  env:
+    ANSIBLE_VERBOSITY: 1
+  playbooks:
+    converge: ../resources/converge.yml
+    side_effect: ../resources/reset.yml
+    verify: ../resources/verify.yml
+  inventory:
+    links:
+      group_vars: ../../inventory/sample/group_vars
+scenario:
+  test_sequence:
+    - dependency
+    - cleanup
+    - destroy
+    - syntax
+    - create
+    - prepare
+    - converge
+    # idempotence is not possible with the playbook in its current form.
+    - verify
+    # We are repurposing side_effect here to test the reset playbook.
+    # This is why we do not run it before verify (which tests the cluster),
+    # but after the verify step.
+    - side_effect
+    - cleanup
+    - destroy

molecule/kube-vip/overrides.yml

@@ -0,0 +1,19 @@
+---
+- name: Apply overrides
+  hosts: all
+  serial: 1
+  tasks:
+    - name: Override host variables
+      ansible.builtin.set_fact:
+        # See:
+        # https://github.com/flannel-io/flannel/blob/67d603aaf45ef80f5dd39f43714fc5e6f8a637eb/Documentation/troubleshooting.md#Vagrant
+        flannel_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
+        kube_vip_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
+
+        # The test VMs might be a bit slow, so we give them more time to join the cluster:
+        retry_count: 45
+
+        # Make sure that our IP ranges do not collide with those of the other scenarios
+        apiserver_endpoint: 192.168.30.225
+        # Use kube-vip instead of MetalLB
+        kube_vip_lb_ip_range: 192.168.30.110-192.168.30.119

molecule/kube-vip/verify-vars.yml

@@ -0,0 +1,9 @@
+---
+# Durable verify inputs for the kube-vip (flannel CNI + kube-vip LB) scenario.
+verify_cni: flannel
+verify_lb: kube-vip
+# The kube-vip cloud provider tag is not defined in the linked sample group
+# vars (its sample entry is commented out), so it is supplied here.
+verify_kube_vip_cloud_provider_tag: v0.0.12
+verify_lb_ip_range:
+  - 192.168.30.110-192.168.30.119

molecule/resources/verify.yml

@@ -1,5 +1,8 @@
 ---
 - name: Verify
   hosts: all
+  vars_files:
+    - >-
+      {{ lookup("ansible.builtin.env", "MOLECULE_SCENARIO_DIRECTORY") }}/verify-vars.yml
   roles:
     - verify_from_outside

molecule/resources/verify_from_outside/tasks/main.yml

@@ -7,6 +7,8 @@
       ansible.builtin.import_tasks: kubecfg-fetch.yml
     - name: "TEST CASE: Get nodes"
       ansible.builtin.include_tasks: test/get-nodes.yml
+    - name: "TEST CASE: Verify components"
+      ansible.builtin.include_tasks: test/verify-components.yml
     - name: "TEST CASE: Deploy example"
       ansible.builtin.include_tasks: test/deploy-example.yml
   always:

molecule/resources/verify_from_outside/tasks/test/deploy-example.yml

@@ -27,26 +27,95 @@
         name: nginx
         namespace: "{{ testing_namespace }}"
         kubeconfig: "{{ kubecfg_path }}"
-      vars: &load_balancer_metadata
+      vars:
         metallb_ip: status.loadBalancer.ingress[0].ip
         metallb_port: spec.ports[0].port
       register: nginx_services
 
+    - name: Wait for the load balancer address to be assigned
+      ansible.builtin.set_fact:
+        nginx_lb_ip: >-
+          {{
+            nginx_services.resources[0].status.loadBalancer.ingress[0].ip
+            if (nginx_services.resources | length > 0) and
+               (nginx_services.resources[0].status.loadBalancer.ingress is defined) and
+               (nginx_services.resources[0].status.loadBalancer.ingress | length > 0)
+            else ''
+          }}
+
+    - name: Retry until the load balancer service has an external IP
+      block:
+        - name: Refresh nginx service until it has an assigned address
+          kubernetes.core.k8s_info:
+            kind: service
+            name: nginx
+            namespace: "{{ testing_namespace }}"
+            kubeconfig: "{{ kubecfg_path }}"
+          register: nginx_lb_wait
+          until: >-
+            (nginx_lb_wait.resources | length > 0) and
+            (nginx_lb_wait.resources[0].status.loadBalancer.ingress is defined) and
+            (nginx_lb_wait.resources[0].status.loadBalancer.ingress | length > 0)
+          retries: 30
+          delay: 5
+
+        - name: Record the assigned load balancer address
+          ansible.builtin.set_fact:
+            nginx_lb_ip: >-
+              {{ nginx_lb_wait.resources[0].status.loadBalancer.ingress[0].ip }}
+
     - name: Assert that the nginx welcome page is available
       ansible.builtin.uri:
-        url: http://{{ ip | ansible.utils.ipwrap }}:{{ port_ }}/
+        url: http://{{ nginx_lb_ip | ansible.utils.ipwrap }}:{{ port_ }}/
-        return_content: yes
+        return_content: true
       register: result
       failed_when: "'Welcome to nginx!' not in result.content"
       vars:
-        ip: >-
-          {{ nginx_services.resources[0].status.loadBalancer.ingress[0].ip }}
         port_: >-
           {{ nginx_services.resources[0].spec.ports[0].port }}
-      # Deactivated linter rules:
+
-      #   - jinja[invalid]: As of version 6.6.0, ansible-lint complains that the input to ipwrap
+    - name: Initialize load balancer address range check
-      #                     would be undefined. This will not be the case during playbook execution.
+      ansible.builtin.set_fact:
-      # noqa jinja[invalid]
+        lb_addr_in_range: false
+        lb_ip_value: "{{ nginx_lb_ip }}"
+
+    - name: Check load balancer address against start-end pools
+      ansible.builtin.set_fact:
+        lb_addr_in_range: true
+      loop: "{{ verify_lb_ip_range }}"
+      loop_control:
+        label: "{{ item }}"
+      when:
+        - "'-' in item"
+        - "'/' not in item"
+        - >-
+          (lb_ip_value | ansible.utils.ipaddr('int') | int) >=
+          (item.split('-')[0] | ansible.utils.ipaddr('int') | int)
+        - >-
+          (lb_ip_value | ansible.utils.ipaddr('int') | int) <=
+          (item.split('-')[1] | ansible.utils.ipaddr('int') | int)
+
+    - name: Check load balancer address against CIDR pools
+      ansible.builtin.set_fact:
+        lb_addr_in_range: true
+      loop: "{{ verify_lb_ip_range }}"
+      loop_control:
+        label: "{{ item }}"
+      when:
+        - "'/' in item"
+        - (lb_ip_value | ansible.utils.ipaddr(item)) is string
+
+    - name: Assert that the load balancer address is within a configured pool
+      ansible.builtin.assert:
+        that: lb_addr_in_range
+        success_msg: "LoadBalancer address {{ lb_ip_value }} is in a configured range"
+        fail_msg: >-
+          LoadBalancer address {{ lb_ip_value }} is not in a configured
+          range {{ verify_lb_ip_range }}
+  # Deactivated linter rules:
+  #   - jinja[invalid]: As of version 6.6.0, ansible-lint complains that the input to ipwrap
+  #                     would be undefined. This will not be the case during playbook execution.
+  # noqa jinja[invalid]
 
   always:
     - name: "Remove namespace: {{ testing_namespace }}"

molecule/resources/verify_from_outside/tasks/test/get-nodes.yml

@@ -9,7 +9,7 @@
   ansible.builtin.assert:
     that: found_nodes == expected_nodes
     success_msg: "Found nodes as expected: {{ found_nodes }}"
-    fail_msg: "Expected nodes {{ expected_nodes }}, but found nodes {{ found_nodes }}"
+    fail_msg: Expected nodes {{ expected_nodes }}, but found nodes {{ found_nodes }}
   vars:
     found_nodes: >-
       {{ cluster_nodes | json_query('resources[*].metadata.name') | unique | sort }}
@@ -22,7 +22,7 @@
         | unique
         | sort
       }}
-  # Deactivated linter rules:
+ # Deactivated linter rules:
   #   - jinja[invalid]: As of version 6.6.0, ansible-lint complains that the input to ipwrap
   #                     would be undefined. This will not be the case during playbook execution.
   # noqa jinja[invalid]

molecule/resources/verify_from_outside/tasks/test/verify-components.yml

@@ -0,0 +1,313 @@
+---
+# Scenario-aware verification of cluster components and their live image tags.
+# Scenario identity (verify_cni / verify_lb) and expected address range come
+# from each scenario's verify-vars.yml, which is plain inventory data available
+# to the verify play. Converge-time set_fact values are not persisted between
+# the two Ansible processes, so they are never used here.
+- name: Verify cluster components report expected versions
+  block:
+    - name: Get all nodes with their kubelet versions
+      kubernetes.core.k8s_info:
+        kind: node
+        kubeconfig: "{{ kubecfg_path }}"
+      register: verify_nodes
+
+    - name: Assert each node reports the expected Kubernetes version
+      ansible.builtin.assert:
+        that: item.status.nodeInfo.kubeletVersion == k3s_version
+        success_msg: "{{ item.metadata.name }} reports {{ k3s_version }}"
+        fail_msg: >-
+          {{ item.metadata.name }} reports
+          {{ item.status.nodeInfo.kubeletVersion }},
+          expected {{ k3s_version }}
+      loop: "{{ verify_nodes.resources }}"
+      loop_control:
+        label: "{{ item.metadata.name }}"
+
+    - name: Verify Flannel is the active CNI
+      when: verify_cni == 'flannel'
+      block:
+        - name: Assert every node reports Ready
+          ansible.builtin.assert:
+            that: item.status.conditions
+              | selectattr('type', 'equalto', 'Ready')
+              | map(attribute='status') | first | default('') == 'True'
+            success_msg: "{{ item.metadata.name }} is Ready"
+            fail_msg: "{{ item.metadata.name }} is not Ready"
+          loop: "{{ verify_nodes.resources }}"
+          loop_control:
+            label: "{{ item.metadata.name }} ready"
+
+        - name: Get any Calico namespaces with Flannel enabled
+          kubernetes.core.k8s_info:
+            kind: Namespace
+            name: calico-system
+            kubeconfig: "{{ kubecfg_path }}"
+          register: flannel_calico_absent
+
+        - name: Assert there is no Calico system namespace
+          ansible.builtin.assert:
+            that: flannel_calico_absent.resources | length == 0
+            success_msg: "No Calico present with Flannel"
+            fail_msg: "A Calico namespace exists alongside Flannel"
+
+        - name: Get the Cilium namespace with Flannel enabled
+          kubernetes.core.k8s_info:
+            kind: Namespace
+            name: cilium
+            kubeconfig: "{{ kubecfg_path }}"
+          register: flannel_cilium
+
+        - name: Assert the Cilium namespace is absent
+          ansible.builtin.assert:
+            that: flannel_cilium.resources | length == 0
+            success_msg: "No Cilium present with Flannel"
+            fail_msg: "A Cilium namespace exists alongside Flannel"
+
+    - name: Verify Calico is the active CNI
+      when: verify_cni == 'calico'
+      block:
+        - name: Get the Calico node DaemonSet image
+          kubernetes.core.k8s_info:
+            kind: DaemonSet
+            name: calico-node
+            namespace: calico-system
+            kubeconfig: "{{ kubecfg_path }}"
+          register: calico_node_ds
+
+        - name: Assert the Calico node image uses the expected tag
+          ansible.builtin.assert:
+            that:
+              - calico_node_ds.resources | length == 1
+              - calico_node_image | regex_search(':' ~ calico_tag)
+            success_msg: "Calico node image uses tag {{ calico_tag }}"
+            fail_msg: >-
+              Calico node image {{ calico_node_image }},
+              expected {{ calico_tag }}
+          vars:
+            calico_node_image: "{{ calico_node_ds.resources[0].spec.template.spec.containers[0].image }}"
+
+        - name: Get Calico TigeraStatus for calico and apiserver
+          kubernetes.core.k8s_info:
+            api_version: operator.tigera.io/v1
+            kind: TigeraStatus
+            name: "{{ item }}"
+            kubeconfig: "{{ kubecfg_path }}"
+          register: calico_tigerastatus
+          loop:
+            - calico
+            - apiserver
+          loop_control:
+            label: "Tigerastatus/{{ item }}"
+
+        - name: Assert Calico TigeraStatus reports Available
+          ansible.builtin.assert:
+            that: >-
+              item.resources | length == 1 and
+              (item.resources[0].status.conditions
+               | selectattr('type', 'equalto', 'Available')
+               | map(attribute='status') | first | default('')) == 'True'
+            success_msg: "Tigerastatus {{ item.resources[0].metadata.name }} is Available"
+            fail_msg: "Tigerastatus is not Available"
+          loop: "{{ calico_tigerastatus.results }}"
+          loop_control:
+            label: "Tigerastatus Available"
+
+        - name: Get any Flannel DaemonSets with Calico enabled
+          kubernetes.core.k8s_info:
+            kind: DaemonSet
+            namespace: kube-flannel
+            kubeconfig: "{{ kubecfg_path }}"
+          register: no_flannel_ds
+
+        - name: Assert there are no Flannel DaemonSets
+          ansible.builtin.assert:
+            that: no_flannel_ds.resources | length == 0
+            success_msg: "No Flannel DaemonSet present with Calico"
+            fail_msg: "A Flannel DaemonSet exists alongside Calico"
+
+    - name: Verify Cilium is the active CNI
+      when: verify_cni == 'cilium'
+      block:
+        - name: Get the Cilium agent and operator images
+          kubernetes.core.k8s_info:
+            kind: "{{ item.kind }}"
+            name: "{{ item.name }}"
+            namespace: kube-system
+            kubeconfig: "{{ kubecfg_path }}"
+          register: cilium_info
+          loop:
+            - { kind: DaemonSet, name: cilium }
+            - { kind: Deployment, name: cilium-operator }
+          loop_control:
+            label: "{{ item.kind }}/{{ item.name }}"
+
+        - name: Assert Cilium agent and operator use the expected image tag
+          ansible.builtin.assert:
+            that:
+              - cilium_agent_image | regex_search(':' ~ cilium_tag)
+              - cilium_operator_image | regex_search(':' ~ cilium_tag)
+            success_msg: "Cilium agent and operator use {{ cilium_tag }}"
+            fail_msg: >-
+              Cilium agent {{ cilium_agent_image }},
+              operator {{ cilium_operator_image }},
+              expected {{ cilium_tag }}
+          vars:
+            cilium_agent_image: >-
+              {{ (cilium_info.results
+                  | selectattr('resources', 'defined')
+                  | map(attribute='resources')
+                  | list
+                  | map(attribute='0')
+                  | selectattr('kind', 'equalto', 'DaemonSet')
+                  | list)[0].spec.template.spec.containers[0].image }}
+            cilium_operator_image: >-
+              {{ (cilium_info.results
+                  | selectattr('resources', 'defined')
+                  | map(attribute='resources')
+                  | list
+                  | map(attribute='0')
+                  | selectattr('kind', 'equalto', 'Deployment')
+                  | list)[0].spec.template.spec.containers[0].image }}
+
+        - name: Get Hubble relay and UI deployments when enabled
+          kubernetes.core.k8s_info:
+            kind: Deployment
+            name: "{{ item }}"
+            namespace: kube-system
+            kubeconfig: "{{ kubecfg_path }}"
+          register: hubble_info
+          loop:
+            - hubble-relay
+            - hubble-ui
+          loop_control:
+            label: "Deployment/{{ item }}"
+          when: cilium_hubble | bool
+
+        - name: Assert Hubble components are Ready when enabled
+          ansible.builtin.assert:
+            that:
+              - item.resources | length == 1
+              - item.resources[0].status.readyReplicas | default(0) >= 1
+            success_msg: "Hubble deployment {{ item.resources[0].metadata.name }} is Ready"
+            fail_msg: "Hubble deployment is not Ready"
+          loop: "{{ hubble_info.results }}"
+          loop_control:
+            label: "Hubble deployment"
+          when: cilium_hubble | bool
+
+        - name: Get any Flannel DaemonSets with Cilium enabled
+          kubernetes.core.k8s_info:
+            kind: DaemonSet
+            namespace: kube-flannel
+            kubeconfig: "{{ kubecfg_path }}"
+          register: no_flannel_ds_cilium
+
+        - name: Assert there are no Flannel DaemonSets
+          ansible.builtin.assert:
+            that: no_flannel_ds_cilium.resources | length == 0
+            success_msg: "No Flannel DaemonSet present with Cilium"
+            fail_msg: "A Flannel DaemonSet exists alongside Cilium"
+
+    - name: Verify MetalLB is the active load balancer
+      when: verify_lb == 'metallb'
+      block:
+        - name: Get the MetalLB controller and speaker images
+          kubernetes.core.k8s_info:
+            kind: "{{ item.kind }}"
+            name: "{{ item.name }}"
+            namespace: metallb-system
+            kubeconfig: "{{ kubecfg_path }}"
+          register: metallb_info
+          loop:
+            - { kind: Deployment, name: controller }
+            - { kind: DaemonSet, name: speaker }
+          loop_control:
+            label: "{{ item.kind }}/{{ item.name }}"
+
+        - name: Assert MetalLB controller and speaker use the expected image tags
+          ansible.builtin.assert:
+            that:
+              - controller_image | regex_search(metal_lb_controller_tag_version)
+              - speaker_image | regex_search(metal_lb_speaker_tag_version)
+            success_msg: >-
+              MetalLB controller {{ metal_lb_controller_tag_version }},
+              speaker {{ metal_lb_speaker_tag_version }}
+            fail_msg: >-
+              MetalLB controller {{ controller_image }},
+              speaker {{ speaker_image }}
+          vars:
+            controller_image: >-
+              {{ (metallb_info.results
+                  | selectattr('resources', 'defined')
+                  | map(attribute='resources')
+                  | list
+                  | map(attribute='0')
+                  | selectattr('kind', 'equalto', 'Deployment')
+                  | list)[0].spec.template.spec.containers[0].image }}
+            speaker_image: >-
+              {{ (metallb_info.results
+                  | selectattr('resources', 'defined')
+                  | map(attribute='resources')
+                  | list
+                  | map(attribute='0')
+                  | selectattr('kind', 'equalto', 'DaemonSet')
+                  | list)[0].spec.template.spec.containers[0].image }}
+
+    - name: Verify kube-vip is the active load balancer
+      when: verify_lb == 'kube-vip'
+      block:
+        - name: Get the kube-vip and cloud provider images
+          kubernetes.core.k8s_info:
+            kind: "{{ item.kind }}"
+            name: "{{ item.name }}"
+            namespace: kube-system
+            kubeconfig: "{{ kubecfg_path }}"
+          register: kubevip_info
+          loop:
+            - { kind: DaemonSet, name: kube-vip-ds }
+            - { kind: Deployment, name: kube-vip-cloud-provider }
+          loop_control:
+            label: "{{ item.kind }}/{{ item.name }}"
+
+        - name: Assert the kube-vip and cloud provider image tags
+          ansible.builtin.assert:
+            that:
+              - kubevip_image | regex_search(':' ~ kube_vip_tag_version)
+              - cloud_provider_image | regex_search(verify_kube_vip_cloud_provider_tag)
+            success_msg: >-
+              kube-vip {{ kube_vip_tag_version }},
+              cloud provider {{ verify_kube_vip_cloud_provider_tag }}
+            fail_msg: >-
+              kube-vip {{ kubevip_image }},
+              cloud provider {{ cloud_provider_image }}
+          vars:
+            kubevip_image: >-
+              {{ (kubevip_info.results
+                  | selectattr('resources', 'defined')
+                  | map(attribute='resources')
+                  | list
+                  | map(attribute='0')
+                  | selectattr('kind', 'equalto', 'DaemonSet')
+                  | list)[0].spec.template.spec.containers[0].image }}
+            cloud_provider_image: >-
+              {{ (kubevip_info.results
+                  | selectattr('resources', 'defined')
+                  | map(attribute='resources')
+                  | list
+                  | map(attribute='0')
+                  | selectattr('kind', 'equalto', 'Deployment')
+                  | list)[0].spec.template.spec.containers[0].image }}
+
+        - name: Get the MetalLB namespace with kube-vip enabled
+          kubernetes.core.k8s_info:
+            kind: Namespace
+            name: metallb-system
+            kubeconfig: "{{ kubecfg_path }}"
+          register: metallb_absent
+
+        - name: Assert the MetalLB namespace does not exist
+          ansible.builtin.assert:
+            that: metallb_absent.resources | length == 0
+            success_msg: "MetalLB is not installed with kube-vip"
+            fail_msg: "MetalLB namespace exists alongside kube-vip"

molecule/single_node/molecule.yml

@@ -5,14 +5,9 @@ driver:
   name: vagrant
 platforms:
   - name: control1
-    box: generic/ubuntu2204
+    box: bento/ubuntu-26.04
     memory: 4096
     cpus: 4
-    config_options:
-      # We currently can not use public-key based authentication on Ubuntu 22.04,
-      # see: https://github.com/chef/bento/issues/1405
-      ssh.username: "vagrant"
-      ssh.password: "vagrant"
     groups:
       - k3s_cluster
       - master
@@ -21,6 +16,8 @@ platforms:
         ip: 192.168.30.50
 provisioner:
   name: ansible
+  env:
+    ANSIBLE_VERBOSITY: 1
   playbooks:
     converge: ../resources/converge.yml
     side_effect: ../resources/reset.yml
@@ -31,7 +28,6 @@ provisioner:
 scenario:
   test_sequence:
     - dependency
-    - lint
     - cleanup
     - destroy
     - syntax

molecule/single_node/overrides.yml

@@ -1,16 +1,18 @@
 ---
 - name: Apply overrides
   hosts: all
+  serial: 1
   tasks:
     - name: Override host variables
       ansible.builtin.set_fact:
         # See:
         # https://github.com/flannel-io/flannel/blob/67d603aaf45ef80f5dd39f43714fc5e6f8a637eb/Documentation/troubleshooting.md#Vagrant
-        flannel_iface: eth1
+        flannel_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
+        kube_vip_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
 
         # The test VMs might be a bit slow, so we give them more time to join the cluster:
         retry_count: 45
 
         # Make sure that our IP ranges do not collide with those of the default scenario
-        apiserver_endpoint: "192.168.30.223"
+        apiserver_endpoint: 192.168.30.223
-        metal_lb_ip_range: "192.168.30.91-192.168.30.99"
+        metal_lb_ip_range: 192.168.30.91-192.168.30.99

molecule/single_node/verify-vars.yml

@@ -0,0 +1,6 @@
+---
+# Durable verify inputs for the single_node (flannel + MetalLB) scenario.
+verify_cni: flannel
+verify_lb: metallb
+verify_lb_ip_range:
+  - 192.168.30.91-192.168.30.99

reboot.yml

@@ -1,9 +1,10 @@
 ---
 - name: Reboot k3s_cluster
   hosts: k3s_cluster
-  gather_facts: yes
+  gather_facts: true
   tasks:
     - name: Reboot the nodes (and Wait upto 5 mins max)
       become: true
-      reboot:
+      ansible.builtin.reboot:
+        reboot_command: "{{ custom_reboot_command | default(omit) }}"
         reboot_timeout: 300

requirements.in

@@ -1,10 +1,10 @@
-ansible-core>=2.13.5
+ansible-core>=2.16.2
 jmespath>=1.0.1
-jsonpatch>=1.32
+jsonpatch>=1.33
-kubernetes>=25.3.0
+kubernetes>=29.0.0
-molecule-vagrant>=1.0.0
+molecule-plugins[vagrant]
-molecule>=4.0.3
+molecule>=6.0.3
-netaddr>=0.8.0
+netaddr>=0.10.1
-pre-commit>=2.20.0
+pre-commit>=3.6.0
-pre-commit-hooks>=1.3.1
+pre-commit-hooks>=4.5.0
-pyyaml>=6.0
+pyyaml>=6.0.1

requirements.txt

@@ -4,174 +4,165 @@
 #
 #    pip-compile requirements.in
 #
-ansible-compat==3.0.1
+ansible-compat==4.1.11
     # via molecule
-ansible-core==2.15.4
+ansible-core==2.18.0
     # via
     #   -r requirements.in
     #   ansible-compat
-arrow==1.2.3
+    #   molecule
-    # via jinja2-time
+attrs==23.2.0
-attrs==22.1.0
+    # via
-    # via jsonschema
+    #   jsonschema
-binaryornot==0.4.4
+    #   referencing
-    # via cookiecutter
+bracex==2.4
-cachetools==5.2.0
+    # via wcmatch
+cachetools==5.3.2
     # via google-auth
-certifi==2022.9.24
+certifi==2023.11.17
     # via
     #   kubernetes
     #   requests
-cffi==1.15.1
+cffi==1.16.0
     # via cryptography
-cfgv==3.3.1
+cfgv==3.4.0
     # via pre-commit
-chardet==5.0.0
+charset-normalizer==3.3.2
-    # via binaryornot
-charset-normalizer==2.1.1
     # via requests
-click==8.1.3
+click==8.1.7
     # via
     #   click-help-colors
-    #   cookiecutter
     #   molecule
-click-help-colors==0.9.1
+click-help-colors==0.9.4
     # via molecule
-commonmark==0.9.1
+cryptography==41.0.7
-    # via rich
-cookiecutter==2.1.1
-    # via molecule
-cryptography==38.0.3
     # via ansible-core
-distlib==0.3.6
+distlib==0.3.8
     # via virtualenv
-distro==1.8.0
-    # via selinux
 enrich==1.2.7
     # via molecule
-filelock==3.8.0
+filelock==3.13.1
     # via virtualenv
-google-auth==2.14.0
+google-auth==2.26.2
     # via kubernetes
-identify==2.5.8
+identify==2.5.33
     # via pre-commit
-idna==3.4
+idna==3.6
     # via requests
-jinja2==3.1.2
+jinja2==3.1.3
     # via
     #   ansible-core
-    #   cookiecutter
-    #   jinja2-time
     #   molecule
-    #   molecule-vagrant
-jinja2-time==0.2.0
-    # via cookiecutter
 jmespath==1.0.1
     # via -r requirements.in
 jsonpatch==1.33
     # via -r requirements.in
-jsonpointer==2.3
+jsonpointer==2.4
     # via jsonpatch
-jsonschema==4.17.0
+jsonschema==4.21.1
     # via
     #   ansible-compat
     #   molecule
-kubernetes==25.3.0
+jsonschema-specifications==2023.12.1
+    # via jsonschema
+kubernetes==29.0.0
     # via -r requirements.in
-markupsafe==2.1.1
+markdown-it-py==3.0.0
+    # via rich
+markupsafe==2.1.4
     # via jinja2
-molecule==4.0.4
+mdurl==0.1.2
+    # via markdown-it-py
+molecule==6.0.3
     # via
     #   -r requirements.in
-    #   molecule-vagrant
+    #   molecule-plugins
-molecule-vagrant==1.0.0
+molecule-plugins[vagrant]==23.5.3
     # via -r requirements.in
-netaddr==0.9.0
+netaddr==0.10.1
     # via -r requirements.in
-nodeenv==1.7.0
+nodeenv==1.8.0
     # via pre-commit
 oauthlib==3.2.2
-    # via requests-oauthlib
+    # via
-packaging==21.3
+    #   kubernetes
+    #   requests-oauthlib
+packaging==23.2
     # via
     #   ansible-compat
     #   ansible-core
     #   molecule
-platformdirs==2.5.2
+platformdirs==4.1.0
     # via virtualenv
-pluggy==1.0.0
+pluggy==1.3.0
     # via molecule
-pre-commit==2.21.0
+pre-commit==3.8.0
     # via -r requirements.in
-pre-commit-hooks==4.5.0
+pre-commit-hooks==4.6.0
     # via -r requirements.in
-pyasn1==0.4.8
+pyasn1==0.5.1
     # via
     #   pyasn1-modules
     #   rsa
-pyasn1-modules==0.2.8
+pyasn1-modules==0.3.0
     # via google-auth
 pycparser==2.21
     # via cffi
-pygments==2.13.0
+pygments==2.17.2
     # via rich
-pyparsing==3.0.9
-    # via packaging
-pyrsistent==0.19.2
-    # via jsonschema
 python-dateutil==2.8.2
-    # via
+    # via kubernetes
-    #   arrow
-    #   kubernetes
-python-slugify==6.1.2
-    # via cookiecutter
 python-vagrant==1.0.0
-    # via molecule-vagrant
+    # via molecule-plugins
-pyyaml==6.0.1
+pyyaml==6.0.2
     # via
     #   -r requirements.in
     #   ansible-compat
     #   ansible-core
-    #   cookiecutter
     #   kubernetes
     #   molecule
-    #   molecule-vagrant
     #   pre-commit
-requests==2.28.1
+referencing==0.32.1
+    # via
+    #   jsonschema
+    #   jsonschema-specifications
+requests==2.31.0
     # via
-    #   cookiecutter
     #   kubernetes
     #   requests-oauthlib
 requests-oauthlib==1.3.1
     # via kubernetes
-resolvelib==0.8.1
+resolvelib==1.0.1
     # via ansible-core
-rich==12.6.0
+rich==13.7.0
     # via
     #   enrich
     #   molecule
+rpds-py==0.17.1
+    # via
+    #   jsonschema
+    #   referencing
 rsa==4.9
     # via google-auth
-ruamel-yaml==0.17.21
+ruamel-yaml==0.18.5
     # via pre-commit-hooks
-selinux==0.2.1
+ruamel-yaml-clib==0.2.8
-    # via molecule-vagrant
+    # via ruamel-yaml
 six==1.16.0
     # via
-    #   google-auth
     #   kubernetes
     #   python-dateutil
 subprocess-tee==0.4.1
     # via ansible-compat
-text-unidecode==1.3
+urllib3==2.1.0
-    # via python-slugify
-urllib3==1.26.12
     # via
     #   kubernetes
     #   requests
-virtualenv==20.16.6
+virtualenv==20.25.0
     # via pre-commit
-websocket-client==1.4.2
+wcmatch==8.5
+    # via molecule
+websocket-client==1.7.0
     # via kubernetes
 
 # The following packages are considered to be unsafe in a requirements file:

reset.yml

@@ -1,23 +1,24 @@
 ---
 - name: Reset k3s cluster
   hosts: k3s_cluster
-  gather_facts: yes
+  gather_facts: true
   roles:
     - role: reset
       become: true
     - role: raspberrypi
       become: true
-      vars: {state: absent}
+      vars: { state: absent }
   post_tasks:
     - name: Reboot and wait for node to come back up
       become: true
-      reboot:
+      ansible.builtin.reboot:
+        reboot_command: "{{ custom_reboot_command | default(omit) }}"
         reboot_timeout: 3600
 
 - name: Revert changes to Proxmox cluster
   hosts: proxmox
   gather_facts: true
-  become: yes
+  become: true
   remote_user: "{{ proxmox_lxc_ssh_user }}"
   roles:
     - role: reset_proxmox_lxc

roles/download/meta/main.yml

@@ -0,0 +1,8 @@
+---
+argument_specs:
+  main:
+    short_description: Manage the downloading of K3S binaries
+    options:
+      k3s_version:
+        description: The desired version of K3S
+        required: true

roles/download/tasks/main.yml

@@ -1,36 +1,46 @@
 ---
-
 - name: Download k3s binary x64
-  get_url:
+  ansible.builtin.get_url:
     url: https://github.com/k3s-io/k3s/releases/download/{{ k3s_version }}/k3s
     checksum: sha256:https://github.com/k3s-io/k3s/releases/download/{{ k3s_version }}/sha256sum-amd64.txt
     dest: /usr/local/bin/k3s
     owner: root
     group: root
-    mode: 0755
+    mode: "0755"
+  register: k3s_download_x64
+  retries: 5
+  delay: 10
+  until: k3s_download_x64 is succeeded
   when: ansible_facts.architecture == "x86_64"
 
 - name: Download k3s binary arm64
-  get_url:
+  ansible.builtin.get_url:
     url: https://github.com/k3s-io/k3s/releases/download/{{ k3s_version }}/k3s-arm64
     checksum: sha256:https://github.com/k3s-io/k3s/releases/download/{{ k3s_version }}/sha256sum-arm64.txt
     dest: /usr/local/bin/k3s
     owner: root
     group: root
-    mode: 0755
+    mode: "0755"
+  register: k3s_download_arm64
+  retries: 5
+  delay: 10
+  until: k3s_download_arm64 is succeeded
   when:
-    - ( ansible_facts.architecture is search("arm") and
+    - ( ansible_facts.architecture is search("arm") and ansible_facts.userspace_bits == "64" )
-        ansible_facts.userspace_bits == "64" ) or
+      or ansible_facts.architecture is search("aarch64")
-      ansible_facts.architecture is search("aarch64")
 
 - name: Download k3s binary armhf
-  get_url:
+  ansible.builtin.get_url:
     url: https://github.com/k3s-io/k3s/releases/download/{{ k3s_version }}/k3s-armhf
     checksum: sha256:https://github.com/k3s-io/k3s/releases/download/{{ k3s_version }}/sha256sum-arm.txt
     dest: /usr/local/bin/k3s
     owner: root
     group: root
-    mode: 0755
+    mode: "0755"
+  register: k3s_download_armhf
+  retries: 5
+  delay: 10
+  until: k3s_download_armhf is succeeded
   when:
     - ansible_facts.architecture is search("arm")
     - ansible_facts.userspace_bits == "32"

roles/k3s_agent/defaults/main.yml

@@ -0,0 +1,4 @@
+---
+extra_agent_args: ""
+group_name_master: master
+systemd_dir: /etc/systemd/system

roles/k3s_agent/meta/main.yml

@@ -0,0 +1,39 @@
+---
+argument_specs:
+  main:
+    short_description: Setup k3s agents
+    options:
+      apiserver_endpoint:
+        description: Virtual ip-address configured on each master
+        required: true
+
+      extra_agent_args:
+        description: Extra arguments for agents nodes
+
+      group_name_master:
+        description: Name of the master group
+        default: master
+
+      k3s_token:
+        description: Token used to communicate between masters
+
+      proxy_env:
+        type: dict
+        description:
+          - Internet proxy configurations.
+          - See https://docs.k3s.io/advanced#configuring-an-http-proxy for details
+        default: ~
+        options:
+          HTTP_PROXY:
+            description: HTTP internet proxy
+            required: true
+          HTTPS_PROXY:
+            description: HTTPS internet proxy
+            required: true
+          NO_PROXY:
+            description: Addresses that will not use the proxies
+            required: true
+
+      systemd_dir:
+        description: Path to systemd services
+        default: /etc/systemd/system

roles/k3s_agent/tasks/http_proxy.yml

@@ -1,18 +1,18 @@
 ---
-
+- name: Create k3s-node.service.d directory
-- name: Create k3s.service.d directory
+  ansible.builtin.file:
-  file:
+    path: "{{ systemd_dir }}/k3s-node.service.d"
-    path: '{{ systemd_dir }}/k3s.service.d'
     state: directory
     owner: root
     group: root
-    mode: '0755'
+    mode: "0755"
-
+  when: proxy_env is defined
 
 - name: Copy K3s http_proxy conf file
-  template:
+  ansible.builtin.template:
-    src: "http_proxy.conf.j2"
+    src: http_proxy.conf.j2
-    dest: "{{ systemd_dir }}/k3s.service.d/http_proxy.conf"
+    dest: "{{ systemd_dir }}/k3s-node.service.d/http_proxy.conf"
     owner: root
     group: root
-    mode: '0755'
+    mode: "0755"
+  when: proxy_env is defined

roles/k3s_agent/tasks/main.yml

@@ -1,20 +1,36 @@
 ---
+- name: Check for PXE-booted system
+  block:
+    - name: Check if system is PXE-booted
+      ansible.builtin.command:
+        cmd: cat /proc/cmdline
+      register: boot_cmdline
+      changed_when: false
+      check_mode: false
+
+    - name: Set fact for PXE-booted system
+      ansible.builtin.set_fact:
+        is_pxe_booted: "{{ 'root=/dev/nfs' in boot_cmdline.stdout }}"
+      when: boot_cmdline.stdout is defined
+
+    - name: Include http_proxy configuration tasks
+      ansible.builtin.include_tasks: http_proxy.yml
 
 - name: Deploy K3s http_proxy conf
-  include_tasks: http_proxy.yml
+  ansible.builtin.include_tasks: http_proxy.yml
   when: proxy_env is defined
 
-- name: Copy K3s service file
+- name: Configure the k3s service
-  template:
+  ansible.builtin.template:
-    src: "k3s.service.j2"
+    src: k3s.service.j2
     dest: "{{ systemd_dir }}/k3s-node.service"
     owner: root
     group: root
-    mode: 0755
+    mode: "0755"
 
-- name: Enable and check K3s service
+- name: Manage k3s service
-  systemd:
+  ansible.builtin.systemd:
     name: k3s-node
-    daemon_reload: yes
+    daemon_reload: true
     state: restarted
-    enabled: yes
+    enabled: true

roles/k3s_agent/templates/k3s.service.j2

@@ -7,11 +7,14 @@ After=network-online.target
 Type=notify
 ExecStartPre=-/sbin/modprobe br_netfilter
 ExecStartPre=-/sbin/modprobe overlay
-ExecStart=/usr/local/bin/k3s agent --server https://{{ apiserver_endpoint | ansible.utils.ipwrap }}:6443 --token {{ hostvars[groups[group_name_master | default('master')][0]]['token'] | default(k3s_token) }} {{ extra_agent_args | default("") }}
+# Conditional snapshotter based on PXE boot status
+ExecStart=/usr/local/bin/k3s agent \
+  --server https://{{ apiserver_endpoint | ansible.utils.ipwrap }}:6443 \
+  {% if is_pxe_booted | default(false) %}--snapshotter native \
+  {% endif %}--token {{ hostvars[groups[group_name_master | default('master')][0]]['token'] | default(k3s_token) }} \
+  {{ extra_agent_args }}
 KillMode=process
 Delegate=yes
-# Having non-zero Limit*s causes performance problems due to accounting overhead
-# in the kernel. We recommend using cgroups to do container-local accounting.
 LimitNOFILE=1048576
 LimitNPROC=infinity
 LimitCORE=infinity

roles/k3s_custom_registries/defaults/main.yml

@@ -1,6 +0,0 @@
----
-# Indicates whether custom registries for k3s should be configured
-# Possible values:
-#   - present
-#   - absent
-state: present

roles/k3s_custom_registries/meta/main.yml

@@ -0,0 +1,20 @@
+---
+argument_specs:
+  main:
+    short_description: Configure the use of a custom container registry
+    options:
+      custom_registries_yaml:
+        description:
+          - YAML block defining custom registries.
+          - >
+            The following is an example that pulls all images used in
+            this playbook through your private registries.
+          - >
+            It also allows you to pull your own images from your private
+            registry, without having to use imagePullSecrets in your
+            deployments.
+          - >
+            If all you need is your own images and you don't care about
+            caching the docker/quay/ghcr.io images, you can just remove
+            those from the mirrors: section.
+        required: true

roles/k3s_custom_registries/tasks/main.yml

@@ -1,17 +1,16 @@
 ---
-
 - name: Create directory /etc/rancher/k3s
-  file:
+  ansible.builtin.file:
-    path: "/etc/{{ item }}"
+    path: /etc/{{ item }}
     state: directory
-    mode: '0755'
+    mode: "0755"
   loop:
     - rancher
     - rancher/k3s
 
 - name: Insert registries into /etc/rancher/k3s/registries.yaml
-  blockinfile:
+  ansible.builtin.blockinfile:
     path: /etc/rancher/k3s/registries.yaml
     block: "{{ custom_registries_yaml }}"
-    mode: '0600'
+    mode: "0600"
     create: true

roles/k3s_server/defaults/main.yml

@@ -1,12 +1,30 @@
 ---
-# If you want to explicitly define an interface that ALL control nodes
+extra_server_args: ""
-# should use to propagate the VIP, define it here. Otherwise, kube-vip
+
-# will determine the right interface automatically at runtime.
+k3s_kubectl_binary: k3s kubectl
-kube_vip_iface: null
 
-# Name of the master group
 group_name_master: master
 
+kube_vip_arp: true
+kube_vip_iface:
+kube_vip_cloud_provider_tag_version: v0.0.12
+kube_vip_tag_version: v1.2.2
+
+kube_vip_bgp: false
+kube_vip_bgp_routerid: 127.0.0.1
+kube_vip_bgp_as: "64513"
+kube_vip_bgp_peeraddress: 192.168.30.1
+kube_vip_bgp_peeras: "64512"
+
+kube_vip_bgp_peers: []
+kube_vip_bgp_peers_groups: ['k3s_master']
+
+metal_lb_controller_tag_version: v0.16.0
+metal_lb_speaker_tag_version: v0.16.0
+metal_lb_type: native
+
+retry_count: 20
+
 # yamllint disable rule:line-length
 server_init_args: >-
   {% if groups[group_name_master | default('master')] | length > 1 %}
@@ -17,4 +35,6 @@ server_init_args: >-
     {% endif %}
     --token {{ k3s_token }}
   {% endif %}
-  {{ extra_server_args | default('') }}
+  {{ extra_server_args }}
+
+systemd_dir: /etc/systemd/system

roles/k3s_server/meta/main.yml

@@ -0,0 +1,135 @@
+---
+argument_specs:
+  main:
+    short_description: Setup k3s servers
+    options:
+      apiserver_endpoint:
+        description: Virtual ip-address configured on each master
+        required: true
+
+      cilium_bgp:
+        description:
+          - Enable cilium BGP control plane for LB services and pod cidrs.
+          - Disables the use of MetalLB.
+        type: bool
+        default: ~
+
+      cilium_iface:
+        description: The network interface used for when Cilium is enabled
+        default: ~
+
+      extra_server_args:
+        description: Extra arguments for server nodes
+        default: ""
+
+      group_name_master:
+        description: Name of the master group
+        default: master
+
+      k3s_create_kubectl_symlink:
+        description: Create the kubectl -> k3s symlink
+        default: false
+        type: bool
+
+      k3s_create_crictl_symlink:
+        description: Create the crictl -> k3s symlink
+        default: false
+        type: bool
+
+      kube_vip_arp:
+        description: Enables kube-vip ARP broadcasts
+        default: true
+        type: bool
+
+      kube_vip_bgp:
+        description: Enables kube-vip BGP peering
+        default: false
+        type: bool
+
+      kube_vip_bgp_routerid:
+        description: Defines the router ID for the kube-vip BGP server
+        default: "127.0.0.1"
+
+      kube_vip_bgp_as:
+        description: Defines the AS for the kube-vip BGP server
+        default: "64513"
+
+      kube_vip_bgp_peeraddress:
+        description: Defines the address for the kube-vip BGP peer
+        default: "192.168.30.1"
+
+      kube_vip_bgp_peeras:
+        description: Defines the AS for the kube-vip BGP peer
+        default: "64512"
+
+      kube_vip_bgp_peers:
+        description: List of BGP peer ASN & address pairs
+        default: []
+
+      kube_vip_bgp_peers_groups:
+        description: Inventory group in which to search for additional kube_vip_bgp_peers parameters to merge.
+        default: ['k3s_master']
+
+      kube_vip_iface:
+        description:
+          - Explicitly define an interface that ALL control nodes
+          - should use to propagate the VIP, define it here.
+          - Otherwise, kube-vip will determine the right interface
+          - automatically at runtime.
+        default: ~
+
+      kube_vip_tag_version:
+        description: Image tag for kube-vip
+        default: v1.2.2
+
+      kube_vip_cloud_provider_tag_version:
+        description: Tag for kube-vip-cloud-provider manifest when enabled
+        default: v0.0.12
+
+      kube_vip_lb_ip_range:
+        description: IP range for kube-vip load balancer
+        default: ~
+
+      metal_lb_controller_tag_version:
+        description: Image tag for MetalLB
+        default: v0.16.0
+
+      metal_lb_speaker_tag_version:
+        description: Image tag for MetalLB
+        default: v0.16.0
+
+      metal_lb_type:
+        choices:
+          - frr
+          - native
+        default: native
+        description: Use FRR mode or native. Valid values are `frr` and `native`
+
+      proxy_env:
+        type: dict
+        description:
+          - Internet proxy configurations.
+          - See https://docs.k3s.io/advanced#configuring-an-http-proxy for details
+        default: ~
+        options:
+          HTTP_PROXY:
+            description: HTTP internet proxy
+            required: true
+          HTTPS_PROXY:
+            description: HTTPS internet proxy
+            required: true
+          NO_PROXY:
+            description: Addresses that will not use the proxies
+            required: true
+
+      retry_count:
+        description: Amount of retries when verifying that nodes joined
+        type: int
+        default: 20
+
+      server_init_args:
+        description: Arguments for server nodes
+
+      systemd_dir:
+        description: Path to systemd services
+        default: /etc/systemd/system

roles/k3s_server/tasks/fetch_k3s_init_logs.yml

@@ -4,10 +4,110 @@
 
 - name: Fetch k3s-init.service logs
   ansible.builtin.command:
-    cmd: journalctl --all --unit=k3s-init.service
+    cmd: >-
+      timeout --signal=TERM --kill-after=5s 30s journalctl --no-pager
+      --unit=k3s-init.service --since=-30min --lines=5000
   changed_when: false
+  failed_when: false
+  no_log: true
   register: k3s_init_log
 
+- name: Fetch k3s-init.service status
+  ansible.builtin.command:
+    cmd: >-
+      timeout --signal=TERM --kill-after=5s 15s systemctl status
+      k3s-init.service --no-pager --full
+  changed_when: false
+  failed_when: false
+  no_log: true
+  register: k3s_init_status
+
+- name: Fetch IP address state
+  ansible.builtin.command:
+    cmd: ip -br address
+  changed_when: false
+  failed_when: false
+  no_log: true
+  register: k3s_ip_address_state
+
+- name: Fetch IP link state
+  ansible.builtin.command:
+    cmd: ip -br link
+  changed_when: false
+  failed_when: false
+  no_log: true
+  register: k3s_ip_link_state
+
+- name: Fetch IP route state
+  ansible.builtin.command:
+    cmd: ip route show
+  changed_when: false
+  failed_when: false
+  no_log: true
+  register: k3s_ip_route_state
+
+- name: Fetch IP neighbor state
+  ansible.builtin.command:
+    cmd: ip neigh show
+  changed_when: false
+  failed_when: false
+  no_log: true
+  register: k3s_ip_neighbor_state
+
+- name: Fetch IP rule state
+  ansible.builtin.command:
+    cmd: ip rule show
+  changed_when: false
+  failed_when: false
+  no_log: true
+  register: k3s_ip_rule_state
+
+- name: Fetch listening TCP sockets
+  ansible.builtin.command:
+    cmd: ss -ltn
+  changed_when: false
+  failed_when: false
+  no_log: true
+  register: k3s_tcp_listener_state
+
+- name: Fetch guest firewall service state
+  ansible.builtin.service_facts:
+  no_log: true
+  register: k3s_service_facts
+
+- name: Fetch input firewall rules
+  ansible.builtin.command:
+    cmd: iptables -S INPUT
+  changed_when: false
+  failed_when: false
+  no_log: true
+  register: k3s_input_firewall_rules
+
+- name: Ping the primary Kubernetes API address from {{ ansible_hostname }}
+  ansible.builtin.command:
+    argv:
+      - ping
+      - -c
+      - "1"
+      - -W
+      - "1"
+      - "{{ hostvars[groups[group_name_master | default('master')][0]].k3s_node_ip | split(',') | first }}"
+  changed_when: false
+  failed_when: false
+  no_log: true
+  register: k3s_primary_api_ping
+
+- name: Probe primary Kubernetes API from {{ ansible_hostname }}
+  ansible.builtin.wait_for:
+    host: "{{ hostvars[groups[group_name_master | default('master')][0]].k3s_node_ip | split(',') | first }}"
+    port: 6443
+    connect_timeout: 1
+    timeout: 2
+  changed_when: false
+  failed_when: false
+  no_log: true
+  register: k3s_primary_api_probe
+
 - name: Create {{ log_destination }}
   delegate_to: localhost
   run_once: true
@@ -23,6 +123,58 @@
   ansible.builtin.template:
     src: content.j2
     dest: "{{ log_destination }}/k3s-init@{{ ansible_hostname }}.log"
-    mode: 0644
+    mode: "0644"
   vars:
-    content: "{{ k3s_init_log.stdout }}"
+    content: |
+      === k3s-init.service status ===
+      {{ k3s_init_status.stdout | regex_replace('--token(?:=| +)[^ ]+', '--token ***') }}
+
+      === k3s-init.service journal ===
+      {{ k3s_init_log.stdout | regex_replace('--token(?:=| +)[^ ]+', '--token ***') }}
+
+- name: Store network diagnostics to {{ log_destination }}
+  delegate_to: localhost
+  become: false
+  ansible.builtin.template:
+    src: content.j2
+    dest: "{{ log_destination }}/network@{{ ansible_hostname }}.log"
+    mode: "0644"
+  vars:
+    content: |
+      === ip -br address ===
+      {{ k3s_ip_address_state.stdout }}
+
+      === ip -br link ===
+      {{ k3s_ip_link_state.stdout }}
+
+      === ip route show ===
+      {{ k3s_ip_route_state.stdout }}
+
+      === ip neigh show ===
+      {{ k3s_ip_neighbor_state.stdout }}
+
+      === ip rule show ===
+      {{ k3s_ip_rule_state.stdout }}
+
+      === ss -ltn ===
+      {{ k3s_tcp_listener_state.stdout }}
+
+      === guest firewall services ===
+      firewalld={{ k3s_service_facts.ansible_facts.services.get('firewalld.service', {}).get('state', 'not-found') }}
+      nftables={{ k3s_service_facts.ansible_facts.services.get('nftables.service', {}).get('state', 'not-found') }}
+      ufw={{ k3s_service_facts.ansible_facts.services.get('ufw.service', {}).get('state', 'not-found') }}
+
+      === iptables -S INPUT ===
+      rc={{ k3s_input_firewall_rules.rc }}
+      {{ k3s_input_firewall_rules.stdout }}
+      {{ k3s_input_firewall_rules.stderr }}
+
+      === primary API ping ===
+      rc={{ k3s_primary_api_ping.rc }}
+      {{ k3s_primary_api_ping.stdout }}
+      {{ k3s_primary_api_ping.stderr }}
+
+      === primary API probe ===
+      failed={{ k3s_primary_api_probe.failed | default(false) }}
+      elapsed={{ k3s_primary_api_probe.elapsed | default('unknown') }}
+      msg={{ k3s_primary_api_probe.msg | default('connected') }}

roles/k3s_server/tasks/http_proxy.yml

@@ -1,18 +1,16 @@
 ---
-
 - name: Create k3s.service.d directory
-  file:
+  ansible.builtin.file:
-    path: '{{ systemd_dir }}/k3s.service.d'
+    path: "{{ systemd_dir }}/k3s.service.d"
     state: directory
     owner: root
     group: root
-    mode: '0755'
+    mode: "0755"
-
 
 - name: Copy K3s http_proxy conf file
-  template:
+  ansible.builtin.template:
-    src: "http_proxy.conf.j2"
+    src: http_proxy.conf.j2
     dest: "{{ systemd_dir }}/k3s.service.d/http_proxy.conf"
     owner: root
     group: root
-    mode: '0755'
+    mode: "0755"

roles/k3s_server/tasks/join_master.yml

@@ -0,0 +1,34 @@
+---
+- name: Verify primary Kubernetes API reachability from {{ joining_master }}
+  ansible.builtin.wait_for:
+    host: >-
+      {{ hostvars[groups[group_name_master | default('master')][0]].k3s_node_ip
+      | split(',') | first }}
+    port: 6443
+    connect_timeout: 2
+    timeout: 30
+  delegate_to: "{{ joining_master }}"
+
+- name: Join transient k3s-init service for {{ joining_master }}
+  ansible.builtin.command:
+    cmd: >-
+      systemd-run -p RestartSec=2 -p Restart=on-failure
+      -p Delegate=yes -p TasksMax=infinity -p KillMode=process
+      -p LimitNOFILE=1048576 -p LimitNPROC=infinity -p LimitCORE=infinity
+      --unit=k3s-init
+      k3s server {{ hostvars[joining_master].k3s_server_init_args }}
+    creates: "{{ systemd_dir }}/k3s-init.service"
+  delegate_to: "{{ joining_master }}"
+  no_log: true
+
+- name: Wait for primary registration of {{ joining_master }}
+  ansible.builtin.command:
+    cmd: >-
+      {{ k3s_kubectl_binary | default('k3s kubectl') }} get node
+      {{ hostvars[joining_master].ansible_hostname }}
+  delegate_to: "{{ groups[group_name_master | default('master')][0] }}"
+  register: joined_master
+  until: joined_master.rc == 0
+  retries: "{{ retry_count | default(20) }}"
+  delay: 2
+  changed_when: false

roles/k3s_server/tasks/kube-vip.yml

@@ -0,0 +1,27 @@
+---
+- name: Create manifests directory on first master
+  ansible.builtin.file:
+    path: /var/lib/rancher/k3s/server/manifests
+    state: directory
+    owner: root
+    group: root
+    mode: "0644"
+  when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
+
+- name: Download vip cloud provider manifest to first master
+  ansible.builtin.get_url:
+    url: https://raw.githubusercontent.com/kube-vip/kube-vip-cloud-provider/{{ kube_vip_cloud_provider_tag_version | default('v0.0.12') }}/manifest/kube-vip-cloud-controller.yaml # noqa yaml[line-length]
+    dest: /var/lib/rancher/k3s/server/manifests/kube-vip-cloud-controller.yaml
+    owner: root
+    group: root
+    mode: "0644"
+  when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
+
+- name: Copy kubevip configMap manifest to first master
+  ansible.builtin.template:
+    src: kubevip.yaml.j2
+    dest: /var/lib/rancher/k3s/server/manifests/kubevip.yaml
+    owner: root
+    group: root
+    mode: "0644"
+  when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']

roles/k3s_server/tasks/main.yml

@@ -1,159 +1,234 @@
 ---
-
 - name: Stop k3s-init
-  systemd:
+  ansible.builtin.systemd:
     name: k3s-init
     state: stopped
   failed_when: false
 
-- name: Clean previous runs of k3s-init  # noqa command-instead-of-module
+# k3s-init won't work if the port is already in use
+- name: Stop k3s
+  ansible.builtin.systemd:
+    name: k3s
+    state: stopped
+  failed_when: false
+
+- name: Clean previous runs of k3s-init # noqa command-instead-of-module
   # The systemd module does not support "reset-failed", so we need to resort to command.
-  command: systemctl reset-failed k3s-init
+  ansible.builtin.command: systemctl reset-failed k3s-init
   failed_when: false
   changed_when: false
 
 - name: Deploy K3s http_proxy conf
-  include_tasks: http_proxy.yml
+  ansible.builtin.include_tasks: http_proxy.yml
   when: proxy_env is defined
 
 - name: Deploy vip manifest
-  include_tasks: vip.yml
+  ansible.builtin.include_tasks: vip.yml
-
 - name: Deploy metallb manifest
-  include_tasks: metallb.yml
+  ansible.builtin.include_tasks: metallb.yml
   tags: metallb
+  when: kube_vip_lb_ip_range is not defined and (not cilium_bgp or cilium_iface is not defined)
 
-- name: Init cluster inside the transient k3s-init service
+- name: Deploy kube-vip manifest
-  command:
+  ansible.builtin.include_tasks: kube-vip.yml
-    cmd: "systemd-run -p RestartSec=2 \
+  tags: kubevip
-                      -p Restart=on-failure \
+  when: kube_vip_lb_ip_range is defined
-                      --unit=k3s-init \
-                      k3s server {{ server_init_args }}"
-    creates: "{{ systemd_dir }}/k3s.service"
 
-- name: Verification
+- name: Initialize and verify the K3s control plane
+  any_errors_fatal: true
   when: not ansible_check_mode
   block:
-    - name: Verify that all nodes actually joined (check k3s-init.service if this fails)
+    - name: Materialize per-host server initialization arguments
-      command:
+      ansible.builtin.set_fact:
-        cmd: k3s kubectl get nodes -l "node-role.kubernetes.io/master=true" -o=jsonpath="{.items[*].metadata.name}"
+        k3s_server_init_args: "{{ server_init_args }}"
-      register: nodes
+      no_log: true
-      until: nodes.rc == 0 and (nodes.stdout.split() | length) == (groups[group_name_master | default('master')] | length)  # yamllint disable-line rule:line-length
+      when: groups[group_name_master | default('master')] | length > 1
-      retries: "{{ retry_count | default(20) }}"
+
-      delay: 10
+    - name: Orchestrate control-plane initialization from the first master
-      changed_when: false
+      when: inventory_hostname == groups[group_name_master | default('master')][0]
+      block:
+        - name: Init the first master inside the transient k3s-init service
+          ansible.builtin.command:
+            cmd: >-
+              systemd-run -p RestartSec=2 -p Restart=on-failure
+              -p Delegate=yes -p TasksMax=infinity -p KillMode=process
+              -p LimitNOFILE=1048576 -p LimitNPROC=infinity -p LimitCORE=infinity
+              --unit=k3s-init k3s server {{ server_init_args }}
+            creates: "{{ systemd_dir }}/k3s-init.service"
+          no_log: true
+
+        - name: Wait for the first master Kubernetes API
+          ansible.builtin.command:
+            cmd: "{{ k3s_kubectl_binary | default('k3s kubectl') }} get --raw=/readyz"
+          register: first_master_api
+          until: first_master_api.rc == 0
+          retries: "{{ retry_count | default(20) }}"
+          delay: 2
+          changed_when: false
+
+        - name: Verify the first master API listener on its node address
+          ansible.builtin.wait_for:
+            host: >-
+              {{ hostvars[groups[group_name_master | default('master')][0]].k3s_node_ip
+              | split(',') | first }}
+            port: 6443
+            connect_timeout: 2
+            timeout: "{{ retry_count | default(20) * 2 }}"
+
+        - name: Join additional masters one at a time
+          ansible.builtin.include_tasks: join_master.yml
+          loop: "{{ groups[group_name_master | default('master')][1:] }}"
+          loop_control:
+            loop_var: joining_master
+          when: groups[group_name_master | default('master')] | length > 1
+
+        - name: Verify that all expected control-plane nodes registered
+          ansible.builtin.command:
+            cmd: >-
+              {{ k3s_kubectl_binary | default('k3s kubectl') }} get nodes
+              -o=jsonpath='{.items[*].metadata.name}'
+          register: nodes
+          until:
+            - nodes.rc == 0
+            - expected_control_plane_names | difference(nodes.stdout.split()) | length == 0
+          retries: "{{ retry_count | default(20) }}"
+          delay: 2
+          changed_when: false
+          vars:
+            expected_control_plane_names: >-
+              {{
+                groups[group_name_master | default('master')]
+                | map('extract', hostvars, 'ansible_hostname')
+                | list
+              }}
+
+        - name: Wait for K3s bootstrap CRDs to become established
+          ansible.builtin.command:
+            cmd: >-
+              {{ k3s_kubectl_binary | default('k3s kubectl') }} wait
+              --for=condition=Established --timeout=5s
+              crd/addons.k3s.cattle.io
+              crd/helmcharts.helm.cattle.io
+              crd/helmchartconfigs.helm.cattle.io
+          register: bootstrap_crds
+          until: bootstrap_crds.rc == 0
+          retries: "{{ retry_count | default(20) }}"
+          delay: 2
+          changed_when: false
   always:
     - name: Save logs of k3s-init.service
-      include_tasks: fetch_k3s_init_logs.yml
+      ansible.builtin.include_tasks: fetch_k3s_init_logs.yml
       when: log_destination
       vars:
         log_destination: >-
           {{ lookup('ansible.builtin.env', 'ANSIBLE_K3S_LOG_DIR', default=False) }}
     - name: Kill the temporary service used for initialization
-      systemd:
+      ansible.builtin.systemd:
         name: k3s-init
         state: stopped
       failed_when: false
 
 - name: Copy K3s service file
   register: k3s_service
-  template:
+  ansible.builtin.template:
-    src: "k3s.service.j2"
+    src: k3s.service.j2
     dest: "{{ systemd_dir }}/k3s.service"
     owner: root
     group: root
-    mode: 0644
+    mode: "0644"
 
 - name: Enable and check K3s service
-  systemd:
+  ansible.builtin.systemd:
     name: k3s
-    daemon_reload: yes
+    daemon_reload: true
     state: restarted
-    enabled: yes
+    enabled: true
 
 - name: Wait for node-token
-  wait_for:
+  ansible.builtin.wait_for:
     path: /var/lib/rancher/k3s/server/node-token
 
 - name: Register node-token file access mode
-  stat:
+  ansible.builtin.stat:
     path: /var/lib/rancher/k3s/server
   register: p
 
 - name: Change file access node-token
-  file:
+  ansible.builtin.file:
     path: /var/lib/rancher/k3s/server
-    mode: "g+rx,o+rx"
+    mode: g+rx,o+rx
 
 - name: Read node-token from master
-  slurp:
+  ansible.builtin.slurp:
     src: /var/lib/rancher/k3s/server/node-token
   register: node_token
 
 - name: Store Master node-token
-  set_fact:
+  ansible.builtin.set_fact:
     token: "{{ node_token.content | b64decode | regex_replace('\n', '') }}"
 
 - name: Restore node-token file access
-  file:
+  ansible.builtin.file:
     path: /var/lib/rancher/k3s/server
     mode: "{{ p.stat.mode }}"
 
 - name: Create directory .kube
-  file:
+  ansible.builtin.file:
     path: "{{ ansible_user_dir }}/.kube"
     state: directory
     owner: "{{ ansible_user_id }}"
-    mode: "u=rwx,g=rx,o="
+    mode: u=rwx,g=rx,o=
 
 - name: Copy config file to user home directory
-  copy:
+  ansible.builtin.copy:
     src: /etc/rancher/k3s/k3s.yaml
     dest: "{{ ansible_user_dir }}/.kube/config"
-    remote_src: yes
+    remote_src: true
     owner: "{{ ansible_user_id }}"
-    mode: "u=rw,g=,o="
+    mode: u=rw,g=,o=
 
 - name: Configure kubectl cluster to {{ endpoint_url }}
-  command: >-
+  ansible.builtin.command: >-
-    k3s kubectl config set-cluster default
+    {{ k3s_kubectl_binary | default('k3s kubectl') }} config set-cluster default
       --server={{ endpoint_url }}
       --kubeconfig {{ ansible_user_dir }}/.kube/config
   changed_when: true
   vars:
     endpoint_url: >-
       https://{{ apiserver_endpoint | ansible.utils.ipwrap }}:6443
-  # Deactivated linter rules:
+# Deactivated linter rules:
-  #   - jinja[invalid]: As of version 6.6.0, ansible-lint complains that the input to ipwrap
+#   - jinja[invalid]: As of version 6.6.0, ansible-lint complains that the input to ipwrap
-  #                     would be undefined. This will not be the case during playbook execution.
+#                     would be undefined. This will not be the case during playbook execution.
-  # noqa jinja[invalid]
+# noqa jinja[invalid]
 
 - name: Create kubectl symlink
-  file:
+  ansible.builtin.file:
     src: /usr/local/bin/k3s
     dest: /usr/local/bin/kubectl
     state: link
+  when: k3s_create_kubectl_symlink | default(true) | bool
 
 - name: Create crictl symlink
-  file:
+  ansible.builtin.file:
     src: /usr/local/bin/k3s
     dest: /usr/local/bin/crictl
     state: link
+  when: k3s_create_crictl_symlink | default(true) | bool
 
 - name: Get contents of manifests folder
-  find:
+  ansible.builtin.find:
     paths: /var/lib/rancher/k3s/server/manifests
     file_type: file
   register: k3s_server_manifests
 
 - name: Get sub dirs of manifests folder
-  find:
+  ansible.builtin.find:
     paths: /var/lib/rancher/k3s/server/manifests
     file_type: directory
   register: k3s_server_manifests_directories
 
 - name: Remove manifests and folders that are only needed for bootstrapping cluster so k3s doesn't auto apply on start
-  file:
+  ansible.builtin.file:
     path: "{{ item.path }}"
     state: absent
   with_items:

roles/k3s_server/tasks/metallb.yml

@@ -1,30 +1,56 @@
 ---
 - name: Create manifests directory on first master
-  file:
+  ansible.builtin.file:
     path: /var/lib/rancher/k3s/server/manifests
     state: directory
     owner: root
     group: root
-    mode: 0644
+    mode: "0644"
   when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
 
 - name: "Download to first master: manifest for metallb-{{ metal_lb_type }}"
   ansible.builtin.get_url:
-    url: "https://raw.githubusercontent.com/metallb/metallb/{{ metal_lb_controller_tag_version }}/config/manifests/metallb-{{ metal_lb_type }}.yaml"  # noqa yaml[line-length]
+    url: https://raw.githubusercontent.com/metallb/metallb/{{ metal_lb_controller_tag_version }}/config/manifests/metallb-{{ metal_lb_type }}.yaml # noqa yaml[line-length]
-    dest: "/var/lib/rancher/k3s/server/manifests/metallb-crds.yaml"
+    dest: /var/lib/rancher/k3s/server/manifests/metallb-crds.yaml
     owner: root
     group: root
-    mode: 0644
+    mode: "0644"
   when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
 
 - name: Set image versions in manifest for metallb-{{ metal_lb_type }}
   ansible.builtin.replace:
-    path: "/var/lib/rancher/k3s/server/manifests/metallb-crds.yaml"
+    path: /var/lib/rancher/k3s/server/manifests/metallb-crds.yaml
     regexp: "{{ item.change | ansible.builtin.regex_escape }}"
     replace: "{{ item.to }}"
   with_items:
-    - change: "metallb/speaker:{{ metal_lb_controller_tag_version }}"
+    - change: metallb/speaker:{{ metal_lb_controller_tag_version }}
-      to: "metallb/speaker:{{ metal_lb_speaker_tag_version }}"
+      to: metallb/speaker:{{ metal_lb_speaker_tag_version }}
   loop_control:
     label: "{{ item.change }} => {{ item.to }}"
   when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
+
+- name: Read back MetalLB manifest from first master
+  ansible.builtin.slurp:
+    src: /var/lib/rancher/k3s/server/manifests/metallb-crds.yaml
+  register: metallb_manifest
+  when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
+
+- name: Check the MetalLB speaker image reference is present
+  ansible.builtin.set_fact:
+    metallb_manifest_has_speaker_tag: >-
+      {{
+        ('quay.io/metallb/speaker:' ~ metal_lb_speaker_tag_version) in
+        (metallb_manifest.content | default('') | b64decode)
+      }}
+  when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
+
+- name: Fail if MetalLB speaker tag was not applied to the manifest
+  ansible.builtin.fail:
+    msg: >-
+      MetalLB speaker image reference
+      quay.io/metallb/speaker:{{ metal_lb_speaker_tag_version }}
+      was not found in the downloaded manifest. The upstream image
+      reference may have changed.
+  when:
+    - ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
+    - not metallb_manifest_has_speaker_tag

roles/k3s_server/tasks/vip.yml

@@ -1,27 +1,31 @@
 ---
+- name: Set _kube_vip_bgp_peers fact
+  ansible.builtin.set_fact:
+    _kube_vip_bgp_peers: "{{ lookup('community.general.merge_variables', '^kube_vip_bgp_peers__.+$', initial_value=kube_vip_bgp_peers, groups=kube_vip_bgp_peers_groups) }}"  # yamllint disable-line rule:line-length
+
 - name: Create manifests directory on first master
-  file:
+  ansible.builtin.file:
     path: /var/lib/rancher/k3s/server/manifests
     state: directory
     owner: root
     group: root
-    mode: 0644
+    mode: "0644"
   when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
 
-- name: Download vip rbac manifest to first master
+- name: Copy kube-vip RBAC manifest to first master
-  ansible.builtin.get_url:
+  ansible.builtin.template:
-    url: "https://raw.githubusercontent.com/kube-vip/kube-vip/{{ kube_vip_tag_version }}/docs/manifests/rbac.yaml"
+    src: kube-vip-rbac.yaml.j2
-    dest: "/var/lib/rancher/k3s/server/manifests/vip-rbac.yaml"
+    dest: /var/lib/rancher/k3s/server/manifests/vip-rbac.yaml
     owner: root
     group: root
-    mode: 0644
+    mode: "0644"
   when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
 
 - name: Copy vip manifest to first master
-  template:
+  ansible.builtin.template:
-    src: "vip.yaml.j2"
+    src: vip.yaml.j2
-    dest: "/var/lib/rancher/k3s/server/manifests/vip.yaml"
+    dest: /var/lib/rancher/k3s/server/manifests/vip.yaml
     owner: root
     group: root
-    mode: 0644
+    mode: "0644"
   when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']

roles/k3s_server/templates/kube-vip-rbac.yaml.j2

@@ -0,0 +1,48 @@
+# Repository-owned kube-vip RBAC derived from the official v1.2.2
+# `kube-vip manifest rbac` output (run with `--inCluster`).
+# Source: https://github.com/kube-vip/kube-vip/blob/v1.2.2/pkg/kubevip/config_generator.go
+# Permissions are kept exactly as broad as the upstream manifest.
+apiVersion: v1
+kind: ServiceAccount
+metadata:
+  name: kube-vip
+  namespace: kube-system
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRole
+metadata:
+  annotations:
+    rbac.authorization.kubernetes.io/autoupdate: "true"
+  name: system:kube-vip-role
+rules:
+  - apiGroups: [""]
+    resources: ["services/status"]
+    verbs: ["update"]
+  - apiGroups: [""]
+    resources: ["services", "endpoints"]
+    verbs: ["list", "get", "watch", "update"]
+  - apiGroups: [""]
+    resources: ["nodes"]
+    verbs: ["list", "get", "watch", "update", "patch"]
+  - apiGroups: ["coordination.k8s.io"]
+    resources: ["leases"]
+    verbs: ["list", "get", "watch", "update", "create"]
+  - apiGroups: ["discovery.k8s.io"]
+    resources: ["endpointslices"]
+    verbs: ["list", "get", "watch", "update"]
+  - apiGroups: [""]
+    resources: ["pods"]
+    verbs: ["list"]
+---
+apiVersion: rbac.authorization.k8s.io/v1
+kind: ClusterRoleBinding
+metadata:
+  name: system:kube-vip-binding
+roleRef:
+  apiGroup: rbac.authorization.k8s.io
+  kind: ClusterRole
+  name: system:kube-vip-role
+subjects:
+  - kind: ServiceAccount
+    name: kube-vip
+    namespace: kube-system

roles/k3s_server/templates/kubevip.yaml.j2

@@ -0,0 +1,13 @@
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: kubevip
+  namespace: kube-system
+data:
+{% if kube_vip_lb_ip_range is string %}
+{# kube_vip_lb_ip_range was used in the legacy way: single string instead of a list #}
+{#   => transform to list with single element #}
+{% set kube_vip_lb_ip_range = [kube_vip_lb_ip_range] %}
+{% endif %}
+  range-global: {{ kube_vip_lb_ip_range | join(',') }}

roles/k3s_server/templates/vip.yaml.j2

@@ -27,14 +27,16 @@ spec:
         - manager
         env:
         - name: vip_arp
-          value: "true"
+          value: "{{ 'true' if kube_vip_arp | default(true) | bool else 'false' }}"
+        - name: bgp_enable
+          value: "{{ 'true' if kube_vip_bgp | default(false) | bool else 'false' }}"
         - name: port
           value: "6443"
 {% if kube_vip_iface %}
         - name: vip_interface
           value: {{ kube_vip_iface }}
 {% endif %}
-        - name: vip_cidr
+        - name: vip_subnet
           value: "{{ apiserver_endpoint | ansible.utils.ipsubnet | ansible.utils.ipaddr('prefix') }}"
         - name: cp_enable
           value: "true"
@@ -43,7 +45,7 @@ spec:
         - name: vip_ddns
           value: "false"
         - name: svc_enable
-          value: "false"
+          value: "{{ 'true' if kube_vip_lb_ip_range is defined else 'false' }}"
         - name: vip_leaderelection
           value: "true"
         - name: vip_leaseduration
@@ -54,6 +56,29 @@ spec:
           value: "2"
         - name: address
           value: {{ apiserver_endpoint }}
+{% if kube_vip_bgp | default(false) | bool %}
+{% if kube_vip_bgp_routerid is defined %}
+        - name: bgp_routerid
+          value: "{{ kube_vip_bgp_routerid }}"
+{% endif %}
+{% if _kube_vip_bgp_peers | length > 0 %}
+        - name: bgp_peers
+          value: "{{ _kube_vip_bgp_peers | map(attribute='peer_address') | zip(_kube_vip_bgp_peers | map(attribute='peer_asn')) | map('join', ':') | join(',') }}"  # yamllint disable-line rule:line-length
+{% else %}
+{% if kube_vip_bgp_as is defined %}
+        - name: bgp_as
+          value: "{{ kube_vip_bgp_as }}"
+{% endif %}
+{% if kube_vip_bgp_peeraddress is defined %}
+        - name: bgp_peeraddress
+          value: "{{ kube_vip_bgp_peeraddress }}"
+{% endif %}
+{% if kube_vip_bgp_peeras is defined %}
+        - name: bgp_peeras
+          value: "{{ kube_vip_bgp_peeras }}"
+{% endif %}
+{% endif %}
+{% endif %}
         image: ghcr.io/kube-vip/kube-vip:{{ kube_vip_tag_version }}
         imagePullPolicy: Always
         name: kube-vip

roles/k3s_server_post/defaults/main.yml

@@ -1,6 +1,34 @@
 ---
-# Timeout to wait for MetalLB services to come up
+k3s_kubectl_binary: k3s kubectl
-metal_lb_available_timeout: 120s
 
-# Name of the master group
+bpf_lb_algorithm: maglev
+bpf_lb_mode: hybrid
+
+calico_blockSize: 26  # noqa var-naming
+calico_ebpf: false
+calico_encapsulation: VXLANCrossSubnet
+calico_natOutgoing: Enabled  # noqa var-naming
+calico_nodeSelector: all()  # noqa var-naming
+calico_tag: v3.32.1
+
+cilium_bgp: false
+cilium_exportPodCIDR: true  # noqa var-naming
+cilium_bgp_my_asn: 64513
+cilium_bgp_peer_asn: 64512
+cilium_bgp_neighbors: []
+cilium_bgp_neighbors_groups: ['k3s_all']
+cilium_bgp_lb_cidr: 192.168.31.0/24
+cilium_hubble: true
+cilium_mode: native
+cilium_tag: v1.20.0
+cilium_cli_tag: v0.19.7
+
+cluster_cidr: 10.52.0.0/16
+enable_bpf_masquerade: true
+kube_proxy_replacement: true
 group_name_master: master
+
+metal_lb_mode: layer2
+metal_lb_available_timeout: 240s
+metal_lb_controller_tag_version: v0.16.0
+metal_lb_ip_range: 192.168.30.80-192.168.30.90

roles/k3s_server_post/meta/main.yml

@@ -0,0 +1,165 @@
+---
+argument_specs:
+  main:
+    short_description: Configure k3s cluster
+    options:
+      apiserver_endpoint:
+        description: Virtual ip-address configured on each master
+        required: true
+
+      bpf_lb_algorithm:
+        description: BPF lb algorithm
+        default: maglev
+
+      bpf_lb_mode:
+        description: BPF lb mode
+        default: hybrid
+
+      calico_blockSize:
+        description: IP pool block size
+        type: int
+        default: 26
+
+      calico_ebpf:
+        description: Use eBPF dataplane instead of iptables
+        type: bool
+        default: false
+
+      calico_encapsulation:
+        description: IP pool encapsulation
+        default: VXLANCrossSubnet
+
+      calico_natOutgoing:
+        description: IP pool NAT outgoing
+        default: Enabled
+
+      calico_nodeSelector:
+        description: IP pool node selector
+        default: all()
+
+      calico_iface:
+        description: The network interface used for when Calico is enabled
+        default: ~
+
+      calico_tag:
+        description: Calico version tag
+        default: v3.32.1
+
+      cilium_bgp:
+        description:
+          - Enable cilium BGP control plane for LB services and pod cidrs.
+          - Disables the use of MetalLB.
+        type: bool
+        default: false
+
+      cilium_bgp_my_asn:
+        description: Local ASN for BGP peer
+        type: int
+        default: 64513
+
+      cilium_bgp_peer_asn:
+        description: BGP peer ASN
+        type: int
+        default: 64512
+
+      cilium_bgp_peer_address:
+        description: BGP peer address
+        default: ~
+
+      cilium_bgp_neighbors:
+        description: List of BGP peer ASN & address pairs
+        default: []
+
+      cilium_bgp_neighbors_groups:
+        description: Inventory group in which to search for additional cilium_bgp_neighbors parameters to merge.
+        default: ['k3s_all']
+
+      cilium_bgp_lb_cidr:
+        description: BGP load balancer IP range
+        default: 192.168.31.0/24
+
+      cilium_exportPodCIDR:
+        description: Export pod CIDR
+        type: bool
+        default: true
+
+      cilium_hubble:
+        description: Enable Cilium Hubble
+        type: bool
+        default: true
+
+      cilium_iface:
+        description: The network interface used for when Cilium is enabled
+        default: ~
+
+      cilium_mode:
+        description:
+          - Inner-node communication mode.
+          - Cilium accepts `native` or `tunnel`. `routed` is accepted as a
+            legacy alias and is mapped to `tunnel` at install time.
+        default: native
+        choices:
+          - native
+          - tunnel
+          - routed
+
+      cilium_tag:
+        description: Cilium version tag
+        default: v1.20.0
+
+      cilium_cli_tag:
+        description: Cilium CLI version tag
+        default: v0.19.7
+
+      cluster_cidr:
+        description: Inner-cluster IP range
+        default: 10.52.0.0/16
+
+      enable_bpf_masquerade:
+        description: Use IP masquerading
+        type: bool
+        default: true
+
+      group_name_master:
+        description: Name of the master group
+        default: master
+
+      kube_proxy_replacement:
+        description: Replace the native kube-proxy with Cilium
+        type: bool
+        default: true
+
+      kube_vip_lb_ip_range:
+        description: IP range for kube-vip load balancer
+        default: ~
+
+      metal_lb_available_timeout:
+        description: Wait for MetalLB resources
+        default: 240s
+
+      metal_lb_ip_range:
+        description: MetalLB ip range for load balancer
+        default: 192.168.30.80-192.168.30.90
+
+      metal_lb_controller_tag_version:
+        description: Image tag for MetalLB
+        default: v0.16.0
+
+      metal_lb_mode:
+        description: Metallb mode
+        default: layer2
+        choices:
+          - bgp
+          - layer2
+
+      metal_lb_bgp_my_asn:
+        description: BGP ASN configurations
+        default: ~
+
+      metal_lb_bgp_peer_asn:
+        description: BGP peer ASN configurations
+        default: ~
+
+      metal_lb_bgp_peer_address:
+        description: BGP peer address
+        default: ~

roles/k3s_server_post/tasks/calico.yml

@@ -0,0 +1,163 @@
+---
+- name: Deploy Calico to cluster
+  when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
+  run_once: true
+  block:
+    - name: Create manifests directory on first master
+      ansible.builtin.file:
+        path: /tmp/k3s
+        state: directory
+        owner: root
+        group: root
+        mode: "0755"
+
+    - name: "Download to first master: Calico CRD bundle for {{ calico_tag }}"
+      ansible.builtin.get_url:
+        url: https://raw.githubusercontent.com/projectcalico/calico/{{ calico_tag }}/manifests/v1_crd_projectcalico_org.yaml # noqa yaml[line-length]
+        dest: /tmp/k3s/v1_crd_projectcalico_org.yaml
+        owner: root
+        group: root
+        mode: "0755"
+
+    - name: "Download to first master: manifest for Tigera Operator and Calico CRDs"
+      ansible.builtin.get_url:
+        url: https://raw.githubusercontent.com/projectcalico/calico/{{ calico_tag }}/manifests/tigera-operator.yaml
+        dest: /tmp/k3s/tigera-operator.yaml
+        owner: root
+        group: root
+        mode: "0755"
+
+    - name: Apply Calico CRD bundle with server-side apply
+      ansible.builtin.command: >-
+        {{ k3s_kubectl_binary | default('k3s kubectl') }}
+        apply --server-side --force-conflicts
+        -f /tmp/k3s/v1_crd_projectcalico_org.yaml
+      register: apply_crds
+      changed_when: >-
+        'created' in apply_crds.stdout or
+        'configured' in apply_crds.stdout or
+        'applied' in apply_crds.stdout
+      failed_when: apply_crds.rc != 0
+
+    - name: Apply Tigera Operator manifest idempotently with server-side apply
+      ansible.builtin.command: >-
+        {{ k3s_kubectl_binary | default('k3s kubectl') }}
+        apply --server-side --force-conflicts
+        -f /tmp/k3s/tigera-operator.yaml
+      register: apply_operator
+      changed_when: >-
+        'created' in apply_operator.stdout or
+        'configured' in apply_operator.stdout or
+        'applied' in apply_operator.stdout
+      failed_when: apply_operator.rc != 0
+
+    - name: Wait for Tigera Operator and managed CRDs to become available
+      ansible.builtin.command: >-
+        {{ k3s_kubectl_binary | default('k3s kubectl') }} wait {{ item.type }}/{{ item.name }}
+        {% if item.namespace is defined %}--namespace='{{ item.namespace }}'{% endif %}
+        --for=condition={{ item.condition }}
+        --timeout=30s
+      register: tigera_result
+      changed_when: false
+      until: tigera_result is succeeded
+      retries: 7
+      delay: 7
+      with_items:
+        - { name: tigera-operator, type: deployment, namespace: tigera-operator, condition: Available=True }
+        - { name: installations.operator.tigera.io, type: crd, condition: Established }
+        - { name: apiservers.operator.tigera.io, type: crd, condition: Established }
+      loop_control:
+        label: "{{ item.type }}/{{ item.name }}"
+
+    - name: Copy Calico custom resources manifest to first master
+      ansible.builtin.template:
+        src: calico.crs.j2
+        dest: /tmp/k3s/custom-resources.yaml
+        owner: root
+        group: root
+        mode: "0755"
+
+    - name: Apply Calico custom resources
+      ansible.builtin.command: >-
+        {{ k3s_kubectl_binary | default('k3s kubectl') }}
+        apply -f /tmp/k3s/custom-resources.yaml
+      register: apply_cr
+      changed_when: >-
+        'configured' in apply_cr.stdout or
+        'created' in apply_cr.stdout or
+        'unchanged' in apply_cr.stdout
+      failed_when: apply_cr.rc != 0
+
+    - name: Wait for Calico system resources to be available
+      ansible.builtin.command: >-
+        {% if item.type == 'daemonset' %}
+        {{ k3s_kubectl_binary | default('k3s kubectl') }} wait pods
+        --namespace='{{ item.namespace }}'
+        --selector={{ item.selector }}
+        --for=condition=Ready
+        {% else %}
+        {{ k3s_kubectl_binary | default('k3s kubectl') }} wait {{ item.type }}/{{ item.name }}
+        --namespace='{{ item.namespace }}'
+        --for=condition=Available
+        {% endif %}
+        --timeout=30s
+      register: cr_result
+      changed_when: false
+      until: cr_result is succeeded
+      retries: 60
+      delay: 10
+      with_items:
+        - { name: calico-typha, type: deployment, namespace: calico-system }
+        - { name: calico-kube-controllers, type: deployment, namespace: calico-system }
+        - name: csi-node-driver
+          type: daemonset
+          selector: k8s-app=csi-node-driver
+          namespace: calico-system
+        - name: calico-node
+          type: daemonset
+          selector: k8s-app=calico-node
+          namespace: calico-system
+      loop_control:
+        label: "{{ item.type }}/{{ item.name }}"
+
+    # The Calico API server is an optional add-on for managing Calico policy via
+    # kubectl (the projectcalico.org/v3 Kubernetes API). It is not required for
+    # Calico CNI data plane operation and does not reconcile on K3s with recent
+    # Calico releases, so it is treated as best-effort here.
+    - name: Wait for the optional Calico API server (best effort)
+      ansible.builtin.command: >-
+        {{ k3s_kubectl_binary | default('k3s kubectl') }} wait deployment/calico-apiserver
+        --namespace=calico-apiserver --for=condition=Available --timeout=30s
+      register: cr_apiserver
+      changed_when: false
+      until: cr_apiserver is succeeded
+      retries: 30
+      delay: 10
+      ignore_errors: true
+
+    - name: Verify Calico TigeraStatus reports Available
+      ansible.builtin.command: >-
+        {{ k3s_kubectl_binary | default('k3s kubectl') }}
+        get tigerastatus {{ item }}
+        -o jsonpath='{.status.conditions[?(@.type=="Available")].status}'
+      register: tigera_status
+      changed_when: false
+      until: tigera_status.stdout | trim == 'True'
+      retries: 30
+      delay: 7
+      failed_when: tigera_status.rc != 0 or (tigera_status.stdout | trim) != 'True'
+      with_items:
+        - calico
+      loop_control:
+        label: "tigerastatus/{{ item }}"
+
+    - name: Patch Felix configuration for eBPF mode
+      ansible.builtin.command:
+        cmd: >
+          {{ k3s_kubectl_binary | default('k3s kubectl') }} patch felixconfiguration default
+          --type='merge'
+          --patch='{"spec": {"bpfKubeProxyIptablesCleanupEnabled": false}}'
+      register: patch_result
+      changed_when: "'felixconfiguration.projectcalico.org/default patched' in patch_result.stdout"
+      failed_when: "'Error' in patch_result.stderr"
+      when: calico_ebpf

roles/k3s_server_post/tasks/cilium.yml

@@ -0,0 +1,275 @@
+---
+- name: Prepare Cilium CLI on first master and deploy CNI
+  when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
+  run_once: true
+  block:
+    - name: Create tmp directory on first master
+      ansible.builtin.file:
+        path: /tmp/k3s
+        state: directory
+        owner: root
+        group: root
+        mode: "0755"
+
+    - name: Check if Cilium CLI is installed
+      ansible.builtin.command: cilium version
+      register: cilium_cli_installed
+      failed_when: false
+      changed_when: false
+      ignore_errors: true
+
+    - name: Check for Cilium CLI version in command output
+      ansible.builtin.set_fact:
+        installed_cli_version: >-
+          {{
+            cilium_cli_installed.stdout_lines
+            | join(' ')
+            | regex_findall('cilium-cli: (v\d+\.\d+\.\d+)')
+            | first
+            | default('unknown')
+          }}
+      when: cilium_cli_installed.rc == 0
+
+    - name: Log installed Cilium CLI version
+      ansible.builtin.debug:
+        msg: "Installed Cilium CLI version: {{ installed_cli_version | default('Not installed') }}"
+
+    - name: Log pinned Cilium CLI version
+      ansible.builtin.debug:
+        msg: "Pinned Cilium CLI version: {{ cilium_cli_tag }}"
+
+    - name: Determine if Cilium CLI needs installation or update
+      ansible.builtin.set_fact:
+        cilium_cli_needs_update: >-
+          {{
+            cilium_cli_installed.rc != 0 or
+            (cilium_cli_installed.rc == 0 and
+            installed_cli_version != cilium_cli_tag)
+          }}
+
+    - name: Install or update Cilium CLI
+      when: cilium_cli_needs_update
+      block:
+        - name: Set architecture variable
+          ansible.builtin.set_fact:
+            cli_arch: "{{ 'arm64' if ansible_architecture == 'aarch64' else 'amd64' }}"
+
+        - name: Download Cilium CLI and checksum
+          ansible.builtin.get_url:
+            url: "{{ cilium_base_url }}/cilium-linux-{{ cli_arch }}{{ item }}"
+            dest: /tmp/k3s/cilium-linux-{{ cli_arch }}{{ item }}
+            owner: root
+            group: root
+            mode: "0755"
+          loop:
+            - .tar.gz
+            - .tar.gz.sha256sum
+          vars:
+            cilium_base_url: https://github.com/cilium/cilium-cli/releases/download/{{ cilium_cli_tag }}
+
+        - name: Verify the downloaded tarball
+          ansible.builtin.shell: |
+            cd /tmp/k3s && sha256sum --check cilium-linux-{{ cli_arch }}.tar.gz.sha256sum
+          args:
+            executable: /bin/bash
+          changed_when: false
+
+        - name: Extract Cilium CLI to /usr/local/bin
+          ansible.builtin.unarchive:
+            src: /tmp/k3s/cilium-linux-{{ cli_arch }}.tar.gz
+            dest: /usr/local/bin
+            remote_src: true
+
+        - name: Remove downloaded tarball and checksum file
+          ansible.builtin.file:
+            path: "{{ item }}"
+            state: absent
+          loop:
+            - /tmp/k3s/cilium-linux-{{ cli_arch }}.tar.gz
+            - /tmp/k3s/cilium-linux-{{ cli_arch }}.tar.gz.sha256sum
+
+    - name: Wait for connectivity to kube VIP
+      ansible.builtin.command: ping -c 1 {{ apiserver_endpoint }}
+      register: ping_result
+      until: ping_result.rc == 0
+      retries: 21
+      delay: 1
+      ignore_errors: true
+      changed_when: false
+
+    - name: Fail if kube VIP not reachable
+      ansible.builtin.fail:
+        msg: API endpoint {{ apiserver_endpoint }} is not reachable
+      when: ping_result.rc != 0
+
+    - name: Test for existing Cilium install
+      ansible.builtin.command: |
+        {{ k3s_kubectl_binary | default('k3s kubectl') }} -n kube-system get daemonsets cilium
+      register: cilium_installed
+      failed_when: false
+      changed_when: false
+      ignore_errors: true
+
+    - name: Check existing Cilium install
+      when: cilium_installed.rc == 0
+      block:
+        - name: Check Cilium version
+          ansible.builtin.command: cilium version
+          register: cilium_version
+          failed_when: false
+          changed_when: false
+          ignore_errors: true
+
+        - name: Parse installed Cilium version
+          ansible.builtin.set_fact:
+            installed_cilium_version: >-
+              {{
+                cilium_version.stdout_lines
+                | join(' ')
+                | regex_findall('cilium image.+(\d+\.\d+\.\d+)')
+                | first
+                | default('unknown')
+              }}
+
+        - name: Determine if Cilium needs update
+          ansible.builtin.set_fact:
+            cilium_needs_update: >-
+              {{ 'v' + installed_cilium_version != cilium_tag }}
+
+        - name: Log result
+          ansible.builtin.debug:
+            msg: >
+              Installed Cilium version: {{ installed_cilium_version }},
+              Target Cilium version: {{ cilium_tag }},
+              Update needed: {{ cilium_needs_update }}
+
+    - name: Map the legacy routed mode to Cilium tunnel mode
+      ansible.builtin.set_fact:
+        # Cilium 1.20 accepts `native` or `tunnel`. `routed` was the name used
+        # by older releases and is kept only as a backward-compatible alias.
+        cilium_routing_mode: >-
+          {{ 'tunnel' if cilium_mode == 'routed' else cilium_mode }}
+
+    - name: Install Cilium
+      ansible.builtin.command: >-
+        {% if cilium_installed.rc != 0 %}
+        cilium install
+        {% else %}
+        cilium upgrade
+        {% endif %}
+        --version "{{ cilium_tag }}"
+        --helm-set operator.replicas="1"
+        {{ '--helm-set devices=' + cilium_iface if cilium_iface != 'auto' else '' }}
+        --helm-set ipam.operator.clusterPoolIPv4PodCIDRList={{ cluster_cidr }}
+        {% if cilium_routing_mode == "native" or (cilium_bgp and cilium_exportPodCIDR != 'false') %}
+        --helm-set ipv4NativeRoutingCIDR={{ cluster_cidr }}
+        {% endif %}
+        --helm-set k8sServiceHost="127.0.0.1"
+        --helm-set k8sServicePort="6444"
+        --helm-set routingMode={{ cilium_routing_mode }}
+        --helm-set autoDirectNodeRoutes={{ "true" if cilium_routing_mode == "native" else "false" }}
+        --helm-set kubeProxyReplacement={{ kube_proxy_replacement }}
+        --helm-set bpf.masquerade={{ enable_bpf_masquerade }}
+        --helm-set bgpControlPlane.enabled={{ cilium_bgp | default("false") }}
+        --helm-set hubble.enabled={{ "true" if cilium_hubble else "false" }}
+        --helm-set hubble.relay.enabled={{ "true" if cilium_hubble else "false" }}
+        --helm-set hubble.ui.enabled={{ "true" if cilium_hubble else "false" }}
+        {% if kube_proxy_replacement is not false %}
+        --helm-set loadBalancer.algorithm={{ bpf_lb_algorithm }}
+        --helm-set loadBalancer.mode={{ bpf_lb_mode }}
+        {% endif %}
+      environment:
+        KUBECONFIG: "{{ ansible_user_dir }}/.kube/config"
+      register: cilium_install_result
+      changed_when: cilium_install_result.rc == 0
+      when: cilium_installed.rc != 0 or cilium_needs_update
+
+    - name: Wait for Cilium resources
+      ansible.builtin.command: >-
+        {% if item.type == 'daemonset' %}
+        {{ k3s_kubectl_binary | default('k3s kubectl') }} wait pods
+        --namespace=kube-system
+        --selector='k8s-app=cilium'
+        --for=condition=Ready
+        {% else %}
+        {{ k3s_kubectl_binary | default('k3s kubectl') }} wait {{ item.type }}/{{ item.name }}
+        --namespace=kube-system
+        --for=condition=Available
+        {% endif %}
+        --timeout=30s
+      register: cr_result
+      changed_when: false
+      until: cr_result is succeeded
+      retries: 30
+      delay: 7
+      with_items:
+        - { name: cilium-operator, type: deployment }
+        - { name: cilium, type: daemonset, selector: k8s-app=cilium }
+        - { name: hubble-relay, type: deployment, check_hubble: true }
+        - { name: hubble-ui, type: deployment, check_hubble: true }
+      loop_control:
+        label: "{{ item.type }}/{{ item.name }}"
+      when: >-
+        not item.check_hubble | default(false) or (item.check_hubble | default(false) and cilium_hubble)
+
+    - name: Wait for Cilium status to be healthy
+      ansible.builtin.command: cilium status --wait
+      environment:
+        KUBECONFIG: "{{ ansible_user_dir }}/.kube/config"
+      register: cilium_status
+      changed_when: false
+      until: cilium_status.rc == 0
+      retries: 30
+      delay: 7
+
+    - name: Configure Cilium BGP
+      when: cilium_bgp
+      block:
+        - name: Set _cilium_bgp_neighbors fact
+          ansible.builtin.set_fact:
+            _cilium_bgp_neighbors: "{{ lookup('community.general.merge_variables', '^cilium_bgp_neighbors__.+$', initial_value=cilium_bgp_neighbors, groups=cilium_bgp_neighbors_groups) }}"  # yamllint disable-line rule:line-length
+
+        - name: Copy BGP manifests to first master
+          ansible.builtin.template:
+            src: cilium.crs.j2
+            dest: /tmp/k3s/cilium-bgp.yaml
+            owner: root
+            group: root
+            mode: "0755"
+
+        - name: Preflight validate rendered BGP manifests against installed CRDs
+          ansible.builtin.command: >-
+            {{ k3s_kubectl_binary | default('k3s kubectl') }}
+            apply --dry-run=server -f /tmp/k3s/cilium-bgp.yaml
+          register: preflight_cr
+          changed_when: false
+          failed_when: preflight_cr.rc != 0
+
+        - name: Apply BGP manifests
+          ansible.builtin.command: >-
+            {{ k3s_kubectl_binary | default('k3s kubectl') }}
+            apply -f /tmp/k3s/cilium-bgp.yaml
+          register: apply_cr
+          changed_when: "'configured' in apply_cr.stdout or 'created' in apply_cr.stdout"
+          failed_when: apply_cr.rc != 0
+
+        - name: Remove deprecated CiliumBGPPeeringPolicy after v2 resources are accepted
+          ansible.builtin.command: >-
+            {{ k3s_kubectl_binary | default('k3s kubectl') }}
+            delete CiliumBGPPeeringPolicy.cilium.io 01-bgp-peering-policy
+          register: delete_old_policy
+          changed_when: "'deleted' in delete_old_policy.stdout"
+          # The policy (and possibly its CRD) may already be absent; this is
+          # intentionally tolerated, not a command whose failure must be hidden.
+          failed_when: false
+
+        - name: Test for BGP config resources
+          ansible.builtin.command: "{{ item }}"
+          loop:
+            - "{{ k3s_kubectl_binary | default('k3s kubectl') }} get CiliumBGPClusterConfig.cilium.io"
+            - "{{ k3s_kubectl_binary | default('k3s kubectl') }} get CiliumBGPPeerConfig.cilium.io"
+            - "{{ k3s_kubectl_binary | default('k3s kubectl') }} get CiliumBGPAdvertisement.cilium.io"
+            - "{{ k3s_kubectl_binary | default('k3s kubectl') }} get CiliumLoadBalancerIPPool.cilium.io"
+          changed_when: false
+          loop_control:
+            label: "{{ item }}"

roles/k3s_server_post/tasks/main.yml

@@ -1,9 +1,20 @@
 ---
+- name: Deploy calico
+  ansible.builtin.include_tasks: calico.yml
+  tags: calico
+  when: calico_iface is defined and cilium_iface is not defined
+
+- name: Deploy cilium
+  ansible.builtin.include_tasks: cilium.yml
+  tags: cilium
+  when: cilium_iface is defined
+
 - name: Deploy metallb pool
-  include_tasks: metallb.yml
+  ansible.builtin.include_tasks: metallb.yml
   tags: metallb
+  when: kube_vip_lb_ip_range is not defined and (not cilium_bgp or cilium_iface is not defined)
 
 - name: Remove tmp directory used for manifests
-  file:
+  ansible.builtin.file:
     path: /tmp/k3s
     state: absent

roles/k3s_server_post/tasks/metallb.yml

@@ -1,32 +1,53 @@
 ---
 - name: Create manifests directory for temp configuration
-  file:
+  ansible.builtin.file:
     path: /tmp/k3s
     state: directory
     owner: "{{ ansible_user_id }}"
-    mode: 0755
+    mode: "0755"
   with_items: "{{ groups[group_name_master | default('master')] }}"
   run_once: true
 
+- name: Delete outdated metallb replicas
+  ansible.builtin.shell: |-
+    set -o pipefail
+
+    REPLICAS=$({{ k3s_kubectl_binary | default('k3s kubectl') }} --namespace='metallb-system' get replicasets \
+      -l 'component=controller,app=metallb' \
+      -o jsonpath='{.items[0].spec.template.spec.containers[0].image}, {.items[0].metadata.name}' 2>/dev/null || true)
+    REPLICAS_SETS=$(echo ${REPLICAS} | grep -v '{{ metal_lb_controller_tag_version }}' | sed -e "s/^.*\s//g")
+    if [ -n "${REPLICAS_SETS}" ] ; then
+      for REPLICAS in "${REPLICAS_SETS}"
+      do
+        {{ k3s_kubectl_binary | default('k3s kubectl') }} --namespace='metallb-system' \
+          delete rs "${REPLICAS}"
+      done
+    fi
+  args:
+    executable: /bin/bash
+  changed_when: false
+  run_once: true
+  with_items: "{{ groups[group_name_master | default('master')] }}"
+
 - name: Copy metallb CRs manifest to first master
-  template:
+  ansible.builtin.template:
-    src: "metallb.crs.j2"
+    src: metallb.crs.j2
-    dest: "/tmp/k3s/metallb-crs.yaml"
+    dest: /tmp/k3s/metallb-crs.yaml
     owner: "{{ ansible_user_id }}"
-    mode: 0755
+    mode: "0755"
   with_items: "{{ groups[group_name_master | default('master')] }}"
   run_once: true
 
 - name: Test metallb-system namespace
-  command: >-
+  ansible.builtin.command: >-
-    k3s kubectl -n metallb-system
+    {{ k3s_kubectl_binary | default('k3s kubectl') }} -n metallb-system
   changed_when: false
   with_items: "{{ groups[group_name_master | default('master')] }}"
   run_once: true
 
 - name: Wait for MetalLB resources
-  command: >-
+  ansible.builtin.command: >-
-    k3s kubectl wait {{ item.resource }}
+    {{ k3s_kubectl_binary | default('k3s kubectl') }} wait {{ item.resource }}
     --namespace='metallb-system'
     {% if item.name | default(False) -%}{{ item.name }}{%- endif %}
     {% if item.selector | default(False) -%}--selector='{{ item.selector }}'{%- endif %}
@@ -62,16 +83,22 @@
   loop_control:
     label: "{{ item.description }}"
 
+- name: Set metallb webhook service name
+  ansible.builtin.set_fact:
+    # Inspected the v0.16.0 manifest: the newer webhook Service name is used
+    # by every supported MetalLB release, so the old pre-0.14.4 branch is gone.
+    metallb_webhook_service_name: metallb-webhook-service
+
 - name: Test metallb-system webhook-service endpoint
-  command: >-
+  ansible.builtin.command: >-
-    k3s kubectl -n metallb-system get endpoints webhook-service
+    {{ k3s_kubectl_binary | default('k3s kubectl') }} -n metallb-system get endpoints {{ metallb_webhook_service_name }}
   changed_when: false
   with_items: "{{ groups[group_name_master | default('master')] }}"
   run_once: true
 
 - name: Apply metallb CRs
-  command: >-
+  ansible.builtin.command: >-
-    k3s kubectl apply -f /tmp/k3s/metallb-crs.yaml
+    {{ k3s_kubectl_binary | default('k3s kubectl') }} apply -f /tmp/k3s/metallb-crs.yaml
     --timeout='{{ metal_lb_available_timeout }}'
   register: this
   changed_when: false
@@ -80,8 +107,8 @@
   retries: 5
 
 - name: Test metallb-system resources for Layer 2 configuration
-  command: >-
+  ansible.builtin.command: >-
-    k3s kubectl -n metallb-system get {{ item }}
+    {{ k3s_kubectl_binary | default('k3s kubectl') }} -n metallb-system get {{ item }}
   changed_when: false
   run_once: true
   when: metal_lb_mode == "layer2"
@@ -90,8 +117,8 @@
     - L2Advertisement
 
 - name: Test metallb-system resources for BGP configuration
-  command: >-
+  ansible.builtin.command: >-
-    k3s kubectl -n metallb-system get {{ item }}
+    {{ k3s_kubectl_binary | default('k3s kubectl') }} -n metallb-system get {{ item }}
   changed_when: false
   run_once: true
   when: metal_lb_mode == "bgp"

roles/k3s_server_post/templates/calico.crs.j2

@@ -0,0 +1,41 @@
+# This section includes base Calico installation configuration.
+# For more information, see: https://docs.tigera.io/calico/latest/reference/installation/api#operator.tigera.io/v1.Installation
+apiVersion: operator.tigera.io/v1
+kind: Installation
+metadata:
+  name: default
+spec:
+  # Configures Calico networking.
+  calicoNetwork:
+    # Note: The ipPools section cannot be modified post-install.
+    ipPools:
+    - blockSize: {{ calico_blockSize }}
+      cidr: {{ cluster_cidr }}
+      encapsulation: {{ calico_encapsulation }}
+      natOutgoing: {{ calico_natOutgoing }}
+      nodeSelector: {{ calico_nodeSelector }}
+    nodeAddressAutodetectionV4:
+      interface: {{ calico_iface  }}
+    linuxDataplane: {{ 'BPF' if calico_ebpf else 'Iptables' }}
+
+---
+
+# This section configures the Calico API server.
+# For more information, see: https://docs.tigera.io/calico/latest/reference/installation/api#operator.tigera.io/v1.APIServer
+apiVersion: operator.tigera.io/v1
+kind: APIServer
+metadata:
+  name: default
+spec: {}
+
+{% if calico_ebpf %}
+---
+kind: ConfigMap
+apiVersion: v1
+metadata:
+  name: kubernetes-services-endpoint
+  namespace: tigera-operator
+data:
+  KUBERNETES_SERVICE_HOST: '{{ apiserver_endpoint }}'
+  KUBERNETES_SERVICE_PORT: '6443'
+{% endif %}

roles/k3s_server_post/templates/cilium.crs.j2

@@ -0,0 +1,92 @@
+# Cilium BGP Control Plane v2 resources.
+# Replace the deprecated v2alpha1 CiliumBGPPeeringPolicy that was removed
+# in Cilium 1.19.
+{% set _cilium_default_peer = {'peer_address': cilium_bgp_peer_address, 'peer_asn': cilium_bgp_peer_asn} %}
+{% set _cilium_peers = _cilium_bgp_neighbors if _cilium_bgp_neighbors | length > 0 else [_cilium_default_peer] %}
+apiVersion: cilium.io/v2
+kind: CiliumBGPPeerConfig
+metadata:
+  name: cilium-peer
+spec:
+  # Matches the timers and multihop used by the previous v2alpha1 policy.
+  ebgpMultihop: 10
+  timers:
+    connectRetryTimeSeconds: 120
+    holdTimeSeconds: 90
+    keepAliveTimeSeconds: 30
+  gracefulRestart:
+    enabled: true
+    restartTimeSeconds: 120
+  families:
+    - afi: ipv4
+      safi: unicast
+      advertisements:
+        matchLabels:
+          advertise: "bgp"
+---
+apiVersion: cilium.io/v2
+kind: CiliumBGPClusterConfig
+metadata:
+  name: cilium-bgp
+spec:
+  # Explicitly select every node so the BGP instance runs across the cluster.
+  nodeSelector:
+    matchExpressions:
+      - key: somekey
+        operator: NotIn
+        values: ['never-used-value']
+  bgpInstances:
+    - name: "instance-{{ cilium_bgp_my_asn }}"
+      localASN: {{ cilium_bgp_my_asn }}
+      peers:
+{% for peer in _cilium_peers %}
+        - name: "peer-{{ peer.peer_asn }}-{{ loop.index }}"
+          peerASN: {{ peer.peer_asn }}
+          peerAddress: {{ peer.peer_address }}
+          peerConfigRef:
+            name: cilium-peer
+{% endfor %}
+{% if cilium_exportPodCIDR %}
+---
+apiVersion: cilium.io/v2
+kind: CiliumBGPAdvertisement
+metadata:
+  name: cilium-pod-cidrs
+  labels:
+    advertise: "bgp"
+spec:
+  advertisements:
+    - advertisementType: "PodCIDR"
+{% endif %}
+---
+apiVersion: cilium.io/v2
+kind: CiliumBGPAdvertisement
+metadata:
+  name: cilium-lb-services
+  labels:
+    advertise: "bgp"
+spec:
+  advertisements:
+    - advertisementType: "Service"
+      service:
+        addresses:
+          - LoadBalancerIP
+      # Advertise all Services carrying an ingress address from the pool.
+      selector:
+        matchExpressions:
+          - key: somekey
+            operator: NotIn
+            values: ['never-used-value']
+---
+apiVersion: cilium.io/v2
+kind: CiliumLoadBalancerIPPool
+metadata:
+  name: "01-lb-pool"
+spec:
+  blocks:
+{% if "/" in cilium_bgp_lb_cidr %}
+  - cidr: {{ cilium_bgp_lb_cidr }}
+{% else %}
+  - start: {{ cilium_bgp_lb_cidr.split('-')[0] }}
+    stop: {{ cilium_bgp_lb_cidr.split('-')[1] }}
+{% endif %}

roles/lxc/handlers/main.yml

@@ -1,5 +1,6 @@
 ---
 - name: Reboot server
   become: true
-  reboot:
+  ansible.builtin.reboot:
+    reboot_command: "{{ custom_reboot_command | default(omit) }}"
   listen: reboot server

roles/lxc/meta/main.yml

@@ -0,0 +1,8 @@
+---
+argument_specs:
+  main:
+    short_description: Configure LXC
+    options:
+      custom_reboot_command:
+        default: ~
+        description: Command to run on reboot

roles/lxc/tasks/main.yml

@@ -1,20 +1,20 @@
 ---
 - name: Check for rc.local file
-  stat:
+  ansible.builtin.stat:
     path: /etc/rc.local
   register: rcfile
 
 - name: Create rc.local if needed
-  lineinfile:
+  ansible.builtin.lineinfile:
     path: /etc/rc.local
     line: "#!/bin/sh -e"
     create: true
     insertbefore: BOF
-    mode: "u=rwx,g=rx,o=rx"
+    mode: u=rwx,g=rx,o=rx
   when: not rcfile.stat.exists
 
 - name: Write rc.local file
-  blockinfile:
+  ansible.builtin.blockinfile:
     path: /etc/rc.local
     content: "{{ lookup('template', 'templates/rc.local.j2') }}"
     state: present

roles/prereq/defaults/main.yml

@@ -1,4 +1,4 @@
 ---
 secure_path:
-  RedHat: '/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin'
+  RedHat: /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
-  Suse: '/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin'
+  Suse: /usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin

roles/prereq/meta/main.yml

@@ -0,0 +1,7 @@
+---
+argument_specs:
+  main:
+    short_description: Prerequisites
+    options:
+      system_timezone:
+        description: Timezone to be set on all nodes

roles/prereq/tasks/main.yml

@@ -14,7 +14,7 @@
     name: net.ipv4.ip_forward
     value: "1"
     state: present
-    reload: yes
+    reload: true
   tags: sysctl
 
 - name: Enable IPv6 forwarding
@@ -22,7 +22,7 @@
     name: net.ipv6.conf.all.forwarding
     value: "1"
     state: present
-    reload: yes
+    reload: true
   tags: sysctl
 
 - name: Enable IPv6 router advertisements
@@ -30,14 +30,14 @@
     name: net.ipv6.conf.all.accept_ra
     value: "2"
     state: present
-    reload: yes
+    reload: true
   tags: sysctl
 
 - name: Add br_netfilter to /etc/modules-load.d/
-  copy:
+  ansible.builtin.copy:
-    content: "br_netfilter"
+    content: br_netfilter
     dest: /etc/modules-load.d/br_netfilter.conf
-    mode: "u=rw,g=,o="
+    mode: u=rw,g=,o=
   when: ansible_os_family == "RedHat"
 
 - name: Load br_netfilter
@@ -51,7 +51,7 @@
     name: "{{ item }}"
     value: "1"
     state: present
-    reload: yes
+    reload: true
   when: ansible_os_family == "RedHat"
   loop:
     - net.bridge.bridge-nf-call-iptables
@@ -59,11 +59,11 @@
   tags: sysctl
 
 - name: Add /usr/local/bin to sudo secure_path
-  lineinfile:
+  ansible.builtin.lineinfile:
-    line: 'Defaults    secure_path = {{ secure_path[ansible_os_family] }}'
+    line: Defaults    secure_path = {{ secure_path[ansible_os_family] }}
-    regexp: "Defaults(\\s)*secure_path(\\s)*="
+    regexp: Defaults(\s)*secure_path(\s)*=
     state: present
     insertafter: EOF
     path: /etc/sudoers
-    validate: 'visudo -cf %s'
+    validate: visudo -cf %s
   when: ansible_os_family in [ "RedHat", "Suse" ]

roles/proxmox_lxc/handlers/main.yml

@@ -2,12 +2,12 @@
 - name: Reboot containers
   block:
     - name: Get container ids from filtered files
-      set_fact:
+      ansible.builtin.set_fact:
         proxmox_lxc_filtered_ids: >-
           {{ proxmox_lxc_filtered_files | map("split", "/") | map("last") | map("split", ".") | map("first") }}
       listen: reboot containers
     - name: Reboot container
-      command: "pct reboot {{ item }}"
+      ansible.builtin.command: pct reboot {{ item }}
       loop: "{{ proxmox_lxc_filtered_ids }}"
       changed_when: true
       listen: reboot containers