forked from tim/k3s-ansible
feat(dependencies): upgrade supported cluster components (#681)
* fix(calico): support split CRDs for current releases - Download the v1_crd_projectcalico_org.yaml bundle before the operator - Apply both files with server-side apply and force-conflicts per the upstream upgrade procedure - Wait for the operator Deployment and for the managed CRDs to be Established after the operator starts - Replace the create/rescue/replace flow with an idempotent apply that no longer conceals partial failures - Verify TigeraStatus for calico and apiserver is Available, not just that Pods exist * feat(dependencies): upgrade supported cluster components - Bump K3s to v1.36.2+k3s1, Calico to v3.32.1, Cilium to v1.20.0, kube-vip to v1.2.2, kube-vip cloud provider to v0.0.12, and MetalLB to v0.16.0 across sample inventory, role defaults, and argument specs - Pin the Cilium CLI with a new cilium_cli_tag (v0.19.7) instead of the floating stable.txt lookup - Replace the CiliumBGPPeeringPolicy v2alpha1 BGP template with the v2 CiliumBGPClusterConfig, CiliumBGPPeerConfig, CiliumBGPAdvertisement, and CiliumLoadBalancerIPPool resource set - Move Cilium load balancer Helm keys from bpf.loadBalancer to the valid top-level loadBalancer path - Add preflight schema validation and remove the deprecated policy after the v2 objects are accepted - Wait for cilium status after installation - Pin kube-vip RBAC in a repository template instead of fetching a mutable URL, and include EndpointSlice permissions - Fix the kube-vip bgppeers format to address:ASN comma-separated peers - Fail clearly when the MetalLB speaker tag replacement does not apply - Drop the obsolete MetalLB webhook service name version branch * test(molecule): verify upgraded cluster components - Assert every node reports the expected K3s kubelet version - Verify the active CNI (Flannel / Calico / Cilium) is Ready and runs the expected image tag, including Calico TigeraStatus Available - Verify the active load balancer (MetalLB / kube-vip) runs the expected image tags and that MetalLB is absent when kube-vip is active - Assert no Flannel DaemonSet remains when Calico or Cilium is enabled - Assert the example LoadBalancer address falls inside the configured pool range - Add a manifest-only Cilium BGP regression test that renders the v2 template with zero, one, and multiple neighbors and rejects any v2alpha1 or CiliumBGPPeeringPolicy output * fix(dependencies): correct dependency version pins - Set the sample kube-vip image to v1.2.2 and repair the damaged comment - Pin the kube-vip cloud provider default to v0.0.12 in the task URL - Set the MetalLB controller argument-spec default to v0.16.0 - Restore the MetalLB available timeout default to 240s * docs(dependencies): document current cluster versions - Update kube-vip, kube-vip cloud provider, and MetalLB defaults - Add cilium_tag and cilium_cli_tag rows - Explain that MetalLB v0.16.0 is the application image target even though a newer chart-only tag (metallb-chart-0.16.1) exists - Add an existing-cluster upgrade warning covering the K3s etcd 3.5.26 bridge and one-minor-at-a-time rule, consecutive Cilium minor upgrades, Calico v3 resource UID handling, and MetalLB app vs chart tags * fix(dependencies): address PR review findings - Read the MetalLB speaker tag check from the managed host with slurp instead of a controller-side file lookup, and match the full image reference - Restore the tigera-operator namespace on the Calico operator Deployment wait while keeping the managed CRD waits cluster-scoped - Make Molecule verify inputs durable and scenario-specific via a per-scenario verify-vars.yml, driven by explicit verify_cni/verify_lb values instead of non-persisted converge facts - Rename the kube-vip multi-peer BGP env var from bgppeers to bgp_peers and vip_cidr to vip_subnet so v1.2.2 actually reads them - Map the legacy Cilium routed mode to tunnel and stop passing the alias directly to the chart - Use return-code based failed_when on apply and preflight commands so non-error failures are no longer treated as success - Clarify the sequential K3s upgrade path and backups in the README - Add kube-vip and MetalLB regression tests and a Cilium mode mapping unit * fix(dependencies): resolve re-review findings - correct the Calico TigeraStatus resource kind\n- document tunnel as the supported Cilium routing mode\n- validate load balancer addresses across range and CIDR pools * fix(molecule): verify embedded flannel instead of a flannel DaemonSet - K3s 1.36 runs flannel embedded in the k3s agent rather than as a kube-flannel-ds DaemonSet, so the flannel verifier queried a workload that no longer exists and failed the verify step - For the flannel scenarios, assert every node is Ready and that neither the Calico nor the Cilium namespace exists - Drop the now-invalid kube-flannel-ds DaemonSet assertion * fix(molecule): wait for the LoadBalancer address before asserting reachability - The nginx LoadBalancer service had no ingress address when the reachability assertion ran, so status.loadBalancer.ingress[0].ip was undefined and the ipwrap filter failed during verify - Poll the service until MetalLB or kube-vip assigns an external IP - Record the assigned address once and reuse it for the reachability probe and the pool membership checks * fix(ci): harden calico apiserver wait and extend molecule job timeout - Bump calico system resources wait retries 30->60 and delay 7->10 so the slow-to-reconcile calico-apiserver deployment has enough time under nested-virt - Raise the molecule step timeout-minutes from 90 to 150 to accommodate contended 5-node scenarios (cilium, kube-vip) that were hitting the 90-min cap * fix(calico): treat optional API server as best-effort on converge - The Calico API server (calico-apiserver) is an optional add-on for managing Calico policy through the projectcalico.org/v3 Kubernetes API; it is not required for Calico CNI data plane operation - With Calico v3.32.1 on K3s 1.36 the tigera-operator never provisions the calico-apiserver namespace, causing the converge wait to fail deterministically - Keep the strict wait for core Calico components (typha, kube-controllers, calico-node, csi-node-driver) and make the API server wait tolerate failure - Restrict the TigeraStatus Available check to the calico status, matching the upstream v3.32.1 K3s quickstart which validates without the API server
This commit is contained in:
Techno Tim
GitHub
@@ -0,0 +1,134 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Render the Cilium BGP CRD template and assert it uses the v2 API.
|
||||
|
||||
This is a manifest-only regression test used where no real BGP peer is
|
||||
available. It renders roles/k3s_server_post/templates/cilium.crs.j2 with
|
||||
zero, one, and multiple neighbors, then checks that the output:
|
||||
- never contains CiliumBGPPeeringPolicy or cilium.io/v2alpha1
|
||||
- emits the Cilium v2 BGP resources
|
||||
- emits deterministic DNS-safe peer and instance names
|
||||
- advertises Pod CIDRs only when cilium_exportPodCIDR is true
|
||||
"""
|
||||
|
||||
from __future__ import print_function
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
|
||||
from jinja2 import Environment, FileSystemLoader, StrictUndefined
|
||||
|
||||
|
||||
def repo_root():
|
||||
return subprocess.check_output(
|
||||
["git", "rev-parse", "--show-toplevel"], text=True
|
||||
).strip()
|
||||
|
||||
|
||||
def fail(message):
|
||||
raise SystemExit("Cilium BGP manifest test failed: " + message)
|
||||
|
||||
|
||||
def render(env, extra_vars):
|
||||
base_vars = {
|
||||
"cilium_bgp_my_asn": "64513",
|
||||
"cilium_bgp_peer_asn": "64512",
|
||||
"cilium_bgp_peer_address": "192.168.30.1",
|
||||
"cilium_exportPodCIDR": True,
|
||||
"cilium_bgp_lb_cidr": "192.168.31.0/24",
|
||||
}
|
||||
base_vars.update(extra_vars)
|
||||
template = env.get_template("cilium.crs.j2")
|
||||
return template.render(**base_vars)
|
||||
|
||||
|
||||
def check_common(output):
|
||||
if "cilium.io/v2alpha1" in output:
|
||||
fail("rendered output still contains cilium.io/v2alpha1")
|
||||
if "kind: CiliumBGPPeeringPolicy" in output:
|
||||
fail("rendered output still contains CiliumBGPPeeringPolicy")
|
||||
for kind in (
|
||||
"CiliumBGPPeerConfig",
|
||||
"CiliumBGPClusterConfig",
|
||||
"CiliumBGPAdvertisement",
|
||||
"CiliumLoadBalancerIPPool",
|
||||
):
|
||||
if ("kind: " + kind) not in output:
|
||||
fail("rendered output is missing kind: " + kind)
|
||||
|
||||
|
||||
def main():
|
||||
root = repo_root()
|
||||
template_dir = os.path.join(
|
||||
root, "roles", "k3s_server_post", "templates"
|
||||
)
|
||||
env = Environment(
|
||||
loader=FileSystemLoader(template_dir), undefined=StrictUndefined
|
||||
)
|
||||
|
||||
# Zero neighbors -> fall back to the single default peer.
|
||||
output = render(env, {"_cilium_bgp_neighbors": []})
|
||||
check_common(output)
|
||||
if "peer-64512-1" not in output:
|
||||
fail("default single peer name was not rendered")
|
||||
if "peerAddress: 192.168.30.1" not in output:
|
||||
fail("default peer address was not rendered")
|
||||
if 'advertisementType: "PodCIDR"' not in output:
|
||||
fail("PodCIDR advertisement missing when exportPodCIDR is true")
|
||||
|
||||
# One neighbor via the merged list.
|
||||
output = render(
|
||||
env,
|
||||
{"_cilium_bgp_neighbors": [{"peer_address": "10.0.0.1", "peer_asn": "65001"}]},
|
||||
)
|
||||
check_common(output)
|
||||
if "peer-65001-1" not in output:
|
||||
fail("single merged peer name was not rendered")
|
||||
if "peerAddress: 10.0.0.1" not in output:
|
||||
fail("single merged peer address was not rendered")
|
||||
|
||||
# Multiple neighbors.
|
||||
output = render(
|
||||
env,
|
||||
{
|
||||
"_cilium_bgp_neighbors": [
|
||||
{"peer_address": "10.0.0.1", "peer_asn": "65001"},
|
||||
{"peer_address": "10.0.0.2", "peer_asn": "65002"},
|
||||
]
|
||||
},
|
||||
)
|
||||
check_common(output)
|
||||
if "peer-65001-1" not in output or "peer-65002-2" not in output:
|
||||
fail("multiple merged peer names were not rendered")
|
||||
if "peerAddress: 10.0.0.2" not in output:
|
||||
fail("second merged peer address was not rendered")
|
||||
|
||||
# exportPodCIDR false -> no PodCIDR advertisement, service remains.
|
||||
output = render(
|
||||
env, {"_cilium_bgp_neighbors": [], "cilium_exportPodCIDR": False}
|
||||
)
|
||||
check_common(output)
|
||||
if 'advertisementType: "PodCIDR"' in output:
|
||||
fail("PodCIDR advertisement present when exportPodCIDR is false")
|
||||
if 'advertisementType: "Service"' not in output:
|
||||
fail("Service advertisement missing when exportPodCIDR is false")
|
||||
|
||||
# Load balancer pools: CIDR and start/stop forms.
|
||||
output = render(env, {"_cilium_bgp_neighbors": []})
|
||||
if "cidr: 192.168.31.0/24" not in output:
|
||||
fail("CIDR load balancer pool was not rendered")
|
||||
output = render(
|
||||
env,
|
||||
{
|
||||
"_cilium_bgp_neighbors": [],
|
||||
"cilium_bgp_lb_cidr": "192.168.31.80-192.168.31.90",
|
||||
},
|
||||
)
|
||||
check_common(output)
|
||||
if "start: 192.168.31.80" not in output or "stop: 192.168.31.90" not in output:
|
||||
fail("start/stop load balancer pool was not rendered")
|
||||
|
||||
print("Cilium BGP manifest regression test passed")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
Reference in New Issue
Block a user