36 Commits

Author SHA1 Message Date
Techno Tim c82f2e0415 chore(deps): apply dependency updates in one combined change (#694)
* chore(deps): apply dependency updates in one combined change

- Bump ansible-core to 2.19.11 and jmespath to 1.1.0 in requirements.in
- Regenerate the Python 3.11 pip-compile lock in requirements.txt
- Bump molecule-plugins to 23.6.0 while keeping molecule on the stable 6.x
  series (avoids the molecule-plugins 26 major jump that broke vagrant module
  resolution in CI)
- Bump ruamel-yaml-clib to 0.2.15
- Bump the zgosalvez/github-actions-ensure-sha-pinned-actions action to 5.0.6
  (SHA-pinned) in lint.yml

* fix(server): make log_destination conditional boolean for ansible-core 2.19

- The always block's 'Save logs of k3s-init.service' task used when: log_destination
  where log_destination is a path string derived from an env var
- ansible-core 2.19 rejects string-derived conditionals; evaluate the path as a
  real boolean (non-empty) check so the conditional is a true boolean
- Required to keep the k3s_server role working with ansible-core 2.19.11 (the
  dependency bump in this change)

* fix(verify): coerce regex_search assertions to bool for ansible-core 2.19

- ansible-core 2.19 requires assert conditionals to be boolean; regex_search
  returns a string, which is now rejected
- Wrap all regex_search results used in assert.that with | bool so the calico,
  cilium, metallb, and kube-vip image-tag checks produce boolean results

* fix(verify): use boolean is regex_search test instead of | bool

- | bool on a regex_search result coerces a tag string like v0.16.0 to False
  in ansible-core 2.19, failing the image-tag assertions
- Use the is regex_search test which returns a real boolean without string
  coercion for the calico, cilium, metallb, and kube-vip image assertions

* fix(verify): use is not none for regex_search assertions

- ansible-core 2.19 has no "is regex_search" test and rejects bool string
  coercion, so use the regex_search filter with an "is not none" comparison,
  which yields a real boolean for the image-tag assertions
- Applies to calico, cilium, metallb, and kube-vip image checks

* fix(metallb): retry transient apiserver resets in config tests

- The Layer 2 and BGP final configuration checks ran a kubectl get per
  resource with no retry, so a transient connection refused from the kube
  API could abort converge
- Mirror the download_retries/download_delay retry pattern used by the
  'Wait for MetalLB resources' task so these checks survive api server
  resets on slow runners
2026-08-04 17:54:58 -05:00
Techno Tim f5483cdabe fix(flannel): default the interface to each host's default IPv4 interface (#689)
* fix(flannel): default the interface to each host's default IPv4 interface

- Replace the hardcoded flannel_iface: eth0 with a per-host default derived
  from ansible_facts.default_ipv4.interface
- KVM/cloud hosts that are not named eth0 (e.g. enp1s0, ens3) now resolve the
  interface automatically instead of failing the k3s_node_ip lookup
- Update the commented calico_iface / cilium_iface examples to match
- Co-authored-by: Fritz Dunkel <677609+FinalDoom@users.noreply.github.com>
- Fixes #621

* test(flannel): add regression test for per-host interface default

- Add a focused test that renders the sample inventory's flannel_iface
  expression against fake ansible facts
- Assert a non-eth0 host (enp1s0, ens3) resolves its own interface and that
  the expression defaults from ansible_facts.default_ipv4.interface
- Wire it as a local pre-commit hook (default-interface-test)

* test(molecule): assert nodes register a non-loopback InternalIP

- Add a flannel-scenario verify assertion that every node reports an
  InternalIP derived from its configured flannel_iface
- Guards against k3s binding to 127.0.0.1 instead of the cluster interface
- Complements the pre-commit default-interface test with an end-to-end check
2026-08-03 17:01:50 +00:00
Techno Tim db85fa960c fix(metallb): guard cilium_bgp when undefined; clarify apiserver_endpoint docs (#683)
* fix(metallb): guard cilium_bgp variable before evaluating

- The 'Deploy metallb manifest' and 'Deploy metallb pool' conditionals evaluate
  'not cilium_bgp' directly, which raises an undefined-variable error when the
  k3s_server role runs without cilium_bgp in scope and no Cilium variables are set
- Guard with 'cilium_bgp is not defined' so the condition resolves cleanly when
  Cilium BGP is not configured
- Fixes #644

* docs(apiserver): clarify apiserver_endpoint must be a free routable IP

- Note that apiserver_endpoint must be an unassigned, routable IP on the
  network and that it is exposed by kube-vip / MetalLB
- Fixes #678

* fix(molecule): wait for MetalLB resources before asserting images

- The MetalLB image-tag assertion crashed with 'list object has no element 0'
  when the controller Deployment was not yet observable at verify time
- Retry the MetalLB controller/speaker lookup until the resources appear
- Fail with a clear message if MetalLB is genuinely absent
2026-08-03 04:52:27 +00:00
Techno Tim bb006cf157 feat(dependencies): upgrade supported cluster components (#681)
* fix(calico): support split CRDs for current releases

- Download the v1_crd_projectcalico_org.yaml bundle before the operator
- Apply both files with server-side apply and force-conflicts per the
  upstream upgrade procedure
- Wait for the operator Deployment and for the managed CRDs to be
  Established after the operator starts
- Replace the create/rescue/replace flow with an idempotent apply that
  no longer conceals partial failures
- Verify TigeraStatus for calico and apiserver is Available, not just
  that Pods exist

* feat(dependencies): upgrade supported cluster components

- Bump K3s to v1.36.2+k3s1, Calico to v3.32.1, Cilium to v1.20.0,
  kube-vip to v1.2.2, kube-vip cloud provider to v0.0.12, and MetalLB to
  v0.16.0 across sample inventory, role defaults, and argument specs
- Pin the Cilium CLI with a new cilium_cli_tag (v0.19.7) instead of the
  floating stable.txt lookup
- Replace the CiliumBGPPeeringPolicy v2alpha1 BGP template with the
  v2 CiliumBGPClusterConfig, CiliumBGPPeerConfig, CiliumBGPAdvertisement,
  and CiliumLoadBalancerIPPool resource set
- Move Cilium load balancer Helm keys from bpf.loadBalancer to the valid
  top-level loadBalancer path
- Add preflight schema validation and remove the deprecated policy after
  the v2 objects are accepted
- Wait for cilium status after installation
- Pin kube-vip RBAC in a repository template instead of fetching a
  mutable URL, and include EndpointSlice permissions
- Fix the kube-vip bgppeers format to address:ASN comma-separated peers
- Fail clearly when the MetalLB speaker tag replacement does not apply
- Drop the obsolete MetalLB webhook service name version branch

* test(molecule): verify upgraded cluster components

- Assert every node reports the expected K3s kubelet version
- Verify the active CNI (Flannel / Calico / Cilium) is Ready and runs
  the expected image tag, including Calico TigeraStatus Available
- Verify the active load balancer (MetalLB / kube-vip) runs the expected
  image tags and that MetalLB is absent when kube-vip is active
- Assert no Flannel DaemonSet remains when Calico or Cilium is enabled
- Assert the example LoadBalancer address falls inside the configured
  pool range
- Add a manifest-only Cilium BGP regression test that renders the v2
  template with zero, one, and multiple neighbors and rejects any v2alpha1
  or CiliumBGPPeeringPolicy output

* fix(dependencies): correct dependency version pins

- Set the sample kube-vip image to v1.2.2 and repair the damaged comment
- Pin the kube-vip cloud provider default to v0.0.12 in the task URL
- Set the MetalLB controller argument-spec default to v0.16.0
- Restore the MetalLB available timeout default to 240s

* docs(dependencies): document current cluster versions

- Update kube-vip, kube-vip cloud provider, and MetalLB defaults
- Add cilium_tag and cilium_cli_tag rows
- Explain that MetalLB v0.16.0 is the application image target even though
  a newer chart-only tag (metallb-chart-0.16.1) exists
- Add an existing-cluster upgrade warning covering the K3s etcd 3.5.26
  bridge and one-minor-at-a-time rule, consecutive Cilium minor upgrades,
  Calico v3 resource UID handling, and MetalLB app vs chart tags

* fix(dependencies): address PR review findings

- Read the MetalLB speaker tag check from the managed host with slurp
  instead of a controller-side file lookup, and match the full image
  reference
- Restore the tigera-operator namespace on the Calico operator Deployment
  wait while keeping the managed CRD waits cluster-scoped
- Make Molecule verify inputs durable and scenario-specific via a
  per-scenario verify-vars.yml, driven by explicit verify_cni/verify_lb
  values instead of non-persisted converge facts
- Rename the kube-vip multi-peer BGP env var from bgppeers to bgp_peers
  and vip_cidr to vip_subnet so v1.2.2 actually reads them
- Map the legacy Cilium routed mode to tunnel and stop passing the alias
  directly to the chart
- Use return-code based failed_when on apply and preflight commands so
  non-error failures are no longer treated as success
- Clarify the sequential K3s upgrade path and backups in the README
- Add kube-vip and MetalLB regression tests and a Cilium mode mapping unit

* fix(dependencies): resolve re-review findings

- correct the Calico TigeraStatus resource kind\n- document tunnel as the supported Cilium routing mode\n- validate load balancer addresses across range and CIDR pools

* fix(molecule): verify embedded flannel instead of a flannel DaemonSet

- K3s 1.36 runs flannel embedded in the k3s agent rather than as a
  kube-flannel-ds DaemonSet, so the flannel verifier queried a workload
  that no longer exists and failed the verify step
- For the flannel scenarios, assert every node is Ready and that neither
  the Calico nor the Cilium namespace exists
- Drop the now-invalid kube-flannel-ds DaemonSet assertion

* fix(molecule): wait for the LoadBalancer address before asserting reachability

- The nginx LoadBalancer service had no ingress address when the
  reachability assertion ran, so status.loadBalancer.ingress[0].ip was
  undefined and the ipwrap filter failed during verify
- Poll the service until MetalLB or kube-vip assigns an external IP
- Record the assigned address once and reuse it for the reachability probe
  and the pool membership checks

* fix(ci): harden calico apiserver wait and extend molecule job timeout

- Bump calico system resources wait retries 30->60 and delay 7->10 so the
  slow-to-reconcile calico-apiserver deployment has enough time under nested-virt
- Raise the molecule step timeout-minutes from 90 to 150 to accommodate
  contended 5-node scenarios (cilium, kube-vip) that were hitting the 90-min cap

* fix(calico): treat optional API server as best-effort on converge

- The Calico API server (calico-apiserver) is an optional add-on for managing
  Calico policy through the projectcalico.org/v3 Kubernetes API; it is not
  required for Calico CNI data plane operation
- With Calico v3.32.1 on K3s 1.36 the tigera-operator never provisions the
  calico-apiserver namespace, causing the converge wait to fail deterministically
- Keep the strict wait for core Calico components (typha, kube-controllers,
  calico-node, csi-node-driver) and make the API server wait tolerate failure
- Restrict the TigeraStatus Available check to the calico status, matching the
  upstream v3.32.1 K3s quickstart which validates without the API server
2026-08-02 12:44:50 -05:00
Timothy Stewart b6363cdfc5 fix(molecule): use stable startup script path
- Resolve the bounded Vagrant startup helper relative to the playbook.
- Avoid relying on an unavailable Molecule project directory variable.
2026-08-01 19:13:48 -05:00
Timothy Stewart 83f205177d fix(molecule): always start guests in bounded batches
- Reuse bounded Vagrant startup batches when machine state already exists.
- Avoid falling back to a concurrent five-guest startup during retries.
- Keep instance reconciliation after all batches complete.
2026-08-01 19:13:48 -05:00
Timothy Stewart 5c288e8f3e fix(molecule): serialize guest fact gathering
- Avoid concurrent SSH startup bursts against the disposable guests.
- Ensure every guest gathers interface facts before network setup.
- Prevent unreachable hosts from falling back to the default eth0 interface.
2026-08-01 19:13:48 -05:00
Timothy Stewart dfcfbc1f3f fix(molecule): select distro-specific private interfaces
- Use eth1 for Ubuntu and Debian Bento guests when present.
- Keep enp0s8 for Rocky Linux guests and other images.
- Apply the selection consistently across all Molecule scenarios.
2026-08-01 19:13:48 -05:00
Timothy Stewart c3606a7847 fix(molecule): use Bento private interface name
- Replace legacy eth1 overrides with the enp0s8 private NIC used by Bento guests.\n- Keep flannel, Calico, Cilium, and kube-vip scenarios on the Vagrant private network.
2026-08-01 19:13:48 -05:00
Timothy Stewart 68d03acc68 test(molecule): update Vagrant boxes to current releases
- Use published Bento boxes for Ubuntu 26.04, Debian 13, and Rocky Linux 10.1.\n- Pin exact VirtualBox amd64 box versions in CI.\n- Remove the obsolete Ubuntu SSH workaround and update fixtures.
2026-08-01 19:13:48 -05:00
Timothy Stewart 10bde4eff0 feat(ci): optimize Molecule VM creation
- reuse validated runner-owned Vagrant linked-clone masters\n- create the default guests in bounded two-machine batches\n- capture create timing and runner utilization diagnostics
2026-08-01 12:27:12 -05:00
Timothy Stewart fb9a0bebd1 fix(molecule): stabilize host-only peer identity
- restore host-only routing required by outside verification
- assign deterministic adapter MACs to the five default guests
- pin disposable full-mesh neighbor entries from live Ansible facts
2026-08-01 12:27:12 -05:00
Timothy Stewart 4990157d35 fix(molecule): isolate default cluster network
- move the five-node cluster NICs to a named VirtualBox internal network
- preserve NAT connectivity for provisioning and downloads
- retain serialized creation and early neighbor identity validation
2026-08-01 12:27:12 -05:00
Timothy Stewart 73fae0826c fix(molecule): stabilize default scenario networking
- serialize five-node Vagrant creation to avoid VirtualBox host-only races
- refresh and verify the primary neighbor mapping before convergence
- include interface MAC details in failure diagnostics
2026-08-01 12:27:12 -05:00
Timothy Stewart 57a22e364d fix(k3s-server): harden isolated control-plane bootstrap
- pin kube-vip and cluster traffic to the private guest interface\n- disable disposable guest firewalls and verify API reachability before joins\n- keep control-plane orchestration on the primary and preserve failure diagnostics
2026-08-01 12:27:12 -05:00
Timothy Stewart 9b220c1629 fix(k3s-server): serialize master joins and capture diagnostics
- validate host-only connectivity before cluster bootstrap\n- materialize per-host join arguments and wait for sequential registration\n- collect bounded network diagnostics when k3s initialization fails
2026-08-01 12:27:12 -05:00
Timothy Stewart 87ea8160c1 fix(ci): harden Molecule download and cleanup
- Retry transient k3s release downloads with bounded backoff.\n- Bound failure diagnostics and validate guest release connectivity.\n- Discover repository-owned Molecule state under the actual project root.
2026-08-01 12:27:12 -05:00
Richard Holmboe b077a49e1f Change to FQCN with ansible-lint fixer (#553)
* Change to FQCN with ansible-lint fixer

Since ansible-base 2.10 (later ansible-core), FQCN is the new way to go.

Updated .ansible-lint with a production profile and removed fqcn in skip_list.
Updated .yamllint with rules needed.

Ran ansible-lint --fix=all, then manually applied some minor changes.

* Changed octal value in molecule/ipv6/prepare.yml
2024-08-12 22:59:59 -05:00
sholdee 6ffc25dfe5 Add Cilium CNI option (#435)
* Add Cilium CNI option

* Tweak version checks and add BGP resource verify

* Update metallb detection for kube-vip feat compat
2024-01-29 19:29:13 -06:00
Gereon Vey bcd37a6904 add kube-vip as a service load balancer (#432)
* add kube-vip as a service load balancer

* add molecule scenario kube-vip

---------

Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-29 09:13:13 -06:00
sholdee 0f23e7e258 Add Calico CNI option (#414)
* Add Tigera Operator/Calico CNI option

Small tweak to reduce delta from head

Set calico option to be disabled by default

Add rescue blocks in case updating existing

Refactor items and update comments

Refactor and consolidate calico.yml into block

Refactor to use template for Calico CRs

Revert use_calico to false

Template blockSize

Align default cidr in template with all.yml sample

Apply upstream version tags

Revert to current ver tags. Upstream's don't work.

Update template address detection

Add Tigera Operator/Calico CNI option

* Add calico-apiserver check

* Add eBPF dataplane option

* Add kube svc endpoint configmap when ebpf enabled

* Add /etc/cni/net.d to reset task

* Refactor based on comments

* Add molecule scenario

* Fix lint

---------

Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-26 18:53:27 -06:00
Techno Tim cda7c92203 feat(k3s): Updated to v1.27 (#294)
* feat(k3s): Updated to v1.27.1+k3s1

* feat(k3s): Updated to v1.27.1+k3s1

* feat(k3s): Updated to v1.27.4+k3s1

* feat(k3s): Updated to v1.27.9+k3s1
2024-01-26 18:54:58 +00:00
Timothy Stewart 5ae8fd1223 fix(molecule): lower resources for nodes 2024-01-25 09:30:02 -06:00
Techno Tim e2e9881f0f Fix CI (#389)
did all the things to make it work
2024-01-24 22:26:38 -06:00
Balázs Hasprai cddbfc8e40 Update truthy values to true/false only, #204 (#387)
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-15 12:43:44 -06:00
Simon Leiner 33ae0d4970 Fix CI (#332)
* Update pre-commit actions

This was done by running "pre-commit autoupdate --freeze".

* Remove pre-commit only dependencies from requirements.in

Including them in the file would create the illusion that those were the
versions actually used in CI, but they are not. The exact versions are
determined by the pre-commit hooks which are pinned in
.pre-commit-config.yaml.

* Ansible Lint: Fix role-name[path]

* Ansible Lint: Fix name[play]

* Ansible Lint: Fix key-order[task]

* Ansible Lint: Fix jinja[spacing]

* Ansible Lint: Fix no-free-form

* Ansible Lint: Fix var-naming[no-reserved]

* Ansible Lint: Fix yaml[comments]

* Ansible Lint: Fix yaml[line-length]

* Ansible Lint: Fix name[casing]

* Ansible Lint: Fix no-changed-when

* Ansible Lint: Fix fqcn[action]

* Ansible Lint: Fix args[module]

* Improve task naming
2023-07-20 10:50:02 -05:00
Simon Leiner 6448948e9f Fix dual-stack clusters with multiple master nodes (#237)
* Test IPv6 scenario with two master nodes

* Fix IPv6 multimaster setup

---------

Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2023-02-20 05:24:19 +00:00
Simon Leiner 7bc198ab26 Pick kube-vip interface automatically by default (#238)
* Pick kube-vip interface automatically by default

* molecule: Fix ipv6 scenario

* Choose a more restrictive molecule timeout in CI
2023-02-20 04:08:36 +00:00
Techno Tim a1c7175bd1 fix(requirements.txt): Use pip-compile (#148)
* fix(requirements.txt): Use pip-compile

* fix(lint): Remove anchors from molecule since they aren't yet supported via lint

* fix(lint): Remove anchors from molecule since they aren't yet supported via lint
2022-11-05 18:37:46 -05:00
Techno Tim cb2fa7c441 k3s, metallb, kube-vip updates (#119)
* feat(k3s): Updated to v1.24.6+k3s1

* feat(kube-vip): Update to v0.5.5

* feat(metal-lb): Update to v0.13.6

* fix(pip): Freeze requirements

* fix(lint): Fixed ansible-lint
2022-10-15 12:23:50 -05:00
Ioannis Angelakopoulos cd76fa05a7 fix master taint implementation - linting problems (#95)
* add virtual-ip to certificate SAN entries

Adds the kube-vip IP as a Subject Alternative Name in the TLS cert. It is needed otherwise you cannot access the cluster.

* fixes bug with master taints (#1)

- improves taint logic

* fixes typo

* fixes formatting

* fixes undefined group['node'] if missing from hosts.ini (#2)

* fixes undefined group['node'] if missing from hosts.ini

- improves application of master taint by centralizing code

* improves molecule testing, fixes linting

* hacking at linter problems, small tweaks

- increases the metallb timeout error due to intermittent testing errors in GitHub actions

* improves context by renaming taint variable

- makes variable boolean

* fix bug

* removes linting hacks

Co-authored-by: Ioannis Angelakopoulos <ioangel@gmail.com>
2022-09-24 20:12:24 -05:00
Simon Leiner d5b37acd8a Drop support for CentOS, test Rocky and Debian in CI (#92)
* Test CentOS 7 in CI

* Drop support for CentOS, test on Rocky and Debian

* Fix reset playbook for Rocky Linux

* Fix typo

* Disable firewalld during testing

Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2022-09-24 05:10:55 +00:00
Simon Leiner 5225493ca0 CI: Fix linting job for ansible-lint 6.6.0 (#96)
* CI: Fix linting job for ansible-lint 6.6.0

* Increase MetalLB timeout to mitigate CI flakiness
2022-09-23 23:28:21 -05:00
Simon Leiner a1ef590442 Add support for API servers on IPv6 addresses (#48)
* Remove duplicate file for deletion

* Add support for IPv6 clusters

To correctly escape IPv6 addresses when ports are used, they must be
wrapped in square brackets [1]. This patch adds support for that,
using Ansible's ipwrap filter [2].

[1]: https://datatracker.ietf.org/doc/html/rfc4038#section-5.1
[2]: http://docs.ansible.com/ansible/latest/collections/ansible/utils/docsite/filters_ipaddr.html#wrapping-ipv6-addresses-in-brackets

* Do not abort other molecule jobs on failure

* Fix cache keys for Vagrant boxes

* Molecule: Derive overrides.yml location from scenario dir

# Conflicts:
#	molecule/default/molecule.yml
#	molecule/ipv6/molecule.yml
2022-09-10 12:57:38 -05:00
Simon Leiner 9ff3bb6b87 Test single-node cluster (#78)
* Molecule: Derive overrides.yml location from scenario dir

# Conflicts:
#	molecule/default/molecule.yml
#	molecule/ipv6/molecule.yml

* Molecule: Add single_node scenario

* Fix get_nodes test for the case of empty groups
2022-09-09 11:47:26 -05:00
Simon Leiner a6b2a95b7e Test playbook using molecule (#67)
* Test cluster using molecule

* Fix detection of first control node

* Include --flannel-iface and --node-ip as k3s arguments

* Store logs of k3s-init.service as GitHub job artifacts
2022-09-03 10:36:28 -05:00