Compare commits

..

154 Commits

Author SHA1 Message Date
Techno Tim bb006cf157 feat(dependencies): upgrade supported cluster components (#681)
* fix(calico): support split CRDs for current releases

- Download the v1_crd_projectcalico_org.yaml bundle before the operator
- Apply both files with server-side apply and force-conflicts per the
  upstream upgrade procedure
- Wait for the operator Deployment and for the managed CRDs to be
  Established after the operator starts
- Replace the create/rescue/replace flow with an idempotent apply that
  no longer conceals partial failures
- Verify TigeraStatus for calico and apiserver is Available, not just
  that Pods exist

* feat(dependencies): upgrade supported cluster components

- Bump K3s to v1.36.2+k3s1, Calico to v3.32.1, Cilium to v1.20.0,
  kube-vip to v1.2.2, kube-vip cloud provider to v0.0.12, and MetalLB to
  v0.16.0 across sample inventory, role defaults, and argument specs
- Pin the Cilium CLI with a new cilium_cli_tag (v0.19.7) instead of the
  floating stable.txt lookup
- Replace the CiliumBGPPeeringPolicy v2alpha1 BGP template with the
  v2 CiliumBGPClusterConfig, CiliumBGPPeerConfig, CiliumBGPAdvertisement,
  and CiliumLoadBalancerIPPool resource set
- Move Cilium load balancer Helm keys from bpf.loadBalancer to the valid
  top-level loadBalancer path
- Add preflight schema validation and remove the deprecated policy after
  the v2 objects are accepted
- Wait for cilium status after installation
- Pin kube-vip RBAC in a repository template instead of fetching a
  mutable URL, and include EndpointSlice permissions
- Fix the kube-vip bgppeers format to address:ASN comma-separated peers
- Fail clearly when the MetalLB speaker tag replacement does not apply
- Drop the obsolete MetalLB webhook service name version branch

* test(molecule): verify upgraded cluster components

- Assert every node reports the expected K3s kubelet version
- Verify the active CNI (Flannel / Calico / Cilium) is Ready and runs
  the expected image tag, including Calico TigeraStatus Available
- Verify the active load balancer (MetalLB / kube-vip) runs the expected
  image tags and that MetalLB is absent when kube-vip is active
- Assert no Flannel DaemonSet remains when Calico or Cilium is enabled
- Assert the example LoadBalancer address falls inside the configured
  pool range
- Add a manifest-only Cilium BGP regression test that renders the v2
  template with zero, one, and multiple neighbors and rejects any v2alpha1
  or CiliumBGPPeeringPolicy output

* fix(dependencies): correct dependency version pins

- Set the sample kube-vip image to v1.2.2 and repair the damaged comment
- Pin the kube-vip cloud provider default to v0.0.12 in the task URL
- Set the MetalLB controller argument-spec default to v0.16.0
- Restore the MetalLB available timeout default to 240s

* docs(dependencies): document current cluster versions

- Update kube-vip, kube-vip cloud provider, and MetalLB defaults
- Add cilium_tag and cilium_cli_tag rows
- Explain that MetalLB v0.16.0 is the application image target even though
  a newer chart-only tag (metallb-chart-0.16.1) exists
- Add an existing-cluster upgrade warning covering the K3s etcd 3.5.26
  bridge and one-minor-at-a-time rule, consecutive Cilium minor upgrades,
  Calico v3 resource UID handling, and MetalLB app vs chart tags

* fix(dependencies): address PR review findings

- Read the MetalLB speaker tag check from the managed host with slurp
  instead of a controller-side file lookup, and match the full image
  reference
- Restore the tigera-operator namespace on the Calico operator Deployment
  wait while keeping the managed CRD waits cluster-scoped
- Make Molecule verify inputs durable and scenario-specific via a
  per-scenario verify-vars.yml, driven by explicit verify_cni/verify_lb
  values instead of non-persisted converge facts
- Rename the kube-vip multi-peer BGP env var from bgppeers to bgp_peers
  and vip_cidr to vip_subnet so v1.2.2 actually reads them
- Map the legacy Cilium routed mode to tunnel and stop passing the alias
  directly to the chart
- Use return-code based failed_when on apply and preflight commands so
  non-error failures are no longer treated as success
- Clarify the sequential K3s upgrade path and backups in the README
- Add kube-vip and MetalLB regression tests and a Cilium mode mapping unit

* fix(dependencies): resolve re-review findings

- correct the Calico TigeraStatus resource kind\n- document tunnel as the supported Cilium routing mode\n- validate load balancer addresses across range and CIDR pools

* fix(molecule): verify embedded flannel instead of a flannel DaemonSet

- K3s 1.36 runs flannel embedded in the k3s agent rather than as a
  kube-flannel-ds DaemonSet, so the flannel verifier queried a workload
  that no longer exists and failed the verify step
- For the flannel scenarios, assert every node is Ready and that neither
  the Calico nor the Cilium namespace exists
- Drop the now-invalid kube-flannel-ds DaemonSet assertion

* fix(molecule): wait for the LoadBalancer address before asserting reachability

- The nginx LoadBalancer service had no ingress address when the
  reachability assertion ran, so status.loadBalancer.ingress[0].ip was
  undefined and the ipwrap filter failed during verify
- Poll the service until MetalLB or kube-vip assigns an external IP
- Record the assigned address once and reuse it for the reachability probe
  and the pool membership checks

* fix(ci): harden calico apiserver wait and extend molecule job timeout

- Bump calico system resources wait retries 30->60 and delay 7->10 so the
  slow-to-reconcile calico-apiserver deployment has enough time under nested-virt
- Raise the molecule step timeout-minutes from 90 to 150 to accommodate
  contended 5-node scenarios (cilium, kube-vip) that were hitting the 90-min cap

* fix(calico): treat optional API server as best-effort on converge

- The Calico API server (calico-apiserver) is an optional add-on for managing
  Calico policy through the projectcalico.org/v3 Kubernetes API; it is not
  required for Calico CNI data plane operation
- With Calico v3.32.1 on K3s 1.36 the tigera-operator never provisions the
  calico-apiserver namespace, causing the converge wait to fail deterministically
- Keep the strict wait for core Calico components (typha, kube-controllers,
  calico-node, csi-node-driver) and make the API server wait tolerate failure
- Restrict the TigeraStatus Available check to the calico status, matching the
  upstream v3.32.1 K3s quickstart which validates without the API server
2026-08-02 12:44:50 -05:00
Timothy Stewart b6363cdfc5 fix(molecule): use stable startup script path
- Resolve the bounded Vagrant startup helper relative to the playbook.
- Avoid relying on an unavailable Molecule project directory variable.
2026-08-01 19:13:48 -05:00
Timothy Stewart 83f205177d fix(molecule): always start guests in bounded batches
- Reuse bounded Vagrant startup batches when machine state already exists.
- Avoid falling back to a concurrent five-guest startup during retries.
- Keep instance reconciliation after all batches complete.
2026-08-01 19:13:48 -05:00
Timothy Stewart 5c288e8f3e fix(molecule): serialize guest fact gathering
- Avoid concurrent SSH startup bursts against the disposable guests.
- Ensure every guest gathers interface facts before network setup.
- Prevent unreachable hosts from falling back to the default eth0 interface.
2026-08-01 19:13:48 -05:00
Timothy Stewart dfcfbc1f3f fix(molecule): select distro-specific private interfaces
- Use eth1 for Ubuntu and Debian Bento guests when present.
- Keep enp0s8 for Rocky Linux guests and other images.
- Apply the selection consistently across all Molecule scenarios.
2026-08-01 19:13:48 -05:00
Timothy Stewart c3606a7847 fix(molecule): use Bento private interface name
- Replace legacy eth1 overrides with the enp0s8 private NIC used by Bento guests.\n- Keep flannel, Calico, Cilium, and kube-vip scenarios on the Vagrant private network.
2026-08-01 19:13:48 -05:00
Timothy Stewart 35939315cd ci(workflows): allow manual CI runs
- Restore workflow dispatch for manual validation on master.\n- Keep automatic CI limited to pull requests targeting master.
2026-08-01 19:13:48 -05:00
Timothy Stewart 997ea63a3b ci(workflows): run CI only for master pull requests
- Remove manual workflow dispatch from the CI entrypoint.\n- Limit CI pull request events to the master branch.
2026-08-01 19:13:48 -05:00
Timothy Stewart 68d03acc68 test(molecule): update Vagrant boxes to current releases
- Use published Bento boxes for Ubuntu 26.04, Debian 13, and Rocky Linux 10.1.\n- Pin exact VirtualBox amd64 box versions in CI.\n- Remove the obsolete Ubuntu SSH workaround and update fixtures.
2026-08-01 19:13:48 -05:00
Timothy Stewart 2babd39c89 fix(ci): align Vagrant prewarm profile
- Match the minimum CPU and memory used by Ubuntu Molecule platforms
- Restore Vagrant's normal key insertion after password authentication
- Cover the canonical prewarm configuration in the fixture
2026-08-01 12:27:12 -05:00
Timothy Stewart bbca35331d fix(ci): configure Ubuntu box prewarming
- Use the Molecule Ubuntu SSH settings when creating its linked-clone master
- Bound a prewarm SSH failure to ten minutes
- Verify the generated Vagrantfile in the master-preparation fixture
2026-08-01 12:27:12 -05:00
Timothy Stewart 4c50fbbe10 fix(k3s-server): stabilize single-server bootstrap
- delegate cgroups for transient K3s server units\n- verify inventory node registration without legacy role labels\n- wait for bootstrap CRDs before replacing the transient service
2026-08-01 12:27:12 -05:00
Timothy Stewart 10bde4eff0 feat(ci): optimize Molecule VM creation
- reuse validated runner-owned Vagrant linked-clone masters\n- create the default guests in bounded two-machine batches\n- capture create timing and runner utilization diagnostics
2026-08-01 12:27:12 -05:00
Timothy Stewart 57f234c8da fix(ci): make Vagrant box cache version-aware
- key immutable caches from a pinned box manifest instead of scenario YAML
- skip producer restoration and downloads on exact cache hits
- validate lock drift, exact versions, architectures, and missing boxes
2026-08-01 12:27:12 -05:00
Timothy Stewart fb9a0bebd1 fix(molecule): stabilize host-only peer identity
- restore host-only routing required by outside verification
- assign deterministic adapter MACs to the five default guests
- pin disposable full-mesh neighbor entries from live Ansible facts
2026-08-01 12:27:12 -05:00
Timothy Stewart 4990157d35 fix(molecule): isolate default cluster network
- move the five-node cluster NICs to a named VirtualBox internal network
- preserve NAT connectivity for provisioning and downloads
- retain serialized creation and early neighbor identity validation
2026-08-01 12:27:12 -05:00
Timothy Stewart 73fae0826c fix(molecule): stabilize default scenario networking
- serialize five-node Vagrant creation to avoid VirtualBox host-only races
- refresh and verify the primary neighbor mapping before convergence
- include interface MAC details in failure diagnostics
2026-08-01 12:27:12 -05:00
Timothy Stewart 57a22e364d fix(k3s-server): harden isolated control-plane bootstrap
- pin kube-vip and cluster traffic to the private guest interface\n- disable disposable guest firewalls and verify API reachability before joins\n- keep control-plane orchestration on the primary and preserve failure diagnostics
2026-08-01 12:27:12 -05:00
Timothy Stewart 9b220c1629 fix(k3s-server): serialize master joins and capture diagnostics
- validate host-only connectivity before cluster bootstrap\n- materialize per-host join arguments and wait for sequential registration\n- collect bounded network diagnostics when k3s initialization fails
2026-08-01 12:27:12 -05:00
Timothy Stewart a0d78ff317 ci(test): stop matrix after first failure
- Enable fail-fast for the serial Molecule matrix\n- Avoid running single-node scenarios after a default scenario failure
2026-08-01 12:27:12 -05:00
Timothy Stewart 999cf3ee05 ci(test): run default scenario first
- Prioritize the multi-node HA scenario in the serial Molecule matrix\n- Fail fast before running the single-node scenario variants
2026-08-01 12:27:12 -05:00
Timothy Stewart 1402f33108 fix(k3s-server): build delegated join arguments
- Preserve explicit per-host server initialization overrides\n- Build default join arguments from the delegated host variables\n- Keep initialization commands out of normal task output
2026-08-01 12:27:12 -05:00
Timothy Stewart 94dbffaef7 fix(k3s-server): serialize control-plane joins
- Wait for each additional master to register before starting the next\n- Keep k3s initialization diagnostics bounded and sanitized
2026-08-01 12:27:12 -05:00
Timothy Stewart a52e2ea72c fix(k3s-server): sequence HA bootstrap
- Wait for the initial control-plane API before starting joining masters\n- Bound and redact k3s-init failure diagnostics
2026-08-01 12:27:12 -05:00
Timothy Stewart bb3843dbb1 fix(ci): skip removed stale Molecule state
- Avoid rereading state files after removing a stale scenario directory.\n- Extend cleanup fixture coverage to multiple stale machine states.
2026-08-01 12:27:12 -05:00
Timothy Stewart 87ea8160c1 fix(ci): harden Molecule download and cleanup
- Retry transient k3s release downloads with bounded backoff.\n- Bound failure diagnostics and validate guest release connectivity.\n- Discover repository-owned Molecule state under the actual project root.
2026-08-01 12:27:12 -05:00
Timothy Stewart 5bc347aed4 fix(ci): support installed yq syntax
- Use portable yq raw-output syntax when enumerating Molecule boxes\n- Preserve repository-scoped Vagrant cache discovery
2026-08-01 12:27:12 -05:00
Timothy Stewart 665e274820 fix(ci): make Ansible collection lint resilient
- Retry Galaxy collection installation on transient network failures\n- Cache installed collections before running pre-commit\n- Run Ansible-lint offline after collection provisioning
2026-08-01 12:27:12 -05:00
Timothy Stewart 5747bfce0e chore(ci): update pinned GitHub Actions
- Update checkout, setup-python, cache, upload-artifact, and pin enforcement actions\n- Keep every third-party action pinned to an immutable release commit\n- Replace the deprecated cache action reference that blocked CI
2026-08-01 12:27:12 -05:00
Timothy Stewart 29b7aa1b72 ci(runner): restore isolated Molecule CI
- Scope VirtualBox cleanup to repository-owned Molecule resources\n- Route lint to hosted runners and harden self-hosted workflow execution\n- Add diagnostics, fixture coverage, and reproducible lint validation
2026-08-01 12:27:12 -05:00
Timothy Stewart 2fad0a8db6 chore(technotimcom): Updated url 2026-01-07 15:31:56 -06:00
Timothy Stewart 5cbbf7371b cupre(repo): switching to personal repo (timothystewart6) 2025-08-24 15:50:44 -05:00
4t1l4 422621c69c Variable documentation (#606)
* docs: first modules' variable docs table

* docs: variables for k3s_server_post

* docs: lxc and prereq vars in README

* style: lint errors

* docs: argument_specs for proxmox_lxc

* docs: last variables found added to the README
2024-11-06 10:27:19 -06:00
dependabot[bot] 39988a9bee chore(deps): bump ansible-core from 2.17.5 to 2.18.0 (#607)
Bumps [ansible-core](https://github.com/ansible/ansible) from 2.17.5 to 2.18.0.
- [Release notes](https://github.com/ansible/ansible/releases)
- [Commits](https://github.com/ansible/ansible/compare/v2.17.5...v2.18.0)

---
updated-dependencies:
- dependency-name: ansible-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-05 04:24:57 +00:00
dependabot[bot] 133a84b564 chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions (#605)
Bumps [zgosalvez/github-actions-ensure-sha-pinned-actions](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions) from 3.0.15 to 3.0.16.
- [Release notes](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/releases)
- [Commits](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/compare/ed00f72a3ca5b6eff8ad4d3ffdcacedb67a21db1...38608ef4fb69adae7f1eac6eeb88e67b7d083bfd)

---
updated-dependencies:
- dependency-name: zgosalvez/github-actions-ensure-sha-pinned-actions
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-04 05:34:04 +00:00
dependabot[bot] 6b79057f6c chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions (#601)
Bumps [zgosalvez/github-actions-ensure-sha-pinned-actions](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions) from 3.0.14 to 3.0.15.
- [Release notes](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/releases)
- [Commits](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/compare/40ba2d51b6b6d8695f2b6bd74e785172d4f8d00f...ed00f72a3ca5b6eff8ad4d3ffdcacedb67a21db1)

---
updated-dependencies:
- dependency-name: zgosalvez/github-actions-ensure-sha-pinned-actions
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-28 19:04:08 +00:00
Christian Berendt 4c0b1ee8f3 k3s_server: add kube_vip_bgp_peers parameter (#599)
With the kube_vip_bgp_peers it is possible to define
multiple BGP peer ASN & address pairs for kube-vip.

Sample:

```
kube_vip_bgp_peers:
  - peer_address: 192.168.128.10
    peer_asn: 64512
  - peer_address: 192.168.128.11
    peer_asn: 64512
  - peer_address: 192.168.128.12
    peer_asn: 64512
```

It is possible to merge further lists with kube_vip_bgp_peers__*
parameters.

Sample:

```
kube_vip_bgp_peers__extra:
  - peer_address: 192.168.128.10
    peer_asn: 64512
kube_vip_bgp_peers:
  - peer_address: 192.168.128.11
    peer_asn: 64512
  - peer_address: 192.168.128.12
    peer_asn: 64512
```

This will result in the following list of BGP peer ASN & address pairs:

```
- peer_address: 192.168.128.10
  peer_asn: 64512
- peer_address: 192.168.128.11
  peer_asn: 64512
- peer_address: 192.168.128.12
  peer_asn: 64512
```

Signed-off-by: Christian Berendt <berendt@osism.tech>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-10-27 20:15:48 +00:00
Christian Berendt 11f9505460 k3s_server_post: always set _cilium_bgp_neighbors (#597)
Related to eddbcbfb76

Signed-off-by: Christian Berendt <berendt@osism.tech>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-10-27 17:08:40 +00:00
dependabot[bot] 850301fbc4 chore(deps): bump actions/cache from 4.1.0 to 4.1.2 (#596)
Bumps [actions/cache](https://github.com/actions/cache) from 4.1.0 to 4.1.2.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/2cdf405574d6ef1f33a1d12acccd3ae82f47b3f2...6849a6489940f00c2f30c0fb92c6274307ccb58a)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-27 10:28:50 -05:00
dependabot[bot] 983e11322e chore(deps): bump actions/checkout from 4.2.0 to 4.2.2 (#598)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.2.0 to 4.2.2.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/d632683dd7b4114ad314bca15554477dd762a938...11bd71901bbe5b1630ceea73d27597364c9af683)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-26 18:45:18 +00:00
dependabot[bot] a4df16cf87 chore(deps): bump actions/upload-artifact from 4.4.1 to 4.4.3 (#592)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.4.1 to 4.4.3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/604373da6381bf24206979c74d06a550515601b9...b4b15b8c7c6ac21ea08fcf65892d2ee8f75cf882)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-26 12:12:58 -05:00
dependabot[bot] f8ababb7bf chore(deps): bump actions/setup-python from 5.2.0 to 5.3.0 (#600)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.2.0 to 5.3.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/f677139bbe7f9c59b41e40162b753c062f5d49a3...0b93645e9fea7318ecaed2b359559ac225c90a2b)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-25 23:22:15 -05:00
dependabot[bot] 90eb5e4b41 chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions (#585)
Bumps [zgosalvez/github-actions-ensure-sha-pinned-actions](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions) from 3.0.13 to 3.0.14.
- [Release notes](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/releases)
- [Commits](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/compare/b8f9a25a51fe633d9215ac7734854dc11cd299cb...40ba2d51b6b6d8695f2b6bd74e785172d4f8d00f)

---
updated-dependencies:
- dependency-name: zgosalvez/github-actions-ensure-sha-pinned-actions
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-10-25 14:22:17 -05:00
dependabot[bot] 97ed29b4a2 chore(deps): bump actions/upload-artifact from 4.4.0 to 4.4.1 (#587)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.4.0 to 4.4.1.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/50769540e7f4bd5e21e526ee35c689e35e0d6874...604373da6381bf24206979c74d06a550515601b9)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-10-08 09:24:06 -05:00
dependabot[bot] fc2225ab8d chore(deps): bump ansible-core from 2.17.4 to 2.17.5 (#589)
Bumps [ansible-core](https://github.com/ansible/ansible) from 2.17.4 to 2.17.5.
- [Release notes](https://github.com/ansible/ansible/releases)
- [Commits](https://github.com/ansible/ansible/compare/v2.17.4...v2.17.5)

---
updated-dependencies:
- dependency-name: ansible-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-08 07:28:24 -05:00
dependabot[bot] d99f6a96f2 chore(deps): bump actions/cache from 4.0.2 to 4.1.0 (#584)
Bumps [actions/cache](https://github.com/actions/cache) from 4.0.2 to 4.1.0.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/0c45773b623bea8c8e75f6c82b208c3cf94ea4f9...2cdf405574d6ef1f33a1d12acccd3ae82f47b3f2)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-10-07 21:23:25 -05:00
dependabot[bot] fab302fd91 chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions (#582)
Bumps [zgosalvez/github-actions-ensure-sha-pinned-actions](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions) from 3.0.12 to 3.0.13.
- [Release notes](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/releases)
- [Commits](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/compare/0901cf7b71c7ea6261ec69a3dc2bd3f9264f893e...b8f9a25a51fe633d9215ac7734854dc11cd299cb)

---
updated-dependencies:
- dependency-name: zgosalvez/github-actions-ensure-sha-pinned-actions
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-10-01 21:01:29 -05:00
Christian Berendt eddbcbfb76 k3s_server_post: add cilium_bgp_neighbors parameter (#579)
With the cilium_bgp_neighbors parameter it is possible to define
multiple BGP peer ASN & address pairs for Cilium.

Sample:

```
cilium_bgp_neighbors:
  - peer_address: 192.168.128.10
    peer_asn: 64512
  - peer_address: 192.168.128.11
    peer_asn: 64512
  - peer_address: 192.168.128.12
    peer_asn: 64512
```

It is possible to merge further lists with cilium_bgp_neighbors__*
parameters.

Sample:

```
cilium_bgp_neighbors__extra:
  - peer_address: 192.168.128.10
    peer_asn: 64512
cilium_bgp_neighbors:
  - peer_address: 192.168.128.11
    peer_asn: 64512
  - peer_address: 192.168.128.12
    peer_asn: 64512
```

This will result in the following list of BGP peer ASN & address pairs:

```
- peer_address: 192.168.128.10
  peer_asn: 64512
- peer_address: 192.168.128.11
  peer_asn: 64512
- peer_address: 192.168.128.12
  peer_asn: 64512
```

Signed-off-by: Christian Berendt <berendt@osism.tech>
2024-09-30 16:10:57 -05:00
dependabot[bot] 03ae8de0d5 chore(deps): bump actions/checkout from 4.1.7 to 4.2.0 (#580)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.7 to 4.2.0.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/692973e3d937129bcbf40652eb9f2f61becf3332...d632683dd7b4114ad314bca15554477dd762a938)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-29 14:35:36 -05:00
dependabot[bot] d136fa4486 chore(deps): bump ansible-core from 2.17.3 to 2.17.4 (#578)
Bumps [ansible-core](https://github.com/ansible/ansible) from 2.17.3 to 2.17.4.
- [Release notes](https://github.com/ansible/ansible/releases)
- [Commits](https://github.com/ansible/ansible/compare/v2.17.3...v2.17.4)

---
updated-dependencies:
- dependency-name: ansible-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-09-13 11:47:51 -05:00
Vincent Link b906cfbf72 fix: fix raspberry pi preparation for ubuntu 24.04 (#576)
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-09-13 09:43:28 -05:00
dependabot[bot] 2c04f38e2c chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions (#577)
Bumps [zgosalvez/github-actions-ensure-sha-pinned-actions](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions) from 3.0.11 to 3.0.12.
- [Release notes](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/releases)
- [Commits](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/compare/3c16e895bb662b4d7e284f032cbe8835a57773cc...0901cf7b71c7ea6261ec69a3dc2bd3f9264f893e)

---
updated-dependencies:
- dependency-name: zgosalvez/github-actions-ensure-sha-pinned-actions
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-12 15:12:15 -05:00
dependabot[bot] 3435f43748 chore(deps): bump actions/upload-artifact from 4.3.6 to 4.4.0 (#574)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.3.6 to 4.4.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/834a144ee995460fba8ed112a2fc961b36a5ec5a...50769540e7f4bd5e21e526ee35c689e35e0d6874)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-11 11:09:29 -05:00
dependabot[bot] 924a2f528c chore(deps): bump actions/setup-python from 5.1.1 to 5.2.0 (#573)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.1.1 to 5.2.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/39cd14951b08e74b54015e9e001cdefcf80e669f...f677139bbe7f9c59b41e40162b753c062f5d49a3)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-31 23:15:52 -05:00
dependabot[bot] 2892ac3858 chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions (#571)
Bumps [zgosalvez/github-actions-ensure-sha-pinned-actions](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions) from 3.0.10 to 3.0.11.
- [Release notes](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/releases)
- [Commits](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/compare/b88cd0aad2c36a63e42c71f81cb1958fed95ac87...3c16e895bb662b4d7e284f032cbe8835a57773cc)

---
updated-dependencies:
- dependency-name: zgosalvez/github-actions-ensure-sha-pinned-actions
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-31 14:33:37 -05:00
Christian Berendt df8e8dd591 Make kubectl binary configurable with the k3s_kubectl_binary parameter (#567)
Closes techno-tim/k3s-ansible#566

Signed-off-by: Christian Berendt <berendt@osism.tech>
2024-08-22 17:58:15 -05:00
dependabot[bot] 3a0303d130 chore(deps): bump ansible-core from 2.17.2 to 2.17.3 (#564)
Bumps [ansible-core](https://github.com/ansible/ansible) from 2.17.2 to 2.17.3.
- [Release notes](https://github.com/ansible/ansible/releases)
- [Commits](https://github.com/ansible/ansible/compare/v2.17.2...v2.17.3)

---
updated-dependencies:
- dependency-name: ansible-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-08-13 06:14:10 +00:00
Richard Holmboe b077a49e1f Change to FQCN with ansible-lint fixer (#553)
* Change to FQCN with ansible-lint fixer

Since ansible-base 2.10 (later ansible-core), FQCN is the new way to go.

Updated .ansible-lint with a production profile and removed fqcn in skip_list.
Updated .yamllint with rules needed.

Ran ansible-lint --fix=all, then manually applied some minor changes.

* Changed octal value in molecule/ipv6/prepare.yml
2024-08-12 22:59:59 -05:00
dependabot[bot] 635f0b21b3 chore(deps): bump actions/upload-artifact from 4.3.5 to 4.3.6 (#561)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.3.5 to 4.3.6.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/89ef406dd8d7e03cfd12d9e0a4a378f454709029...834a144ee995460fba8ed112a2fc961b36a5ec5a)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-08-07 20:22:17 +00:00
dependabot[bot] 4a64ad42df chore(deps): bump pyyaml from 6.0.1 to 6.0.2 (#562)
Bumps [pyyaml](https://github.com/yaml/pyyaml) from 6.0.1 to 6.0.2.
- [Release notes](https://github.com/yaml/pyyaml/releases)
- [Changelog](https://github.com/yaml/pyyaml/blob/main/CHANGES)
- [Commits](https://github.com/yaml/pyyaml/compare/6.0.1...6.0.2)

---
updated-dependencies:
- dependency-name: pyyaml
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-08-07 18:16:52 +00:00
Christian Berendt d0537736de k3s_server: add missing parameter descriptions (#559)
The commit 3a20500f9c has introduced
argument specs in the role meta information. These two parameters
were still missing there.

Realted to 2d0596209e

Signed-off-by: Christian Berendt <berendt@osism.tech>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-08-07 16:10:16 +00:00
Christian Berendt 2149827800 k3s_server: add kube-vip BGP support (#554)
With the kube_vip_bgp parameter it is possible to enable the kube-vip
BGP support (https://kube-vip.io/docs/modes/bgp/).

The configuration is possible with the following new parameters:

* kube_vip_bgp_routerid
* kube_vip_bgp_as
* kube_vip_bgp_peeraddress
* kube_vip_bgp_peeras

Signed-off-by: Christian Berendt <berendt@osism.tech>
2024-08-07 09:36:05 -05:00
Christian Berendt 2d0596209e Make it possible to disable the creation of the kubectl/crictl symlinks (#558)
If k3s_create_kubectl_symlink is set to false the kubectl symlink will
not be created.

If k3s_create_crictl_symlink is set to false the crictl symlink will not
be created.

By default the symlinks will be created. The default behavior is not
changed.

Signed-off-by: Christian Berendt <berendt@osism.tech>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-08-05 21:19:57 -05:00
Dov Benyomin Sohacheski 3a20500f9c Add default values to roles (#509)
*  Add default values to roles

* 🚚 Move to use meta files for roles

* 🛠 Fix descriptions

*  Add meta for server

* 🚧 WIP

* 🌟 Complete

* 🧹 Ran and fix lint errors

* 🔨 Fix required and default conflict

---------

Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-08-05 17:00:24 -05:00
dependabot[bot] 9ce9fecc5b chore(deps): bump actions/upload-artifact from 4.3.4 to 4.3.5 (#555)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.3.4 to 4.3.5.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/0b2256b8c012f0828dc542b3febcab082c67f72b...89ef406dd8d7e03cfd12d9e0a4a378f454709029)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-08-05 13:55:38 -05:00
Christian Berendt 668d7fb896 k3s_server: add kube_vip_arp parameter (#550)
With the kube_vip_arp parameter it is possible to set or unset the
vip_arp environment variable of the kube-vip-ds daemonset. The value of
the kube_vip_arp is true by default to not change the existing default.

Signed-off-by: Christian Berendt <berendt@osism.tech>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-07-29 16:05:30 -05:00
dependabot[bot] 6cee0e9051 chore(deps): bump pre-commit from 3.7.1 to 3.8.0 (#549)
Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 3.7.1 to 3.8.0.
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pre-commit/pre-commit/compare/v3.7.1...v3.8.0)

---
updated-dependencies:
- dependency-name: pre-commit
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-07-29 13:24:20 -05:00
dependabot[bot] 6823ad51d5 chore(deps): bump ansible-core from 2.17.0 to 2.17.2 (#544)
Bumps [ansible-core](https://github.com/ansible/ansible) from 2.17.0 to 2.17.2.
- [Release notes](https://github.com/ansible/ansible/releases)
- [Commits](https://github.com/ansible/ansible/compare/v2.17.0...v2.17.2)

---
updated-dependencies:
- dependency-name: ansible-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-07-29 11:11:43 -05:00
dependabot[bot] 1a521ea0d9 chore(deps): bump actions/setup-python from 5.1.0 to 5.1.1 (#541)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.1.0 to 5.1.1.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/82c7e631bb3cdc910f68e0081d67478d79c6982d...39cd14951b08e74b54015e9e001cdefcf80e669f)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-07-29 09:13:20 -05:00
dependabot[bot] e48bb6df26 chore(deps): bump actions/upload-artifact from 4.3.3 to 4.3.4 (#538)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.3.3 to 4.3.4.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/65462800fd760344b1a7b4382951275a0abb4808...0b2256b8c012f0828dc542b3febcab082c67f72b)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-07-28 18:06:27 -05:00
dependabot[bot] 36893c27fb chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions (#536)
Bumps [zgosalvez/github-actions-ensure-sha-pinned-actions](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions) from 3.0.6 to 3.0.10.
- [Release notes](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/releases)
- [Commits](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/compare/2f2ebc6d914ab515939dc13f570f91baeb2c194c...b88cd0aad2c36a63e42c71f81cb1958fed95ac87)

---
updated-dependencies:
- dependency-name: zgosalvez/github-actions-ensure-sha-pinned-actions
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-07-28 16:27:55 -05:00
Techno Tim e8cd10d49b chore(deps): Updated all k8s components (#532)
* chore(deps): Updated all k8s components

* Metallb 0.14.4 support (#533)

* Added support for metallb >= 0.14.4

* update gpg

* Added support for metallb >= 0.14.4

* remove extra file

---------

Co-authored-by: Konstantin Kornienko <k.kornienko@postgrespro.ru>
Co-authored-by: Konstantin Kornienko <konstantin.kornienko@gmail.com>

* chore(deps): Updated actions

* chore(dependencies): Roll back ansible-core

* chore(deps): Updated kube-vip

* chore(deps): Updated dependencies

* chore(deps): Updated dependencies

* fix(cilium): fixing logic

* fix(cilium): fixing logic

---------

Co-authored-by: Konstantin Kornienko <k.kornienko@postgrespro.ru>
Co-authored-by: Konstantin Kornienko <konstantin.kornienko@gmail.com>
2024-07-28 14:29:32 -05:00
Timothy Stewart b86156b995 chore(deps): Updated actions 2024-06-19 12:33:21 -05:00
Rina Y 072f1a321d fixed broken when statement for Deploy metallb manifest task (#513)
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-06-03 23:39:54 +00:00
dependabot[bot] 2f46a54240 chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions (#519)
Bumps [zgosalvez/github-actions-ensure-sha-pinned-actions](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions) from 3.0.5 to 3.0.6.
- [Release notes](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/releases)
- [Commits](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/compare/40e45e738b3cad2729f599d8afc6ed02184e1dbd...2f2ebc6d914ab515939dc13f570f91baeb2c194c)

---
updated-dependencies:
- dependency-name: zgosalvez/github-actions-ensure-sha-pinned-actions
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-06-03 22:37:27 +00:00
dependabot[bot] bf0418d77f --- (#517)
updated-dependencies:
- dependency-name: ansible-core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-06-03 21:43:14 +00:00
dependabot[bot] d88eb80df0 chore(deps): bump pre-commit from 3.7.0 to 3.7.1 (#511)
Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 3.7.0 to 3.7.1.
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pre-commit/pre-commit/compare/v3.7.0...v3.7.1)

---
updated-dependencies:
- dependency-name: pre-commit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-06-03 15:50:27 -05:00
dependabot[bot] f50d335451 chore(deps): bump actions/checkout from 4.1.5 to 4.1.6 (#515)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.5 to 4.1.6.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/44c2b7a8a4ea60a981eaca3cf939b5f4305c123b...a5ac7e51b41094c92402da3b24376905380afc29)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-17 12:38:25 -05:00
Dov Benyomin Sohacheski d6597150c7 Allow IP range for cilium_bgp_lb_cidr (#507)
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-05-10 21:45:29 -05:00
dependabot[bot] 353f7ab641 chore(deps): bump actions/checkout from 4.1.4 to 4.1.5 (#505)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.4 to 4.1.5.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/0ad4b8fadaa221de15dcec353f45205ec38ea70b...44c2b7a8a4ea60a981eaca3cf939b5f4305c123b)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-08 12:23:32 -05:00
dependabot[bot] c7c727c3dc chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions (#504)
Bumps [zgosalvez/github-actions-ensure-sha-pinned-actions](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions) from 3.0.4 to 3.0.5.
- [Release notes](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/releases)
- [Commits](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/compare/19ebcb0babbd282ae1822a0b9c28f3f1f25cea45...40e45e738b3cad2729f599d8afc6ed02184e1dbd)

---
updated-dependencies:
- dependency-name: zgosalvez/github-actions-ensure-sha-pinned-actions
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-05-06 09:55:54 -05:00
dependabot[bot] 0422bfa2ac chore(deps): bump actions/checkout from 4.1.3 to 4.1.4 (#499)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.3 to 4.1.4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/1d96c772d19495a3b5c517cd2bc0cb401ea0529f...0ad4b8fadaa221de15dcec353f45205ec38ea70b)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-26 03:27:36 +00:00
dependabot[bot] 0333406725 chore(deps): bump actions/upload-artifact from 4.3.2 to 4.3.3 (#498)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.3.2 to 4.3.3.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/1746f4ab65b179e0ea60a494b83293b640dd5bba...65462800fd760344b1a7b4382951275a0abb4808)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-24 21:30:53 -05:00
dependabot[bot] f4a19d368b chore(deps): bump actions/checkout from 4.1.2 to 4.1.3 (#497)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.2 to 4.1.3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/9bb56186c3b09b4f86b1c65136769dd318469633...1d96c772d19495a3b5c517cd2bc0cb401ea0529f)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-22 17:02:39 +00:00
dependabot[bot] 02d212c007 chore(deps): bump actions/upload-artifact from 4.3.1 to 4.3.2 (#495)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.3.1 to 4.3.2.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/5d5d22a31266ced268874388b861e4b58bb5c2f3...1746f4ab65b179e0ea60a494b83293b640dd5bba)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-19 03:54:30 +00:00
dependabot[bot] 80095250e9 chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions (#493)
Bumps [zgosalvez/github-actions-ensure-sha-pinned-actions](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions) from 3.0.3 to 3.0.4.
- [Release notes](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/releases)
- [Commits](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/compare/ba37328d4ea95eaf8b3bd6c6cef308f709a5f2ec...19ebcb0babbd282ae1822a0b9c28f3f1f25cea45)

---
updated-dependencies:
- dependency-name: zgosalvez/github-actions-ensure-sha-pinned-actions
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-04-17 02:27:50 +00:00
dependabot[bot] 4fe2c92795 chore(deps): bump ansible-core from 2.16.5 to 2.16.6 (#494)
Bumps [ansible-core](https://github.com/ansible/ansible) from 2.16.5 to 2.16.6.
- [Release notes](https://github.com/ansible/ansible/releases)
- [Commits](https://github.com/ansible/ansible/compare/v2.16.5...v2.16.6)

---
updated-dependencies:
- dependency-name: ansible-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-16 23:57:42 +00:00
dependabot[bot] b3f2a4addc chore(deps): bump pre-commit-hooks from 4.5.0 to 4.6.0 (#491)
Bumps [pre-commit-hooks](https://github.com/pre-commit/pre-commit-hooks) from 4.5.0 to 4.6.0.
- [Release notes](https://github.com/pre-commit/pre-commit-hooks/releases)
- [Changelog](https://github.com/pre-commit/pre-commit-hooks/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pre-commit/pre-commit-hooks/compare/v4.5.0...v4.6.0)

---
updated-dependencies:
- dependency-name: pre-commit-hooks
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-10 21:03:34 -05:00
dependabot[bot] cb03ee829e chore(deps): bump actions/setup-python from 5.0.0 to 5.1.0 (#488)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5.0.0 to 5.1.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/0a5c61591373683505ea898e09a3ea4f39ef2b9c...82c7e631bb3cdc910f68e0081d67478d79c6982d)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-03-28 14:25:10 +00:00
dependabot[bot] 9e2e82faeb chore(deps): bump ansible-core from 2.16.4 to 2.16.5 (#486)
Bumps [ansible-core](https://github.com/ansible/ansible) from 2.16.4 to 2.16.5.
- [Release notes](https://github.com/ansible/ansible/releases)
- [Commits](https://github.com/ansible/ansible/compare/v2.16.4...v2.16.5)

---
updated-dependencies:
- dependency-name: ansible-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-28 08:42:19 -05:00
dependabot[bot] 7c1f6cbe42 chore(deps): bump pre-commit from 3.6.2 to 3.7.0 (#485)
Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 3.6.2 to 3.7.0.
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pre-commit/pre-commit/compare/v3.6.2...v3.7.0)

---
updated-dependencies:
- dependency-name: pre-commit
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-03-25 18:58:53 -05:00
Timothy Stewart 604eb7a6e6 fix(script): fixed yq 2024-03-25 13:36:08 -05:00
Timothy Stewart a204ed5169 fix(script): fixed yq 2024-03-25 13:28:28 -05:00
Timothy Stewart b6608ca3e4 fix(script): fixed yq 2024-03-25 13:21:32 -05:00
Timothy Stewart 8252a45dfd fix(script): fixed yq 2024-03-25 13:07:31 -05:00
Timothy Stewart c99f098c2e fix(script): fixed yq 2024-03-25 13:02:16 -05:00
Timothy Stewart 7867b87d85 fix(script): fixed yq 2024-03-25 11:34:40 -05:00
Timothy Stewart dfe19f3731 fix(script): fixed yq 2024-03-25 11:31:19 -05:00
Techno Tim a46d97a28d chore(deps): Updated cilium to v1.15.2 (#484)
* chore(deps): Updated cilium to v1.15.2

* fix(molecule): increase timeout for slow CI
2024-03-22 14:16:31 -05:00
dependabot[bot] dc9d571f17 chore(deps): bump actions/cache from 4.0.1 to 4.0.2 (#481)
Bumps [actions/cache](https://github.com/actions/cache) from 4.0.1 to 4.0.2.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/ab5e6d0c87105b4c9c2047343972218f562e4319...0c45773b623bea8c8e75f6c82b208c3cf94ea4f9)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-22 10:36:35 -05:00
dependabot[bot] 6742551e5c chore(deps): bump actions/checkout from 4.1.1 to 4.1.2 (#477)
Bumps [actions/checkout](https://github.com/actions/checkout) from 4.1.1 to 4.1.2.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/b4ffde65f46336ab88eb53be808477a3936bae11...9bb56186c3b09b4f86b1c65136769dd318469633)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-03-13 02:57:45 +00:00
Techno Tim fb3478a086 feat(k3s): Updated to v1.29.2+k3s1 (#474) 2024-03-12 02:29:46 +00:00
Techno Tim 518c5bb62a fix(molecule): Disable ipv6 test (#476) 2024-03-11 20:42:47 -05:00
Techno Tim 3f5d8dfe9f chore(deps): Updated kube-vip (#471) 2024-03-10 21:03:20 +00:00
Luke D Iremadze efbfadcb93 Update Kube Config instructions (#470)
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-03-05 22:21:40 -06:00
Phil Bolduc f81ec04ba2 Support defining a custom reboot command (#467)
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-03-06 03:52:24 +00:00
Techno Tim 8432d3bc66 chore(deps): Updated (#469)
* chore(deps): Updated

* fix(kube-vipe): changed rbac location
2024-03-05 04:18:23 +00:00
Phil Bolduc 14ae9df1bc Change path for KUBECONFIG (#465) 2024-03-04 16:58:52 -06:00
dependabot[bot] f175716339 chore(deps): bump actions/cache from 4.0.0 to 4.0.1 (#460)
Bumps [actions/cache](https://github.com/actions/cache) from 4.0.0 to 4.0.1.
- [Release notes](https://github.com/actions/cache/releases)
- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)
- [Commits](https://github.com/actions/cache/compare/13aacd865c20de90d75de3b17ebe84f7a17d57d2...ab5e6d0c87105b4c9c2047343972218f562e4319)

---
updated-dependencies:
- dependency-name: actions/cache
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-03-02 22:20:03 +00:00
Daniel SP 955c6f6b4a Fix cgroups cmdline path (#456)
* Fix cgroups cmdline path

* Add check and variable so that we write to the correct file

* Add LSB release checks. This is untested atm.

* Break test into multiple lines so that we can pass lint checks

* Flip logic on its head and check for existence only rather than content per contributor suggestion

---------

Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-03-02 15:14:41 -06:00
dependabot[bot] 3b74985767 chore(deps): bump ansible-core from 2.16.3 to 2.16.4 (#459)
Bumps [ansible-core](https://github.com/ansible/ansible) from 2.16.3 to 2.16.4.
- [Release notes](https://github.com/ansible/ansible/releases)
- [Commits](https://github.com/ansible/ansible/compare/v2.16.3...v2.16.4)

---
updated-dependencies:
- dependency-name: ansible-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-27 04:05:58 +00:00
dependabot[bot] 9ace193ade chore(deps): bump pre-commit from 3.6.1 to 3.6.2 (#453)
Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 3.6.1 to 3.6.2.
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pre-commit/pre-commit/compare/v3.6.1...v3.6.2)

---
updated-dependencies:
- dependency-name: pre-commit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-26 19:31:14 +00:00
Angel Garza 83a0be3afd Added pre_task to verify minimum Ansible version (#454) 2024-02-24 19:20:57 -06:00
dependabot[bot] 029eba6102 chore(deps): bump pre-commit from 3.6.0 to 3.6.1 (#450)
Bumps [pre-commit](https://github.com/pre-commit/pre-commit) from 3.6.0 to 3.6.1.
- [Release notes](https://github.com/pre-commit/pre-commit/releases)
- [Changelog](https://github.com/pre-commit/pre-commit/blob/main/CHANGELOG.md)
- [Commits](https://github.com/pre-commit/pre-commit/compare/v3.6.0...v3.6.1)

---
updated-dependencies:
- dependency-name: pre-commit
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-11 23:06:15 -06:00
dependabot[bot] 0c8253b3a5 chore(deps): bump molecule-plugins[vagrant] from 23.5.0 to 23.5.3 (#449)
Bumps [molecule-plugins[vagrant]](https://github.com/ansible-community/molecule-plugins) from 23.5.0 to 23.5.3.
- [Release notes](https://github.com/ansible-community/molecule-plugins/releases)
- [Commits](https://github.com/ansible-community/molecule-plugins/compare/v23.5.0...v23.5.3)

---
updated-dependencies:
- dependency-name: molecule-plugins[vagrant]
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-08 22:41:07 -06:00
dependabot[bot] 326b71dfa2 chore(deps): bump pre-commit/action from 3.0.0 to 3.0.1 (#447)
Bumps [pre-commit/action](https://github.com/pre-commit/action) from 3.0.0 to 3.0.1.
- [Release notes](https://github.com/pre-commit/action/releases)
- [Commits](https://github.com/pre-commit/action/compare/646c83fcd040023954eafda54b4db0192ce70507...2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd)

---
updated-dependencies:
- dependency-name: pre-commit/action
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-02-07 23:03:23 -06:00
dependabot[bot] b95d6dd2cc chore(deps): bump actions/upload-artifact from 4.3.0 to 4.3.1 (#443)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 4.3.0 to 4.3.1.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/26f96dfa697d77e81fd5907df203aa23a56210a8...5d5d22a31266ced268874388b861e4b58bb5c2f3)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-02-06 21:40:15 +00:00
Jayson Grace e4146b4ca9 Add PXE boot support to k3s_agent role (#409)
* Add conditional snapshotter for PXE-booted systems

**Added:**

- PXE Boot Check - Introduced tasks to check if the system is PXE-booted by
  analyzing `/proc/cmdline` in `roles/k3s_agent/tasks/main.yml`.
- Conditional Snapshotter in Template - Added logic in `k3s.service.j2` template
  to conditionally set `--snapshotter native` for PXE-booted systems.

**Changed:**

- `k3s.service.j2` Template Update - Modified the `ExecStart` line to include a
  conditional check for `is_pxe_booted` fact, dynamically setting the
  `--snapshotter` option for NFS-mounted systems.
- `main.yml` Task Modification - Added tasks to set `is_pxe_booted` fact based
  on the presence of `root=/dev/nfs` in the system's boot command line.

This update allows k3s agents on PXE-booted systems to use the native snapshotter
when running on NFS, addressing compatibility issues with OverlayFS.

* Introduce idiomatic practices for affected areas from previous commits

**Added:**

- Structured HTTP Proxy Configuration Block - Added a structured block in
  `http_proxy.yml` for managing HTTP proxy settings, aligning with Ansible's
  recommended practices. This includes creating directories and deploying
  configuration files in a clear, modular fashion.
- Conditional Execution for Proxy Setup - Implemented conditional execution
  for the proxy setup in `http_proxy.yml`, utilizing `proxy_env` to adhere
  to Ansible's best practices for conditional tasks.
- Improved PXE-Boot System Check Block - Introduced a more structured approach
  in `main.yml` for checking PXE-booted systems, enhancing readability and
  maintainability.

**Changed:**

- Adopted Ansible Builtin Modules - Transitioned existing tasks to use
  `ansible.builtin` modules, ensuring compatibility and future-proofing the
  role.
- Refined Task Grouping - Reorganized tasks into logical blocks, improving
  the overall structure and readability, and showcasing Ansible's capabilities
  for efficient task management.
- Updated K3s Service Configuration - Modified the K3s service configuration
  task in `main.yml` for a more streamlined approach using Ansible's template
  module, reflecting community-driven best practices.

**Removed:**

- Streamlined Task Definitions - Optimized task definitions to reduce
  redundancy, focusing on clarity and adherence to the evolving Ansible
  community standards.

* Added missing checks causing failures for agents
2024-02-06 14:24:31 -06:00
Devin Buhl 1fb10faf7f fix(cilium): k3s has an API server proxy at 127.0.0.1:6444 (#442)
There is no need to use kube-vip LB
2024-02-05 04:35:51 +00:00
Techno Tim ea3b3c776a chore(deps) pre-commit updates (#438)
* chore(deps): Updated pre-commit

* fix(actions): cleaning up comments
2024-01-30 11:54:28 -06:00
dependabot[bot] 5beca87783 chore(deps): bump ansible-core from 2.16.2 to 2.16.3 (#436)
Bumps [ansible-core](https://github.com/ansible/ansible) from 2.16.2 to 2.16.3.
- [Release notes](https://github.com/ansible/ansible/releases)
- [Commits](https://github.com/ansible/ansible/compare/v2.16.2...v2.16.3)

---
updated-dependencies:
- dependency-name: ansible-core
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-29 21:29:07 -06:00
sholdee 6ffc25dfe5 Add Cilium CNI option (#435)
* Add Cilium CNI option

* Tweak version checks and add BGP resource verify

* Update metallb detection for kube-vip feat compat
2024-01-29 19:29:13 -06:00
Gereon Vey bcd37a6904 add kube-vip as a service load balancer (#432)
* add kube-vip as a service load balancer

* add molecule scenario kube-vip

---------

Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-29 09:13:13 -06:00
Techno Tim 8dd3ffc825 fix(ci): Don't run CI for certain files (#433)
* fix(ci): Don't run CI for certain files

* fix(ci): Don't run CI for certain files
2024-01-28 20:42:28 +00:00
dependabot[bot] f6ba208b5c chore(deps): bump actions/upload-artifact from 3.1.1 to 4.3.0 (#426)
Bumps [actions/upload-artifact](https://github.com/actions/upload-artifact) from 3.1.1 to 4.3.0.
- [Release notes](https://github.com/actions/upload-artifact/releases)
- [Commits](https://github.com/actions/upload-artifact/compare/83fd05a356d7e2593de66fc9913b3002723633cb...26f96dfa697d77e81fd5907df203aa23a56210a8)

---
updated-dependencies:
- dependency-name: actions/upload-artifact
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-28 19:40:47 +00:00
dependabot[bot] a22d8f7aaf chore(deps): bump zgosalvez/github-actions-ensure-sha-pinned-actions (#425)
Bumps [zgosalvez/github-actions-ensure-sha-pinned-actions](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions) from 2.0.1 to 3.0.3.
- [Release notes](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/releases)
- [Commits](https://github.com/zgosalvez/github-actions-ensure-sha-pinned-actions/compare/af2eb3226618e2494e3d9084f515ad6dcf16e229...ba37328d4ea95eaf8b3bd6c6cef308f709a5f2ec)

---
updated-dependencies:
- dependency-name: zgosalvez/github-actions-ensure-sha-pinned-actions
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-28 17:20:46 +00:00
dependabot[bot] 05fb6b566d chore(deps): bump actions/setup-python from 2.3.3 to 5.0.0 (#423) 2024-01-28 01:57:41 +00:00
egandro 3aeb7d69ea added fix for metallb version upgrades (#394)
* added fix for metallb version upgrades

* use bash to allow pipefail

---------

Co-authored-by: Harald Fielker <harald.fielker@gmail.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-28 00:50:13 +00:00
dependabot[bot] 61bf3971ef chore(deps): bump actions/checkout from 2.5.0 to 4.1.1 (#424)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2.5.0 to 4.1.1.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/e2f20e631ae6d7dd3b768f56a5d2af784dd54791...b4ffde65f46336ab88eb53be808477a3936bae11)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-27 17:26:12 -06:00
Gereon Vey 3f06a11c8d fetch kubeconfig from master after deployment (#431)
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-27 16:30:13 -06:00
Techno Tim 3888a29bb1 fix(ci): only run CI for PRs (#430)
* fix(ci): only run CI for PRs

* fix(ci): ensure that branch is up to date

* fix(ci): ensure that branch is up to date
2024-01-27 15:35:47 -06:00
Timothy Stewart 98ef696f31 fix(ci): fixes for ephemeral nodes 2024-01-26 23:12:50 -06:00
Timothy Stewart de26a79a4c fix(ci): fixes for ephemeral nodes 2024-01-26 23:09:30 -06:00
Timothy Stewart ab7ca9b551 fix(ci): fixes for ephemeral nodes 2024-01-26 23:06:02 -06:00
Timothy Stewart c5f71c9e2e fix(ci): fixes for ephemeral nodes 2024-01-26 22:52:19 -06:00
sholdee 0f23e7e258 Add Calico CNI option (#414)
* Add Tigera Operator/Calico CNI option

Small tweak to reduce delta from head

Set calico option to be disabled by default

Add rescue blocks in case updating existing

Refactor items and update comments

Refactor and consolidate calico.yml into block

Refactor to use template for Calico CRs

Revert use_calico to false

Template blockSize

Align default cidr in template with all.yml sample

Apply upstream version tags

Revert to current ver tags. Upstream's don't work.

Update template address detection

Add Tigera Operator/Calico CNI option

* Add calico-apiserver check

* Add eBPF dataplane option

* Add kube svc endpoint configmap when ebpf enabled

* Add /etc/cni/net.d to reset task

* Refactor based on comments

* Add molecule scenario

* Fix lint

---------

Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-26 18:53:27 -06:00
Techno Tim 121061d875 chore(deps) Updated LBs (#428)
* chore(deps): Updated metallb

* chore(deps): Updated kube-vip
2024-01-26 23:54:33 +00:00
João Gonçalves db53f595fd feat(k3s): added support for latest raspberrypi os (debian 12 bookworm) (#404)
* feat(k3s): added support for latest raspberrypi os (debian 12 bookworm)

* Update test.yml

* Revert test workflow

---------

Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-26 22:20:06 +00:00
Techno Tim 7b6b24ce4d feat(k3s): Updated to v1.29.0+k3s1 (#421) 2024-01-26 14:49:24 -06:00
Techno Tim a5728da35e feat(k3s): Updated to v1.28 (#420)
* feat(k3s): Updated to v1.28.5+k3s1
2024-01-26 13:10:21 -06:00
Techno Tim cda7c92203 feat(k3s): Updated to v1.27 (#294)
* feat(k3s): Updated to v1.27.1+k3s1

* feat(k3s): Updated to v1.27.1+k3s1

* feat(k3s): Updated to v1.27.4+k3s1

* feat(k3s): Updated to v1.27.9+k3s1
2024-01-26 18:54:58 +00:00
Techno Tim d910b83bf3 fix(molecule): Cleanup all artifacts, side effects, and actions in case nodes are not ephemeral (#427) 2024-01-26 17:16:26 +00:00
Techno Tim 101313f880 feat(dependabot): Added docker docker and github actions (#422) 2024-01-26 16:19:42 +00:00
Techno Tim 12be355867 feat(k3s): Updated to v1.26 (#207)
* feat(k3s): Updated to v1.26.0+k3s2

* feat(k3s): Updated to v1.26.2+k3s1

* feat(k3s): Updated to v1.26.3+k3s1

* feat(k3s): Updated to v1.26.4+k3s1

* feat(k3s): Updated to v1.26.7+k3s1

* feat(k3s): Updated to v1.26.11+k3s2

* feat(k3s): Updated to v1.26.12+k3s1
2024-01-25 22:09:08 +00:00
Gabor A aa09e3e9df fix: typos (#416)
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-25 20:40:56 +00:00
sholdee 511c410451 Add Debian Bookworm support and refactor Pi OS detection (#415)
* Refactor Pi OS detection and add Debian Bookworm support

* Add bullseye back

---------

Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-25 19:20:02 +00:00
Balázs Hasprai df9c6f3014 Fix http_proxy service dir in k3s_agent role (#400)
* Fix http_proxy service dir in k3s_agent role

* Fix http_proxy reset: rm conf files before dirs

* Fix http_proxy reset rm order

---------

Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-25 11:34:46 -06:00
Timothy Stewart 5ae8fd1223 fix(molecule): lower resources for nodes 2024-01-25 09:30:02 -06:00
Techno Tim e2e9881f0f Fix CI (#389)
did all the things to make it work
2024-01-24 22:26:38 -06:00
egandro edf0c9eebd fix for recreating new control planes (2nd run) (#393)
Co-authored-by: Harald Fielker <harald.fielker@gmail.com>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-19 08:37:14 -06:00
egandro 7669fd4721 initial galaxy.yml (#388)
* initial galaxy.yml

* added readme

* lint fix

* Updated description

Co-authored-by: Dov Benyomin Sohacheski <b@kloud.email>

* Updated license_file section

Co-authored-by: Dov Benyomin Sohacheski <b@kloud.email>

* Updated tags section

Co-authored-by: Dov Benyomin Sohacheski <b@kloud.email>

* Updated dependencies section

Co-authored-by: Dov Benyomin Sohacheski <b@kloud.email>

* removed extra empty line galaxy created

---------

Co-authored-by: Harald Fielker <harald.fielker@gmail.com>
Co-authored-by: Dov Benyomin Sohacheski <b@kloud.email>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-18 18:35:19 -06:00
Balázs Hasprai cddbfc8e40 Update truthy values to true/false only, #204 (#387)
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
2024-01-15 12:43:44 -06:00
Techno Tim 70e658cf98 feat(k3s): Updated to v1.25.16+k3s4 (#407) 2024-01-12 21:34:23 -06:00
dependabot[bot] 7badfbd7bd chore(deps): bump netaddr from 0.9.0 to 0.10.0 (#411)
Bumps [netaddr](https://github.com/drkjam/netaddr) from 0.9.0 to 0.10.0.
- [Release notes](https://github.com/drkjam/netaddr/releases)
- [Changelog](https://github.com/netaddr/netaddr/blob/master/CHANGELOG)
- [Commits](https://github.com/drkjam/netaddr/compare/0.9.0...0.10.0)

---
updated-dependencies:
- dependency-name: netaddr
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-01-01 20:46:52 -06:00
116 changed files with 4771 additions and 570 deletions
+22 -8
View File
@@ -1,17 +1,31 @@
---
profile: production
exclude_paths:
# default paths
- '.cache/'
- '.github/'
- 'test/fixtures/formatting-before/'
- 'test/fixtures/formatting-prettier/'
- .cache/
- .ansible/
- .github/
- test/fixtures/formatting-before/
- test/fixtures/formatting-prettier/
# The "converge" and "reset" playbooks use import_playbook in
# conjunction with the "env" lookup plugin, which lets the
# syntax check of ansible-lint fail.
- 'molecule/**/converge.yml'
- 'molecule/**/prepare.yml'
- 'molecule/**/reset.yml'
- molecule/**/converge.yml
- molecule/**/prepare.yml
- molecule/**/reset.yml
# Scenario verify inputs are plain variable files, not playbooks. They are
# loaded as vars, not executed, so ansible-lint must not treat them as plays.
- molecule/**/verify-vars.yml
# The file was generated by galaxy ansible - don't mess with it.
- galaxy.yml
skip_list:
- 'fqcn-builtins'
- var-naming[no-role-prefix]
# The Molecule Vagrant driver injects this module at runtime. The custom create
# playbook is syntax-checked separately against the exact pinned plugin module.
mock_modules:
- vagrant
+10 -2
View File
@@ -1,5 +1,5 @@
<!-- It's a good idea to check this post first for general troubleshooting https://github.com/techno-tim/k3s-ansible/discussions/19 -->
<!-- It's a good idea to check this post first for general troubleshooting https://github.com/timothystewart6/k3s-ansible/discussions/19 -->
<!--- Provide a general summary of the issue in the Title above -->
@@ -37,6 +37,11 @@ systemd_dir: ""
flannel_iface: ""
#calico_iface: ""
calico_ebpf: ""
calico_cidr: ""
calico_tag: ""
apiserver_endpoint: ""
k3s_token: "NA"
@@ -46,6 +51,9 @@ extra_agent_args: ""
kube_vip_tag_version: ""
kube_vip_cloud_provider_tag_version: ""
kube_vip_lb_ip_range: ""
metal_lb_speaker_tag_version: ""
metal_lb_controller_tag_version: ""
@@ -74,4 +82,4 @@ node
## Possible Solution
<!--- Not obligatory, but suggest a fix/reason for the bug, -->
- [ ] I've checked the [General Troubleshooting Guide](https://github.com/techno-tim/k3s-ansible/discussions/20)
- [ ] I've checked the [General Troubleshooting Guide](https://github.com/timothystewart6/k3s-ansible/discussions/20)
+6
View File
@@ -0,0 +1,6 @@
---
self-hosted-runner:
labels:
- k3s-ci
- virtualbox
- nested-virt
+15
View File
@@ -9,3 +9,18 @@ updates:
ignore:
- dependency-name: "*"
update-types: ["version-update:semver-major"]
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
rebase-strategy: "auto"
- package-ecosystem: "docker"
directory: "/"
schedule:
interval: "daily"
rebase-strategy: "auto"
ignore:
- dependency-name: "*"
update-types: ["version-update:semver-major"]
+75 -21
View File
@@ -1,37 +1,91 @@
#!/bin/bash
# download-boxes.sh
# Check all molecule.yml files for required Vagrant boxes and download the ones that are not
# already present on the system.
# Validate the pinned Vagrant box set and download exact versions that are not
# already present in VAGRANT_HOME.
set -euo pipefail
GIT_ROOT=$(git rev-parse --show-toplevel)
PROVIDER=virtualbox
LOCK_FILE="${VAGRANT_BOX_LOCK_FILE:-${GIT_ROOT}/.github/vagrant-boxes.lock}"
# Read all boxes for all platforms from the "molecule.yml" files
all_boxes=$(cat "${GIT_ROOT}"/molecule/*/molecule.yml |
yq -r '.platforms[].box' | # Read the "box" property of each node under "platforms"
grep --invert-match --regexp=--- | # Filter out file separators
sort |
uniq)
MOLECULE_YML_PATH=("${GIT_ROOT}"/molecule/*/molecule.yml)
# Read the boxes that are currently present on the system (for the current provider)
# Extract the unique boxes referenced by the scenarios.
declared_boxes=$(for file in "${MOLECULE_YML_PATH[@]}"; do
yq -r '.platforms[].box' "$file"
done | sort -u)
if [[ ! -r "$LOCK_FILE" ]]; then
printf 'Vagrant box lock file is missing or unreadable: %s\n' "$LOCK_FILE" >&2
exit 1
fi
lock_entries=$(awk '
/^[[:space:]]*#/ || NF == 0 { next }
NF != 3 {
printf "Invalid lock entry on line %d: expected box, version, architecture\n", NR > "/dev/stderr"
invalid = 1
next
}
{ print $1 " " $2 " " $3 }
END { exit invalid }
' "$LOCK_FILE")
duplicate_boxes=$(printf '%s\n' "$lock_entries" | awk '{ print $1 }' | sort | uniq -d)
if [[ -n "$duplicate_boxes" ]]; then
printf 'Duplicate Vagrant box lock entries:\n%s\n' "$duplicate_boxes" >&2
exit 1
fi
locked_boxes=$(printf '%s\n' "$lock_entries" | sort)
locked_names=$(printf '%s\n' "$locked_boxes" | awk '{ print $1 }')
missing_locks=$(comm -23 <(printf '%s\n' "$declared_boxes") <(printf '%s\n' "$locked_names"))
unused_locks=$(comm -13 <(printf '%s\n' "$declared_boxes") <(printf '%s\n' "$locked_names"))
if [[ -n "$missing_locks" || -n "$unused_locks" ]]; then
if [[ -n "$missing_locks" ]]; then
printf 'Scenario boxes missing from the lock file:\n%s\n' "$missing_locks" >&2
fi
if [[ -n "$unused_locks" ]]; then
printf 'Lock entries not referenced by a scenario:\n%s\n' "$unused_locks" >&2
fi
exit 1
fi
printf 'Pinned Vagrant boxes:\n%s\n' "$locked_boxes"
# Read exact box, provider, version, and architecture tuples already present.
present_boxes=$(
(vagrant box list |
grep "${PROVIDER}" | # Filter by boxes available for the current provider
awk '{print $1;}' | # The box name is the first word in each line
sort |
uniq) ||
echo "" # In case any of these commands fails, just use an empty list
vagrant box list --machine-readable |
awk -F, -v expected_provider="$PROVIDER" '
$3 == "box-name" { name = $4; next }
$3 == "box-provider" { provider = $4; next }
$3 == "box-version" { version = $4; next }
$3 == "box-architecture" {
architecture = $4
if (provider == expected_provider) {
print name " " version " " architecture
}
name = provider = version = architecture = ""
}
' |
sort -u
)
# The boxes that we need to download are the ones present in $all_boxes, but not $present_boxes.
download_boxes=$(comm -2 -3 <(echo "${all_boxes}") <(echo "${present_boxes}"))
download_boxes=$(comm -23 \
<(printf '%s\n' "$locked_boxes") \
<(printf '%s\n' "$present_boxes"))
# Actually download the necessary boxes
if [ -n "${download_boxes}" ]; then
echo "${download_boxes}" | while IFS= read -r box; do
vagrant box add --provider "${PROVIDER}" "${box}"
if [[ -n "$download_boxes" ]]; then
printf '%s\n' "$download_boxes" | while read -r box version architecture; do
vagrant box add \
--provider "$PROVIDER" \
--box-version "$version" \
--architecture "$architecture" \
"$box"
done
else
printf 'All pinned Vagrant boxes are already present.\n'
fi
+249
View File
@@ -0,0 +1,249 @@
#!/usr/bin/env bash
set -Eeuo pipefail
usage() {
printf '%s\n' \
'Usage: cleanup-runner-resources.sh [--snapshot|--dry-run|--apply]' \
'' \
'Discover and, with --apply, remove only VirtualBox resources referenced by' \
'repository-owned Molecule Vagrant state. The default is --dry-run.'
}
mode="dry-run"
case "${1:-}" in
"") ;;
--snapshot) mode="snapshot" ;;
--dry-run) mode="dry-run" ;;
--apply) mode="apply" ;;
--help|-h) usage; exit 0 ;;
*) usage >&2; exit 2 ;;
esac
home_dir="${HOME:?HOME must be set}"
molecule_root="${K3S_CI_MOLECULE_ROOT:-${home_dir}/.cache/molecule}"
repository_name="${K3S_CI_MOLECULE_PROJECT:-k3s-ansible}"
virtualbox_root="${K3S_CI_VIRTUALBOX_ROOT:-${home_dir}/VirtualBox VMs}"
hostonly_marker="${K3S_CI_HOSTONLY_MARKER:-${home_dir}/.cache/k3s-ci/hostonly-interfaces}"
record_hostonly() {
local marker_dir="${hostonly_marker%/*}"
local marker_tmp="${hostonly_marker}.tmp"
local hostonly_inventory
if ! hostonly_inventory="$(VBoxManage list hostonlyifs)"; then
fail_closed 'unable to inventory VirtualBox host-only interfaces'
fi
mkdir -p -- "$marker_dir"
awk -F': ' '
/^Name:/ { name=$2 }
/^IPAddress:/ { print name "|" $2 }
' <<< "$hostonly_inventory" > "$marker_tmp"
mv -- "$marker_tmp" "$hostonly_marker"
chmod 600 "$hostonly_marker"
printf 'Recorded host-only interface baseline: %s\n' "$hostonly_marker"
}
cleanup_hostonly() {
local hostonly_inventory
if [[ ! -f "$hostonly_marker" ]]; then
printf 'No host-only interface baseline found; leaving interfaces unchanged.\n'
return 0
fi
if ! hostonly_inventory="$(VBoxManage list hostonlyifs)"; then
fail_closed 'unable to inventory VirtualBox host-only interfaces'
fi
while IFS='|' read -r interface_name interface_ip; do
[[ "$interface_name" == vboxnet* ]] || continue
[[ "$interface_ip" == 192.168.30.* || "$interface_ip" == fdad:bad:ba55:* ]] || continue
if grep -Fqx "${interface_name}|${interface_ip}" "$hostonly_marker"; then
continue
fi
if [[ "$mode" == apply ]]; then
VBoxManage hostonlyif remove "$interface_name"
printf 'Removed host-only interface %s (%s)\n' "$interface_name" "$interface_ip"
else
printf 'Would remove host-only interface %s (%s)\n' "$interface_name" "$interface_ip"
fi
done < <(awk -F': ' '
/^Name:/ { name=$2 }
/^IPAddress:/ { print name "|" $2 }
' <<< "$hostonly_inventory")
}
if [[ "$mode" == snapshot ]]; then
record_hostonly
exit 0
fi
resolve_existing_dir() {
local candidate="$1"
if [[ ! -d "$candidate" ]]; then
return 1
fi
readlink -f -- "$candidate"
}
root_contains() {
local root="$1"
local path="$2"
[[ "$path" == "$root"/* ]]
}
is_supported_scenario() {
case "$1" in
default|single_node|calico|cilium|kube-vip|ipv6) return 0 ;;
*) return 1 ;;
esac
}
is_unregistered_vm_error() {
grep -Eq 'Could not find a registered machine|VBOX_E_OBJECT_NOT_FOUND'
}
fail_closed() {
printf 'cleanup refused: %s\n' "$1" >&2
exit 3
}
if [[ ! "$repository_name" =~ ^[A-Za-z0-9._-]+$ ]]; then
fail_closed 'invalid Molecule repository name'
fi
print_inventory() {
local phase="$1"
printf '%s VirtualBox inventory:\n' "$phase"
VBoxManage list vms || true
VBoxManage list hdds || true
VBoxManage list hostonlyifs || true
}
molecule_root_real="$(resolve_existing_dir "$molecule_root" || true)"
if [[ -z "$molecule_root_real" ]]; then
printf 'No Molecule root exists: %s\n' "$molecule_root"
cleanup_hostonly
exit 0
fi
print_inventory before
repository_root_real="$(resolve_existing_dir "$molecule_root_real/$repository_name" || true)"
if [[ -z "$repository_root_real" ]]; then
printf 'No repository Molecule state root exists: %s\n' "$molecule_root_real/$repository_name"
cleanup_hostonly
print_inventory after
exit 0
fi
if ! root_contains "$molecule_root_real" "$repository_root_real"; then
fail_closed "repository Molecule state root is outside Molecule root: $repository_root_real"
fi
declare -a state_files=()
while IFS= read -r -d '' state_file; do
state_files+=("$state_file")
done < <(find "$repository_root_real" -mindepth 6 -maxdepth 6 -type f \
-path '*/.vagrant/machines/*/virtualbox/id' -print0 2>/dev/null)
if ((${#state_files[@]} == 0)); then
printf 'No repository-owned Molecule Vagrant state found under %s\n' "$repository_root_real"
cleanup_hostonly
print_inventory after
exit 0
fi
virtualbox_root_real="$(resolve_existing_dir "$virtualbox_root" || true)"
declare -a vm_records=()
for state_file in "${state_files[@]}"; do
if [[ ! -f "$state_file" ]]; then
printf 'Skipping Vagrant state removed with its stale scenario directory: %s\n' "$state_file"
continue
fi
state_file_real="$(readlink -f -- "$state_file")"
state_dir="${state_file_real%/.vagrant/machines/*/virtualbox/id}"
machine_dir="${state_file_real%/virtualbox/id}"
machine_name="${machine_dir##*/}"
scenario_name="${state_dir##*/}"
if ! root_contains "$repository_root_real" "$state_dir"; then
fail_closed "state path is outside the repository Molecule root: $state_file_real"
fi
if ! is_supported_scenario "$scenario_name"; then
fail_closed "unexpected Molecule scenario: $scenario_name"
fi
if [[ "$machine_name" != control* && "$machine_name" != node* ]]; then
fail_closed "unexpected Molecule machine name: $machine_name"
fi
vm_uuid="$(tr -d '[:space:]' < "$state_file_real")"
if [[ ! "$vm_uuid" =~ ^[0-9a-fA-F-]{36}$ ]]; then
fail_closed "invalid VirtualBox UUID in $state_file_real"
fi
if ! vm_info="$(VBoxManage showvminfo "$vm_uuid" --machinereadable 2>&1)"; then
if is_unregistered_vm_error <<< "$vm_info"; then
printf 'Stale Vagrant state without a registered VM: %s (%s)\n' "$machine_name" "$vm_uuid"
if [[ "$mode" == apply ]]; then
rm -rf -- "${state_dir}/.vagrant"
printf 'Removed stale Vagrant state: %s\n' "${state_dir}/.vagrant"
fi
continue
fi
fail_closed "unable to inspect VirtualBox VM $vm_uuid: $vm_info"
fi
if [[ -z "$virtualbox_root_real" ]]; then
fail_closed "VirtualBox VM root does not exist: $virtualbox_root"
fi
cfg_file="$(awk -F= '$1 == "CfgFile" {gsub(/"/, "", $2); print $2; exit}' <<< "$vm_info")"
if [[ -z "$cfg_file" ]]; then
fail_closed "VirtualBox configuration path missing for $vm_uuid"
fi
cfg_file_real="$(readlink -f -- "$cfg_file")"
if ! root_contains "$virtualbox_root_real" "$cfg_file_real"; then
fail_closed "VM configuration is outside VirtualBox root: $cfg_file_real"
fi
while IFS= read -r disk_path; do
[[ -z "$disk_path" ]] && continue
disk_path_real="$(readlink -f -- "$disk_path" 2>/dev/null || true)"
if [[ -z "$disk_path_real" ]] || ! root_contains "$virtualbox_root_real" "$disk_path_real"; then
fail_closed "attached disk is outside VirtualBox root: $disk_path"
fi
done < <(awk -F= '$1 ~ /^(SATA|IDE|SCSI|SAS|VirtioSCSI|NVMe)-[0-9]+-[0-9]+$/ {gsub(/"/, "", $2); print $2}' <<< "$vm_info")
vm_records+=("$vm_uuid|$machine_name|$cfg_file_real")
done
if ((${#vm_records[@]} == 0)); then
printf 'No live repository-owned VirtualBox resources found\n'
cleanup_hostonly
print_inventory after
exit 0
fi
for record in "${vm_records[@]}"; do
IFS='|' read -r vm_uuid machine_name cfg_file_real <<< "$record"
if [[ "$mode" == dry-run ]]; then
printf 'Would remove VM %s (%s) config=%s\n' "$machine_name" "$vm_uuid" "$cfg_file_real"
continue
fi
vm_state="$(VBoxManage showvminfo "$vm_uuid" --machinereadable | awk -F= '$1 == "VMState" {gsub(/"/, "", $2); print $2; exit}')"
if [[ "$vm_state" != poweroff && "$vm_state" != saved ]]; then
VBoxManage controlvm "$vm_uuid" poweroff
fi
VBoxManage unregistervm "$vm_uuid" --delete
printf 'Removed VM %s (%s)\n' "$machine_name" "$vm_uuid"
done
cleanup_hostonly
print_inventory after
if [[ "$mode" == apply ]]; then
printf 'Repository-owned VM and host-only interface cleanup complete.\n'
else
printf 'Dry run complete. No resources were modified.\n'
fi
+47
View File
@@ -0,0 +1,47 @@
#!/usr/bin/env bash
set -Eeuo pipefail
output_dir="${1:-${RUNNER_TEMP:-/tmp}/k3s-ci-diagnostics}"
mkdir -p -- "$output_dir"
umask 077
run_capture() {
local output_file="$1"
shift
{
printf '$'
printf ' %q' "$@"
printf '\n'
"$@"
} > "$output_dir/$output_file" 2>&1 || true
}
run_capture system.txt uname -a
run_capture runner-user.txt id
run_capture memory.txt free -h
run_capture disk.txt df -h
run_capture virtualbox-version VBoxManage --version
run_capture virtualbox-vms VBoxManage list vms
run_capture virtualbox-running-vms VBoxManage list runningvms
run_capture virtualbox-disks VBoxManage list hdds
run_capture virtualbox-hostonlyifs VBoxManage list hostonlyifs
run_capture virtualbox-groups VBoxManage list groups
run_capture vagrant-status vagrant global-status
run_capture molecule-state find "${HOME}/.cache/molecule" -maxdepth 6 -type f -path '*/.vagrant/machines/*/virtualbox/id' -print
scenario_name="${K3S_CI_SCENARIO_NAME:-}"
if [[ "$scenario_name" =~ ^[A-Za-z0-9_-]+$ ]]; then
molecule_state_dir="${HOME}/.cache/molecule/k3s-ansible/${scenario_name}"
for log_name in vagrant.out vagrant.err; do
if [[ -r "${molecule_state_dir}/${log_name}" ]]; then
cp -- "${molecule_state_dir}/${log_name}" "$output_dir/${scenario_name}-${log_name}"
fi
done
fi
if [[ -r /etc/vbox/networks.conf ]]; then
cp -- /etc/vbox/networks.conf "$output_dir/virtualbox-networks.conf"
fi
printf 'Diagnostics written to %s\n' "$output_dir"
+44
View File
@@ -0,0 +1,44 @@
#!/usr/bin/env bash
set -Eeuo pipefail
output_dir="${1:?output directory is required}"
interval="${2:-10}"
[[ "$interval" =~ ^[1-9][0-9]*$ ]] || {
printf 'monitor interval must be a positive integer\n' >&2
exit 2
}
mkdir -p -- "$output_dir"
umask 077
free -h > "$output_dir/memory-before.txt"
df -h > "$output_dir/disk-before.txt"
vmstat -w "$interval" > "$output_dir/vmstat.txt" &
vmstat_pid=$!
iostat_pid=""
if command -v iostat >/dev/null 2>&1; then
iostat -dx "$interval" > "$output_dir/iostat.txt" &
iostat_pid=$!
else
printf 'iostat is not installed on this runner\n' > "$output_dir/iostat-unavailable.txt"
fi
cleanup() {
local rc=$?
trap - EXIT INT TERM
kill "$vmstat_pid" 2>/dev/null || true
[[ -z "$iostat_pid" ]] || kill "$iostat_pid" 2>/dev/null || true
wait "$vmstat_pid" 2>/dev/null || true
[[ -z "$iostat_pid" ]] || wait "$iostat_pid" 2>/dev/null || true
free -h > "$output_dir/memory-after.txt"
df -h > "$output_dir/disk-after.txt"
exit "$rc"
}
trap cleanup EXIT INT TERM
while :; do
sleep 3600 &
wait $!
done
+211
View File
@@ -0,0 +1,211 @@
#!/usr/bin/env bash
set -Eeuo pipefail
fail() {
printf 'Vagrant box master preparation refused: %s\n' "$1" >&2
exit 3
}
root_contains() {
local root="$1"
local path="$2"
[[ "$path" == "$root"/* ]]
}
read_machine_value() {
local machine_info="$1"
local key="$2"
awk -F= -v key="$key" '$1 == key {gsub(/"/, "", $2); print $2; exit}' <<< "$machine_info"
}
read_extra_data() {
local uuid="$1"
local key="$2"
local value
value="$(VBoxManage getextradata "$uuid" "$key" 2>/dev/null || true)"
[[ "$value" == 'Value: '* ]] || return 1
printf '%s\n' "${value#Value: }"
}
validate_owned_master() {
local uuid="$1"
local box="$2"
local version="$3"
local architecture="$4"
local machine_info cfg_file cfg_file_real vm_state groups disk_path disk_path_real
[[ "$uuid" =~ ^[0-9a-fA-F-]{36}$ ]] || return 1
machine_info="$(VBoxManage showvminfo "$uuid" --machinereadable 2>/dev/null)" || return 1
vm_state="$(read_machine_value "$machine_info" VMState)"
groups="$(read_machine_value "$machine_info" groups)"
cfg_file="$(read_machine_value "$machine_info" CfgFile)"
[[ "$vm_state" == poweroff ]] || return 1
[[ ",$groups," == *,/k3s-ansible/box-masters,* ]] || return 1
[[ -n "$cfg_file" ]] || return 1
cfg_file_real="$(readlink -f -- "$cfg_file" 2>/dev/null || true)"
[[ -n "$cfg_file_real" ]] || return 1
root_contains "$virtualbox_root_real" "$cfg_file_real" || return 1
[[ "$(read_extra_data "$uuid" k3s-ansible/owner || true)" == box-master ]] || return 1
[[ "$(read_extra_data "$uuid" k3s-ansible/box || true)" == "$box" ]] || return 1
[[ "$(read_extra_data "$uuid" k3s-ansible/version || true)" == "$version" ]] || return 1
[[ "$(read_extra_data "$uuid" k3s-ansible/architecture || true)" == "$architecture" ]] || return 1
while IFS= read -r disk_path; do
[[ -z "$disk_path" || "$disk_path" == none ]] && continue
disk_path_real="$(readlink -f -- "$disk_path" 2>/dev/null || true)"
[[ -n "$disk_path_real" ]] || return 1
root_contains "$virtualbox_root_real" "$disk_path_real" || return 1
done < <(awk -F= '$1 ~ /^(SATA|IDE|SCSI|SAS|VirtioSCSI|NVMe)-[0-9]+-[0-9]+$/ {
gsub(/"/, "", $2); print $2
}' <<< "$machine_info")
}
write_prewarm_vagrantfile() {
local destination="$1"
local box="$2"
local version="$3"
{
printf '%s\n' "Vagrant.configure('2') do |config|"
printf ' config.vm.box = "%s"\n' "$box"
printf ' config.vm.box_version = "%s"\n' "$version"
printf '%s\n' \
' config.vm.synced_folder ".", "/vagrant", disabled: true' \
' config.vm.hostname = "k3s-ansible-box-prewarm"' \
' config.vm.boot_timeout = 600' \
' config.vm.provider "virtualbox" do |virtualbox|' \
' virtualbox.linked_clone = true' \
' virtualbox.memory = 1024' \
' virtualbox.cpus = 2' \
' end' \
'end'
} > "$destination"
}
cleanup_prewarm() {
local rc=$?
trap - EXIT
if [[ -n "${prewarm_dir:-}" && -d "$prewarm_dir" ]]; then
VAGRANT_CWD="$prewarm_dir" vagrant destroy --force >/dev/null 2>&1 || true
rm -rf -- "$prewarm_dir"
fi
exit "$rc"
}
create_owned_master() {
local box="$1"
local version="$2"
local architecture="$3"
local master_id_file="$4"
local mapping_file="$5"
local uuid machine_info cfg_file cfg_file_real vm_state mapping_tmp
# A master_id restored from an immutable cache is only a hint. Without the
# runner-local ownership record and matching VirtualBox metadata it is not
# trusted, adopted, modified, or deleted.
rm -f -- "$master_id_file"
prewarm_dir="$(mktemp -d "${master_root}/prewarm.XXXXXX")"
trap cleanup_prewarm EXIT
write_prewarm_vagrantfile "$prewarm_dir/Vagrantfile" "$box" "$version"
printf 'Creating runner-owned linked-clone master for %s %s %s\n' \
"$box" "$version" "$architecture"
VAGRANT_CWD="$prewarm_dir" vagrant up --provider virtualbox --no-provision
[[ -r "$master_id_file" ]] || fail "Vagrant did not record a master UUID for $box"
uuid="$(tr -d '[:space:]' < "$master_id_file")"
[[ "$uuid" =~ ^[0-9a-fA-F-]{36}$ ]] || fail "Vagrant recorded an invalid master UUID for $box"
machine_info="$(VBoxManage showvminfo "$uuid" --machinereadable 2>/dev/null)" || \
fail "Vagrant master $uuid for $box is not registered"
vm_state="$(read_machine_value "$machine_info" VMState)"
cfg_file="$(read_machine_value "$machine_info" CfgFile)"
cfg_file_real="$(readlink -f -- "$cfg_file" 2>/dev/null || true)"
[[ "$vm_state" == poweroff ]] || fail "new Vagrant master $uuid is not powered off"
if [[ -z "$cfg_file_real" ]] || ! root_contains "$virtualbox_root_real" "$cfg_file_real"; then
fail "new Vagrant master $uuid is outside the runner VirtualBox root"
fi
VAGRANT_CWD="$prewarm_dir" vagrant destroy --force
VBoxManage modifyvm "$uuid" --groups /k3s-ansible/box-masters
VBoxManage setextradata "$uuid" k3s-ansible/owner box-master
VBoxManage setextradata "$uuid" k3s-ansible/box "$box"
VBoxManage setextradata "$uuid" k3s-ansible/version "$version"
VBoxManage setextradata "$uuid" k3s-ansible/architecture "$architecture"
validate_owned_master "$uuid" "$box" "$version" "$architecture" || \
fail "new Vagrant master $uuid failed ownership validation"
rm -rf -- "$prewarm_dir"
prewarm_dir=""
trap - EXIT
mapping_tmp="${mapping_file}.tmp"
printf '%s\n' "$uuid" > "$mapping_tmp"
chmod 600 "$mapping_tmp"
mv -- "$mapping_tmp" "$mapping_file"
printf '%s\n' "$uuid" > "$master_id_file"
chmod 600 "$master_id_file"
printf 'Created and recorded owned master %s for %s\n' "$uuid" "$box"
}
repository_root="${K3S_CI_REPOSITORY_ROOT:-$(git rev-parse --show-toplevel)}"
lock_file="${VAGRANT_BOX_LOCK_FILE:-${repository_root}/.github/vagrant-boxes.lock}"
vagrant_home="${VAGRANT_HOME:?VAGRANT_HOME must be set}"
master_root="${K3S_CI_VAGRANT_MASTER_ROOT:-${HOME:?HOME must be set}/.cache/k3s-ci/vagrant-masters}"
virtualbox_root="${K3S_CI_VIRTUALBOX_ROOT:-${HOME}/VirtualBox VMs}"
[[ -r "$lock_file" ]] || fail "box lock file is missing or unreadable: $lock_file"
[[ -d "$vagrant_home/boxes" ]] || fail "Vagrant box directory is missing: $vagrant_home/boxes"
[[ -d "$virtualbox_root" ]] || fail "VirtualBox root is missing: $virtualbox_root"
box_root_real="$(readlink -f -- "$vagrant_home/boxes")"
virtualbox_root_real="$(readlink -f -- "$virtualbox_root")"
mkdir -p -- "$master_root"
chmod 700 "$master_root"
exec 9> "${master_root}/prepare.lock"
flock 9
lock_entries="$(awk '
/^[[:space:]]*#/ || NF == 0 { next }
NF != 3 { invalid = 1; next }
{ print $1 " " $2 " " $3 }
END { exit invalid }
' "$lock_file")" || fail 'invalid Vagrant box lock entry'
[[ -n "$lock_entries" ]] || fail 'Vagrant box lock is empty'
while read -r box version architecture; do
[[ "$box" =~ ^[A-Za-z0-9._-]+/[A-Za-z0-9._-]+$ ]] || fail "invalid box name: $box"
[[ "$version" =~ ^[A-Za-z0-9._-]+$ ]] || fail "invalid box version: $version"
[[ "$architecture" =~ ^[A-Za-z0-9._-]+$ ]] || fail "invalid box architecture: $architecture"
box_slug="${box//\//-VAGRANTSLASH-}"
record_slug="${box//\//_}-${version}-${architecture}"
box_dir="${vagrant_home}/boxes/${box_slug}/${version}/${architecture}/virtualbox"
[[ -d "$box_dir" ]] || fail "pinned box is not installed: $box $version $architecture"
box_dir_real="$(readlink -f -- "$box_dir")"
root_contains "$box_root_real" "$box_dir_real" || fail "box directory is outside VAGRANT_HOME: $box_dir_real"
master_id_file="${box_dir_real}/master_id"
mapping_file="${master_root}/${record_slug}.uuid"
uuid=""
if [[ -r "$mapping_file" ]]; then
uuid="$(tr -d '[:space:]' < "$mapping_file")"
fi
if [[ -n "$uuid" ]] && validate_owned_master "$uuid" "$box" "$version" "$architecture"; then
printf '%s\n' "$uuid" > "$master_id_file"
chmod 600 "$master_id_file"
printf 'Reusing owned master %s for %s %s %s\n' "$uuid" "$box" "$version" "$architecture"
continue
fi
if [[ -e "$mapping_file" ]]; then
printf 'Owned master record is stale for %s; rebuilding without deleting any VM or disk.\n' "$box"
fi
create_owned_master "$box" "$version" "$architecture" "$master_id_file" "$mapping_file"
done <<< "$lock_entries"
printf 'All pinned Vagrant box masters are ready.\n'
+134
View File
@@ -0,0 +1,134 @@
#!/usr/bin/env python3
"""Render the Cilium BGP CRD template and assert it uses the v2 API.
This is a manifest-only regression test used where no real BGP peer is
available. It renders roles/k3s_server_post/templates/cilium.crs.j2 with
zero, one, and multiple neighbors, then checks that the output:
- never contains CiliumBGPPeeringPolicy or cilium.io/v2alpha1
- emits the Cilium v2 BGP resources
- emits deterministic DNS-safe peer and instance names
- advertises Pod CIDRs only when cilium_exportPodCIDR is true
"""
from __future__ import print_function
import os
import subprocess
from jinja2 import Environment, FileSystemLoader, StrictUndefined
def repo_root():
return subprocess.check_output(
["git", "rev-parse", "--show-toplevel"], text=True
).strip()
def fail(message):
raise SystemExit("Cilium BGP manifest test failed: " + message)
def render(env, extra_vars):
base_vars = {
"cilium_bgp_my_asn": "64513",
"cilium_bgp_peer_asn": "64512",
"cilium_bgp_peer_address": "192.168.30.1",
"cilium_exportPodCIDR": True,
"cilium_bgp_lb_cidr": "192.168.31.0/24",
}
base_vars.update(extra_vars)
template = env.get_template("cilium.crs.j2")
return template.render(**base_vars)
def check_common(output):
if "cilium.io/v2alpha1" in output:
fail("rendered output still contains cilium.io/v2alpha1")
if "kind: CiliumBGPPeeringPolicy" in output:
fail("rendered output still contains CiliumBGPPeeringPolicy")
for kind in (
"CiliumBGPPeerConfig",
"CiliumBGPClusterConfig",
"CiliumBGPAdvertisement",
"CiliumLoadBalancerIPPool",
):
if ("kind: " + kind) not in output:
fail("rendered output is missing kind: " + kind)
def main():
root = repo_root()
template_dir = os.path.join(
root, "roles", "k3s_server_post", "templates"
)
env = Environment(
loader=FileSystemLoader(template_dir), undefined=StrictUndefined
)
# Zero neighbors -> fall back to the single default peer.
output = render(env, {"_cilium_bgp_neighbors": []})
check_common(output)
if "peer-64512-1" not in output:
fail("default single peer name was not rendered")
if "peerAddress: 192.168.30.1" not in output:
fail("default peer address was not rendered")
if 'advertisementType: "PodCIDR"' not in output:
fail("PodCIDR advertisement missing when exportPodCIDR is true")
# One neighbor via the merged list.
output = render(
env,
{"_cilium_bgp_neighbors": [{"peer_address": "10.0.0.1", "peer_asn": "65001"}]},
)
check_common(output)
if "peer-65001-1" not in output:
fail("single merged peer name was not rendered")
if "peerAddress: 10.0.0.1" not in output:
fail("single merged peer address was not rendered")
# Multiple neighbors.
output = render(
env,
{
"_cilium_bgp_neighbors": [
{"peer_address": "10.0.0.1", "peer_asn": "65001"},
{"peer_address": "10.0.0.2", "peer_asn": "65002"},
]
},
)
check_common(output)
if "peer-65001-1" not in output or "peer-65002-2" not in output:
fail("multiple merged peer names were not rendered")
if "peerAddress: 10.0.0.2" not in output:
fail("second merged peer address was not rendered")
# exportPodCIDR false -> no PodCIDR advertisement, service remains.
output = render(
env, {"_cilium_bgp_neighbors": [], "cilium_exportPodCIDR": False}
)
check_common(output)
if 'advertisementType: "PodCIDR"' in output:
fail("PodCIDR advertisement present when exportPodCIDR is false")
if 'advertisementType: "Service"' not in output:
fail("Service advertisement missing when exportPodCIDR is false")
# Load balancer pools: CIDR and start/stop forms.
output = render(env, {"_cilium_bgp_neighbors": []})
if "cidr: 192.168.31.0/24" not in output:
fail("CIDR load balancer pool was not rendered")
output = render(
env,
{
"_cilium_bgp_neighbors": [],
"cilium_bgp_lb_cidr": "192.168.31.80-192.168.31.90",
},
)
check_common(output)
if "start: 192.168.31.80" not in output or "stop: 192.168.31.90" not in output:
fail("start/stop load balancer pool was not rendered")
print("Cilium BGP manifest regression test passed")
if __name__ == "__main__":
main()
+129
View File
@@ -0,0 +1,129 @@
#!/usr/bin/env bash
# shellcheck disable=SC2016
set -Eeuo pipefail
repo_root="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/../.." && pwd)"
test_root="$(mktemp -d "${TMPDIR:-/tmp}/k3s-ci-cleanup-test.XXXXXX")"
trap 'rm -rf -- "$test_root"' EXIT
molecule_root="$test_root/molecule"
virtualbox_root="$test_root/VirtualBox VMs"
fake_bin="$test_root/bin"
mkdir -p -- "$molecule_root/k3s-ansible/single_node/.vagrant/machines/control1/virtualbox" \
"$molecule_root/k3s-ansible/single_node/.vagrant/machines/control2/virtualbox" \
"$virtualbox_root/control1" "$virtualbox_root/unmarked" "$fake_bin"
printf '%s\n' '11111111-1111-1111-1111-111111111111' \
> "$molecule_root/k3s-ansible/single_node/.vagrant/machines/control1/virtualbox/id"
printf '%s\n' '22222222-2222-2222-2222-222222222222' \
> "$molecule_root/k3s-ansible/single_node/.vagrant/machines/control2/virtualbox/id"
touch "$virtualbox_root/control1/control1.vbox" "$virtualbox_root/control1/disk.vdi" \
"$virtualbox_root/unmarked/unmarked.vbox"
printf '%s\n' \
'#!/usr/bin/env bash' \
'set -Eeuo pipefail' \
'case "${1:-}" in' \
' list)' \
' if [[ "${2:-}" == hostonlyifs && "${FAKE_HOSTONLY_FAIL:-false}" == true ]]; then exit 1; fi' \
' exit 0' \
' ;;' \
' showvminfo)' \
' if [[ "${FAKE_VM_MODE:-normal}" == missing ]]; then' \
' printf '\''VBoxManage: error: Could not find a registered machine named "missing"\n'\'' >&2' \
' exit 1' \
' fi' \
' if [[ "${FAKE_VM_MODE:-normal}" == fault ]]; then' \
' printf '\''VBoxManage: error: VirtualBox service is unavailable\n'\'' >&2' \
' exit 1' \
' fi' \
' printf '\''CfgFile="%s"\n'\'' "${FAKE_VBOX_ROOT}/control1/control1.vbox"' \
' printf '\''SATA-0-0="%s"\n'\'' "${FAKE_VBOX_ROOT}/control1/disk.vdi"' \
' printf '\''VMState="running"\n'\''' \
' ;;' \
' controlvm) printf '\''controlvm %s\n'\'' "$*" >> "${FAKE_LOG}" ;;' \
' unregistervm)' \
' printf '\''unregistervm %s\n'\'' "$*" >> "${FAKE_LOG}"' \
' rm -f -- "${FAKE_VBOX_ROOT}/control1/control1.vbox" "${FAKE_VBOX_ROOT}/control1/disk.vdi"' \
' ;;' \
' *) : ;;' \
'esac' > "$fake_bin/VBoxManage"
chmod 700 "$fake_bin/VBoxManage"
output="$test_root/output.txt"
if PATH="$fake_bin:$PATH" \
HOME="$test_root/home" \
K3S_CI_MOLECULE_ROOT="$molecule_root" \
K3S_CI_MOLECULE_PROJECT=k3s-ansible \
K3S_CI_VIRTUALBOX_ROOT="$virtualbox_root" \
K3S_CI_HOSTONLY_MARKER="$test_root/hostonly-baseline" \
FAKE_VM_MODE=fault \
FAKE_VBOX_ROOT="$virtualbox_root" \
FAKE_LOG="$test_root/vbox.log" \
bash "$repo_root/.github/scripts/cleanup-runner-resources.sh" --apply > "$output" 2>&1; then
printf '%s\n' 'cleanup unexpectedly accepted a VirtualBox inspection failure' >&2
exit 1
fi
grep -Fq 'cleanup refused: unable to inspect VirtualBox VM' "$output"
[[ -f "$molecule_root/k3s-ansible/single_node/.vagrant/machines/control1/virtualbox/id" ]]
printf '%s\n' 'vboxnet0|192.168.30.1' > "$test_root/hostonly-baseline"
if PATH="$fake_bin:$PATH" \
HOME="$test_root/home" \
K3S_CI_MOLECULE_ROOT="$molecule_root" \
K3S_CI_MOLECULE_PROJECT=k3s-ansible \
K3S_CI_VIRTUALBOX_ROOT="$virtualbox_root" \
K3S_CI_HOSTONLY_MARKER="$test_root/hostonly-baseline" \
FAKE_HOSTONLY_FAIL=true \
FAKE_VBOX_ROOT="$virtualbox_root" \
FAKE_LOG="$test_root/vbox.log" \
bash "$repo_root/.github/scripts/cleanup-runner-resources.sh" --dry-run > "$output" 2>&1; then
printf '%s\n' 'cleanup unexpectedly accepted a host-only inventory failure' >&2
exit 1
fi
grep -Fq 'cleanup refused: unable to inventory VirtualBox host-only interfaces' "$output"
PATH="$fake_bin:$PATH" \
HOME="$test_root/home" \
K3S_CI_MOLECULE_ROOT="$molecule_root" \
K3S_CI_MOLECULE_PROJECT=k3s-ansible \
K3S_CI_VIRTUALBOX_ROOT="$virtualbox_root" \
K3S_CI_HOSTONLY_MARKER="$test_root/hostonly-baseline" \
FAKE_VBOX_ROOT="$virtualbox_root" \
FAKE_LOG="$test_root/vbox.log" \
bash "$repo_root/.github/scripts/cleanup-runner-resources.sh" --dry-run > "$output"
grep -Fq 'Would remove VM control1 (11111111-1111-1111-1111-111111111111)' "$output"
[[ ! -e "$test_root/vbox.log" ]]
PATH="$fake_bin:$PATH" \
HOME="$test_root/home" \
K3S_CI_MOLECULE_ROOT="$molecule_root" \
K3S_CI_MOLECULE_PROJECT=k3s-ansible \
K3S_CI_VIRTUALBOX_ROOT="$virtualbox_root" \
K3S_CI_HOSTONLY_MARKER="$test_root/hostonly-baseline" \
FAKE_VBOX_ROOT="$virtualbox_root" \
FAKE_LOG="$test_root/vbox.log" \
bash "$repo_root/.github/scripts/cleanup-runner-resources.sh" --apply > "$output"
grep -Fq 'controlvm 11111111-1111-1111-1111-111111111111 poweroff' "$test_root/vbox.log"
grep -Fq 'unregistervm unregistervm 11111111-1111-1111-1111-111111111111 --delete' "$test_root/vbox.log"
[[ ! -e "$virtualbox_root/control1/control1.vbox" ]]
[[ -e "$virtualbox_root/unmarked/unmarked.vbox" ]]
PATH="$fake_bin:$PATH" \
HOME="$test_root/home" \
K3S_CI_MOLECULE_ROOT="$molecule_root" \
K3S_CI_MOLECULE_PROJECT=k3s-ansible \
K3S_CI_VIRTUALBOX_ROOT="$virtualbox_root" \
K3S_CI_HOSTONLY_MARKER="$test_root/hostonly-baseline" \
FAKE_VM_MODE=missing \
FAKE_VBOX_ROOT="$virtualbox_root" \
FAKE_LOG="$test_root/vbox.log" \
bash "$repo_root/.github/scripts/cleanup-runner-resources.sh" --apply > "$output"
grep -Fq 'Stale Vagrant state without a registered VM' "$output"
[[ ! -d "$molecule_root/k3s-ansible/single_node/.vagrant" ]]
printf 'cleanup-runner-resources fixture test passed\n'
+4
View File
@@ -0,0 +1,4 @@
#!/usr/bin/env bash
set -Eeuo pipefail
[[ "${1:-}" =~ ^[0-9]+$ ]]
+30
View File
@@ -0,0 +1,30 @@
#!/usr/bin/env bash
set -Eeuo pipefail
printf '%s\n' "$*" >> "$MOCK_VAGRANT_LOG"
case "${1:-}" in
up)
if [[ -n "${MOCK_VAGRANTFILE_CAPTURE:-}" ]]; then
cp -- "$VAGRANT_CWD/Vagrantfile" "$MOCK_VAGRANTFILE_CAPTURE"
fi
counter_file="$MOCK_VBOX_STATE/counter"
counter=0
[[ ! -r "$counter_file" ]] || counter="$(cat "$counter_file")"
counter=$((counter + 1))
printf '%s\n' "$counter" > "$counter_file"
uuid="00000000-0000-4000-8000-$(printf '%012d' "$counter")"
vm_dir="$MOCK_VBOX_ROOT/master-$counter"
mkdir -p -- "$vm_dir"
: > "$vm_dir/master.vbox"
: > "$vm_dir/master.vdi"
printf '%s\n' \
'VMState="poweroff"' \
'groups="/"' \
"CfgFile=\"$vm_dir/master.vbox\"" \
"SATA-0-0=\"$vm_dir/master.vdi\"" > "$MOCK_VBOX_STATE/vm-$uuid"
printf '%s\n' "$uuid" > "$MOCK_BOX_DIR/master_id"
;;
destroy) ;;
*) exit 2 ;;
esac
+40
View File
@@ -0,0 +1,40 @@
#!/usr/bin/env bash
set -Eeuo pipefail
printf '%s\n' "$*" >> "$MOCK_VBOX_LOG"
command_name="${1:-}"
uuid="${2:-}"
case "$command_name" in
showvminfo)
[[ -r "$MOCK_VBOX_STATE/vm-$uuid" ]] || exit 1
cat "$MOCK_VBOX_STATE/vm-$uuid"
;;
getextradata)
key_slug="${3//\//_}"
if [[ ! -r "$MOCK_VBOX_STATE/extra-$uuid-$key_slug" ]]; then
printf '%s\n' 'No value set!'
exit 0
fi
printf 'Value: '
cat "$MOCK_VBOX_STATE/extra-$uuid-$key_slug"
;;
modifyvm)
[[ "${3:-}" == --groups ]]
awk -v groups="${4:-}" '
$1 !~ /^groups=/ { print }
END { printf "groups=\"%s\"\n", groups }
' "$MOCK_VBOX_STATE/vm-$uuid" > "$MOCK_VBOX_STATE/vm-$uuid.tmp"
mv "$MOCK_VBOX_STATE/vm-$uuid.tmp" "$MOCK_VBOX_STATE/vm-$uuid"
;;
setextradata)
key_slug="${3//\//_}"
printf '%s\n' "${4:-}" > "$MOCK_VBOX_STATE/extra-$uuid-$key_slug"
;;
unregistervm|closemedium)
printf '%s\n' 'destructive VirtualBox command invoked' >&2
exit 99
;;
*) exit 2 ;;
esac
+35
View File
@@ -0,0 +1,35 @@
#!/usr/bin/env bash
set -Eeuo pipefail
repo_root="$(git rev-parse --show-toplevel)"
main_tasks="$repo_root/roles/k3s_server/tasks/main.yml"
join_tasks="$repo_root/roles/k3s_server/tasks/join_master.yml"
for task_file in "$main_tasks" "$join_tasks"; do
for property in \
'Delegate=yes' \
'TasksMax=infinity' \
'KillMode=process' \
'LimitNOFILE=1048576' \
'LimitNPROC=infinity' \
'LimitCORE=infinity'; do
grep -Fq -- "$property" "$task_file" || {
printf '%s is missing transient K3s property %s\n' "$task_file" "$property" >&2
exit 1
}
done
done
if grep -Fq -- "node-role.kubernetes.io/master=true' -o=jsonpath" "$main_tasks"; then
printf 'control-plane registration still depends on the optional legacy master role key\n' >&2
exit 1
fi
grep -Fq -- "map('extract', hostvars, 'ansible_hostname')" "$main_tasks"
grep -Fq -- 'difference(nodes.stdout.split())' "$main_tasks"
grep -Fq -- 'crd/addons.k3s.cattle.io' "$main_tasks"
grep -Fq -- 'crd/helmcharts.helm.cattle.io' "$main_tasks"
grep -Fq -- 'crd/helmchartconfigs.helm.cattle.io' "$main_tasks"
printf 'K3s transient bootstrap regression test passed\n'
+130
View File
@@ -0,0 +1,130 @@
#!/usr/bin/env python3
"""Render the kube-vip DaemonSet template and assert env key correctness.
kube-vip v1.2.2 reads `bgp_peers` and `vip_subnet`; it ignores the older
`bgppeers` and `vip_cidr` names. This test proves the rendered manifest uses
the keys the target image actually parses.
"""
from __future__ import print_function
import os
import subprocess
from jinja2 import Environment, FileSystemLoader, StrictUndefined
def repo_root():
return subprocess.check_output(
["git", "rev-parse", "--show-toplevel"], text=True
).strip()
def fail(message):
raise SystemExit("kube-vip manifest test failed: " + message)
def fake_ipsubnet(value):
# ansible.utils.ipsubnet -> network of the address as x.y.z.0/24
parts = value.split(".")
return ".".join(parts[:3]) + ".0/24"
def fake_ipaddr(_value, expr=None):
# ansible.utils.ipaddr('prefix') -> prefix length
return "24"
def fake_bool(value):
# Minimal stand-in for Ansible's truthiness filter used by the template.
if isinstance(value, bool):
return value
return str(value).lower() in ("1", "true", "yes", "on")
def fake_map(seq, *args, **kwargs):
# Minimal stand-in for Ansible's map() filter in the two forms used by the
# template: map(attribute='x') on a list of dicts, and map('join', sep) on
# a list of sequences.
if "attribute" in kwargs:
return [item[kwargs["attribute"]] for item in seq]
if kwargs:
# e.g. map(default='x') not used here; ignore unknown kwargs.
return list(seq)
if args:
filter_name = args[0]
sep = args[1] if len(args) > 1 else ""
if filter_name == "join":
return [sep.join(str(x) for x in item) for item in seq]
return list(seq)
def fake_zip(*seqs):
return list(zip(*seqs))
def render(env, extra_vars):
base_vars = {
"apiserver_endpoint": "192.168.30.222",
"kube_vip_iface": "",
"kube_vip_arp": True,
"kube_vip_bgp": True,
"kube_vip_bgp_routerid": "127.0.0.1",
"_kube_vip_bgp_peers": [
{"peer_address": "192.168.30.1", "peer_asn": "64512"},
{"peer_address": "192.168.30.2", "peer_asn": "64513"},
],
"kube_vip_tag_version": "v1.2.2",
}
base_vars.update(extra_vars)
template = env.get_template("vip.yaml.j2")
return template.render(**base_vars)
def main():
root = repo_root()
template_dir = os.path.join(root, "roles", "k3s_server", "templates")
env = Environment(
loader=FileSystemLoader(template_dir), undefined=StrictUndefined
)
env.filters["ansible.utils.ipsubnet"] = fake_ipsubnet
env.filters["ansible.utils.ipaddr"] = fake_ipaddr
env.filters["bool"] = fake_bool
env.filters["map"] = fake_map
env.filters["zip"] = fake_zip
# Multi-peer BGP armed: must emit bgp_peers, never bgppeers.
output = render(env, {})
if "name: bgp_peers" not in output:
fail("rendered manifest is missing bgp_peers")
if "name: bgppeers" in output:
fail("rendered manifest still uses the ignored bgppeers key")
if "name: vip_subnet" not in output:
fail("rendered manifest is missing vip_subnet")
if "name: vip_cidr" in output:
fail("rendered manifest still uses the ignored vip_cidr key")
if "192.168.30.1:64512,192.168.30.2:64513" not in output:
fail("bgp_peers value is not comma-separated address:ASN entries")
if "ghcr.io/kube-vip/kube-vip:v1.2.2" not in output:
fail("kube-vip image tag is not v1.2.2")
# BGP enabled with no merged peers: single-peer fallback vars, no bgp_peers.
output = render(
env,
{
"_kube_vip_bgp_peers": [],
"kube_vip_bgp_as": "64513",
"kube_vip_bgp_peeraddress": "192.168.30.1",
"kube_vip_bgp_peeras": "64512",
},
)
if "name: bgp_as" not in output:
fail("single-peer bgp_as was not rendered")
if "name: bgp_peers" in output:
fail("bgp_peers present even though the peer list is empty")
print("kube-vip manifest regression test passed")
if __name__ == "__main__":
main()
+29
View File
@@ -0,0 +1,29 @@
#!/usr/bin/env bash
set -Eeuo pipefail
repo_root="$(git rev-parse --show-toplevel)"
metallb_task="$repo_root/roles/k3s_server/tasks/metallb.yml"
# The speaker tag verification must read the rendered manifest on the managed
# host with slurp. A controller-side lookup('ansible.builtin.file', ...) would
# read from the Ansible control node, which does not have the file, and would
# fail on every MetalLB scenario.
grep -Fq -- 'ansible.builtin.slurp' "$metallb_task" || {
printf 'MetalLB speaker tag check does not use slurp on the managed host\n' >&2
exit 1
}
grep -Eq -- 'lookup\(.?ansible\.builtin\.file' "$metallb_task" && {
printf 'MetalLB speaker tag check uses a controller-side file lookup\n' >&2
exit 1
}
# The check must reference the full image reference, not just a bare version
# string that could appear anywhere in the manifest.
grep -Fq -- 'quay.io/metallb/speaker:' "$metallb_task" || {
printf 'MetalLB speaker tag check does not match the full image reference\n' >&2
exit 1
}
printf 'MetalLB remote manifest read regression test passed\n'
+89
View File
@@ -0,0 +1,89 @@
#!/usr/bin/env bash
set -Eeuo pipefail
repo_root="$(git rev-parse --show-toplevel)"
fixture="$(mktemp -d)"
trap 'rm -rf -- "$fixture"' EXIT
mock_bin="$fixture/bin"
mock_state="$fixture/state"
mock_home="$fixture/home"
mock_vagrant_home="$fixture/vagrant-home"
mock_box_dir="$mock_vagrant_home/boxes/bento-VAGRANTSLASH-ubuntu-26.04/202606.01.0/amd64/virtualbox"
mock_vbox_root="$mock_home/VirtualBox VMs"
mock_master_root="$mock_home/.cache/k3s-ci/vagrant-masters"
lock_file="$fixture/vagrant-boxes.lock"
mkdir -p -- "$mock_bin" "$mock_state" "$mock_box_dir" "$mock_vbox_root"
printf '%s\n' 'bento/ubuntu-26.04 202606.01.0 amd64' > "$lock_file"
ln -s "$repo_root/.github/scripts/test-fixtures/mock-vboxmanage" "$mock_bin/VBoxManage"
ln -s "$repo_root/.github/scripts/test-fixtures/mock-vagrant" "$mock_bin/vagrant"
ln -s "$repo_root/.github/scripts/test-fixtures/mock-flock" "$mock_bin/flock"
export PATH="$mock_bin:$PATH"
export HOME="$mock_home"
export VAGRANT_HOME="$mock_vagrant_home"
export VAGRANT_BOX_LOCK_FILE="$lock_file"
export K3S_CI_REPOSITORY_ROOT="$repo_root"
export K3S_CI_VAGRANT_MASTER_ROOT="$mock_master_root"
export K3S_CI_VIRTUALBOX_ROOT="$mock_vbox_root"
export MOCK_VBOX_STATE="$mock_state"
export MOCK_VBOX_ROOT="$mock_vbox_root"
export MOCK_BOX_DIR="$mock_box_dir"
export MOCK_VBOX_LOG="$fixture/vbox.log"
export MOCK_VAGRANT_LOG="$fixture/vagrant.log"
export MOCK_VAGRANTFILE_CAPTURE="$fixture/prewarm-Vagrantfile"
: > "$MOCK_VBOX_LOG"
: > "$MOCK_VAGRANT_LOG"
unowned_uuid='99999999-9999-4999-8999-999999999999'
printf '%s\n' "$unowned_uuid" > "$mock_box_dir/master_id"
script="$repo_root/.github/scripts/prepare-vagrant-box-masters.sh"
first_output="$fixture/first-output"
second_output="$fixture/second-output"
third_output="$fixture/third-output"
"$script" > "$first_output"
if grep -Fq 'config.ssh.insert_key' "$MOCK_VAGRANTFILE_CAPTURE"; then
printf 'prewarm Vagrantfile unexpectedly overrides Vagrant SSH key insertion\n' >&2
exit 1
fi
grep -Fq 'virtualbox.memory = 1024' "$MOCK_VAGRANTFILE_CAPTURE"
grep -Fq 'virtualbox.cpus = 2' "$MOCK_VAGRANTFILE_CAPTURE"
grep -Fq 'config.vm.boot_timeout = 600' "$MOCK_VAGRANTFILE_CAPTURE"
mapping_file="$mock_master_root/bento_ubuntu-26.04-202606.01.0-amd64.uuid"
test -s "$mapping_file"
cmp -s "$mapping_file" "$mock_box_dir/master_id"
grep -Fq 'Created and recorded owned master' "$first_output"
grep -Fq 'modifyvm' "$MOCK_VBOX_LOG"
grep -Fq 'setextradata' "$MOCK_VBOX_LOG"
if grep -Fq "$unowned_uuid" "$MOCK_VBOX_LOG"; then
printf 'unowned cached master UUID was unexpectedly inspected or modified\n' >&2
exit 1
fi
if grep -Eq 'unregistervm|closemedium' "$MOCK_VBOX_LOG"; then
printf 'master preparation invoked a destructive VirtualBox command\n' >&2
exit 1
fi
: > "$MOCK_VAGRANT_LOG"
"$script" > "$second_output"
grep -Fq 'Reusing owned master' "$second_output"
if grep -Fq 'up ' "$MOCK_VAGRANT_LOG"; then
printf 'valid owned master was unexpectedly rebuilt\n' >&2
exit 1
fi
stale_uuid="$(tr -d '[:space:]' < "$mapping_file")"
rm -f -- "$mock_state/vm-$stale_uuid"
: > "$MOCK_VAGRANT_LOG"
"$script" > "$third_output"
grep -Fq 'rebuilding without deleting any VM or disk' "$third_output"
grep -Fq 'up ' "$MOCK_VAGRANT_LOG"
if grep -Eq 'unregistervm|closemedium' "$MOCK_VBOX_LOG"; then
printf 'stale master recovery invoked a destructive VirtualBox command\n' >&2
exit 1
fi
printf 'Vagrant box master preparation fixture test passed\n'
+24
View File
@@ -0,0 +1,24 @@
#!/usr/bin/env bash
set -Eeuo pipefail
if (($# < 2)); then
printf 'Usage: vagrant-up-timed.sh WORKDIR MACHINE [MACHINE ...]\n' >&2
exit 2
fi
workdir="$1"
shift
timing_log="${K3S_CI_CREATE_TIMING_LOG:-${RUNNER_TEMP:-/tmp}/k3s-ci-create-timing.log}"
mkdir -p -- "${timing_log%/*}"
printf '%s batch-start machines=%s\n' "$(date --iso-8601=ns)" "$*" | tee -a "$timing_log"
set +e
VAGRANT_CWD="$workdir" vagrant up "$@" --provider virtualbox --no-provision 2>&1 |
while IFS= read -r line; do
printf '%s %s\n' "$(date --iso-8601=ns)" "$line"
done | tee -a "$timing_log"
rc=${PIPESTATUS[0]}
set -e
printf '%s batch-end rc=%d machines=%s\n' "$(date --iso-8601=ns)" "$rc" "$*" | tee -a "$timing_log"
exit "$rc"
+98
View File
@@ -0,0 +1,98 @@
#!/usr/bin/env bash
# The single-quoted expressions below are written into fake executables and
# intentionally expand only when those executables run.
# shellcheck disable=SC2016
set -Eeuo pipefail
repo_root=$(git rev-parse --show-toplevel)
test_root=$(mktemp -d)
fake_bin="$test_root/bin"
fake_log="$test_root/vagrant.log"
output="$test_root/output.txt"
mkdir -p "$fake_bin"
trap 'rm -rf "$test_root"' EXIT
printf '%s\n' \
'#!/usr/bin/env bash' \
'set -Eeuo pipefail' \
'printf "%s\n" bento/debian-13 bento/rockylinux-10.1 bento/ubuntu-26.04' \
>"$fake_bin/yq"
printf '%s\n' \
'#!/usr/bin/env bash' \
'set -Eeuo pipefail' \
'emit_box() {' \
' case "$1" in' \
' bento/debian-13) version=202510.26.0 ;;' \
' bento/rockylinux-10.1) version=202512.01.0 ;;' \
' bento/ubuntu-26.04) version=202606.01.0 ;;' \
' *) printf "Unexpected box: %s\n" "$1" >&2; exit 1 ;;' \
' esac' \
' printf "0,,box-name,%s\n" "$1"' \
' printf "0,,box-provider,virtualbox\n"' \
' printf "0,,box-version,%s\n" "$version"' \
' printf "0,,box-architecture,amd64\n"' \
'}' \
'if [[ "${1:-}" == box && "${2:-}" == list ]]; then' \
' emit_box bento/debian-13' \
' emit_box bento/rockylinux-10.1' \
' if [[ "${FAKE_PRESENT_MODE:-all}" == all ]]; then' \
' emit_box bento/ubuntu-26.04' \
' fi' \
'elif [[ "${1:-}" == box && "${2:-}" == add ]]; then' \
' printf "%s\n" "$*" >>"${FAKE_VAGRANT_LOG:?}"' \
'else' \
' printf "Unexpected vagrant arguments: %s\n" "$*" >&2' \
' exit 1' \
'fi' \
>"$fake_bin/vagrant"
chmod +x "$fake_bin/yq" "$fake_bin/vagrant"
PATH="$fake_bin:$PATH" \
FAKE_VAGRANT_LOG="$fake_log" \
"$repo_root/.github/download-boxes.sh" >"$output"
grep -Fq 'All pinned Vagrant boxes are already present.' "$output"
[[ ! -e "$fake_log" ]]
PATH="$fake_bin:$PATH" \
FAKE_PRESENT_MODE=partial \
FAKE_VAGRANT_LOG="$fake_log" \
"$repo_root/.github/download-boxes.sh" >"$output"
grep -Fxq \
'box add --provider virtualbox --box-version 202606.01.0 --architecture amd64 bento/ubuntu-26.04' \
"$fake_log"
incomplete_lock="$test_root/incomplete.lock"
printf '%s\n' \
'bento/debian-13 202510.26.0 amd64' \
'bento/rockylinux-10.1 202512.01.0 amd64' \
>"$incomplete_lock"
if PATH="$fake_bin:$PATH" \
VAGRANT_BOX_LOCK_FILE="$incomplete_lock" \
FAKE_VAGRANT_LOG="$fake_log" \
"$repo_root/.github/download-boxes.sh" >"$output" 2>&1; then
printf 'Download script accepted a lock missing a scenario box.\n' >&2
exit 1
fi
grep -Fq 'Scenario boxes missing from the lock file:' "$output"
grep -Fq 'bento/ubuntu-26.04' "$output"
duplicate_lock="$test_root/duplicate.lock"
printf '%s\n' \
'bento/debian-13 202510.26.0 amd64' \
'bento/debian-13 202508.10.0 amd64' \
'bento/rockylinux-10.1 202512.01.0 amd64' \
'bento/ubuntu-26.04 202606.01.0 amd64' \
>"$duplicate_lock"
if PATH="$fake_bin:$PATH" \
VAGRANT_BOX_LOCK_FILE="$duplicate_lock" \
FAKE_VAGRANT_LOG="$fake_log" \
"$repo_root/.github/download-boxes.sh" >"$output" 2>&1; then
printf 'Download script accepted duplicate box lock entries.\n' >&2
exit 1
fi
grep -Fq 'Duplicate Vagrant box lock entries:' "$output"
printf 'Vagrant box download tests passed.\n'
+4
View File
@@ -0,0 +1,4 @@
# box version architecture
bento/debian-13 202510.26.0 amd64
bento/rockylinux-10.1 202512.01.0 amd64
bento/ubuntu-26.04 202606.01.0 amd64
+57
View File
@@ -0,0 +1,57 @@
---
name: "Cache"
on:
workflow_call:
jobs:
molecule:
name: cache
runs-on: [self-hosted, linux, x64, k3s-ci, virtualbox, nested-virt]
env:
PYTHON_VERSION: "3.11"
VAGRANT_DEFAULT_PROVIDER: virtualbox
VAGRANT_HOME: ${{ github.workspace }}/.vagrant-home
steps:
- name: Check out the codebase
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # 7.0.1
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Check nested VirtualBox platform
run: |
set -Eeuo pipefail
grep -Eq 'vmx|svm' /proc/cpuinfo
test -c /dev/kvm
test -c /dev/vboxdrv
VBoxManage --version
vagrant --version
test -r /etc/vbox/networks.conf
test "$(stat -c '%u' /etc/vbox/networks.conf)" -eq 0
free -h
df -Pk "${RUNNER_TEMP}"
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # 7.0.0
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'pip' # caching pip dependencies
- name: Cache Vagrant boxes
id: cache-vagrant
uses: actions/cache@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # 6.1.0
with:
# This producer only needs to know whether the immutable cache exists.
# Molecule jobs restore it after this job completes.
lookup-only: true
path: |
.vagrant-home/boxes
key: vagrant-boxes-${{ runner.name }}-${{ runner.os }}-${{ runner.arch }}-virtualbox-7.2-vagrant-2.4-${{ hashFiles('.github/vagrant-boxes.lock') }} # yamllint disable-line rule:line-length
- name: Download Vagrant boxes for all scenarios
# An exact hit skips both cache restoration and upstream downloads.
# A lock change builds and saves one clean, version-pinned cache.
if: steps.cache-vagrant.outputs.cache-hit != 'true'
run: |
./.github/download-boxes.sh
./.github/scripts/prepare-vagrant-box-masters.sh
vagrant box list
+28 -3
View File
@@ -2,14 +2,39 @@
name: "CI"
on:
pull_request:
push:
branches:
- master
types:
- opened
- synchronize
- reopened
- ready_for_review
paths-ignore:
- '**/README.md'
- '**/.gitignore'
- '**/FUNDING.yml'
- '**/host.ini'
- '**/*.md'
- '**/.editorconfig'
- '**/ansible.example.cfg'
- '**/deploy.sh'
- '**/LICENSE'
- '**/reboot.sh'
- '**/reset.sh'
workflow_dispatch:
permissions:
contents: read
concurrency:
group: ci-${{ github.event.pull_request.number || github.run_id }}
cancel-in-progress: true
jobs:
pre:
uses: ./.github/workflows/cache.yml
needs: [lint]
lint:
uses: ./.github/workflows/lint.yml
test:
uses: ./.github/workflows/test.yml
needs: [lint]
needs: [pre, lint]
+30 -23
View File
@@ -7,35 +7,26 @@ jobs:
name: Pre-Commit
runs-on: ubuntu-latest
env:
PYTHON_VERSION: "3.11"
PYTHON_VERSION: "3.12"
steps:
- name: Check out the codebase
uses: actions/checkout@e2f20e631ae6d7dd3b768f56a5d2af784dd54791 # v3 2.5.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # 7.0.1
with:
ref: ${{ github.event.pull_request.head.sha }}
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@75f3110429a8c05be0e1bf360334e4cced2b63fa # 2.3.3
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # 7.0.0
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'pip' # caching pip dependencies
- name: Cache pip
uses: actions/cache@9b0c1fce7a93df8e3bb8926b0d6e9d89e92f20a7 # 3.0.11
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('./requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Cache Ansible
uses: actions/cache@9b0c1fce7a93df8e3bb8926b0d6e9d89e92f20a7 # 3.0.11
- name: Restore Ansible cache
id: cache-ansible
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # 6.1.0
with:
path: ~/.ansible/collections
key: ${{ runner.os }}-ansible-${{ hashFiles('collections/requirements.txt') }}
restore-keys: |
${{ runner.os }}-ansible-
key: ansible-${{ hashFiles('collections/requirements.yml') }}
- name: Install dependencies
run: |
@@ -47,21 +38,37 @@ jobs:
python3 -m pip install -r requirements.txt
echo "::endgroup::"
echo "::group::Install Ansible role requirements from collections/requirements.yml"
ansible-galaxy install -r collections/requirements.yml
echo "::endgroup::"
- name: Install Ansible collections with retries
if: steps.cache-ansible.outputs.cache-hit != 'true'
run: |
set -Eeuo pipefail
for attempt in 1 2 3 4 5; do
if ansible-galaxy collection install -r collections/requirements.yml; then
exit 0
fi
echo "Ansible Galaxy attempt ${attempt} failed; retrying."
sleep $((attempt * 10))
done
exit 1
- name: Save Ansible collection cache
if: steps.cache-ansible.outputs.cache-hit != 'true'
uses: actions/cache/save@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # 6.1.0
with:
path: ~/.ansible/collections
key: ansible-${{ hashFiles('collections/requirements.yml') }}
- name: Run pre-commit
uses: pre-commit/action@646c83fcd040023954eafda54b4db0192ce70507 # 3.0.0
uses: pre-commit/action@2c7b3805fd2a0fd8c1884dcaebf91fc102a13ecd # 3.0.1
ensure-pinned-actions:
name: Ensure SHA Pinned Actions
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@e2f20e631ae6d7dd3b768f56a5d2af784dd54791 # v3 2.5.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # 7.0.1
- name: Ensure SHA pinned actions
uses: zgosalvez/github-actions-ensure-sha-pinned-actions@af2eb3226618e2494e3d9084f515ad6dcf16e229 # 2.0.1
uses: zgosalvez/github-actions-ensure-sha-pinned-actions@3db98c0363e2fa5df3e1c4c471777a7c10b24cc9 # 5.0.5
with:
allowlist: |
aws-actions/
+69 -43
View File
@@ -5,60 +5,64 @@ on:
jobs:
molecule:
name: Molecule
runs-on: macos-12
runs-on: [self-hosted, linux, x64, k3s-ci, virtualbox, nested-virt]
strategy:
matrix:
scenario:
- default
- ipv6
- single_node
fail-fast: false
- calico
- cilium
- kube-vip
# - ipv6
fail-fast: true
max-parallel: 1
env:
PYTHON_VERSION: "3.11"
VAGRANT_DEFAULT_PROVIDER: virtualbox
VAGRANT_HOME: ${{ github.workspace }}/.vagrant-home
steps:
- name: Check out the codebase
uses: actions/checkout@e2f20e631ae6d7dd3b768f56a5d2af784dd54791 # v3 2.5.0
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # 7.0.1
with:
ref: ${{ github.event.pull_request.head.sha }}
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- name: Configure VirtualBox
run: |-
sudo mkdir -p /etc/vbox
cat <<EOF | sudo tee -a /etc/vbox/networks.conf > /dev/null
* 192.168.30.0/24
* fdad:bad:ba55::/64
EOF
- name: Clean repository-owned resources before testing
run: ./.github/scripts/cleanup-runner-resources.sh --apply
- name: Cache pip
uses: actions/cache@9b0c1fce7a93df8e3bb8926b0d6e9d89e92f20a7 # 3.0.11
with:
path: ~/.cache/pip
key: ${{ runner.os }}-pip-${{ hashFiles('./requirements.txt') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Record host-only network baseline
run: ./.github/scripts/cleanup-runner-resources.sh --snapshot
- name: Cache Vagrant boxes
uses: actions/cache@9b0c1fce7a93df8e3bb8926b0d6e9d89e92f20a7 # 3.0.11
with:
path: |
~/.vagrant.d/boxes
key: vagrant-boxes-${{ hashFiles('**/molecule.yml') }}
restore-keys: |
vagrant-boxes
- name: Download Vagrant boxes for all scenarios
# To save some cache space, all scenarios share the same cache key.
# On the other hand, this means that the cache contents should be
# the same across all scenarios. This step ensures that.
run: ./.github/download-boxes.sh
- name: Check nested VirtualBox platform
run: |
set -Eeuo pipefail
grep -Eq 'vmx|svm' /proc/cpuinfo
test -c /dev/kvm
test -c /dev/vboxdrv
VBoxManage --version
vagrant --version
test -r /etc/vbox/networks.conf
test "$(stat -c '%u' /etc/vbox/networks.conf)" -eq 0
free -h
df -Pk "${RUNNER_TEMP}"
- name: Set up Python ${{ env.PYTHON_VERSION }}
uses: actions/setup-python@75f3110429a8c05be0e1bf360334e4cced2b63fa # 2.3.3
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # 7.0.0
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'pip' # caching pip dependencies
- name: Restore vagrant Boxes cache
uses: actions/cache/restore@55cc8345863c7cc4c66a329aec7e433d2d1c52a9 # 6.1.0
with:
path: .vagrant-home/boxes
key: vagrant-boxes-${{ runner.name }}-${{ runner.os }}-${{ runner.arch }}-virtualbox-7.2-vagrant-2.4-${{ hashFiles('.github/vagrant-boxes.lock') }} # yamllint disable-line rule:line-length
fail-on-cache-miss: true
- name: Prepare runner-owned Vagrant box masters
run: ./.github/scripts/prepare-vagrant-box-masters.sh
- name: Install dependencies
run: |
echo "::group::Upgrade pip"
@@ -70,23 +74,45 @@ jobs:
echo "::endgroup::"
- name: Test with molecule
run: molecule test --scenario-name ${{ matrix.scenario }}
timeout-minutes: 90
run: |
set -Eeuo pipefail
resource_dir="${RUNNER_TEMP}/logs/resources/${{ matrix.scenario }}"
timing_file="${RUNNER_TEMP}/logs/timing/${{ matrix.scenario }}.txt"
mkdir -p -- "${timing_file%/*}"
./.github/scripts/monitor-runner-resources.sh "$resource_dir" 10 &
monitor_pid=$!
stop_monitor() {
kill -TERM "$monitor_pid" 2>/dev/null || true
wait "$monitor_pid" 2>/dev/null || true
}
trap stop_monitor EXIT
/usr/bin/time -v -o "$timing_file" \
molecule test --scenario-name ${{ matrix.scenario }}
timeout-minutes: 150
env:
ANSIBLE_K3S_LOG_DIR: ${{ runner.temp }}/logs/k3s-ansible/${{ matrix.scenario }}
ANSIBLE_SSH_RETRIES: 4
ANSIBLE_TIMEOUT: 60
ANSIBLE_TIMEOUT: 120
PY_COLORS: 1
ANSIBLE_FORCE_COLOR: 1
K3S_CI_CREATE_TIMING_LOG: ${{ runner.temp }}/logs/timing/${{ matrix.scenario }}-create.log
- name: Collect runner diagnostics
if: always()
run: ./.github/scripts/collect-runner-diagnostics.sh "${RUNNER_TEMP}/logs/runner"
env:
K3S_CI_SCENARIO_NAME: ${{ matrix.scenario }}
- name: Clean repository-owned resources after testing
if: always()
run: ./.github/scripts/cleanup-runner-resources.sh --apply
- name: Upload log files
if: always() # do this even if a step before has failed
uses: actions/upload-artifact@83fd05a356d7e2593de66fc9913b3002723633cb # 3.1.1
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # 7.0.1
with:
name: logs
name: logs-${{ matrix.scenario }}-${{ github.run_id }}-${{ github.run_attempt }}
path: |
${{ runner.temp }}/logs
- name: Delete old box versions
if: always() # do this even if a step before has failed
run: vagrant box prune --force
if-no-files-found: warn
retention-days: 14
+3
View File
@@ -1,3 +1,6 @@
.env/
*.log
ansible.cfg
.ansible/
kubeconfig
zIgnore/
+57 -6
View File
@@ -1,7 +1,7 @@
---
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: f71fa2c1f9cf5cb705f73dffe4b21f7c61470ba9 # frozen: v4.4.0
rev: v4.5.0
hooks:
- id: requirements-txt-fixer
- id: sort-simple-yaml
@@ -12,24 +12,75 @@ repos:
- id: trailing-whitespace
args: [--markdown-linebreak-ext=md]
- repo: https://github.com/adrienverge/yamllint.git
rev: b05e028c5881819161d11cb543fd96a30c06cceb # frozen: v1.32.0
rev: v1.33.0
hooks:
- id: yamllint
args: [-c=.yamllint]
- repo: https://github.com/ansible-community/ansible-lint.git
rev: 3293b64b939c0de16ef8cb81dd49255e475bf89a # frozen: v6.17.2
rev: v6.22.2
hooks:
- id: ansible-lint
additional_dependencies: [ansible-core==2.18.0]
language_version: python3.12
args: [--offline]
- repo: https://github.com/shellcheck-py/shellcheck-py
rev: 375289a39f5708101b1f916eb729e8d6da96993f # frozen: v0.9.0.5
rev: v0.9.0.6
hooks:
- id: shellcheck
- repo: https://github.com/Lucas-C/pre-commit-hooks
rev: 12885e376b93dc4536ad68d156065601e4433665 # frozen: v1.5.1
rev: v1.5.4
hooks:
- id: remove-crlf
- id: remove-tabs
- repo: https://github.com/sirosen/texthooks
rev: c4ffd3e31669dd4fa4d31a23436cc13839730084 # frozen: 0.5.0
rev: 0.6.4
hooks:
- id: fix-smartquotes
- repo: local
hooks:
- id: cleanup-runner-resources-test
name: cleanup runner resources test
entry: .github/scripts/test-cleanup-runner-resources.sh
language: system
pass_filenames: false
files: ^\.github/scripts/(cleanup-runner-resources|test-cleanup-runner-resources)\.sh$
- id: download-vagrant-boxes-test
name: Vagrant box download test
entry: .github/test-download-boxes.sh
language: system
pass_filenames: false
files: ^\.github/(download-boxes|test-download-boxes)\.sh$|^\.github/vagrant-boxes\.lock$
- id: prepare-vagrant-box-masters-test
name: Vagrant box master preparation test
entry: .github/scripts/test-prepare-vagrant-box-masters.sh
language: system
pass_filenames: false
files: ^\.github/scripts/(prepare-vagrant-box-masters|test-prepare-vagrant-box-masters)\.sh$
- id: k3s-server-bootstrap-test
name: K3s transient bootstrap test
entry: .github/scripts/test-k3s-server-bootstrap.sh
language: system
pass_filenames: false
files: ^roles/k3s_server/tasks/(main|join_master)\.yml$|^\.github/scripts/test-k3s-server-bootstrap\.sh$
- id: cilium-bgp-manifest-test
name: Cilium BGP manifest test
entry: python3 .github/scripts/test-cilium-bgp-manifest.py
language: python
additional_dependencies:
- Jinja2>=3.1
pass_filenames: false
files: ^roles/k3s_server_post/templates/cilium\.crs\.j2$|^\.github/scripts/test-cilium-bgp-manifest\.py$
- id: kube-vip-manifest-test
name: kube-vip manifest test
entry: python3 .github/scripts/test-kube-vip-manifest.py
language: python
additional_dependencies:
- Jinja2>=3.1
pass_filenames: false
files: ^roles/k3s_server/templates/vip\.yaml\.j2$|^\.github/scripts/test-kube-vip-manifest\.py$
- id: metallb-remote-read-test
name: MetalLB remote read test
entry: .github/scripts/test-metallb-remote-read.sh
language: system
pass_filenames: false
files: ^roles/k3s_server/tasks/metallb\.yml$|^\.github/scripts/test-metallb-remote-read\.sh$
+12 -1
View File
@@ -2,8 +2,19 @@
extends: default
rules:
comments:
min-spaces-from-content: 1
comments-indentation: false
braces:
max-spaces-inside: 1
octal-values:
forbid-implicit-octal: true
forbid-explicit-octal: true
line-length:
max: 120
level: warning
truthy:
allowed-values: ['true', 'false', 'yes', 'no']
allowed-values: ["true", "false"]
ignore:
- galaxy.yml
+143 -8
View File
@@ -8,7 +8,7 @@ This is based on the work from [this fork](https://github.com/212850a/k3s-ansibl
If you want more context on how this works, see:
📄 [Documentation](https://technotim.live/posts/k3s-etcd-ansible/) (including example commands)
📄 [Documentation](https://technotim.com/posts/k3s-etcd-ansible/) (including example commands)
📺 [Watch the Video](https://www.youtube.com/watch?v=CbkEWcUZ7zM)
@@ -16,9 +16,9 @@ If you want more context on how this works, see:
Build a Kubernetes cluster using Ansible with k3s. The goal is easily install a HA Kubernetes cluster on machines running:
- [x] Debian (tested on version 11)
- [x] Ubuntu (tested on version 22.04)
- [x] Rocky (tested on version 9)
- [x] Debian (tested on version 13)
- [x] Ubuntu (tested on version 26.04 LTS)
- [x] Rocky (tested on version 10)
on processor architecture:
@@ -28,7 +28,7 @@ on processor architecture:
## ✅ System requirements
- Control Node (the machine you are running `ansible` commands) must have Ansible 2.11+ If you need a quick primer on Ansible [you can check out my docs and setting up Ansible](https://technotim.live/posts/ansible-automation/).
- Control Node (the machine you are running `ansible` commands) must have Ansible 2.11+ If you need a quick primer on Ansible [you can check out my docs and setting up Ansible](https://technotim.com/posts/ansible-automation/).
- You will also need to install collections that this playbook uses by running `ansible-galaxy collection install -r ./collections/requirements.yml` (important❗)
@@ -91,21 +91,134 @@ ansible-playbook reset.yml -i inventory/my-cluster/hosts.ini
>You should also reboot these nodes due to the VIP not being destroyed
## 🔁 Upgrading an existing cluster
These version variables select the components used for a **fresh** installation.
They are not a supported direct in-place upgrade path for an existing cluster.
K3s, Calico, and Cilium each require staged upgrades for long-lived clusters.
- **K3s**: do not jump an embedded-etcd cluster straight to Kubernetes 1.36.
Upgrade one Kubernetes minor version at a time. From the sample default
(`v1.30.2+k3s2`) the sequence is: the latest supported 1.30 patch, then 1.31,
1.32, a 1.33 patch that contains etcd 3.5.26 (for example `v1.33.7+k3s3`),
then 1.34, 1.35, and finally 1.36. Upgrade servers one at a time before
agents. Take backups and confirm cluster health at each step; this playbook
does not automate the upgrade, so those remain manual operational steps. See
[K3s manual upgrades](https://docs.k3s.io/upgrades/manual) and the
[v1.34 release notes](https://docs.k3s.io/release-notes/v1.34.X).
- **Cilium**: upstream supports only consecutive minor upgrades. Update to the
latest patch of the current minor, then upgrade 1.17, 1.18, 1.19, and 1.20 in
order, reading each version's upgrade notes and running preflight checks.
Do not attempt a direct upgrade from an old Cilium to 1.20.
- **Calico**: starting with 3.28 the v3 resource UID behavior changed. If you
have operators with OwnerReferences pointing to `projectcalico.org/v3`
resources, remove and recreate those references around an in-place upgrade.
- **MetalLB**: this project installs application tag `v0.16.0`. A newer
chart-only tag such as `metallb-chart-0.16.1` is not an application or image
release and must not be used as the controller or speaker image tag.
## ⚙️ Kube Config
To copy your `kube config` locally so that you can access your **Kubernetes** cluster run:
```bash
scp debian@master_ip:~/.kube/config ~/.kube/config
scp debian@master_ip:/etc/rancher/k3s/k3s.yaml ~/.kube/config
```
If you get file Permission denied, go into the node and temporarly run:
```bash
sudo chmod 777 /etc/rancher/k3s/k3s.yaml
```
Then copy with the scp command and reset the permissions back to:
```bash
sudo chmod 600 /etc/rancher/k3s/k3s.yaml
```
You'll then want to modify the config to point to master IP by running:
```bash
sudo nano ~/.kube/config
```
Then change `server: https://127.0.0.1:6443` to match your master IP: `server: https://192.168.1.222:6443`
### 🔨 Testing your cluster
See the commands [here](https://technotim.live/posts/k3s-etcd-ansible/#testing-your-cluster).
See the commands [here](https://technotim.com/posts/k3s-etcd-ansible/#testing-your-cluster).
### Variables
| Role(s) | Variable | Type | Default | Required | Description |
|---|---|---|---|---|---|
| `download` | `k3s_version` | string | ❌ | Required | K3s binaries version |
| `k3s_agent`, `k3s_server`, `k3s_server_post` | `apiserver_endpoint` | string | ❌ | Required | Virtual ip-address configured on each master |
| `k3s_agent` | `extra_agent_args` | string | `null` | Not required | Extra arguments for agents nodes |
| `k3s_agent`, `k3s_server` | `group_name_master` | string | `null` | Not required | Name othe master group |
| `k3s_agent` | `k3s_token` | string | `null` | Not required | Token used to communicate between masters |
| `k3s_agent`, `k3s_server` | `proxy_env` | dict | `null` | Not required | Internet proxy configurations |
| `k3s_agent`, `k3s_server` | `proxy_env.HTTP_PROXY` | string | ❌ | Required | HTTP internet proxy |
| `k3s_agent`, `k3s_server` | `proxy_env.HTTPS_PROXY` | string | ❌ | Required | HTTP internet proxy |
| `k3s_agent`, `k3s_server` | `proxy_env.NO_PROXY` | string | ❌ | Required | Addresses that will not use the proxies |
| `k3s_agent`, `k3s_server`, `reset` | `systemd_dir` | string | `/etc/systemd/system` | Not required | Path to systemd services |
| `k3s_custom_registries` | `custom_registries_yaml` | string | ❌ | Required | YAML block defining custom registries. The following is an example that pulls all images used in this playbook through your private registries. It also allows you to pull your own images from your private registry, without having to use imagePullSecrets in your deployments. If all you need is your own images and you don't care about caching the docker/quay/ghcr.io images, you can just remove those from the mirrors: section. |
| `k3s_server`, `k3s_server_post` | `cilium_bgp` | bool | `~` | Not required | Enable cilium BGP control plane for LB services and pod cidrs. Disables the use of MetalLB. |
| `k3s_server`, `k3s_server_post` | `cilium_iface` | string | ❌ | Not required | The network interface used for when Cilium is enabled |
| `k3s_server` | `extra_server_args` | string | `""` | Not required | Extra arguments for server nodes |
| `k3s_server` | `k3s_create_kubectl_symlink` | bool | `false` | Not required | Create the kubectl -> k3s symlink |
| `k3s_server` | `k3s_create_crictl_symlink` | bool | `true` | Not required | Create the crictl -> k3s symlink |
| `k3s_server` | `kube_vip_arp` | bool | `true` | Not required | Enables kube-vip ARP broadcasts |
| `k3s_server` | `kube_vip_bgp` | bool | `false` | Not required | Enables kube-vip BGP peering |
| `k3s_server` | `kube_vip_bgp_routerid` | string | `"127.0.0.1"` | Not required | Defines the router ID for the kube-vip BGP server |
| `k3s_server` | `kube_vip_bgp_as` | string | `"64513"` | Not required | Defines the AS for the kube-vip BGP server |
| `k3s_server` | `kube_vip_bgp_peeraddress` | string | `"192.168.30.1"` | Not required | Defines the address for the kube-vip BGP peer |
| `k3s_server` | `kube_vip_bgp_peeras` | string | `"64512"` | Not required | Defines the AS for the kube-vip BGP peer |
| `k3s_server` | `kube_vip_bgp_peers` | list | `[]` | Not required | List of BGP peer ASN & address pairs |
| `k3s_server` | `kube_vip_bgp_peers_groups` | list | `['k3s_master']` | Not required | Inventory group in which to search for additional `kube_vip_bgp_peers` parameters to merge. |
| `k3s_server` | `kube_vip_iface` | string | `~` | Not required | Explicitly define an interface that ALL control nodes should use to propagate the VIP, define it here. Otherwise, kube-vip will determine the right interface automatically at runtime. |
| `k3s_server` | `kube_vip_tag_version` | string | `v1.2.2` | Not required | Image tag for kube-vip |
| `k3s_server` | `kube_vip_cloud_provider_tag_version` | string | `v0.0.12` | Not required | Tag for kube-vip-cloud-provider manifest when enable |
| `k3s_server`, `k3_server_post` | `kube_vip_lb_ip_range` | string | `~` | Not required | IP range for kube-vip load balancer |
| `k3s_server`, `k3s_server_post` | `metal_lb_controller_tag_version` | string | `v0.16.0` | Not required | Image tag for MetalLB |
| `k3s_server` | `metal_lb_speaker_tag_version` | string | `v0.16.0` | Not required | Image tag for MetalLB |
| `k3s_server` | `metal_lb_type` | string | `native` | Not required | Use FRR mode or native. Valid values are `frr` and `native` |
| `k3s_server` | `retry_count` | int | `20` | Not required | Amount of retries when verifying that nodes joined |
| `k3s_server` | `server_init_args` | string | ❌ | Not required | Arguments for server nodes |
| `k3s_server_post` | `bpf_lb_algorithm` | string | `maglev` | Not required | BPF lb algorithm |
| `k3s_server_post` | `bpf_lb_mode` | string | `hybrid` | Not required | BPF lb mode |
| `k3s_server_post` | `calico_blocksize` | int | `26` | Not required | IP pool block size |
| `k3s_server_post` | `calico_ebpf` | bool | `false` | Not required | Use eBPF dataplane instead of iptables |
| `k3s_server_post` | `calico_encapsulation` | string | `VXLANCrossSubnet` | Not required | IP pool encapsulation |
| `k3s_server_post` | `calico_natOutgoing` | string | `Enabled` | Not required | IP pool NAT outgoing |
| `k3s_server_post` | `calico_nodeSelector` | string | `all()` | Not required | IP pool node selector |
| `k3s_server_post` | `calico_iface` | string | `~` | Not required | The network interface used for when Calico is enabled |
| `k3s_server_post` | `calico_tag` | string | `v3.32.1` | Not required | Calico version tag |
| `k3s_server_post` | `cilium_bgp_my_asn` | int | `64513` | Not required | Local ASN for BGP peer |
| `k3s_server_post` | `cilium_bgp_peer_asn` | int | `64512` | Not required | BGP peer ASN |
| `k3s_server_post` | `cilium_bgp_peer_address` | string | `~` | Not required | BGP peer address |
| `k3s_server_post` | `cilium_bgp_neighbors` | list | `[]` | Not required | List of BGP peer ASN & address pairs |
| `k3s_server_post` | `cilium_bgp_neighbors_groups` | list | `['k3s_all']` | Not required | Inventory group in which to search for additional `cilium_bgp_neighbors` parameters to merge. |
| `k3s_server_post` | `cilium_bgp_lb_cidr` | string | `192.168.31.0/24` | Not required | BGP load balancer IP range |
| `k3s_server_post` | `cilium_exportPodCIDR` | bool | `true` | Not required | Export pod CIDR |
| `k3s_server_post` | `cilium_hubble` | bool | `true` | Not required | Enable Cilium Hubble |
| `k3s_server_post` | `cilium_mode` | string | `native` | Not required | Inner-node communication mode (choices are `native` and `tunnel`; `routed` is a deprecated alias for `tunnel`) |
| `k3s_server_post` | `cilium_tag` | string | `v1.20.0` | Not required | Cilium version tag |
| `k3s_server_post` | `cilium_cli_tag` | string | `v0.19.7` | Not required | Cilium CLI version tag |
| `k3s_server_post` | `cluster_cidr` | string | `10.52.0.0/16` | Not required | Inner-cluster IP range |
| `k3s_server_post` | `enable_bpf_masquerade` | bool | `true` | Not required | Use IP masquerading |
| `k3s_server_post` | `kube_proxy_replacement` | bool | `true` | Not required | Replace the native kube-proxy with Cilium |
| `k3s_server_post` | `metal_lb_available_timeout` | string | `240s` | Not required | Wait for MetalLB resources |
| `k3s_server_post` | `metal_lb_ip_range` | string | `192.168.30.80-192.168.30.90` | Not required | MetalLB ip range for load balancer |
| `k3s_server_post` | `metal_lb_controller_tag_version` | string | `v0.16.0` | Not required | Image tag for MetalLB |
| `k3s_server_post` | `metal_lb_mode` | string | `layer2` | Not required | Metallb mode (choices are `bgp` and `layer2`) |
| `k3s_server_post` | `metal_lb_bgp_my_asn` | string | `~` | Not required | BGP ASN configurations |
| `k3s_server_post` | `metal_lb_bgp_peer_asn` | string | `~` | Not required | BGP peer ASN configurations |
| `k3s_server_post` | `metal_lb_bgp_peer_address` | string | `~` | Not required | BGP peer address |
| `lxc` | `custom_reboot_command` | string | `~` | Not required | Command to run on reboot |
| `prereq` | `system_timezone` | string | `null` | Not required | Timezone to be set on all nodes |
| `proxmox_lxc`, `reset_proxmox_lxc` | `proxmox_lxc_ct_ids` | list | ❌ | Required | Proxmox container ID list |
| `raspberrypi` | `state` | string | `present` | Not required | Indicates whether the k3s prerequisites for Raspberry Pi should be set up (possible values are `present` and `absent`) |
### Troubleshooting
Be sure to see [this post](https://github.com/techno-tim/k3s-ansible/discussions/20) on how to troubleshoot common problems
Be sure to see [this post](https://github.com/timothystewart6/k3s-ansible/discussions/20) on how to troubleshoot common problems
### Testing the playbook using molecule
@@ -118,6 +231,28 @@ You can find more information about it [here](molecule/README.md).
This repo uses `pre-commit` and `pre-commit-hooks` to lint and fix common style and syntax errors. Be sure to install python packages and then run `pre-commit install`. For more information, see [pre-commit](https://pre-commit.com/)
## 🌌 Ansible Galaxy
This collection can now be used in larger ansible projects.
Instructions:
- create or modify a file `collections/requirements.yml` in your project
```yml
collections:
- name: ansible.utils
- name: community.general
- name: ansible.posix
- name: kubernetes.core
- name: https://github.com/timothystewart6/k3s-ansible.git
type: git
version: master
```
- install via `ansible-galaxy collection install -r ./collections/requirements.yml`
- every role is now available via the prefix `techno_tim.k3s_ansible.` e.g. `techno_tim.k3s_ansible.lxc`
## Thanks 🤝
This repo is really standing on the shoulders of giants. Thank you to all those who have contributed and thanks to these repos for code and ideas:
+81
View File
@@ -0,0 +1,81 @@
### REQUIRED
# The namespace of the collection. This can be a company/brand/organization or product namespace under which all
# content lives. May only contain alphanumeric lowercase characters and underscores. Namespaces cannot start with
# underscores or numbers and cannot contain consecutive underscores
namespace: techno_tim
# The name of the collection. Has the same character restrictions as 'namespace'
name: k3s_ansible
# The version of the collection. Must be compatible with semantic versioning
version: 1.0.0
# The path to the Markdown (.md) readme file. This path is relative to the root of the collection
readme: README.md
# A list of the collection's content authors. Can be just the name or in the format 'Full Name <email> (url)
# @nicks:irc/im.site#channel'
authors:
- your name <example@domain.com>
### OPTIONAL but strongly recommended
# A short summary description of the collection
description: >
The easiest way to bootstrap a self-hosted High Availability Kubernetes
cluster. A fully automated HA k3s etcd install with kube-vip, MetalLB,
and more.
# Either a single license or a list of licenses for content inside of a collection. Ansible Galaxy currently only
# accepts L(SPDX,https://spdx.org/licenses/) licenses. This key is mutually exclusive with 'license_file'
license:
- Apache-2.0
# A list of tags you want to associate with the collection for indexing/searching. A tag name has the same character
# requirements as 'namespace' and 'name'
tags:
- etcd
- high-availability
- k8s
- k3s
- k3s-cluster
- kube-vip
- kubernetes
- metallb
- rancher
# Collections that this collection requires to be installed for it to be usable. The key of the dict is the
# collection label 'namespace.name'. The value is a version range
# L(specifiers,https://python-semanticversion.readthedocs.io/en/latest/#requirement-specification). Multiple version
# range specifiers can be set and are separated by ','
dependencies:
ansible.utils: '*'
ansible.posix: '*'
community.general: '*'
kubernetes.core: '*'
# The URL of the originating SCM repository
repository: https://github.com/timothystewart6/k3s-ansible
# The URL to any online docs
documentation: https://github.com/timothystewart6/k3s-ansible
# The URL to the homepage of the collection/project
homepage: https://www.youtube.com/watch?v=CbkEWcUZ7zM
# The URL to the collection issue tracker
issues: https://github.com/timothystewart6/k3s-ansible/issues
# A list of file glob-like patterns used to filter any files or directories that should not be included in the build
# artifact. A pattern is matched from the relative path of the file or directory of the collection directory. This
# uses 'fnmatch' to match the files or directories. Some directories and files like 'galaxy.yml', '*.pyc', '*.retry',
# and '.git' are always filtered. Mutually exclusive with 'manifest'
build_ignore: []
# A dict controlling use of manifest directives used in building the collection artifact. The key 'directives' is a
# list of MANIFEST.in style
# L(directives,https://packaging.python.org/en/latest/guides/using-manifest-in/#manifest-in-commands). The key
# 'omit_default_directives' is a boolean that controls whether the default directives are used. Mutually exclusive
# with 'build_ignore'
# manifest: null
+70 -15
View File
@@ -1,53 +1,104 @@
---
k3s_version: v1.26.11+k3s2
k3s_version: v1.36.2+k3s1
# this is the user that has ssh access to these machines
ansible_user: ansibleuser
systemd_dir: /etc/systemd/system
# Set your timezone
system_timezone: "Your/Timezone"
system_timezone: Your/Timezone
# interface which will be used for flannel
flannel_iface: "eth0"
flannel_iface: eth0
# uncomment calico_iface to use tigera operator/calico cni instead of flannel https://docs.tigera.io/calico/latest/about
# calico_iface: "eth0"
calico_ebpf: false # use eBPF dataplane instead of iptables
calico_tag: v3.32.1 # calico version tag
# uncomment cilium_iface to use cilium cni instead of flannel or calico
# ensure v4.19.57, v5.1.16, v5.2.0 or more recent kernel
# cilium_iface: "eth0"
cilium_mode: native # native when nodes are on the same subnet or use BGP, otherwise set tunnel
cilium_tag: v1.20.0 # cilium version tag
cilium_cli_tag: v0.19.7 # cilium cli version tag
cilium_hubble: true # enable hubble observability relay and ui
# if using calico or cilium, you may specify the cluster pod cidr pool
cluster_cidr: 10.52.0.0/16
# enable cilium bgp control plane for lb services and pod cidrs. disables metallb.
cilium_bgp: false
# bgp parameters for cilium cni. only active when cilium_iface is defined and cilium_bgp is true.
cilium_bgp_my_asn: "64513"
cilium_bgp_peer_asn: "64512"
cilium_bgp_peer_address: 192.168.30.1
cilium_bgp_lb_cidr: 192.168.31.0/24 # cidr for cilium loadbalancer ipam
# enable kube-vip ARP broadcasts
kube_vip_arp: true
# enable kube-vip BGP peering
kube_vip_bgp: false
# bgp parameters for kube-vip
kube_vip_bgp_routerid: "127.0.0.1" # Defines the router ID for the BGP server
kube_vip_bgp_as: "64513" # Defines the AS for the BGP server
kube_vip_bgp_peeraddress: "192.168.30.1" # Defines the address for the BGP peer
kube_vip_bgp_peeras: "64512" # Defines the AS for the BGP peer
# apiserver_endpoint is virtual ip-address which will be configured on each master
apiserver_endpoint: "192.168.30.222"
apiserver_endpoint: 192.168.30.222
# k3s_token is required masters can talk together securely
# this token should be alpha numeric only
k3s_token: "some-SUPER-DEDEUPER-secret-password"
k3s_token: some-SUPER-DEDEUPER-secret-password
# The IP on which the node is reachable in the cluster.
# Here, a sensible default is provided, you can still override
# it for each of your hosts, though.
k3s_node_ip: '{{ ansible_facts[flannel_iface]["ipv4"]["address"] }}'
k3s_node_ip: "{{ ansible_facts[(cilium_iface | default(calico_iface | default(flannel_iface)))]['ipv4']['address'] }}"
# Disable the taint manually by setting: k3s_master_taint = false
k3s_master_taint: "{{ true if groups['node'] | default([]) | length >= 1 else false }}"
# these arguments are recommended for servers as well as agents:
extra_args: >-
--flannel-iface={{ flannel_iface }}
{{ '--flannel-iface=' + flannel_iface if calico_iface is not defined and cilium_iface is not defined else '' }}
--node-ip={{ k3s_node_ip }}
# change these to your liking, the only required are: --disable servicelb, --tls-san {{ apiserver_endpoint }}
# the contents of the if block is also required if using calico or cilium
extra_server_args: >-
{{ extra_args }}
{{ '--node-taint node-role.kubernetes.io/master=true:NoSchedule' if k3s_master_taint else '' }}
{% if calico_iface is defined or cilium_iface is defined %}
--flannel-backend=none
--disable-network-policy
--cluster-cidr={{ cluster_cidr | default('10.52.0.0/16') }}
{% endif %}
--tls-san {{ apiserver_endpoint }}
--disable servicelb
--disable traefik
extra_agent_args: >-
{{ extra_args }}
# image tag for kube-vip
kube_vip_tag_version: "v0.5.12"
kube_vip_tag_version: v1.2.2
# tag for kube-vip-cloud-provider manifest
# kube_vip_cloud_provider_tag_version: "v0.0.12"
# kube-vip ip range for load balancer
# (uncomment to use kube-vip for services instead of MetalLB)
# kube_vip_lb_ip_range: "192.168.30.80-192.168.30.90"
# metallb type frr or native
metal_lb_type: "native"
metal_lb_type: native
# metallb mode layer2 or bgp
metal_lb_mode: "layer2"
metal_lb_mode: layer2
# bgp options
# metal_lb_bgp_my_asn: "64513"
@@ -55,20 +106,20 @@ metal_lb_mode: "layer2"
# metal_lb_bgp_peer_address: "192.168.30.1"
# image tag for metal lb
metal_lb_speaker_tag_version: "v0.13.9"
metal_lb_controller_tag_version: "v0.13.9"
metal_lb_speaker_tag_version: v0.16.0
metal_lb_controller_tag_version: v0.16.0
# metallb ip range for load balancer
metal_lb_ip_range: "192.168.30.80-192.168.30.90"
metal_lb_ip_range: 192.168.30.80-192.168.30.90
# Only enable if your nodes are proxmox LXC nodes, make sure to configure your proxmox nodes
# in your hosts.ini file.
# Please read https://gist.github.com/triangletodd/02f595cd4c0dc9aac5f7763ca2264185 before using this.
# Most notably, your containers must be privileged, and must not have nesting set to true.
# Please note this script disables most of the security of lxc containers, with the trade off being that lxc
# containers are significantly more resource efficent compared to full VMs.
# containers are significantly more resource efficient compared to full VMs.
# Mixing and matching VMs and lxc containers is not supported, ymmv if you want to do this.
# I would only really recommend using this if you have partiularly low powered proxmox nodes where the overhead of
# I would only really recommend using this if you have particularly low powered proxmox nodes where the overhead of
# VMs would use a significant portion of your available resources.
proxmox_lxc_configure: false
# the user that you would use to ssh into the host, for example if you run ssh some-user@my-proxmox-host,
@@ -122,6 +173,10 @@ custom_registries_yaml: |
username: yourusername
password: yourpassword
# On some distros like Diet Pi, there is no dbus installed. dbus required by the default reboot command.
# Uncomment if you need a custom reboot command
# custom_reboot_command: /usr/sbin/shutdown -r now
# Only enable and configure these if you access the internet through a proxy
# proxy_env:
# HTTP_PROXY: "http://proxy.domain.local:3128"
+1 -1
View File
@@ -1,2 +1,2 @@
---
ansible_user: '{{ proxmox_lxc_ssh_user }}'
ansible_user: "{{ proxmox_lxc_ssh_user }}"
+6
View File
@@ -13,6 +13,12 @@ We have these scenarios:
To save a bit of test time, this cluster is _not_ highly available, it consists of only one control and one worker node.
- **single_node**:
Very similar to the default scenario, but uses only a single node for all cluster functionality.
- **calico**:
The same as single node, but uses calico cni instead of flannel.
- **cilium**:
The same as single node, but uses cilium cni instead of flannel.
- **kube-vip**
The same as single node, but uses kube-vip as service loadbalancer instead of MetalLB
## How to execute
+44
View File
@@ -0,0 +1,44 @@
---
dependency:
name: galaxy
driver:
name: vagrant
platforms:
- name: control1
box: bento/ubuntu-26.04
memory: 4096
cpus: 4
groups:
- k3s_cluster
- master
interfaces:
- network_name: private_network
ip: 192.168.30.62
provisioner:
name: ansible
env:
ANSIBLE_VERBOSITY: 1
playbooks:
converge: ../resources/converge.yml
side_effect: ../resources/reset.yml
verify: ../resources/verify.yml
inventory:
links:
group_vars: ../../inventory/sample/group_vars
scenario:
test_sequence:
- dependency
- cleanup
- destroy
- syntax
- create
- prepare
- converge
# idempotence is not possible with the playbook in its current form.
- verify
# We are repurposing side_effect here to test the reset playbook.
# This is why we do not run it before verify (which tests the cluster),
# but after the verify step.
- side_effect
- cleanup
- destroy
+18
View File
@@ -0,0 +1,18 @@
---
- name: Apply overrides
hosts: all
serial: 1
tasks:
- name: Override host variables
ansible.builtin.set_fact:
# See:
# https://github.com/flannel-io/flannel/blob/67d603aaf45ef80f5dd39f43714fc5e6f8a637eb/Documentation/troubleshooting.md#Vagrant
calico_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
kube_vip_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
# The test VMs might be a bit slow, so we give them more time to join the cluster:
retry_count: 45
# Make sure that our IP ranges do not collide with those of the other scenarios
apiserver_endpoint: 192.168.30.224
metal_lb_ip_range: 192.168.30.100-192.168.30.109
+6
View File
@@ -0,0 +1,6 @@
---
# Durable verify inputs for the calico (Calico CNI + MetalLB) scenario.
verify_cni: calico
verify_lb: metallb
verify_lb_ip_range:
- 192.168.30.100-192.168.30.109
+44
View File
@@ -0,0 +1,44 @@
---
dependency:
name: galaxy
driver:
name: vagrant
platforms:
- name: control1
box: bento/ubuntu-26.04
memory: 4096
cpus: 4
groups:
- k3s_cluster
- master
interfaces:
- network_name: private_network
ip: 192.168.30.63
provisioner:
name: ansible
env:
ANSIBLE_VERBOSITY: 1
playbooks:
converge: ../resources/converge.yml
side_effect: ../resources/reset.yml
verify: ../resources/verify.yml
inventory:
links:
group_vars: ../../inventory/sample/group_vars
scenario:
test_sequence:
- dependency
- cleanup
- destroy
- syntax
- create
- prepare
- converge
# idempotence is not possible with the playbook in its current form.
- verify
# We are repurposing side_effect here to test the reset playbook.
# This is why we do not run it before verify (which tests the cluster),
# but after the verify step.
- side_effect
- cleanup
- destroy
+18
View File
@@ -0,0 +1,18 @@
---
- name: Apply overrides
hosts: all
serial: 1
tasks:
- name: Override host variables
ansible.builtin.set_fact:
# See:
# https://github.com/flannel-io/flannel/blob/67d603aaf45ef80f5dd39f43714fc5e6f8a637eb/Documentation/troubleshooting.md#Vagrant
cilium_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
kube_vip_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
# The test VMs might be a bit slow, so we give them more time to join the cluster:
retry_count: 45
# Make sure that our IP ranges do not collide with those of the other scenarios
apiserver_endpoint: 192.168.30.225
metal_lb_ip_range: 192.168.30.110-192.168.30.119
+6
View File
@@ -0,0 +1,6 @@
---
# Durable verify inputs for the cilium (Cilium CNI + MetalLB) scenario.
verify_cni: cilium
verify_lb: metallb
verify_lb_ip_range:
- 192.168.30.110-192.168.30.119
+80
View File
@@ -0,0 +1,80 @@
---
- name: Create
hosts: localhost
connection: local
gather_facts: false
no_log: "{{ molecule_no_log }}"
vars:
create_batches:
- [control1, control2]
- [control3, node1]
- [node2]
tasks:
- name: Verify that bounded batches cover the configured platforms exactly once
ansible.builtin.assert:
that:
- create_batches | flatten | sort == molecule_yml.platforms | map(attribute='name') | sort
- create_batches | flatten | length == create_batches | flatten | unique | length
- create_batches | map('length') | max <= 2
fail_msg: Bounded create batches do not match the configured default platforms.
- name: Generate the complete Vagrant configuration # noqa fqcn[action]
vagrant:
instances: "{{ molecule_yml.platforms }}"
default_box: "{{ molecule_yml.driver.default_box | default('generic/alpine316') }}"
provider_name: "{{ molecule_yml.driver.provider.name | default(omit, true) }}"
provision: "{{ molecule_yml.driver.provision | default(omit) }}"
cachier: "{{ molecule_yml.driver.cachier | default(omit) }}"
parallel: false
state: halt
changed_when: false
- name: Start clean Vagrant guests in bounded batches
ansible.builtin.command:
argv: >-
{{
[playbook_dir + '/../../.github/scripts/vagrant-up-timed.sh',
molecule_ephemeral_directory] + item
}}
loop: "{{ create_batches }}"
loop_control:
label: "{{ item | join(', ') }}"
changed_when: true
- name: Reconcile all instances and collect their connection configuration # noqa fqcn[action]
vagrant:
instances: "{{ molecule_yml.platforms }}"
default_box: "{{ molecule_yml.driver.default_box | default('generic/alpine316') }}"
provider_name: "{{ molecule_yml.driver.provider.name | default(omit, true) }}"
provision: "{{ molecule_yml.driver.provision | default(omit) }}"
cachier: "{{ molecule_yml.driver.cachier | default(omit) }}"
parallel: false
state: up
register: server
no_log: false
- name: Populate instance configuration dictionaries
ansible.builtin.set_fact:
instance_conf_dict:
instance: "{{ item.Host }}"
address: "{{ item.HostName }}"
user: "{{ item.User }}"
port: "{{ item.Port }}"
identity_file: "{{ item.IdentityFile }}"
loop: "{{ server.results }}"
register: instance_config_dict
- name: Convert instance configuration dictionaries to a list
ansible.builtin.set_fact:
instance_conf: >-
{{
instance_config_dict.results
| map(attribute='ansible_facts.instance_conf_dict')
| list
}}
- name: Write Molecule instance configuration
ansible.builtin.copy:
content: "{{ instance_conf | to_json | from_json | to_yaml }}"
dest: "{{ molecule_instance_config }}"
mode: "0600"
+28 -22
View File
@@ -3,76 +3,83 @@ dependency:
name: galaxy
driver:
name: vagrant
# The Vagrant driver warns that parallel VirtualBox creation can cause
# platform issues. Keep this five-node, mixed-distribution scenario serial.
parallel: false
platforms:
- name: control1
box: generic/ubuntu2204
memory: 2048
box: bento/ubuntu-26.04
memory: 1024
cpus: 2
groups:
- k3s_cluster
- master
# Keep adapter 2 stable across linked-clone rebuilds so stale host-only
# neighbor state still identifies the current scenario guest.
provider_raw_config_args:
- "customize ['modifyvm', :id, '--mac-address2', '080027A13038']"
interfaces:
- network_name: private_network
ip: 192.168.30.38
config_options:
# We currently can not use public-key based authentication on Ubuntu 22.04,
# see: https://github.com/chef/bento/issues/1405
ssh.username: "vagrant"
ssh.password: "vagrant"
- name: control2
box: generic/debian11
memory: 2048
box: bento/debian-13
memory: 1024
cpus: 2
groups:
- k3s_cluster
- master
provider_raw_config_args:
- "customize ['modifyvm', :id, '--mac-address2', '080027A13039']"
interfaces:
- network_name: private_network
ip: 192.168.30.39
- name: control3
box: generic/rocky9
memory: 2048
box: bento/rockylinux-10.1
memory: 1024
cpus: 2
groups:
- k3s_cluster
- master
provider_raw_config_args:
- "customize ['modifyvm', :id, '--mac-address2', '080027A13040']"
interfaces:
- network_name: private_network
ip: 192.168.30.40
- name: node1
box: generic/ubuntu2204
memory: 2048
box: bento/ubuntu-26.04
memory: 1024
cpus: 2
groups:
- k3s_cluster
- node
provider_raw_config_args:
- "customize ['modifyvm', :id, '--mac-address2', '080027A13041']"
interfaces:
- network_name: private_network
ip: 192.168.30.41
config_options:
# We currently can not use public-key based authentication on Ubuntu 22.04,
# see: https://github.com/chef/bento/issues/1405
ssh.username: "vagrant"
ssh.password: "vagrant"
- name: node2
box: generic/rocky9
memory: 2048
box: bento/rockylinux-10.1
memory: 1024
cpus: 2
groups:
- k3s_cluster
- node
provider_raw_config_args:
- "customize ['modifyvm', :id, '--mac-address2', '080027A13042']"
interfaces:
- network_name: private_network
ip: 192.168.30.42
provisioner:
name: ansible
env:
ANSIBLE_VERBOSITY: 1
playbooks:
create: create.yml
converge: ../resources/converge.yml
side_effect: ../resources/reset.yml
verify: ../resources/verify.yml
@@ -82,7 +89,6 @@ provisioner:
scenario:
test_sequence:
- dependency
- lint
- cleanup
- destroy
- syntax
+5 -1
View File
@@ -1,12 +1,16 @@
---
- name: Apply overrides
hosts: all
serial: 1
tasks:
- name: Override host variables
ansible.builtin.set_fact:
# See:
# https://github.com/flannel-io/flannel/blob/67d603aaf45ef80f5dd39f43714fc5e6f8a637eb/Documentation/troubleshooting.md#Vagrant
flannel_iface: eth1
flannel_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
# kube-vip cannot infer the cluster interface in these multi-NIC
# Vagrant guests because the default route is on eth0.
kube_vip_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
# The test VMs might be a bit slow, so we give them more time to join the cluster:
retry_count: 45
+127 -8
View File
@@ -5,18 +5,137 @@
- name: Network setup
hosts: all
vars:
primary_master: "{{ groups[group_name_master | default('master')][0] }}"
primary_cluster_ip: >-
{{ hostvars[primary_master].k3s_node_ip | split(',') | first }}
cluster_interface: >-
{{ cilium_iface | default(calico_iface | default(flannel_iface)) }}
primary_cluster_interface: >-
{{ hostvars[primary_master].cilium_iface
| default(hostvars[primary_master].calico_iface
| default(hostvars[primary_master].flannel_iface)) }}
primary_cluster_mac: >-
{{ hostvars[primary_master].ansible_facts[primary_cluster_interface].macaddress }}
tasks:
- name: Disable firewalld
when: ansible_distribution == "Rocky"
# Rocky Linux comes with firewalld enabled. It blocks some of the network
# connections needed for our k3s cluster. For our test setup, we just disable
# it since the VM host's firewall is still active for connections to and from
# the Internet.
- name: Gather service facts
ansible.builtin.service_facts:
- name: Disable guest firewall services
# The disposable test guests use an isolated VirtualBox network. A distro
# firewall can allow ICMP while silently blocking the inter-node Kubernetes
# API connection, so disable the known guest firewalls consistently.
# When building your own cluster, please DO NOT blindly copy this. Instead,
# please create a custom firewall configuration that fits your network design
# and security needs.
ansible.builtin.systemd:
name: firewalld
enabled: no
name: "{{ item }}"
enabled: false
state: stopped
become: true
loop:
- firewalld.service
- nftables.service
- ufw.service
when: item in ansible_facts.services
- name: Verify the private cluster interface
ansible.builtin.assert:
that:
- flannel_iface in ansible_facts
- ansible_facts[flannel_iface].ipv4 is defined
- ansible_facts[flannel_iface].ipv4.address is defined
fail_msg: >-
The Vagrant private interface {{ flannel_iface }} does not have an
IPv4 address on {{ inventory_hostname }}.
- name: Pin disposable cluster peer neighbor entries
ansible.builtin.command:
argv:
- ip
- neigh
- replace
- "{{ peer_cluster_ip }}"
- lladdr
- "{{ peer_cluster_mac }}"
- nud
- permanent
- dev
- "{{ cluster_interface }}"
become: true
changed_when: false
loop: "{{ groups['k3s_cluster'] }}"
loop_control:
label: "{{ inventory_hostname }} -> {{ item }}"
vars:
peer_cluster_interface: >-
{{ hostvars[item].cilium_iface
| default(hostvars[item].calico_iface
| default(hostvars[item].flannel_iface)) }}
peer_cluster_ip: >-
{{ hostvars[item].k3s_node_ip | split(',') | first }}
peer_cluster_mac: >-
{{ hostvars[item].ansible_facts[peer_cluster_interface].macaddress }}
when: item != inventory_hostname
- name: Verify guest-to-guest cluster network reachability
ansible.builtin.command:
argv:
- ping
- -c
- "1"
- -W
- "1"
- "{{ primary_cluster_ip }}"
register: primary_cluster_ping
until: primary_cluster_ping.rc == 0
retries: 6
delay: 2
changed_when: false
- name: Read the primary neighbor entry
ansible.builtin.command:
argv:
- ip
- neigh
- show
- to
- "{{ primary_cluster_ip }}"
- dev
- "{{ cluster_interface }}"
register: primary_cluster_neighbor
changed_when: false
when: inventory_hostname != primary_master
- name: Verify the primary neighbor identity
ansible.builtin.assert:
that:
- (primary_cluster_mac | lower) in (primary_cluster_neighbor.stdout | lower)
fail_msg: >-
{{ inventory_hostname }} resolved primary {{ primary_cluster_ip }} to
an unexpected MAC on {{ cluster_interface }}. Expected
{{ primary_cluster_mac }}, got: {{ primary_cluster_neighbor.stdout }}
when: inventory_hostname != primary_master
- name: Verify GitHub release host DNS
ansible.builtin.getent:
database: hosts
key: github.com
register: github_dns
retries: 6
delay: 5
until: github_dns is succeeded
- name: Verify k3s checksum URL is reachable
ansible.builtin.uri:
url: >-
https://github.com/k3s-io/k3s/releases/download/{{ k3s_version
}}/sha256sum-amd64.txt
method: HEAD
follow_redirects: safe
status_code: [200, 302]
timeout: 15
register: k3s_checksum_request
retries: 3
delay: 5
until: k3s_checksum_request.status in [200, 302]
+9
View File
@@ -0,0 +1,9 @@
---
# Durable verify inputs for the default (flannel + MetalLB) scenario.
# These are plain inventory vars linked into the shared Molecule inventory so
# the verify play can see them even though the converge play's set_fact values
# are not persisted between the two Ansible processes.
verify_cni: flannel
verify_lb: metallb
verify_lb_ip_range:
- 192.168.30.80-192.168.30.90
+8 -22
View File
@@ -5,8 +5,8 @@ driver:
name: vagrant
platforms:
- name: control1
box: generic/ubuntu2204
memory: 2048
box: bento/ubuntu-26.04
memory: 1024
cpus: 2
groups:
- k3s_cluster
@@ -14,15 +14,10 @@ platforms:
interfaces:
- network_name: private_network
ip: fdad:bad:ba55::de:11
config_options:
# We currently can not use public-key based authentication on Ubuntu 22.04,
# see: https://github.com/chef/bento/issues/1405
ssh.username: "vagrant"
ssh.password: "vagrant"
- name: control2
box: generic/ubuntu2204
memory: 2048
box: bento/ubuntu-26.04
memory: 1024
cpus: 2
groups:
- k3s_cluster
@@ -30,15 +25,10 @@ platforms:
interfaces:
- network_name: private_network
ip: fdad:bad:ba55::de:12
config_options:
# We currently can not use public-key based authentication on Ubuntu 22.04,
# see: https://github.com/chef/bento/issues/1405
ssh.username: "vagrant"
ssh.password: "vagrant"
- name: node1
box: generic/ubuntu2204
memory: 2048
box: bento/ubuntu-26.04
memory: 1024
cpus: 2
groups:
- k3s_cluster
@@ -46,13 +36,10 @@ platforms:
interfaces:
- network_name: private_network
ip: fdad:bad:ba55::de:21
config_options:
# We currently can not use public-key based authentication on Ubuntu 22.04,
# see: https://github.com/chef/bento/issues/1405
ssh.username: "vagrant"
ssh.password: "vagrant"
provisioner:
name: ansible
env:
ANSIBLE_VERBOSITY: 1
playbooks:
converge: ../resources/converge.yml
side_effect: ../resources/reset.yml
@@ -63,7 +50,6 @@ provisioner:
scenario:
test_sequence:
- dependency
- lint
- cleanup
- destroy
- syntax
+3 -2
View File
@@ -1,17 +1,18 @@
---
- name: Apply overrides
hosts: all
serial: 1
tasks:
- name: Override host variables (1/2)
ansible.builtin.set_fact:
# See:
# https://github.com/flannel-io/flannel/blob/67d603aaf45ef80f5dd39f43714fc5e6f8a637eb/Documentation/troubleshooting.md#Vagrant
flannel_iface: eth1
flannel_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
# In this scenario, we have multiple interfaces that the VIP could be
# broadcasted on. Since we have assigned a dedicated private network
# here, let's make sure that it is used.
kube_vip_iface: eth1
kube_vip_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
# The test VMs might be a bit slow, so we give them more time to join the cluster:
retry_count: 45
+1 -1
View File
@@ -38,7 +38,7 @@
dest: /etc/netplan/55-flannel-ipv4.yaml
owner: root
group: root
mode: 0644
mode: "0644"
register: netplan_template
- name: Apply netplan configuration
+7
View File
@@ -0,0 +1,7 @@
---
# Durable verify inputs for the ipv6 (flannel CNI + MetalLB) scenario.
verify_cni: flannel
verify_lb: metallb
verify_lb_ip_range:
- fdad:bad:ba55::1b:0/112
- 192.168.123.80-192.168.123.90
+44
View File
@@ -0,0 +1,44 @@
---
dependency:
name: galaxy
driver:
name: vagrant
platforms:
- name: control1
box: bento/ubuntu-26.04
memory: 4096
cpus: 4
groups:
- k3s_cluster
- master
interfaces:
- network_name: private_network
ip: 192.168.30.62
provisioner:
name: ansible
env:
ANSIBLE_VERBOSITY: 1
playbooks:
converge: ../resources/converge.yml
side_effect: ../resources/reset.yml
verify: ../resources/verify.yml
inventory:
links:
group_vars: ../../inventory/sample/group_vars
scenario:
test_sequence:
- dependency
- cleanup
- destroy
- syntax
- create
- prepare
- converge
# idempotence is not possible with the playbook in its current form.
- verify
# We are repurposing side_effect here to test the reset playbook.
# This is why we do not run it before verify (which tests the cluster),
# but after the verify step.
- side_effect
- cleanup
- destroy
+19
View File
@@ -0,0 +1,19 @@
---
- name: Apply overrides
hosts: all
serial: 1
tasks:
- name: Override host variables
ansible.builtin.set_fact:
# See:
# https://github.com/flannel-io/flannel/blob/67d603aaf45ef80f5dd39f43714fc5e6f8a637eb/Documentation/troubleshooting.md#Vagrant
flannel_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
kube_vip_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
# The test VMs might be a bit slow, so we give them more time to join the cluster:
retry_count: 45
# Make sure that our IP ranges do not collide with those of the other scenarios
apiserver_endpoint: 192.168.30.225
# Use kube-vip instead of MetalLB
kube_vip_lb_ip_range: 192.168.30.110-192.168.30.119
+9
View File
@@ -0,0 +1,9 @@
---
# Durable verify inputs for the kube-vip (flannel CNI + kube-vip LB) scenario.
verify_cni: flannel
verify_lb: kube-vip
# The kube-vip cloud provider tag is not defined in the linked sample group
# vars (its sample entry is commented out), so it is supplied here.
verify_kube_vip_cloud_provider_tag: v0.0.12
verify_lb_ip_range:
- 192.168.30.110-192.168.30.119
+3
View File
@@ -1,5 +1,8 @@
---
- name: Verify
hosts: all
vars_files:
- >-
{{ lookup("ansible.builtin.env", "MOLECULE_SCENARIO_DIRECTORY") }}/verify-vars.yml
roles:
- verify_from_outside
@@ -7,6 +7,8 @@
ansible.builtin.import_tasks: kubecfg-fetch.yml
- name: "TEST CASE: Get nodes"
ansible.builtin.include_tasks: test/get-nodes.yml
- name: "TEST CASE: Verify components"
ansible.builtin.include_tasks: test/verify-components.yml
- name: "TEST CASE: Deploy example"
ansible.builtin.include_tasks: test/deploy-example.yml
always:
@@ -27,26 +27,95 @@
name: nginx
namespace: "{{ testing_namespace }}"
kubeconfig: "{{ kubecfg_path }}"
vars: &load_balancer_metadata
vars:
metallb_ip: status.loadBalancer.ingress[0].ip
metallb_port: spec.ports[0].port
register: nginx_services
- name: Wait for the load balancer address to be assigned
ansible.builtin.set_fact:
nginx_lb_ip: >-
{{
nginx_services.resources[0].status.loadBalancer.ingress[0].ip
if (nginx_services.resources | length > 0) and
(nginx_services.resources[0].status.loadBalancer.ingress is defined) and
(nginx_services.resources[0].status.loadBalancer.ingress | length > 0)
else ''
}}
- name: Retry until the load balancer service has an external IP
block:
- name: Refresh nginx service until it has an assigned address
kubernetes.core.k8s_info:
kind: service
name: nginx
namespace: "{{ testing_namespace }}"
kubeconfig: "{{ kubecfg_path }}"
register: nginx_lb_wait
until: >-
(nginx_lb_wait.resources | length > 0) and
(nginx_lb_wait.resources[0].status.loadBalancer.ingress is defined) and
(nginx_lb_wait.resources[0].status.loadBalancer.ingress | length > 0)
retries: 30
delay: 5
- name: Record the assigned load balancer address
ansible.builtin.set_fact:
nginx_lb_ip: >-
{{ nginx_lb_wait.resources[0].status.loadBalancer.ingress[0].ip }}
- name: Assert that the nginx welcome page is available
ansible.builtin.uri:
url: http://{{ ip | ansible.utils.ipwrap }}:{{ port_ }}/
return_content: yes
url: http://{{ nginx_lb_ip | ansible.utils.ipwrap }}:{{ port_ }}/
return_content: true
register: result
failed_when: "'Welcome to nginx!' not in result.content"
vars:
ip: >-
{{ nginx_services.resources[0].status.loadBalancer.ingress[0].ip }}
port_: >-
{{ nginx_services.resources[0].spec.ports[0].port }}
# Deactivated linter rules:
# - jinja[invalid]: As of version 6.6.0, ansible-lint complains that the input to ipwrap
# would be undefined. This will not be the case during playbook execution.
# noqa jinja[invalid]
- name: Initialize load balancer address range check
ansible.builtin.set_fact:
lb_addr_in_range: false
lb_ip_value: "{{ nginx_lb_ip }}"
- name: Check load balancer address against start-end pools
ansible.builtin.set_fact:
lb_addr_in_range: true
loop: "{{ verify_lb_ip_range }}"
loop_control:
label: "{{ item }}"
when:
- "'-' in item"
- "'/' not in item"
- >-
(lb_ip_value | ansible.utils.ipaddr('int') | int) >=
(item.split('-')[0] | ansible.utils.ipaddr('int') | int)
- >-
(lb_ip_value | ansible.utils.ipaddr('int') | int) <=
(item.split('-')[1] | ansible.utils.ipaddr('int') | int)
- name: Check load balancer address against CIDR pools
ansible.builtin.set_fact:
lb_addr_in_range: true
loop: "{{ verify_lb_ip_range }}"
loop_control:
label: "{{ item }}"
when:
- "'/' in item"
- (lb_ip_value | ansible.utils.ipaddr(item)) is string
- name: Assert that the load balancer address is within a configured pool
ansible.builtin.assert:
that: lb_addr_in_range
success_msg: "LoadBalancer address {{ lb_ip_value }} is in a configured range"
fail_msg: >-
LoadBalancer address {{ lb_ip_value }} is not in a configured
range {{ verify_lb_ip_range }}
# Deactivated linter rules:
# - jinja[invalid]: As of version 6.6.0, ansible-lint complains that the input to ipwrap
# would be undefined. This will not be the case during playbook execution.
# noqa jinja[invalid]
always:
- name: "Remove namespace: {{ testing_namespace }}"
@@ -9,7 +9,7 @@
ansible.builtin.assert:
that: found_nodes == expected_nodes
success_msg: "Found nodes as expected: {{ found_nodes }}"
fail_msg: "Expected nodes {{ expected_nodes }}, but found nodes {{ found_nodes }}"
fail_msg: Expected nodes {{ expected_nodes }}, but found nodes {{ found_nodes }}
vars:
found_nodes: >-
{{ cluster_nodes | json_query('resources[*].metadata.name') | unique | sort }}
@@ -22,7 +22,7 @@
| unique
| sort
}}
# Deactivated linter rules:
# Deactivated linter rules:
# - jinja[invalid]: As of version 6.6.0, ansible-lint complains that the input to ipwrap
# would be undefined. This will not be the case during playbook execution.
# noqa jinja[invalid]
@@ -0,0 +1,313 @@
---
# Scenario-aware verification of cluster components and their live image tags.
# Scenario identity (verify_cni / verify_lb) and expected address range come
# from each scenario's verify-vars.yml, which is plain inventory data available
# to the verify play. Converge-time set_fact values are not persisted between
# the two Ansible processes, so they are never used here.
- name: Verify cluster components report expected versions
block:
- name: Get all nodes with their kubelet versions
kubernetes.core.k8s_info:
kind: node
kubeconfig: "{{ kubecfg_path }}"
register: verify_nodes
- name: Assert each node reports the expected Kubernetes version
ansible.builtin.assert:
that: item.status.nodeInfo.kubeletVersion == k3s_version
success_msg: "{{ item.metadata.name }} reports {{ k3s_version }}"
fail_msg: >-
{{ item.metadata.name }} reports
{{ item.status.nodeInfo.kubeletVersion }},
expected {{ k3s_version }}
loop: "{{ verify_nodes.resources }}"
loop_control:
label: "{{ item.metadata.name }}"
- name: Verify Flannel is the active CNI
when: verify_cni == 'flannel'
block:
- name: Assert every node reports Ready
ansible.builtin.assert:
that: item.status.conditions
| selectattr('type', 'equalto', 'Ready')
| map(attribute='status') | first | default('') == 'True'
success_msg: "{{ item.metadata.name }} is Ready"
fail_msg: "{{ item.metadata.name }} is not Ready"
loop: "{{ verify_nodes.resources }}"
loop_control:
label: "{{ item.metadata.name }} ready"
- name: Get any Calico namespaces with Flannel enabled
kubernetes.core.k8s_info:
kind: Namespace
name: calico-system
kubeconfig: "{{ kubecfg_path }}"
register: flannel_calico_absent
- name: Assert there is no Calico system namespace
ansible.builtin.assert:
that: flannel_calico_absent.resources | length == 0
success_msg: "No Calico present with Flannel"
fail_msg: "A Calico namespace exists alongside Flannel"
- name: Get the Cilium namespace with Flannel enabled
kubernetes.core.k8s_info:
kind: Namespace
name: cilium
kubeconfig: "{{ kubecfg_path }}"
register: flannel_cilium
- name: Assert the Cilium namespace is absent
ansible.builtin.assert:
that: flannel_cilium.resources | length == 0
success_msg: "No Cilium present with Flannel"
fail_msg: "A Cilium namespace exists alongside Flannel"
- name: Verify Calico is the active CNI
when: verify_cni == 'calico'
block:
- name: Get the Calico node DaemonSet image
kubernetes.core.k8s_info:
kind: DaemonSet
name: calico-node
namespace: calico-system
kubeconfig: "{{ kubecfg_path }}"
register: calico_node_ds
- name: Assert the Calico node image uses the expected tag
ansible.builtin.assert:
that:
- calico_node_ds.resources | length == 1
- calico_node_image | regex_search(':' ~ calico_tag)
success_msg: "Calico node image uses tag {{ calico_tag }}"
fail_msg: >-
Calico node image {{ calico_node_image }},
expected {{ calico_tag }}
vars:
calico_node_image: "{{ calico_node_ds.resources[0].spec.template.spec.containers[0].image }}"
- name: Get Calico TigeraStatus for calico and apiserver
kubernetes.core.k8s_info:
api_version: operator.tigera.io/v1
kind: TigeraStatus
name: "{{ item }}"
kubeconfig: "{{ kubecfg_path }}"
register: calico_tigerastatus
loop:
- calico
- apiserver
loop_control:
label: "Tigerastatus/{{ item }}"
- name: Assert Calico TigeraStatus reports Available
ansible.builtin.assert:
that: >-
item.resources | length == 1 and
(item.resources[0].status.conditions
| selectattr('type', 'equalto', 'Available')
| map(attribute='status') | first | default('')) == 'True'
success_msg: "Tigerastatus {{ item.resources[0].metadata.name }} is Available"
fail_msg: "Tigerastatus is not Available"
loop: "{{ calico_tigerastatus.results }}"
loop_control:
label: "Tigerastatus Available"
- name: Get any Flannel DaemonSets with Calico enabled
kubernetes.core.k8s_info:
kind: DaemonSet
namespace: kube-flannel
kubeconfig: "{{ kubecfg_path }}"
register: no_flannel_ds
- name: Assert there are no Flannel DaemonSets
ansible.builtin.assert:
that: no_flannel_ds.resources | length == 0
success_msg: "No Flannel DaemonSet present with Calico"
fail_msg: "A Flannel DaemonSet exists alongside Calico"
- name: Verify Cilium is the active CNI
when: verify_cni == 'cilium'
block:
- name: Get the Cilium agent and operator images
kubernetes.core.k8s_info:
kind: "{{ item.kind }}"
name: "{{ item.name }}"
namespace: kube-system
kubeconfig: "{{ kubecfg_path }}"
register: cilium_info
loop:
- { kind: DaemonSet, name: cilium }
- { kind: Deployment, name: cilium-operator }
loop_control:
label: "{{ item.kind }}/{{ item.name }}"
- name: Assert Cilium agent and operator use the expected image tag
ansible.builtin.assert:
that:
- cilium_agent_image | regex_search(':' ~ cilium_tag)
- cilium_operator_image | regex_search(':' ~ cilium_tag)
success_msg: "Cilium agent and operator use {{ cilium_tag }}"
fail_msg: >-
Cilium agent {{ cilium_agent_image }},
operator {{ cilium_operator_image }},
expected {{ cilium_tag }}
vars:
cilium_agent_image: >-
{{ (cilium_info.results
| selectattr('resources', 'defined')
| map(attribute='resources')
| list
| map(attribute='0')
| selectattr('kind', 'equalto', 'DaemonSet')
| list)[0].spec.template.spec.containers[0].image }}
cilium_operator_image: >-
{{ (cilium_info.results
| selectattr('resources', 'defined')
| map(attribute='resources')
| list
| map(attribute='0')
| selectattr('kind', 'equalto', 'Deployment')
| list)[0].spec.template.spec.containers[0].image }}
- name: Get Hubble relay and UI deployments when enabled
kubernetes.core.k8s_info:
kind: Deployment
name: "{{ item }}"
namespace: kube-system
kubeconfig: "{{ kubecfg_path }}"
register: hubble_info
loop:
- hubble-relay
- hubble-ui
loop_control:
label: "Deployment/{{ item }}"
when: cilium_hubble | bool
- name: Assert Hubble components are Ready when enabled
ansible.builtin.assert:
that:
- item.resources | length == 1
- item.resources[0].status.readyReplicas | default(0) >= 1
success_msg: "Hubble deployment {{ item.resources[0].metadata.name }} is Ready"
fail_msg: "Hubble deployment is not Ready"
loop: "{{ hubble_info.results }}"
loop_control:
label: "Hubble deployment"
when: cilium_hubble | bool
- name: Get any Flannel DaemonSets with Cilium enabled
kubernetes.core.k8s_info:
kind: DaemonSet
namespace: kube-flannel
kubeconfig: "{{ kubecfg_path }}"
register: no_flannel_ds_cilium
- name: Assert there are no Flannel DaemonSets
ansible.builtin.assert:
that: no_flannel_ds_cilium.resources | length == 0
success_msg: "No Flannel DaemonSet present with Cilium"
fail_msg: "A Flannel DaemonSet exists alongside Cilium"
- name: Verify MetalLB is the active load balancer
when: verify_lb == 'metallb'
block:
- name: Get the MetalLB controller and speaker images
kubernetes.core.k8s_info:
kind: "{{ item.kind }}"
name: "{{ item.name }}"
namespace: metallb-system
kubeconfig: "{{ kubecfg_path }}"
register: metallb_info
loop:
- { kind: Deployment, name: controller }
- { kind: DaemonSet, name: speaker }
loop_control:
label: "{{ item.kind }}/{{ item.name }}"
- name: Assert MetalLB controller and speaker use the expected image tags
ansible.builtin.assert:
that:
- controller_image | regex_search(metal_lb_controller_tag_version)
- speaker_image | regex_search(metal_lb_speaker_tag_version)
success_msg: >-
MetalLB controller {{ metal_lb_controller_tag_version }},
speaker {{ metal_lb_speaker_tag_version }}
fail_msg: >-
MetalLB controller {{ controller_image }},
speaker {{ speaker_image }}
vars:
controller_image: >-
{{ (metallb_info.results
| selectattr('resources', 'defined')
| map(attribute='resources')
| list
| map(attribute='0')
| selectattr('kind', 'equalto', 'Deployment')
| list)[0].spec.template.spec.containers[0].image }}
speaker_image: >-
{{ (metallb_info.results
| selectattr('resources', 'defined')
| map(attribute='resources')
| list
| map(attribute='0')
| selectattr('kind', 'equalto', 'DaemonSet')
| list)[0].spec.template.spec.containers[0].image }}
- name: Verify kube-vip is the active load balancer
when: verify_lb == 'kube-vip'
block:
- name: Get the kube-vip and cloud provider images
kubernetes.core.k8s_info:
kind: "{{ item.kind }}"
name: "{{ item.name }}"
namespace: kube-system
kubeconfig: "{{ kubecfg_path }}"
register: kubevip_info
loop:
- { kind: DaemonSet, name: kube-vip-ds }
- { kind: Deployment, name: kube-vip-cloud-provider }
loop_control:
label: "{{ item.kind }}/{{ item.name }}"
- name: Assert the kube-vip and cloud provider image tags
ansible.builtin.assert:
that:
- kubevip_image | regex_search(':' ~ kube_vip_tag_version)
- cloud_provider_image | regex_search(verify_kube_vip_cloud_provider_tag)
success_msg: >-
kube-vip {{ kube_vip_tag_version }},
cloud provider {{ verify_kube_vip_cloud_provider_tag }}
fail_msg: >-
kube-vip {{ kubevip_image }},
cloud provider {{ cloud_provider_image }}
vars:
kubevip_image: >-
{{ (kubevip_info.results
| selectattr('resources', 'defined')
| map(attribute='resources')
| list
| map(attribute='0')
| selectattr('kind', 'equalto', 'DaemonSet')
| list)[0].spec.template.spec.containers[0].image }}
cloud_provider_image: >-
{{ (kubevip_info.results
| selectattr('resources', 'defined')
| map(attribute='resources')
| list
| map(attribute='0')
| selectattr('kind', 'equalto', 'Deployment')
| list)[0].spec.template.spec.containers[0].image }}
- name: Get the MetalLB namespace with kube-vip enabled
kubernetes.core.k8s_info:
kind: Namespace
name: metallb-system
kubeconfig: "{{ kubecfg_path }}"
register: metallb_absent
- name: Assert the MetalLB namespace does not exist
ansible.builtin.assert:
that: metallb_absent.resources | length == 0
success_msg: "MetalLB is not installed with kube-vip"
fail_msg: "MetalLB namespace exists alongside kube-vip"
+3 -7
View File
@@ -5,14 +5,9 @@ driver:
name: vagrant
platforms:
- name: control1
box: generic/ubuntu2204
box: bento/ubuntu-26.04
memory: 4096
cpus: 4
config_options:
# We currently can not use public-key based authentication on Ubuntu 22.04,
# see: https://github.com/chef/bento/issues/1405
ssh.username: "vagrant"
ssh.password: "vagrant"
groups:
- k3s_cluster
- master
@@ -21,6 +16,8 @@ platforms:
ip: 192.168.30.50
provisioner:
name: ansible
env:
ANSIBLE_VERBOSITY: 1
playbooks:
converge: ../resources/converge.yml
side_effect: ../resources/reset.yml
@@ -31,7 +28,6 @@ provisioner:
scenario:
test_sequence:
- dependency
- lint
- cleanup
- destroy
- syntax
+5 -3
View File
@@ -1,16 +1,18 @@
---
- name: Apply overrides
hosts: all
serial: 1
tasks:
- name: Override host variables
ansible.builtin.set_fact:
# See:
# https://github.com/flannel-io/flannel/blob/67d603aaf45ef80f5dd39f43714fc5e6f8a637eb/Documentation/troubleshooting.md#Vagrant
flannel_iface: eth1
flannel_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
kube_vip_iface: "{{ 'eth1' if 'eth1' in ansible_facts.interfaces else 'enp0s8' }}"
# The test VMs might be a bit slow, so we give them more time to join the cluster:
retry_count: 45
# Make sure that our IP ranges do not collide with those of the default scenario
apiserver_endpoint: "192.168.30.223"
metal_lb_ip_range: "192.168.30.91-192.168.30.99"
apiserver_endpoint: 192.168.30.223
metal_lb_ip_range: 192.168.30.91-192.168.30.99
+6
View File
@@ -0,0 +1,6 @@
---
# Durable verify inputs for the single_node (flannel + MetalLB) scenario.
verify_cni: flannel
verify_lb: metallb
verify_lb_ip_range:
- 192.168.30.91-192.168.30.99
+3 -2
View File
@@ -1,9 +1,10 @@
---
- name: Reboot k3s_cluster
hosts: k3s_cluster
gather_facts: yes
gather_facts: true
tasks:
- name: Reboot the nodes (and Wait upto 5 mins max)
become: true
reboot:
ansible.builtin.reboot:
reboot_command: "{{ custom_reboot_command | default(omit) }}"
reboot_timeout: 300
+9 -9
View File
@@ -1,10 +1,10 @@
ansible-core>=2.13.5
ansible-core>=2.16.2
jmespath>=1.0.1
jsonpatch>=1.32
kubernetes>=25.3.0
molecule-vagrant>=1.0.0
molecule>=4.0.3
netaddr>=0.8.0
pre-commit>=2.20.0
pre-commit-hooks>=1.3.1
pyyaml>=6.0
jsonpatch>=1.33
kubernetes>=29.0.0
molecule-plugins[vagrant]
molecule>=6.0.3
netaddr>=0.10.1
pre-commit>=3.6.0
pre-commit-hooks>=4.5.0
pyyaml>=6.0.1
+71 -80
View File
@@ -4,174 +4,165 @@
#
# pip-compile requirements.in
#
ansible-compat==3.0.1
ansible-compat==4.1.11
# via molecule
ansible-core==2.15.4
ansible-core==2.18.0
# via
# -r requirements.in
# ansible-compat
arrow==1.2.3
# via jinja2-time
attrs==22.1.0
# via jsonschema
binaryornot==0.4.4
# via cookiecutter
cachetools==5.2.0
# molecule
attrs==23.2.0
# via
# jsonschema
# referencing
bracex==2.4
# via wcmatch
cachetools==5.3.2
# via google-auth
certifi==2022.9.24
certifi==2023.11.17
# via
# kubernetes
# requests
cffi==1.15.1
cffi==1.16.0
# via cryptography
cfgv==3.3.1
cfgv==3.4.0
# via pre-commit
chardet==5.0.0
# via binaryornot
charset-normalizer==2.1.1
charset-normalizer==3.3.2
# via requests
click==8.1.3
click==8.1.7
# via
# click-help-colors
# cookiecutter
# molecule
click-help-colors==0.9.1
click-help-colors==0.9.4
# via molecule
commonmark==0.9.1
# via rich
cookiecutter==2.1.1
# via molecule
cryptography==38.0.3
cryptography==41.0.7
# via ansible-core
distlib==0.3.6
distlib==0.3.8
# via virtualenv
distro==1.8.0
# via selinux
enrich==1.2.7
# via molecule
filelock==3.8.0
filelock==3.13.1
# via virtualenv
google-auth==2.14.0
google-auth==2.26.2
# via kubernetes
identify==2.5.8
identify==2.5.33
# via pre-commit
idna==3.4
idna==3.6
# via requests
jinja2==3.1.2
jinja2==3.1.3
# via
# ansible-core
# cookiecutter
# jinja2-time
# molecule
# molecule-vagrant
jinja2-time==0.2.0
# via cookiecutter
jmespath==1.0.1
# via -r requirements.in
jsonpatch==1.33
# via -r requirements.in
jsonpointer==2.3
jsonpointer==2.4
# via jsonpatch
jsonschema==4.17.0
jsonschema==4.21.1
# via
# ansible-compat
# molecule
kubernetes==25.3.0
jsonschema-specifications==2023.12.1
# via jsonschema
kubernetes==29.0.0
# via -r requirements.in
markupsafe==2.1.1
markdown-it-py==3.0.0
# via rich
markupsafe==2.1.4
# via jinja2
molecule==4.0.4
mdurl==0.1.2
# via markdown-it-py
molecule==6.0.3
# via
# -r requirements.in
# molecule-vagrant
molecule-vagrant==1.0.0
# molecule-plugins
molecule-plugins[vagrant]==23.5.3
# via -r requirements.in
netaddr==0.9.0
netaddr==0.10.1
# via -r requirements.in
nodeenv==1.7.0
nodeenv==1.8.0
# via pre-commit
oauthlib==3.2.2
# via requests-oauthlib
packaging==21.3
# via
# kubernetes
# requests-oauthlib
packaging==23.2
# via
# ansible-compat
# ansible-core
# molecule
platformdirs==2.5.2
platformdirs==4.1.0
# via virtualenv
pluggy==1.0.0
pluggy==1.3.0
# via molecule
pre-commit==2.21.0
pre-commit==3.8.0
# via -r requirements.in
pre-commit-hooks==4.5.0
pre-commit-hooks==4.6.0
# via -r requirements.in
pyasn1==0.4.8
pyasn1==0.5.1
# via
# pyasn1-modules
# rsa
pyasn1-modules==0.2.8
pyasn1-modules==0.3.0
# via google-auth
pycparser==2.21
# via cffi
pygments==2.13.0
pygments==2.17.2
# via rich
pyparsing==3.0.9
# via packaging
pyrsistent==0.19.2
# via jsonschema
python-dateutil==2.8.2
# via
# arrow
# kubernetes
python-slugify==6.1.2
# via cookiecutter
# via kubernetes
python-vagrant==1.0.0
# via molecule-vagrant
pyyaml==6.0.1
# via molecule-plugins
pyyaml==6.0.2
# via
# -r requirements.in
# ansible-compat
# ansible-core
# cookiecutter
# kubernetes
# molecule
# molecule-vagrant
# pre-commit
requests==2.28.1
referencing==0.32.1
# via
# jsonschema
# jsonschema-specifications
requests==2.31.0
# via
# cookiecutter
# kubernetes
# requests-oauthlib
requests-oauthlib==1.3.1
# via kubernetes
resolvelib==0.8.1
resolvelib==1.0.1
# via ansible-core
rich==12.6.0
rich==13.7.0
# via
# enrich
# molecule
rpds-py==0.17.1
# via
# jsonschema
# referencing
rsa==4.9
# via google-auth
ruamel-yaml==0.17.21
ruamel-yaml==0.18.5
# via pre-commit-hooks
selinux==0.2.1
# via molecule-vagrant
ruamel-yaml-clib==0.2.8
# via ruamel-yaml
six==1.16.0
# via
# google-auth
# kubernetes
# python-dateutil
subprocess-tee==0.4.1
# via ansible-compat
text-unidecode==1.3
# via python-slugify
urllib3==1.26.12
urllib3==2.1.0
# via
# kubernetes
# requests
virtualenv==20.16.6
virtualenv==20.25.0
# via pre-commit
websocket-client==1.4.2
wcmatch==8.5
# via molecule
websocket-client==1.7.0
# via kubernetes
# The following packages are considered to be unsafe in a requirements file:
+5 -4
View File
@@ -1,23 +1,24 @@
---
- name: Reset k3s cluster
hosts: k3s_cluster
gather_facts: yes
gather_facts: true
roles:
- role: reset
become: true
- role: raspberrypi
become: true
vars: {state: absent}
vars: { state: absent }
post_tasks:
- name: Reboot and wait for node to come back up
become: true
reboot:
ansible.builtin.reboot:
reboot_command: "{{ custom_reboot_command | default(omit) }}"
reboot_timeout: 3600
- name: Revert changes to Proxmox cluster
hosts: proxmox
gather_facts: true
become: yes
become: true
remote_user: "{{ proxmox_lxc_ssh_user }}"
roles:
- role: reset_proxmox_lxc
+8
View File
@@ -0,0 +1,8 @@
---
argument_specs:
main:
short_description: Manage the downloading of K3S binaries
options:
k3s_version:
description: The desired version of K3S
required: true
+20 -10
View File
@@ -1,36 +1,46 @@
---
- name: Download k3s binary x64
get_url:
ansible.builtin.get_url:
url: https://github.com/k3s-io/k3s/releases/download/{{ k3s_version }}/k3s
checksum: sha256:https://github.com/k3s-io/k3s/releases/download/{{ k3s_version }}/sha256sum-amd64.txt
dest: /usr/local/bin/k3s
owner: root
group: root
mode: 0755
mode: "0755"
register: k3s_download_x64
retries: 5
delay: 10
until: k3s_download_x64 is succeeded
when: ansible_facts.architecture == "x86_64"
- name: Download k3s binary arm64
get_url:
ansible.builtin.get_url:
url: https://github.com/k3s-io/k3s/releases/download/{{ k3s_version }}/k3s-arm64
checksum: sha256:https://github.com/k3s-io/k3s/releases/download/{{ k3s_version }}/sha256sum-arm64.txt
dest: /usr/local/bin/k3s
owner: root
group: root
mode: 0755
mode: "0755"
register: k3s_download_arm64
retries: 5
delay: 10
until: k3s_download_arm64 is succeeded
when:
- ( ansible_facts.architecture is search("arm") and
ansible_facts.userspace_bits == "64" ) or
ansible_facts.architecture is search("aarch64")
- ( ansible_facts.architecture is search("arm") and ansible_facts.userspace_bits == "64" )
or ansible_facts.architecture is search("aarch64")
- name: Download k3s binary armhf
get_url:
ansible.builtin.get_url:
url: https://github.com/k3s-io/k3s/releases/download/{{ k3s_version }}/k3s-armhf
checksum: sha256:https://github.com/k3s-io/k3s/releases/download/{{ k3s_version }}/sha256sum-arm.txt
dest: /usr/local/bin/k3s
owner: root
group: root
mode: 0755
mode: "0755"
register: k3s_download_armhf
retries: 5
delay: 10
until: k3s_download_armhf is succeeded
when:
- ansible_facts.architecture is search("arm")
- ansible_facts.userspace_bits == "32"
+4
View File
@@ -0,0 +1,4 @@
---
extra_agent_args: ""
group_name_master: master
systemd_dir: /etc/systemd/system
+39
View File
@@ -0,0 +1,39 @@
---
argument_specs:
main:
short_description: Setup k3s agents
options:
apiserver_endpoint:
description: Virtual ip-address configured on each master
required: true
extra_agent_args:
description: Extra arguments for agents nodes
group_name_master:
description: Name of the master group
default: master
k3s_token:
description: Token used to communicate between masters
proxy_env:
type: dict
description:
- Internet proxy configurations.
- See https://docs.k3s.io/advanced#configuring-an-http-proxy for details
default: ~
options:
HTTP_PROXY:
description: HTTP internet proxy
required: true
HTTPS_PROXY:
description: HTTPS internet proxy
required: true
NO_PROXY:
description: Addresses that will not use the proxies
required: true
systemd_dir:
description: Path to systemd services
default: /etc/systemd/system
+10 -10
View File
@@ -1,18 +1,18 @@
---
- name: Create k3s.service.d directory
file:
path: '{{ systemd_dir }}/k3s.service.d'
- name: Create k3s-node.service.d directory
ansible.builtin.file:
path: "{{ systemd_dir }}/k3s-node.service.d"
state: directory
owner: root
group: root
mode: '0755'
mode: "0755"
when: proxy_env is defined
- name: Copy K3s http_proxy conf file
template:
src: "http_proxy.conf.j2"
dest: "{{ systemd_dir }}/k3s.service.d/http_proxy.conf"
ansible.builtin.template:
src: http_proxy.conf.j2
dest: "{{ systemd_dir }}/k3s-node.service.d/http_proxy.conf"
owner: root
group: root
mode: '0755'
mode: "0755"
when: proxy_env is defined
+25 -9
View File
@@ -1,20 +1,36 @@
---
- name: Check for PXE-booted system
block:
- name: Check if system is PXE-booted
ansible.builtin.command:
cmd: cat /proc/cmdline
register: boot_cmdline
changed_when: false
check_mode: false
- name: Set fact for PXE-booted system
ansible.builtin.set_fact:
is_pxe_booted: "{{ 'root=/dev/nfs' in boot_cmdline.stdout }}"
when: boot_cmdline.stdout is defined
- name: Include http_proxy configuration tasks
ansible.builtin.include_tasks: http_proxy.yml
- name: Deploy K3s http_proxy conf
include_tasks: http_proxy.yml
ansible.builtin.include_tasks: http_proxy.yml
when: proxy_env is defined
- name: Copy K3s service file
template:
src: "k3s.service.j2"
- name: Configure the k3s service
ansible.builtin.template:
src: k3s.service.j2
dest: "{{ systemd_dir }}/k3s-node.service"
owner: root
group: root
mode: 0755
mode: "0755"
- name: Enable and check K3s service
systemd:
- name: Manage k3s service
ansible.builtin.systemd:
name: k3s-node
daemon_reload: yes
daemon_reload: true
state: restarted
enabled: yes
enabled: true
+6 -3
View File
@@ -7,11 +7,14 @@ After=network-online.target
Type=notify
ExecStartPre=-/sbin/modprobe br_netfilter
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/local/bin/k3s agent --server https://{{ apiserver_endpoint | ansible.utils.ipwrap }}:6443 --token {{ hostvars[groups[group_name_master | default('master')][0]]['token'] | default(k3s_token) }} {{ extra_agent_args | default("") }}
# Conditional snapshotter based on PXE boot status
ExecStart=/usr/local/bin/k3s agent \
--server https://{{ apiserver_endpoint | ansible.utils.ipwrap }}:6443 \
{% if is_pxe_booted | default(false) %}--snapshotter native \
{% endif %}--token {{ hostvars[groups[group_name_master | default('master')][0]]['token'] | default(k3s_token) }} \
{{ extra_agent_args }}
KillMode=process
Delegate=yes
# Having non-zero Limit*s causes performance problems due to accounting overhead
# in the kernel. We recommend using cgroups to do container-local accounting.
LimitNOFILE=1048576
LimitNPROC=infinity
LimitCORE=infinity
@@ -1,6 +0,0 @@
---
# Indicates whether custom registries for k3s should be configured
# Possible values:
# - present
# - absent
state: present
+20
View File
@@ -0,0 +1,20 @@
---
argument_specs:
main:
short_description: Configure the use of a custom container registry
options:
custom_registries_yaml:
description:
- YAML block defining custom registries.
- >
The following is an example that pulls all images used in
this playbook through your private registries.
- >
It also allows you to pull your own images from your private
registry, without having to use imagePullSecrets in your
deployments.
- >
If all you need is your own images and you don't care about
caching the docker/quay/ghcr.io images, you can just remove
those from the mirrors: section.
required: true
+5 -6
View File
@@ -1,17 +1,16 @@
---
- name: Create directory /etc/rancher/k3s
file:
path: "/etc/{{ item }}"
ansible.builtin.file:
path: /etc/{{ item }}
state: directory
mode: '0755'
mode: "0755"
loop:
- rancher
- rancher/k3s
- name: Insert registries into /etc/rancher/k3s/registries.yaml
blockinfile:
ansible.builtin.blockinfile:
path: /etc/rancher/k3s/registries.yaml
block: "{{ custom_registries_yaml }}"
mode: '0600'
mode: "0600"
create: true
+26 -6
View File
@@ -1,12 +1,30 @@
---
# If you want to explicitly define an interface that ALL control nodes
# should use to propagate the VIP, define it here. Otherwise, kube-vip
# will determine the right interface automatically at runtime.
kube_vip_iface: null
extra_server_args: ""
k3s_kubectl_binary: k3s kubectl
# Name of the master group
group_name_master: master
kube_vip_arp: true
kube_vip_iface:
kube_vip_cloud_provider_tag_version: v0.0.12
kube_vip_tag_version: v1.2.2
kube_vip_bgp: false
kube_vip_bgp_routerid: 127.0.0.1
kube_vip_bgp_as: "64513"
kube_vip_bgp_peeraddress: 192.168.30.1
kube_vip_bgp_peeras: "64512"
kube_vip_bgp_peers: []
kube_vip_bgp_peers_groups: ['k3s_master']
metal_lb_controller_tag_version: v0.16.0
metal_lb_speaker_tag_version: v0.16.0
metal_lb_type: native
retry_count: 20
# yamllint disable rule:line-length
server_init_args: >-
{% if groups[group_name_master | default('master')] | length > 1 %}
@@ -17,4 +35,6 @@ server_init_args: >-
{% endif %}
--token {{ k3s_token }}
{% endif %}
{{ extra_server_args | default('') }}
{{ extra_server_args }}
systemd_dir: /etc/systemd/system
+135
View File
@@ -0,0 +1,135 @@
---
argument_specs:
main:
short_description: Setup k3s servers
options:
apiserver_endpoint:
description: Virtual ip-address configured on each master
required: true
cilium_bgp:
description:
- Enable cilium BGP control plane for LB services and pod cidrs.
- Disables the use of MetalLB.
type: bool
default: ~
cilium_iface:
description: The network interface used for when Cilium is enabled
default: ~
extra_server_args:
description: Extra arguments for server nodes
default: ""
group_name_master:
description: Name of the master group
default: master
k3s_create_kubectl_symlink:
description: Create the kubectl -> k3s symlink
default: false
type: bool
k3s_create_crictl_symlink:
description: Create the crictl -> k3s symlink
default: false
type: bool
kube_vip_arp:
description: Enables kube-vip ARP broadcasts
default: true
type: bool
kube_vip_bgp:
description: Enables kube-vip BGP peering
default: false
type: bool
kube_vip_bgp_routerid:
description: Defines the router ID for the kube-vip BGP server
default: "127.0.0.1"
kube_vip_bgp_as:
description: Defines the AS for the kube-vip BGP server
default: "64513"
kube_vip_bgp_peeraddress:
description: Defines the address for the kube-vip BGP peer
default: "192.168.30.1"
kube_vip_bgp_peeras:
description: Defines the AS for the kube-vip BGP peer
default: "64512"
kube_vip_bgp_peers:
description: List of BGP peer ASN & address pairs
default: []
kube_vip_bgp_peers_groups:
description: Inventory group in which to search for additional kube_vip_bgp_peers parameters to merge.
default: ['k3s_master']
kube_vip_iface:
description:
- Explicitly define an interface that ALL control nodes
- should use to propagate the VIP, define it here.
- Otherwise, kube-vip will determine the right interface
- automatically at runtime.
default: ~
kube_vip_tag_version:
description: Image tag for kube-vip
default: v1.2.2
kube_vip_cloud_provider_tag_version:
description: Tag for kube-vip-cloud-provider manifest when enabled
default: v0.0.12
kube_vip_lb_ip_range:
description: IP range for kube-vip load balancer
default: ~
metal_lb_controller_tag_version:
description: Image tag for MetalLB
default: v0.16.0
metal_lb_speaker_tag_version:
description: Image tag for MetalLB
default: v0.16.0
metal_lb_type:
choices:
- frr
- native
default: native
description: Use FRR mode or native. Valid values are `frr` and `native`
proxy_env:
type: dict
description:
- Internet proxy configurations.
- See https://docs.k3s.io/advanced#configuring-an-http-proxy for details
default: ~
options:
HTTP_PROXY:
description: HTTP internet proxy
required: true
HTTPS_PROXY:
description: HTTPS internet proxy
required: true
NO_PROXY:
description: Addresses that will not use the proxies
required: true
retry_count:
description: Amount of retries when verifying that nodes joined
type: int
default: 20
server_init_args:
description: Arguments for server nodes
systemd_dir:
description: Path to systemd services
default: /etc/systemd/system
+155 -3
View File
@@ -4,10 +4,110 @@
- name: Fetch k3s-init.service logs
ansible.builtin.command:
cmd: journalctl --all --unit=k3s-init.service
cmd: >-
timeout --signal=TERM --kill-after=5s 30s journalctl --no-pager
--unit=k3s-init.service --since=-30min --lines=5000
changed_when: false
failed_when: false
no_log: true
register: k3s_init_log
- name: Fetch k3s-init.service status
ansible.builtin.command:
cmd: >-
timeout --signal=TERM --kill-after=5s 15s systemctl status
k3s-init.service --no-pager --full
changed_when: false
failed_when: false
no_log: true
register: k3s_init_status
- name: Fetch IP address state
ansible.builtin.command:
cmd: ip -br address
changed_when: false
failed_when: false
no_log: true
register: k3s_ip_address_state
- name: Fetch IP link state
ansible.builtin.command:
cmd: ip -br link
changed_when: false
failed_when: false
no_log: true
register: k3s_ip_link_state
- name: Fetch IP route state
ansible.builtin.command:
cmd: ip route show
changed_when: false
failed_when: false
no_log: true
register: k3s_ip_route_state
- name: Fetch IP neighbor state
ansible.builtin.command:
cmd: ip neigh show
changed_when: false
failed_when: false
no_log: true
register: k3s_ip_neighbor_state
- name: Fetch IP rule state
ansible.builtin.command:
cmd: ip rule show
changed_when: false
failed_when: false
no_log: true
register: k3s_ip_rule_state
- name: Fetch listening TCP sockets
ansible.builtin.command:
cmd: ss -ltn
changed_when: false
failed_when: false
no_log: true
register: k3s_tcp_listener_state
- name: Fetch guest firewall service state
ansible.builtin.service_facts:
no_log: true
register: k3s_service_facts
- name: Fetch input firewall rules
ansible.builtin.command:
cmd: iptables -S INPUT
changed_when: false
failed_when: false
no_log: true
register: k3s_input_firewall_rules
- name: Ping the primary Kubernetes API address from {{ ansible_hostname }}
ansible.builtin.command:
argv:
- ping
- -c
- "1"
- -W
- "1"
- "{{ hostvars[groups[group_name_master | default('master')][0]].k3s_node_ip | split(',') | first }}"
changed_when: false
failed_when: false
no_log: true
register: k3s_primary_api_ping
- name: Probe primary Kubernetes API from {{ ansible_hostname }}
ansible.builtin.wait_for:
host: "{{ hostvars[groups[group_name_master | default('master')][0]].k3s_node_ip | split(',') | first }}"
port: 6443
connect_timeout: 1
timeout: 2
changed_when: false
failed_when: false
no_log: true
register: k3s_primary_api_probe
- name: Create {{ log_destination }}
delegate_to: localhost
run_once: true
@@ -23,6 +123,58 @@
ansible.builtin.template:
src: content.j2
dest: "{{ log_destination }}/k3s-init@{{ ansible_hostname }}.log"
mode: 0644
mode: "0644"
vars:
content: "{{ k3s_init_log.stdout }}"
content: |
=== k3s-init.service status ===
{{ k3s_init_status.stdout | regex_replace('--token(?:=| +)[^ ]+', '--token ***') }}
=== k3s-init.service journal ===
{{ k3s_init_log.stdout | regex_replace('--token(?:=| +)[^ ]+', '--token ***') }}
- name: Store network diagnostics to {{ log_destination }}
delegate_to: localhost
become: false
ansible.builtin.template:
src: content.j2
dest: "{{ log_destination }}/network@{{ ansible_hostname }}.log"
mode: "0644"
vars:
content: |
=== ip -br address ===
{{ k3s_ip_address_state.stdout }}
=== ip -br link ===
{{ k3s_ip_link_state.stdout }}
=== ip route show ===
{{ k3s_ip_route_state.stdout }}
=== ip neigh show ===
{{ k3s_ip_neighbor_state.stdout }}
=== ip rule show ===
{{ k3s_ip_rule_state.stdout }}
=== ss -ltn ===
{{ k3s_tcp_listener_state.stdout }}
=== guest firewall services ===
firewalld={{ k3s_service_facts.ansible_facts.services.get('firewalld.service', {}).get('state', 'not-found') }}
nftables={{ k3s_service_facts.ansible_facts.services.get('nftables.service', {}).get('state', 'not-found') }}
ufw={{ k3s_service_facts.ansible_facts.services.get('ufw.service', {}).get('state', 'not-found') }}
=== iptables -S INPUT ===
rc={{ k3s_input_firewall_rules.rc }}
{{ k3s_input_firewall_rules.stdout }}
{{ k3s_input_firewall_rules.stderr }}
=== primary API ping ===
rc={{ k3s_primary_api_ping.rc }}
{{ k3s_primary_api_ping.stdout }}
{{ k3s_primary_api_ping.stderr }}
=== primary API probe ===
failed={{ k3s_primary_api_probe.failed | default(false) }}
elapsed={{ k3s_primary_api_probe.elapsed | default('unknown') }}
msg={{ k3s_primary_api_probe.msg | default('connected') }}
+6 -8
View File
@@ -1,18 +1,16 @@
---
- name: Create k3s.service.d directory
file:
path: '{{ systemd_dir }}/k3s.service.d'
ansible.builtin.file:
path: "{{ systemd_dir }}/k3s.service.d"
state: directory
owner: root
group: root
mode: '0755'
mode: "0755"
- name: Copy K3s http_proxy conf file
template:
src: "http_proxy.conf.j2"
ansible.builtin.template:
src: http_proxy.conf.j2
dest: "{{ systemd_dir }}/k3s.service.d/http_proxy.conf"
owner: root
group: root
mode: '0755'
mode: "0755"
+34
View File
@@ -0,0 +1,34 @@
---
- name: Verify primary Kubernetes API reachability from {{ joining_master }}
ansible.builtin.wait_for:
host: >-
{{ hostvars[groups[group_name_master | default('master')][0]].k3s_node_ip
| split(',') | first }}
port: 6443
connect_timeout: 2
timeout: 30
delegate_to: "{{ joining_master }}"
- name: Join transient k3s-init service for {{ joining_master }}
ansible.builtin.command:
cmd: >-
systemd-run -p RestartSec=2 -p Restart=on-failure
-p Delegate=yes -p TasksMax=infinity -p KillMode=process
-p LimitNOFILE=1048576 -p LimitNPROC=infinity -p LimitCORE=infinity
--unit=k3s-init
k3s server {{ hostvars[joining_master].k3s_server_init_args }}
creates: "{{ systemd_dir }}/k3s-init.service"
delegate_to: "{{ joining_master }}"
no_log: true
- name: Wait for primary registration of {{ joining_master }}
ansible.builtin.command:
cmd: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }} get node
{{ hostvars[joining_master].ansible_hostname }}
delegate_to: "{{ groups[group_name_master | default('master')][0] }}"
register: joined_master
until: joined_master.rc == 0
retries: "{{ retry_count | default(20) }}"
delay: 2
changed_when: false
+27
View File
@@ -0,0 +1,27 @@
---
- name: Create manifests directory on first master
ansible.builtin.file:
path: /var/lib/rancher/k3s/server/manifests
state: directory
owner: root
group: root
mode: "0644"
when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
- name: Download vip cloud provider manifest to first master
ansible.builtin.get_url:
url: https://raw.githubusercontent.com/kube-vip/kube-vip-cloud-provider/{{ kube_vip_cloud_provider_tag_version | default('v0.0.12') }}/manifest/kube-vip-cloud-controller.yaml # noqa yaml[line-length]
dest: /var/lib/rancher/k3s/server/manifests/kube-vip-cloud-controller.yaml
owner: root
group: root
mode: "0644"
when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
- name: Copy kubevip configMap manifest to first master
ansible.builtin.template:
src: kubevip.yaml.j2
dest: /var/lib/rancher/k3s/server/manifests/kubevip.yaml
owner: root
group: root
mode: "0644"
when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
+130 -55
View File
@@ -1,159 +1,234 @@
---
- name: Stop k3s-init
systemd:
ansible.builtin.systemd:
name: k3s-init
state: stopped
failed_when: false
- name: Clean previous runs of k3s-init # noqa command-instead-of-module
# k3s-init won't work if the port is already in use
- name: Stop k3s
ansible.builtin.systemd:
name: k3s
state: stopped
failed_when: false
- name: Clean previous runs of k3s-init # noqa command-instead-of-module
# The systemd module does not support "reset-failed", so we need to resort to command.
command: systemctl reset-failed k3s-init
ansible.builtin.command: systemctl reset-failed k3s-init
failed_when: false
changed_when: false
- name: Deploy K3s http_proxy conf
include_tasks: http_proxy.yml
ansible.builtin.include_tasks: http_proxy.yml
when: proxy_env is defined
- name: Deploy vip manifest
include_tasks: vip.yml
ansible.builtin.include_tasks: vip.yml
- name: Deploy metallb manifest
include_tasks: metallb.yml
ansible.builtin.include_tasks: metallb.yml
tags: metallb
when: kube_vip_lb_ip_range is not defined and (not cilium_bgp or cilium_iface is not defined)
- name: Init cluster inside the transient k3s-init service
command:
cmd: "systemd-run -p RestartSec=2 \
-p Restart=on-failure \
--unit=k3s-init \
k3s server {{ server_init_args }}"
creates: "{{ systemd_dir }}/k3s.service"
- name: Deploy kube-vip manifest
ansible.builtin.include_tasks: kube-vip.yml
tags: kubevip
when: kube_vip_lb_ip_range is defined
- name: Verification
- name: Initialize and verify the K3s control plane
any_errors_fatal: true
when: not ansible_check_mode
block:
- name: Verify that all nodes actually joined (check k3s-init.service if this fails)
command:
cmd: k3s kubectl get nodes -l "node-role.kubernetes.io/master=true" -o=jsonpath="{.items[*].metadata.name}"
register: nodes
until: nodes.rc == 0 and (nodes.stdout.split() | length) == (groups[group_name_master | default('master')] | length) # yamllint disable-line rule:line-length
retries: "{{ retry_count | default(20) }}"
delay: 10
changed_when: false
- name: Materialize per-host server initialization arguments
ansible.builtin.set_fact:
k3s_server_init_args: "{{ server_init_args }}"
no_log: true
when: groups[group_name_master | default('master')] | length > 1
- name: Orchestrate control-plane initialization from the first master
when: inventory_hostname == groups[group_name_master | default('master')][0]
block:
- name: Init the first master inside the transient k3s-init service
ansible.builtin.command:
cmd: >-
systemd-run -p RestartSec=2 -p Restart=on-failure
-p Delegate=yes -p TasksMax=infinity -p KillMode=process
-p LimitNOFILE=1048576 -p LimitNPROC=infinity -p LimitCORE=infinity
--unit=k3s-init k3s server {{ server_init_args }}
creates: "{{ systemd_dir }}/k3s-init.service"
no_log: true
- name: Wait for the first master Kubernetes API
ansible.builtin.command:
cmd: "{{ k3s_kubectl_binary | default('k3s kubectl') }} get --raw=/readyz"
register: first_master_api
until: first_master_api.rc == 0
retries: "{{ retry_count | default(20) }}"
delay: 2
changed_when: false
- name: Verify the first master API listener on its node address
ansible.builtin.wait_for:
host: >-
{{ hostvars[groups[group_name_master | default('master')][0]].k3s_node_ip
| split(',') | first }}
port: 6443
connect_timeout: 2
timeout: "{{ retry_count | default(20) * 2 }}"
- name: Join additional masters one at a time
ansible.builtin.include_tasks: join_master.yml
loop: "{{ groups[group_name_master | default('master')][1:] }}"
loop_control:
loop_var: joining_master
when: groups[group_name_master | default('master')] | length > 1
- name: Verify that all expected control-plane nodes registered
ansible.builtin.command:
cmd: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }} get nodes
-o=jsonpath='{.items[*].metadata.name}'
register: nodes
until:
- nodes.rc == 0
- expected_control_plane_names | difference(nodes.stdout.split()) | length == 0
retries: "{{ retry_count | default(20) }}"
delay: 2
changed_when: false
vars:
expected_control_plane_names: >-
{{
groups[group_name_master | default('master')]
| map('extract', hostvars, 'ansible_hostname')
| list
}}
- name: Wait for K3s bootstrap CRDs to become established
ansible.builtin.command:
cmd: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }} wait
--for=condition=Established --timeout=5s
crd/addons.k3s.cattle.io
crd/helmcharts.helm.cattle.io
crd/helmchartconfigs.helm.cattle.io
register: bootstrap_crds
until: bootstrap_crds.rc == 0
retries: "{{ retry_count | default(20) }}"
delay: 2
changed_when: false
always:
- name: Save logs of k3s-init.service
include_tasks: fetch_k3s_init_logs.yml
ansible.builtin.include_tasks: fetch_k3s_init_logs.yml
when: log_destination
vars:
log_destination: >-
{{ lookup('ansible.builtin.env', 'ANSIBLE_K3S_LOG_DIR', default=False) }}
- name: Kill the temporary service used for initialization
systemd:
ansible.builtin.systemd:
name: k3s-init
state: stopped
failed_when: false
- name: Copy K3s service file
register: k3s_service
template:
src: "k3s.service.j2"
ansible.builtin.template:
src: k3s.service.j2
dest: "{{ systemd_dir }}/k3s.service"
owner: root
group: root
mode: 0644
mode: "0644"
- name: Enable and check K3s service
systemd:
ansible.builtin.systemd:
name: k3s
daemon_reload: yes
daemon_reload: true
state: restarted
enabled: yes
enabled: true
- name: Wait for node-token
wait_for:
ansible.builtin.wait_for:
path: /var/lib/rancher/k3s/server/node-token
- name: Register node-token file access mode
stat:
ansible.builtin.stat:
path: /var/lib/rancher/k3s/server
register: p
- name: Change file access node-token
file:
ansible.builtin.file:
path: /var/lib/rancher/k3s/server
mode: "g+rx,o+rx"
mode: g+rx,o+rx
- name: Read node-token from master
slurp:
ansible.builtin.slurp:
src: /var/lib/rancher/k3s/server/node-token
register: node_token
- name: Store Master node-token
set_fact:
ansible.builtin.set_fact:
token: "{{ node_token.content | b64decode | regex_replace('\n', '') }}"
- name: Restore node-token file access
file:
ansible.builtin.file:
path: /var/lib/rancher/k3s/server
mode: "{{ p.stat.mode }}"
- name: Create directory .kube
file:
ansible.builtin.file:
path: "{{ ansible_user_dir }}/.kube"
state: directory
owner: "{{ ansible_user_id }}"
mode: "u=rwx,g=rx,o="
mode: u=rwx,g=rx,o=
- name: Copy config file to user home directory
copy:
ansible.builtin.copy:
src: /etc/rancher/k3s/k3s.yaml
dest: "{{ ansible_user_dir }}/.kube/config"
remote_src: yes
remote_src: true
owner: "{{ ansible_user_id }}"
mode: "u=rw,g=,o="
mode: u=rw,g=,o=
- name: Configure kubectl cluster to {{ endpoint_url }}
command: >-
k3s kubectl config set-cluster default
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }} config set-cluster default
--server={{ endpoint_url }}
--kubeconfig {{ ansible_user_dir }}/.kube/config
changed_when: true
vars:
endpoint_url: >-
https://{{ apiserver_endpoint | ansible.utils.ipwrap }}:6443
# Deactivated linter rules:
# - jinja[invalid]: As of version 6.6.0, ansible-lint complains that the input to ipwrap
# would be undefined. This will not be the case during playbook execution.
# noqa jinja[invalid]
# Deactivated linter rules:
# - jinja[invalid]: As of version 6.6.0, ansible-lint complains that the input to ipwrap
# would be undefined. This will not be the case during playbook execution.
# noqa jinja[invalid]
- name: Create kubectl symlink
file:
ansible.builtin.file:
src: /usr/local/bin/k3s
dest: /usr/local/bin/kubectl
state: link
when: k3s_create_kubectl_symlink | default(true) | bool
- name: Create crictl symlink
file:
ansible.builtin.file:
src: /usr/local/bin/k3s
dest: /usr/local/bin/crictl
state: link
when: k3s_create_crictl_symlink | default(true) | bool
- name: Get contents of manifests folder
find:
ansible.builtin.find:
paths: /var/lib/rancher/k3s/server/manifests
file_type: file
register: k3s_server_manifests
- name: Get sub dirs of manifests folder
find:
ansible.builtin.find:
paths: /var/lib/rancher/k3s/server/manifests
file_type: directory
register: k3s_server_manifests_directories
- name: Remove manifests and folders that are only needed for bootstrapping cluster so k3s doesn't auto apply on start
file:
ansible.builtin.file:
path: "{{ item.path }}"
state: absent
with_items:
+34 -8
View File
@@ -1,30 +1,56 @@
---
- name: Create manifests directory on first master
file:
ansible.builtin.file:
path: /var/lib/rancher/k3s/server/manifests
state: directory
owner: root
group: root
mode: 0644
mode: "0644"
when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
- name: "Download to first master: manifest for metallb-{{ metal_lb_type }}"
ansible.builtin.get_url:
url: "https://raw.githubusercontent.com/metallb/metallb/{{ metal_lb_controller_tag_version }}/config/manifests/metallb-{{ metal_lb_type }}.yaml" # noqa yaml[line-length]
dest: "/var/lib/rancher/k3s/server/manifests/metallb-crds.yaml"
url: https://raw.githubusercontent.com/metallb/metallb/{{ metal_lb_controller_tag_version }}/config/manifests/metallb-{{ metal_lb_type }}.yaml # noqa yaml[line-length]
dest: /var/lib/rancher/k3s/server/manifests/metallb-crds.yaml
owner: root
group: root
mode: 0644
mode: "0644"
when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
- name: Set image versions in manifest for metallb-{{ metal_lb_type }}
ansible.builtin.replace:
path: "/var/lib/rancher/k3s/server/manifests/metallb-crds.yaml"
path: /var/lib/rancher/k3s/server/manifests/metallb-crds.yaml
regexp: "{{ item.change | ansible.builtin.regex_escape }}"
replace: "{{ item.to }}"
with_items:
- change: "metallb/speaker:{{ metal_lb_controller_tag_version }}"
to: "metallb/speaker:{{ metal_lb_speaker_tag_version }}"
- change: metallb/speaker:{{ metal_lb_controller_tag_version }}
to: metallb/speaker:{{ metal_lb_speaker_tag_version }}
loop_control:
label: "{{ item.change }} => {{ item.to }}"
when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
- name: Read back MetalLB manifest from first master
ansible.builtin.slurp:
src: /var/lib/rancher/k3s/server/manifests/metallb-crds.yaml
register: metallb_manifest
when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
- name: Check the MetalLB speaker image reference is present
ansible.builtin.set_fact:
metallb_manifest_has_speaker_tag: >-
{{
('quay.io/metallb/speaker:' ~ metal_lb_speaker_tag_version) in
(metallb_manifest.content | default('') | b64decode)
}}
when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
- name: Fail if MetalLB speaker tag was not applied to the manifest
ansible.builtin.fail:
msg: >-
MetalLB speaker image reference
quay.io/metallb/speaker:{{ metal_lb_speaker_tag_version }}
was not found in the downloaded manifest. The upstream image
reference may have changed.
when:
- ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
- not metallb_manifest_has_speaker_tag
+15 -11
View File
@@ -1,27 +1,31 @@
---
- name: Set _kube_vip_bgp_peers fact
ansible.builtin.set_fact:
_kube_vip_bgp_peers: "{{ lookup('community.general.merge_variables', '^kube_vip_bgp_peers__.+$', initial_value=kube_vip_bgp_peers, groups=kube_vip_bgp_peers_groups) }}" # yamllint disable-line rule:line-length
- name: Create manifests directory on first master
file:
ansible.builtin.file:
path: /var/lib/rancher/k3s/server/manifests
state: directory
owner: root
group: root
mode: 0644
mode: "0644"
when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
- name: Download vip rbac manifest to first master
ansible.builtin.get_url:
url: "https://raw.githubusercontent.com/kube-vip/kube-vip/{{ kube_vip_tag_version }}/docs/manifests/rbac.yaml"
dest: "/var/lib/rancher/k3s/server/manifests/vip-rbac.yaml"
- name: Copy kube-vip RBAC manifest to first master
ansible.builtin.template:
src: kube-vip-rbac.yaml.j2
dest: /var/lib/rancher/k3s/server/manifests/vip-rbac.yaml
owner: root
group: root
mode: 0644
mode: "0644"
when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
- name: Copy vip manifest to first master
template:
src: "vip.yaml.j2"
dest: "/var/lib/rancher/k3s/server/manifests/vip.yaml"
ansible.builtin.template:
src: vip.yaml.j2
dest: /var/lib/rancher/k3s/server/manifests/vip.yaml
owner: root
group: root
mode: 0644
mode: "0644"
when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
@@ -0,0 +1,48 @@
# Repository-owned kube-vip RBAC derived from the official v1.2.2
# `kube-vip manifest rbac` output (run with `--inCluster`).
# Source: https://github.com/kube-vip/kube-vip/blob/v1.2.2/pkg/kubevip/config_generator.go
# Permissions are kept exactly as broad as the upstream manifest.
apiVersion: v1
kind: ServiceAccount
metadata:
name: kube-vip
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
name: system:kube-vip-role
rules:
- apiGroups: [""]
resources: ["services/status"]
verbs: ["update"]
- apiGroups: [""]
resources: ["services", "endpoints"]
verbs: ["list", "get", "watch", "update"]
- apiGroups: [""]
resources: ["nodes"]
verbs: ["list", "get", "watch", "update", "patch"]
- apiGroups: ["coordination.k8s.io"]
resources: ["leases"]
verbs: ["list", "get", "watch", "update", "create"]
- apiGroups: ["discovery.k8s.io"]
resources: ["endpointslices"]
verbs: ["list", "get", "watch", "update"]
- apiGroups: [""]
resources: ["pods"]
verbs: ["list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: system:kube-vip-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:kube-vip-role
subjects:
- kind: ServiceAccount
name: kube-vip
namespace: kube-system
@@ -0,0 +1,13 @@
---
apiVersion: v1
kind: ConfigMap
metadata:
name: kubevip
namespace: kube-system
data:
{% if kube_vip_lb_ip_range is string %}
{# kube_vip_lb_ip_range was used in the legacy way: single string instead of a list #}
{# => transform to list with single element #}
{% set kube_vip_lb_ip_range = [kube_vip_lb_ip_range] %}
{% endif %}
range-global: {{ kube_vip_lb_ip_range | join(',') }}
+28 -3
View File
@@ -27,14 +27,16 @@ spec:
- manager
env:
- name: vip_arp
value: "true"
value: "{{ 'true' if kube_vip_arp | default(true) | bool else 'false' }}"
- name: bgp_enable
value: "{{ 'true' if kube_vip_bgp | default(false) | bool else 'false' }}"
- name: port
value: "6443"
{% if kube_vip_iface %}
- name: vip_interface
value: {{ kube_vip_iface }}
{% endif %}
- name: vip_cidr
- name: vip_subnet
value: "{{ apiserver_endpoint | ansible.utils.ipsubnet | ansible.utils.ipaddr('prefix') }}"
- name: cp_enable
value: "true"
@@ -43,7 +45,7 @@ spec:
- name: vip_ddns
value: "false"
- name: svc_enable
value: "false"
value: "{{ 'true' if kube_vip_lb_ip_range is defined else 'false' }}"
- name: vip_leaderelection
value: "true"
- name: vip_leaseduration
@@ -54,6 +56,29 @@ spec:
value: "2"
- name: address
value: {{ apiserver_endpoint }}
{% if kube_vip_bgp | default(false) | bool %}
{% if kube_vip_bgp_routerid is defined %}
- name: bgp_routerid
value: "{{ kube_vip_bgp_routerid }}"
{% endif %}
{% if _kube_vip_bgp_peers | length > 0 %}
- name: bgp_peers
value: "{{ _kube_vip_bgp_peers | map(attribute='peer_address') | zip(_kube_vip_bgp_peers | map(attribute='peer_asn')) | map('join', ':') | join(',') }}" # yamllint disable-line rule:line-length
{% else %}
{% if kube_vip_bgp_as is defined %}
- name: bgp_as
value: "{{ kube_vip_bgp_as }}"
{% endif %}
{% if kube_vip_bgp_peeraddress is defined %}
- name: bgp_peeraddress
value: "{{ kube_vip_bgp_peeraddress }}"
{% endif %}
{% if kube_vip_bgp_peeras is defined %}
- name: bgp_peeras
value: "{{ kube_vip_bgp_peeras }}"
{% endif %}
{% endif %}
{% endif %}
image: ghcr.io/kube-vip/kube-vip:{{ kube_vip_tag_version }}
imagePullPolicy: Always
name: kube-vip
+31 -3
View File
@@ -1,6 +1,34 @@
---
# Timeout to wait for MetalLB services to come up
metal_lb_available_timeout: 120s
k3s_kubectl_binary: k3s kubectl
# Name of the master group
bpf_lb_algorithm: maglev
bpf_lb_mode: hybrid
calico_blockSize: 26 # noqa var-naming
calico_ebpf: false
calico_encapsulation: VXLANCrossSubnet
calico_natOutgoing: Enabled # noqa var-naming
calico_nodeSelector: all() # noqa var-naming
calico_tag: v3.32.1
cilium_bgp: false
cilium_exportPodCIDR: true # noqa var-naming
cilium_bgp_my_asn: 64513
cilium_bgp_peer_asn: 64512
cilium_bgp_neighbors: []
cilium_bgp_neighbors_groups: ['k3s_all']
cilium_bgp_lb_cidr: 192.168.31.0/24
cilium_hubble: true
cilium_mode: native
cilium_tag: v1.20.0
cilium_cli_tag: v0.19.7
cluster_cidr: 10.52.0.0/16
enable_bpf_masquerade: true
kube_proxy_replacement: true
group_name_master: master
metal_lb_mode: layer2
metal_lb_available_timeout: 240s
metal_lb_controller_tag_version: v0.16.0
metal_lb_ip_range: 192.168.30.80-192.168.30.90
+165
View File
@@ -0,0 +1,165 @@
---
argument_specs:
main:
short_description: Configure k3s cluster
options:
apiserver_endpoint:
description: Virtual ip-address configured on each master
required: true
bpf_lb_algorithm:
description: BPF lb algorithm
default: maglev
bpf_lb_mode:
description: BPF lb mode
default: hybrid
calico_blockSize:
description: IP pool block size
type: int
default: 26
calico_ebpf:
description: Use eBPF dataplane instead of iptables
type: bool
default: false
calico_encapsulation:
description: IP pool encapsulation
default: VXLANCrossSubnet
calico_natOutgoing:
description: IP pool NAT outgoing
default: Enabled
calico_nodeSelector:
description: IP pool node selector
default: all()
calico_iface:
description: The network interface used for when Calico is enabled
default: ~
calico_tag:
description: Calico version tag
default: v3.32.1
cilium_bgp:
description:
- Enable cilium BGP control plane for LB services and pod cidrs.
- Disables the use of MetalLB.
type: bool
default: false
cilium_bgp_my_asn:
description: Local ASN for BGP peer
type: int
default: 64513
cilium_bgp_peer_asn:
description: BGP peer ASN
type: int
default: 64512
cilium_bgp_peer_address:
description: BGP peer address
default: ~
cilium_bgp_neighbors:
description: List of BGP peer ASN & address pairs
default: []
cilium_bgp_neighbors_groups:
description: Inventory group in which to search for additional cilium_bgp_neighbors parameters to merge.
default: ['k3s_all']
cilium_bgp_lb_cidr:
description: BGP load balancer IP range
default: 192.168.31.0/24
cilium_exportPodCIDR:
description: Export pod CIDR
type: bool
default: true
cilium_hubble:
description: Enable Cilium Hubble
type: bool
default: true
cilium_iface:
description: The network interface used for when Cilium is enabled
default: ~
cilium_mode:
description:
- Inner-node communication mode.
- Cilium accepts `native` or `tunnel`. `routed` is accepted as a
legacy alias and is mapped to `tunnel` at install time.
default: native
choices:
- native
- tunnel
- routed
cilium_tag:
description: Cilium version tag
default: v1.20.0
cilium_cli_tag:
description: Cilium CLI version tag
default: v0.19.7
cluster_cidr:
description: Inner-cluster IP range
default: 10.52.0.0/16
enable_bpf_masquerade:
description: Use IP masquerading
type: bool
default: true
group_name_master:
description: Name of the master group
default: master
kube_proxy_replacement:
description: Replace the native kube-proxy with Cilium
type: bool
default: true
kube_vip_lb_ip_range:
description: IP range for kube-vip load balancer
default: ~
metal_lb_available_timeout:
description: Wait for MetalLB resources
default: 240s
metal_lb_ip_range:
description: MetalLB ip range for load balancer
default: 192.168.30.80-192.168.30.90
metal_lb_controller_tag_version:
description: Image tag for MetalLB
default: v0.16.0
metal_lb_mode:
description: Metallb mode
default: layer2
choices:
- bgp
- layer2
metal_lb_bgp_my_asn:
description: BGP ASN configurations
default: ~
metal_lb_bgp_peer_asn:
description: BGP peer ASN configurations
default: ~
metal_lb_bgp_peer_address:
description: BGP peer address
default: ~
+163
View File
@@ -0,0 +1,163 @@
---
- name: Deploy Calico to cluster
when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
run_once: true
block:
- name: Create manifests directory on first master
ansible.builtin.file:
path: /tmp/k3s
state: directory
owner: root
group: root
mode: "0755"
- name: "Download to first master: Calico CRD bundle for {{ calico_tag }}"
ansible.builtin.get_url:
url: https://raw.githubusercontent.com/projectcalico/calico/{{ calico_tag }}/manifests/v1_crd_projectcalico_org.yaml # noqa yaml[line-length]
dest: /tmp/k3s/v1_crd_projectcalico_org.yaml
owner: root
group: root
mode: "0755"
- name: "Download to first master: manifest for Tigera Operator and Calico CRDs"
ansible.builtin.get_url:
url: https://raw.githubusercontent.com/projectcalico/calico/{{ calico_tag }}/manifests/tigera-operator.yaml
dest: /tmp/k3s/tigera-operator.yaml
owner: root
group: root
mode: "0755"
- name: Apply Calico CRD bundle with server-side apply
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }}
apply --server-side --force-conflicts
-f /tmp/k3s/v1_crd_projectcalico_org.yaml
register: apply_crds
changed_when: >-
'created' in apply_crds.stdout or
'configured' in apply_crds.stdout or
'applied' in apply_crds.stdout
failed_when: apply_crds.rc != 0
- name: Apply Tigera Operator manifest idempotently with server-side apply
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }}
apply --server-side --force-conflicts
-f /tmp/k3s/tigera-operator.yaml
register: apply_operator
changed_when: >-
'created' in apply_operator.stdout or
'configured' in apply_operator.stdout or
'applied' in apply_operator.stdout
failed_when: apply_operator.rc != 0
- name: Wait for Tigera Operator and managed CRDs to become available
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }} wait {{ item.type }}/{{ item.name }}
{% if item.namespace is defined %}--namespace='{{ item.namespace }}'{% endif %}
--for=condition={{ item.condition }}
--timeout=30s
register: tigera_result
changed_when: false
until: tigera_result is succeeded
retries: 7
delay: 7
with_items:
- { name: tigera-operator, type: deployment, namespace: tigera-operator, condition: Available=True }
- { name: installations.operator.tigera.io, type: crd, condition: Established }
- { name: apiservers.operator.tigera.io, type: crd, condition: Established }
loop_control:
label: "{{ item.type }}/{{ item.name }}"
- name: Copy Calico custom resources manifest to first master
ansible.builtin.template:
src: calico.crs.j2
dest: /tmp/k3s/custom-resources.yaml
owner: root
group: root
mode: "0755"
- name: Apply Calico custom resources
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }}
apply -f /tmp/k3s/custom-resources.yaml
register: apply_cr
changed_when: >-
'configured' in apply_cr.stdout or
'created' in apply_cr.stdout or
'unchanged' in apply_cr.stdout
failed_when: apply_cr.rc != 0
- name: Wait for Calico system resources to be available
ansible.builtin.command: >-
{% if item.type == 'daemonset' %}
{{ k3s_kubectl_binary | default('k3s kubectl') }} wait pods
--namespace='{{ item.namespace }}'
--selector={{ item.selector }}
--for=condition=Ready
{% else %}
{{ k3s_kubectl_binary | default('k3s kubectl') }} wait {{ item.type }}/{{ item.name }}
--namespace='{{ item.namespace }}'
--for=condition=Available
{% endif %}
--timeout=30s
register: cr_result
changed_when: false
until: cr_result is succeeded
retries: 60
delay: 10
with_items:
- { name: calico-typha, type: deployment, namespace: calico-system }
- { name: calico-kube-controllers, type: deployment, namespace: calico-system }
- name: csi-node-driver
type: daemonset
selector: k8s-app=csi-node-driver
namespace: calico-system
- name: calico-node
type: daemonset
selector: k8s-app=calico-node
namespace: calico-system
loop_control:
label: "{{ item.type }}/{{ item.name }}"
# The Calico API server is an optional add-on for managing Calico policy via
# kubectl (the projectcalico.org/v3 Kubernetes API). It is not required for
# Calico CNI data plane operation and does not reconcile on K3s with recent
# Calico releases, so it is treated as best-effort here.
- name: Wait for the optional Calico API server (best effort)
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }} wait deployment/calico-apiserver
--namespace=calico-apiserver --for=condition=Available --timeout=30s
register: cr_apiserver
changed_when: false
until: cr_apiserver is succeeded
retries: 30
delay: 10
ignore_errors: true
- name: Verify Calico TigeraStatus reports Available
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }}
get tigerastatus {{ item }}
-o jsonpath='{.status.conditions[?(@.type=="Available")].status}'
register: tigera_status
changed_when: false
until: tigera_status.stdout | trim == 'True'
retries: 30
delay: 7
failed_when: tigera_status.rc != 0 or (tigera_status.stdout | trim) != 'True'
with_items:
- calico
loop_control:
label: "tigerastatus/{{ item }}"
- name: Patch Felix configuration for eBPF mode
ansible.builtin.command:
cmd: >
{{ k3s_kubectl_binary | default('k3s kubectl') }} patch felixconfiguration default
--type='merge'
--patch='{"spec": {"bpfKubeProxyIptablesCleanupEnabled": false}}'
register: patch_result
changed_when: "'felixconfiguration.projectcalico.org/default patched' in patch_result.stdout"
failed_when: "'Error' in patch_result.stderr"
when: calico_ebpf
+275
View File
@@ -0,0 +1,275 @@
---
- name: Prepare Cilium CLI on first master and deploy CNI
when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
run_once: true
block:
- name: Create tmp directory on first master
ansible.builtin.file:
path: /tmp/k3s
state: directory
owner: root
group: root
mode: "0755"
- name: Check if Cilium CLI is installed
ansible.builtin.command: cilium version
register: cilium_cli_installed
failed_when: false
changed_when: false
ignore_errors: true
- name: Check for Cilium CLI version in command output
ansible.builtin.set_fact:
installed_cli_version: >-
{{
cilium_cli_installed.stdout_lines
| join(' ')
| regex_findall('cilium-cli: (v\d+\.\d+\.\d+)')
| first
| default('unknown')
}}
when: cilium_cli_installed.rc == 0
- name: Log installed Cilium CLI version
ansible.builtin.debug:
msg: "Installed Cilium CLI version: {{ installed_cli_version | default('Not installed') }}"
- name: Log pinned Cilium CLI version
ansible.builtin.debug:
msg: "Pinned Cilium CLI version: {{ cilium_cli_tag }}"
- name: Determine if Cilium CLI needs installation or update
ansible.builtin.set_fact:
cilium_cli_needs_update: >-
{{
cilium_cli_installed.rc != 0 or
(cilium_cli_installed.rc == 0 and
installed_cli_version != cilium_cli_tag)
}}
- name: Install or update Cilium CLI
when: cilium_cli_needs_update
block:
- name: Set architecture variable
ansible.builtin.set_fact:
cli_arch: "{{ 'arm64' if ansible_architecture == 'aarch64' else 'amd64' }}"
- name: Download Cilium CLI and checksum
ansible.builtin.get_url:
url: "{{ cilium_base_url }}/cilium-linux-{{ cli_arch }}{{ item }}"
dest: /tmp/k3s/cilium-linux-{{ cli_arch }}{{ item }}
owner: root
group: root
mode: "0755"
loop:
- .tar.gz
- .tar.gz.sha256sum
vars:
cilium_base_url: https://github.com/cilium/cilium-cli/releases/download/{{ cilium_cli_tag }}
- name: Verify the downloaded tarball
ansible.builtin.shell: |
cd /tmp/k3s && sha256sum --check cilium-linux-{{ cli_arch }}.tar.gz.sha256sum
args:
executable: /bin/bash
changed_when: false
- name: Extract Cilium CLI to /usr/local/bin
ansible.builtin.unarchive:
src: /tmp/k3s/cilium-linux-{{ cli_arch }}.tar.gz
dest: /usr/local/bin
remote_src: true
- name: Remove downloaded tarball and checksum file
ansible.builtin.file:
path: "{{ item }}"
state: absent
loop:
- /tmp/k3s/cilium-linux-{{ cli_arch }}.tar.gz
- /tmp/k3s/cilium-linux-{{ cli_arch }}.tar.gz.sha256sum
- name: Wait for connectivity to kube VIP
ansible.builtin.command: ping -c 1 {{ apiserver_endpoint }}
register: ping_result
until: ping_result.rc == 0
retries: 21
delay: 1
ignore_errors: true
changed_when: false
- name: Fail if kube VIP not reachable
ansible.builtin.fail:
msg: API endpoint {{ apiserver_endpoint }} is not reachable
when: ping_result.rc != 0
- name: Test for existing Cilium install
ansible.builtin.command: |
{{ k3s_kubectl_binary | default('k3s kubectl') }} -n kube-system get daemonsets cilium
register: cilium_installed
failed_when: false
changed_when: false
ignore_errors: true
- name: Check existing Cilium install
when: cilium_installed.rc == 0
block:
- name: Check Cilium version
ansible.builtin.command: cilium version
register: cilium_version
failed_when: false
changed_when: false
ignore_errors: true
- name: Parse installed Cilium version
ansible.builtin.set_fact:
installed_cilium_version: >-
{{
cilium_version.stdout_lines
| join(' ')
| regex_findall('cilium image.+(\d+\.\d+\.\d+)')
| first
| default('unknown')
}}
- name: Determine if Cilium needs update
ansible.builtin.set_fact:
cilium_needs_update: >-
{{ 'v' + installed_cilium_version != cilium_tag }}
- name: Log result
ansible.builtin.debug:
msg: >
Installed Cilium version: {{ installed_cilium_version }},
Target Cilium version: {{ cilium_tag }},
Update needed: {{ cilium_needs_update }}
- name: Map the legacy routed mode to Cilium tunnel mode
ansible.builtin.set_fact:
# Cilium 1.20 accepts `native` or `tunnel`. `routed` was the name used
# by older releases and is kept only as a backward-compatible alias.
cilium_routing_mode: >-
{{ 'tunnel' if cilium_mode == 'routed' else cilium_mode }}
- name: Install Cilium
ansible.builtin.command: >-
{% if cilium_installed.rc != 0 %}
cilium install
{% else %}
cilium upgrade
{% endif %}
--version "{{ cilium_tag }}"
--helm-set operator.replicas="1"
{{ '--helm-set devices=' + cilium_iface if cilium_iface != 'auto' else '' }}
--helm-set ipam.operator.clusterPoolIPv4PodCIDRList={{ cluster_cidr }}
{% if cilium_routing_mode == "native" or (cilium_bgp and cilium_exportPodCIDR != 'false') %}
--helm-set ipv4NativeRoutingCIDR={{ cluster_cidr }}
{% endif %}
--helm-set k8sServiceHost="127.0.0.1"
--helm-set k8sServicePort="6444"
--helm-set routingMode={{ cilium_routing_mode }}
--helm-set autoDirectNodeRoutes={{ "true" if cilium_routing_mode == "native" else "false" }}
--helm-set kubeProxyReplacement={{ kube_proxy_replacement }}
--helm-set bpf.masquerade={{ enable_bpf_masquerade }}
--helm-set bgpControlPlane.enabled={{ cilium_bgp | default("false") }}
--helm-set hubble.enabled={{ "true" if cilium_hubble else "false" }}
--helm-set hubble.relay.enabled={{ "true" if cilium_hubble else "false" }}
--helm-set hubble.ui.enabled={{ "true" if cilium_hubble else "false" }}
{% if kube_proxy_replacement is not false %}
--helm-set loadBalancer.algorithm={{ bpf_lb_algorithm }}
--helm-set loadBalancer.mode={{ bpf_lb_mode }}
{% endif %}
environment:
KUBECONFIG: "{{ ansible_user_dir }}/.kube/config"
register: cilium_install_result
changed_when: cilium_install_result.rc == 0
when: cilium_installed.rc != 0 or cilium_needs_update
- name: Wait for Cilium resources
ansible.builtin.command: >-
{% if item.type == 'daemonset' %}
{{ k3s_kubectl_binary | default('k3s kubectl') }} wait pods
--namespace=kube-system
--selector='k8s-app=cilium'
--for=condition=Ready
{% else %}
{{ k3s_kubectl_binary | default('k3s kubectl') }} wait {{ item.type }}/{{ item.name }}
--namespace=kube-system
--for=condition=Available
{% endif %}
--timeout=30s
register: cr_result
changed_when: false
until: cr_result is succeeded
retries: 30
delay: 7
with_items:
- { name: cilium-operator, type: deployment }
- { name: cilium, type: daemonset, selector: k8s-app=cilium }
- { name: hubble-relay, type: deployment, check_hubble: true }
- { name: hubble-ui, type: deployment, check_hubble: true }
loop_control:
label: "{{ item.type }}/{{ item.name }}"
when: >-
not item.check_hubble | default(false) or (item.check_hubble | default(false) and cilium_hubble)
- name: Wait for Cilium status to be healthy
ansible.builtin.command: cilium status --wait
environment:
KUBECONFIG: "{{ ansible_user_dir }}/.kube/config"
register: cilium_status
changed_when: false
until: cilium_status.rc == 0
retries: 30
delay: 7
- name: Configure Cilium BGP
when: cilium_bgp
block:
- name: Set _cilium_bgp_neighbors fact
ansible.builtin.set_fact:
_cilium_bgp_neighbors: "{{ lookup('community.general.merge_variables', '^cilium_bgp_neighbors__.+$', initial_value=cilium_bgp_neighbors, groups=cilium_bgp_neighbors_groups) }}" # yamllint disable-line rule:line-length
- name: Copy BGP manifests to first master
ansible.builtin.template:
src: cilium.crs.j2
dest: /tmp/k3s/cilium-bgp.yaml
owner: root
group: root
mode: "0755"
- name: Preflight validate rendered BGP manifests against installed CRDs
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }}
apply --dry-run=server -f /tmp/k3s/cilium-bgp.yaml
register: preflight_cr
changed_when: false
failed_when: preflight_cr.rc != 0
- name: Apply BGP manifests
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }}
apply -f /tmp/k3s/cilium-bgp.yaml
register: apply_cr
changed_when: "'configured' in apply_cr.stdout or 'created' in apply_cr.stdout"
failed_when: apply_cr.rc != 0
- name: Remove deprecated CiliumBGPPeeringPolicy after v2 resources are accepted
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }}
delete CiliumBGPPeeringPolicy.cilium.io 01-bgp-peering-policy
register: delete_old_policy
changed_when: "'deleted' in delete_old_policy.stdout"
# The policy (and possibly its CRD) may already be absent; this is
# intentionally tolerated, not a command whose failure must be hidden.
failed_when: false
- name: Test for BGP config resources
ansible.builtin.command: "{{ item }}"
loop:
- "{{ k3s_kubectl_binary | default('k3s kubectl') }} get CiliumBGPClusterConfig.cilium.io"
- "{{ k3s_kubectl_binary | default('k3s kubectl') }} get CiliumBGPPeerConfig.cilium.io"
- "{{ k3s_kubectl_binary | default('k3s kubectl') }} get CiliumBGPAdvertisement.cilium.io"
- "{{ k3s_kubectl_binary | default('k3s kubectl') }} get CiliumLoadBalancerIPPool.cilium.io"
changed_when: false
loop_control:
label: "{{ item }}"
+13 -2
View File
@@ -1,9 +1,20 @@
---
- name: Deploy calico
ansible.builtin.include_tasks: calico.yml
tags: calico
when: calico_iface is defined and cilium_iface is not defined
- name: Deploy cilium
ansible.builtin.include_tasks: cilium.yml
tags: cilium
when: cilium_iface is defined
- name: Deploy metallb pool
include_tasks: metallb.yml
ansible.builtin.include_tasks: metallb.yml
tags: metallb
when: kube_vip_lb_ip_range is not defined and (not cilium_bgp or cilium_iface is not defined)
- name: Remove tmp directory used for manifests
file:
ansible.builtin.file:
path: /tmp/k3s
state: absent
+45 -18
View File
@@ -1,32 +1,53 @@
---
- name: Create manifests directory for temp configuration
file:
ansible.builtin.file:
path: /tmp/k3s
state: directory
owner: "{{ ansible_user_id }}"
mode: 0755
mode: "0755"
with_items: "{{ groups[group_name_master | default('master')] }}"
run_once: true
- name: Delete outdated metallb replicas
ansible.builtin.shell: |-
set -o pipefail
REPLICAS=$({{ k3s_kubectl_binary | default('k3s kubectl') }} --namespace='metallb-system' get replicasets \
-l 'component=controller,app=metallb' \
-o jsonpath='{.items[0].spec.template.spec.containers[0].image}, {.items[0].metadata.name}' 2>/dev/null || true)
REPLICAS_SETS=$(echo ${REPLICAS} | grep -v '{{ metal_lb_controller_tag_version }}' | sed -e "s/^.*\s//g")
if [ -n "${REPLICAS_SETS}" ] ; then
for REPLICAS in "${REPLICAS_SETS}"
do
{{ k3s_kubectl_binary | default('k3s kubectl') }} --namespace='metallb-system' \
delete rs "${REPLICAS}"
done
fi
args:
executable: /bin/bash
changed_when: false
run_once: true
with_items: "{{ groups[group_name_master | default('master')] }}"
- name: Copy metallb CRs manifest to first master
template:
src: "metallb.crs.j2"
dest: "/tmp/k3s/metallb-crs.yaml"
ansible.builtin.template:
src: metallb.crs.j2
dest: /tmp/k3s/metallb-crs.yaml
owner: "{{ ansible_user_id }}"
mode: 0755
mode: "0755"
with_items: "{{ groups[group_name_master | default('master')] }}"
run_once: true
- name: Test metallb-system namespace
command: >-
k3s kubectl -n metallb-system
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }} -n metallb-system
changed_when: false
with_items: "{{ groups[group_name_master | default('master')] }}"
run_once: true
- name: Wait for MetalLB resources
command: >-
k3s kubectl wait {{ item.resource }}
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }} wait {{ item.resource }}
--namespace='metallb-system'
{% if item.name | default(False) -%}{{ item.name }}{%- endif %}
{% if item.selector | default(False) -%}--selector='{{ item.selector }}'{%- endif %}
@@ -62,16 +83,22 @@
loop_control:
label: "{{ item.description }}"
- name: Set metallb webhook service name
ansible.builtin.set_fact:
# Inspected the v0.16.0 manifest: the newer webhook Service name is used
# by every supported MetalLB release, so the old pre-0.14.4 branch is gone.
metallb_webhook_service_name: metallb-webhook-service
- name: Test metallb-system webhook-service endpoint
command: >-
k3s kubectl -n metallb-system get endpoints webhook-service
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }} -n metallb-system get endpoints {{ metallb_webhook_service_name }}
changed_when: false
with_items: "{{ groups[group_name_master | default('master')] }}"
run_once: true
- name: Apply metallb CRs
command: >-
k3s kubectl apply -f /tmp/k3s/metallb-crs.yaml
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }} apply -f /tmp/k3s/metallb-crs.yaml
--timeout='{{ metal_lb_available_timeout }}'
register: this
changed_when: false
@@ -80,8 +107,8 @@
retries: 5
- name: Test metallb-system resources for Layer 2 configuration
command: >-
k3s kubectl -n metallb-system get {{ item }}
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }} -n metallb-system get {{ item }}
changed_when: false
run_once: true
when: metal_lb_mode == "layer2"
@@ -90,8 +117,8 @@
- L2Advertisement
- name: Test metallb-system resources for BGP configuration
command: >-
k3s kubectl -n metallb-system get {{ item }}
ansible.builtin.command: >-
{{ k3s_kubectl_binary | default('k3s kubectl') }} -n metallb-system get {{ item }}
changed_when: false
run_once: true
when: metal_lb_mode == "bgp"
@@ -0,0 +1,41 @@
# This section includes base Calico installation configuration.
# For more information, see: https://docs.tigera.io/calico/latest/reference/installation/api#operator.tigera.io/v1.Installation
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
name: default
spec:
# Configures Calico networking.
calicoNetwork:
# Note: The ipPools section cannot be modified post-install.
ipPools:
- blockSize: {{ calico_blockSize }}
cidr: {{ cluster_cidr }}
encapsulation: {{ calico_encapsulation }}
natOutgoing: {{ calico_natOutgoing }}
nodeSelector: {{ calico_nodeSelector }}
nodeAddressAutodetectionV4:
interface: {{ calico_iface }}
linuxDataplane: {{ 'BPF' if calico_ebpf else 'Iptables' }}
---
# This section configures the Calico API server.
# For more information, see: https://docs.tigera.io/calico/latest/reference/installation/api#operator.tigera.io/v1.APIServer
apiVersion: operator.tigera.io/v1
kind: APIServer
metadata:
name: default
spec: {}
{% if calico_ebpf %}
---
kind: ConfigMap
apiVersion: v1
metadata:
name: kubernetes-services-endpoint
namespace: tigera-operator
data:
KUBERNETES_SERVICE_HOST: '{{ apiserver_endpoint }}'
KUBERNETES_SERVICE_PORT: '6443'
{% endif %}
@@ -0,0 +1,92 @@
# Cilium BGP Control Plane v2 resources.
# Replace the deprecated v2alpha1 CiliumBGPPeeringPolicy that was removed
# in Cilium 1.19.
{% set _cilium_default_peer = {'peer_address': cilium_bgp_peer_address, 'peer_asn': cilium_bgp_peer_asn} %}
{% set _cilium_peers = _cilium_bgp_neighbors if _cilium_bgp_neighbors | length > 0 else [_cilium_default_peer] %}
apiVersion: cilium.io/v2
kind: CiliumBGPPeerConfig
metadata:
name: cilium-peer
spec:
# Matches the timers and multihop used by the previous v2alpha1 policy.
ebgpMultihop: 10
timers:
connectRetryTimeSeconds: 120
holdTimeSeconds: 90
keepAliveTimeSeconds: 30
gracefulRestart:
enabled: true
restartTimeSeconds: 120
families:
- afi: ipv4
safi: unicast
advertisements:
matchLabels:
advertise: "bgp"
---
apiVersion: cilium.io/v2
kind: CiliumBGPClusterConfig
metadata:
name: cilium-bgp
spec:
# Explicitly select every node so the BGP instance runs across the cluster.
nodeSelector:
matchExpressions:
- key: somekey
operator: NotIn
values: ['never-used-value']
bgpInstances:
- name: "instance-{{ cilium_bgp_my_asn }}"
localASN: {{ cilium_bgp_my_asn }}
peers:
{% for peer in _cilium_peers %}
- name: "peer-{{ peer.peer_asn }}-{{ loop.index }}"
peerASN: {{ peer.peer_asn }}
peerAddress: {{ peer.peer_address }}
peerConfigRef:
name: cilium-peer
{% endfor %}
{% if cilium_exportPodCIDR %}
---
apiVersion: cilium.io/v2
kind: CiliumBGPAdvertisement
metadata:
name: cilium-pod-cidrs
labels:
advertise: "bgp"
spec:
advertisements:
- advertisementType: "PodCIDR"
{% endif %}
---
apiVersion: cilium.io/v2
kind: CiliumBGPAdvertisement
metadata:
name: cilium-lb-services
labels:
advertise: "bgp"
spec:
advertisements:
- advertisementType: "Service"
service:
addresses:
- LoadBalancerIP
# Advertise all Services carrying an ingress address from the pool.
selector:
matchExpressions:
- key: somekey
operator: NotIn
values: ['never-used-value']
---
apiVersion: cilium.io/v2
kind: CiliumLoadBalancerIPPool
metadata:
name: "01-lb-pool"
spec:
blocks:
{% if "/" in cilium_bgp_lb_cidr %}
- cidr: {{ cilium_bgp_lb_cidr }}
{% else %}
- start: {{ cilium_bgp_lb_cidr.split('-')[0] }}
stop: {{ cilium_bgp_lb_cidr.split('-')[1] }}
{% endif %}
+2 -1
View File
@@ -1,5 +1,6 @@
---
- name: Reboot server
become: true
reboot:
ansible.builtin.reboot:
reboot_command: "{{ custom_reboot_command | default(omit) }}"
listen: reboot server
+8
View File
@@ -0,0 +1,8 @@
---
argument_specs:
main:
short_description: Configure LXC
options:
custom_reboot_command:
default: ~
description: Command to run on reboot
+4 -4
View File
@@ -1,20 +1,20 @@
---
- name: Check for rc.local file
stat:
ansible.builtin.stat:
path: /etc/rc.local
register: rcfile
- name: Create rc.local if needed
lineinfile:
ansible.builtin.lineinfile:
path: /etc/rc.local
line: "#!/bin/sh -e"
create: true
insertbefore: BOF
mode: "u=rwx,g=rx,o=rx"
mode: u=rwx,g=rx,o=rx
when: not rcfile.stat.exists
- name: Write rc.local file
blockinfile:
ansible.builtin.blockinfile:
path: /etc/rc.local
content: "{{ lookup('template', 'templates/rc.local.j2') }}"
state: present
+2 -2
View File
@@ -1,4 +1,4 @@
---
secure_path:
RedHat: '/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin'
Suse: '/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin'
RedHat: /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
Suse: /usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin
+7
View File
@@ -0,0 +1,7 @@
---
argument_specs:
main:
short_description: Prerequisites
options:
system_timezone:
description: Timezone to be set on all nodes
+11 -11
View File
@@ -14,7 +14,7 @@
name: net.ipv4.ip_forward
value: "1"
state: present
reload: yes
reload: true
tags: sysctl
- name: Enable IPv6 forwarding
@@ -22,7 +22,7 @@
name: net.ipv6.conf.all.forwarding
value: "1"
state: present
reload: yes
reload: true
tags: sysctl
- name: Enable IPv6 router advertisements
@@ -30,14 +30,14 @@
name: net.ipv6.conf.all.accept_ra
value: "2"
state: present
reload: yes
reload: true
tags: sysctl
- name: Add br_netfilter to /etc/modules-load.d/
copy:
content: "br_netfilter"
ansible.builtin.copy:
content: br_netfilter
dest: /etc/modules-load.d/br_netfilter.conf
mode: "u=rw,g=,o="
mode: u=rw,g=,o=
when: ansible_os_family == "RedHat"
- name: Load br_netfilter
@@ -51,7 +51,7 @@
name: "{{ item }}"
value: "1"
state: present
reload: yes
reload: true
when: ansible_os_family == "RedHat"
loop:
- net.bridge.bridge-nf-call-iptables
@@ -59,11 +59,11 @@
tags: sysctl
- name: Add /usr/local/bin to sudo secure_path
lineinfile:
line: 'Defaults secure_path = {{ secure_path[ansible_os_family] }}'
regexp: "Defaults(\\s)*secure_path(\\s)*="
ansible.builtin.lineinfile:
line: Defaults secure_path = {{ secure_path[ansible_os_family] }}
regexp: Defaults(\s)*secure_path(\s)*=
state: present
insertafter: EOF
path: /etc/sudoers
validate: 'visudo -cf %s'
validate: visudo -cf %s
when: ansible_os_family in [ "RedHat", "Suse" ]
+2 -2
View File
@@ -2,12 +2,12 @@
- name: Reboot containers
block:
- name: Get container ids from filtered files
set_fact:
ansible.builtin.set_fact:
proxmox_lxc_filtered_ids: >-
{{ proxmox_lxc_filtered_files | map("split", "/") | map("last") | map("split", ".") | map("first") }}
listen: reboot containers
- name: Reboot container
command: "pct reboot {{ item }}"
ansible.builtin.command: pct reboot {{ item }}
loop: "{{ proxmox_lxc_filtered_ids }}"
changed_when: true
listen: reboot containers

Some files were not shown because too many files have changed in this diff Show More