- systemd treats % as a specifier in ExecStart, so token values containing %
fail with 'failed to resolve unit specifiers invalid slot'
- escape % to %% in the agent unit file
- escape % to %% in server_init_args so the multi-master bootstrap path
(systemd-run k3s-init) handles % tokens too
Co-authored-by: finaldoom <677609+FinalDoom@users.noreply.github.com>
* fix(calico): support split CRDs for current releases
- Download the v1_crd_projectcalico_org.yaml bundle before the operator
- Apply both files with server-side apply and force-conflicts per the
upstream upgrade procedure
- Wait for the operator Deployment and for the managed CRDs to be
Established after the operator starts
- Replace the create/rescue/replace flow with an idempotent apply that
no longer conceals partial failures
- Verify TigeraStatus for calico and apiserver is Available, not just
that Pods exist
* feat(dependencies): upgrade supported cluster components
- Bump K3s to v1.36.2+k3s1, Calico to v3.32.1, Cilium to v1.20.0,
kube-vip to v1.2.2, kube-vip cloud provider to v0.0.12, and MetalLB to
v0.16.0 across sample inventory, role defaults, and argument specs
- Pin the Cilium CLI with a new cilium_cli_tag (v0.19.7) instead of the
floating stable.txt lookup
- Replace the CiliumBGPPeeringPolicy v2alpha1 BGP template with the
v2 CiliumBGPClusterConfig, CiliumBGPPeerConfig, CiliumBGPAdvertisement,
and CiliumLoadBalancerIPPool resource set
- Move Cilium load balancer Helm keys from bpf.loadBalancer to the valid
top-level loadBalancer path
- Add preflight schema validation and remove the deprecated policy after
the v2 objects are accepted
- Wait for cilium status after installation
- Pin kube-vip RBAC in a repository template instead of fetching a
mutable URL, and include EndpointSlice permissions
- Fix the kube-vip bgppeers format to address:ASN comma-separated peers
- Fail clearly when the MetalLB speaker tag replacement does not apply
- Drop the obsolete MetalLB webhook service name version branch
* test(molecule): verify upgraded cluster components
- Assert every node reports the expected K3s kubelet version
- Verify the active CNI (Flannel / Calico / Cilium) is Ready and runs
the expected image tag, including Calico TigeraStatus Available
- Verify the active load balancer (MetalLB / kube-vip) runs the expected
image tags and that MetalLB is absent when kube-vip is active
- Assert no Flannel DaemonSet remains when Calico or Cilium is enabled
- Assert the example LoadBalancer address falls inside the configured
pool range
- Add a manifest-only Cilium BGP regression test that renders the v2
template with zero, one, and multiple neighbors and rejects any v2alpha1
or CiliumBGPPeeringPolicy output
* fix(dependencies): correct dependency version pins
- Set the sample kube-vip image to v1.2.2 and repair the damaged comment
- Pin the kube-vip cloud provider default to v0.0.12 in the task URL
- Set the MetalLB controller argument-spec default to v0.16.0
- Restore the MetalLB available timeout default to 240s
* docs(dependencies): document current cluster versions
- Update kube-vip, kube-vip cloud provider, and MetalLB defaults
- Add cilium_tag and cilium_cli_tag rows
- Explain that MetalLB v0.16.0 is the application image target even though
a newer chart-only tag (metallb-chart-0.16.1) exists
- Add an existing-cluster upgrade warning covering the K3s etcd 3.5.26
bridge and one-minor-at-a-time rule, consecutive Cilium minor upgrades,
Calico v3 resource UID handling, and MetalLB app vs chart tags
* fix(dependencies): address PR review findings
- Read the MetalLB speaker tag check from the managed host with slurp
instead of a controller-side file lookup, and match the full image
reference
- Restore the tigera-operator namespace on the Calico operator Deployment
wait while keeping the managed CRD waits cluster-scoped
- Make Molecule verify inputs durable and scenario-specific via a
per-scenario verify-vars.yml, driven by explicit verify_cni/verify_lb
values instead of non-persisted converge facts
- Rename the kube-vip multi-peer BGP env var from bgppeers to bgp_peers
and vip_cidr to vip_subnet so v1.2.2 actually reads them
- Map the legacy Cilium routed mode to tunnel and stop passing the alias
directly to the chart
- Use return-code based failed_when on apply and preflight commands so
non-error failures are no longer treated as success
- Clarify the sequential K3s upgrade path and backups in the README
- Add kube-vip and MetalLB regression tests and a Cilium mode mapping unit
* fix(dependencies): resolve re-review findings
- correct the Calico TigeraStatus resource kind\n- document tunnel as the supported Cilium routing mode\n- validate load balancer addresses across range and CIDR pools
* fix(molecule): verify embedded flannel instead of a flannel DaemonSet
- K3s 1.36 runs flannel embedded in the k3s agent rather than as a
kube-flannel-ds DaemonSet, so the flannel verifier queried a workload
that no longer exists and failed the verify step
- For the flannel scenarios, assert every node is Ready and that neither
the Calico nor the Cilium namespace exists
- Drop the now-invalid kube-flannel-ds DaemonSet assertion
* fix(molecule): wait for the LoadBalancer address before asserting reachability
- The nginx LoadBalancer service had no ingress address when the
reachability assertion ran, so status.loadBalancer.ingress[0].ip was
undefined and the ipwrap filter failed during verify
- Poll the service until MetalLB or kube-vip assigns an external IP
- Record the assigned address once and reuse it for the reachability probe
and the pool membership checks
* fix(ci): harden calico apiserver wait and extend molecule job timeout
- Bump calico system resources wait retries 30->60 and delay 7->10 so the
slow-to-reconcile calico-apiserver deployment has enough time under nested-virt
- Raise the molecule step timeout-minutes from 90 to 150 to accommodate
contended 5-node scenarios (cilium, kube-vip) that were hitting the 90-min cap
* fix(calico): treat optional API server as best-effort on converge
- The Calico API server (calico-apiserver) is an optional add-on for managing
Calico policy through the projectcalico.org/v3 Kubernetes API; it is not
required for Calico CNI data plane operation
- With Calico v3.32.1 on K3s 1.36 the tigera-operator never provisions the
calico-apiserver namespace, causing the converge wait to fail deterministically
- Keep the strict wait for core Calico components (typha, kube-controllers,
calico-node, csi-node-driver) and make the API server wait tolerate failure
- Restrict the TigeraStatus Available check to the calico status, matching the
upstream v3.32.1 K3s quickstart which validates without the API server
- delegate cgroups for transient K3s server units\n- verify inventory node registration without legacy role labels\n- wait for bootstrap CRDs before replacing the transient service
- serialize five-node Vagrant creation to avoid VirtualBox host-only races
- refresh and verify the primary neighbor mapping before convergence
- include interface MAC details in failure diagnostics
- pin kube-vip and cluster traffic to the private guest interface\n- disable disposable guest firewalls and verify API reachability before joins\n- keep control-plane orchestration on the primary and preserve failure diagnostics
- Preserve explicit per-host server initialization overrides\n- Build default join arguments from the delegated host variables\n- Keep initialization commands out of normal task output
- Retry transient k3s release downloads with bounded backoff.\n- Bound failure diagnostics and validate guest release connectivity.\n- Discover repository-owned Molecule state under the actual project root.
* docs: first modules' variable docs table
* docs: variables for k3s_server_post
* docs: lxc and prereq vars in README
* style: lint errors
* docs: argument_specs for proxmox_lxc
* docs: last variables found added to the README
With the kube_vip_bgp_peers it is possible to define
multiple BGP peer ASN & address pairs for kube-vip.
Sample:
```
kube_vip_bgp_peers:
- peer_address: 192.168.128.10
peer_asn: 64512
- peer_address: 192.168.128.11
peer_asn: 64512
- peer_address: 192.168.128.12
peer_asn: 64512
```
It is possible to merge further lists with kube_vip_bgp_peers__*
parameters.
Sample:
```
kube_vip_bgp_peers__extra:
- peer_address: 192.168.128.10
peer_asn: 64512
kube_vip_bgp_peers:
- peer_address: 192.168.128.11
peer_asn: 64512
- peer_address: 192.168.128.12
peer_asn: 64512
```
This will result in the following list of BGP peer ASN & address pairs:
```
- peer_address: 192.168.128.10
peer_asn: 64512
- peer_address: 192.168.128.11
peer_asn: 64512
- peer_address: 192.168.128.12
peer_asn: 64512
```
Signed-off-by: Christian Berendt <berendt@osism.tech>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
With the cilium_bgp_neighbors parameter it is possible to define
multiple BGP peer ASN & address pairs for Cilium.
Sample:
```
cilium_bgp_neighbors:
- peer_address: 192.168.128.10
peer_asn: 64512
- peer_address: 192.168.128.11
peer_asn: 64512
- peer_address: 192.168.128.12
peer_asn: 64512
```
It is possible to merge further lists with cilium_bgp_neighbors__*
parameters.
Sample:
```
cilium_bgp_neighbors__extra:
- peer_address: 192.168.128.10
peer_asn: 64512
cilium_bgp_neighbors:
- peer_address: 192.168.128.11
peer_asn: 64512
- peer_address: 192.168.128.12
peer_asn: 64512
```
This will result in the following list of BGP peer ASN & address pairs:
```
- peer_address: 192.168.128.10
peer_asn: 64512
- peer_address: 192.168.128.11
peer_asn: 64512
- peer_address: 192.168.128.12
peer_asn: 64512
```
Signed-off-by: Christian Berendt <berendt@osism.tech>
* Change to FQCN with ansible-lint fixer
Since ansible-base 2.10 (later ansible-core), FQCN is the new way to go.
Updated .ansible-lint with a production profile and removed fqcn in skip_list.
Updated .yamllint with rules needed.
Ran ansible-lint --fix=all, then manually applied some minor changes.
* Changed octal value in molecule/ipv6/prepare.yml
The commit 3a20500f9c has introduced
argument specs in the role meta information. These two parameters
were still missing there.
Realted to 2d0596209e
Signed-off-by: Christian Berendt <berendt@osism.tech>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
With the kube_vip_bgp parameter it is possible to enable the kube-vip
BGP support (https://kube-vip.io/docs/modes/bgp/).
The configuration is possible with the following new parameters:
* kube_vip_bgp_routerid
* kube_vip_bgp_as
* kube_vip_bgp_peeraddress
* kube_vip_bgp_peeras
Signed-off-by: Christian Berendt <berendt@osism.tech>
If k3s_create_kubectl_symlink is set to false the kubectl symlink will
not be created.
If k3s_create_crictl_symlink is set to false the crictl symlink will not
be created.
By default the symlinks will be created. The default behavior is not
changed.
Signed-off-by: Christian Berendt <berendt@osism.tech>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
* ➕ Add default values to roles
* 🚚 Move to use meta files for roles
* 🛠 Fix descriptions
* ➕ Add meta for server
* 🚧 WIP
* 🌟 Complete
* 🧹 Ran and fix lint errors
* 🔨 Fix required and default conflict
---------
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
With the kube_vip_arp parameter it is possible to set or unset the
vip_arp environment variable of the kube-vip-ds daemonset. The value of
the kube_vip_arp is true by default to not change the existing default.
Signed-off-by: Christian Berendt <berendt@osism.tech>
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
* Fix cgroups cmdline path
* Add check and variable so that we write to the correct file
* Add LSB release checks. This is untested atm.
* Break test into multiple lines so that we can pass lint checks
* Flip logic on its head and check for existence only rather than content per contributor suggestion
---------
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
* Add conditional snapshotter for PXE-booted systems
**Added:**
- PXE Boot Check - Introduced tasks to check if the system is PXE-booted by
analyzing `/proc/cmdline` in `roles/k3s_agent/tasks/main.yml`.
- Conditional Snapshotter in Template - Added logic in `k3s.service.j2` template
to conditionally set `--snapshotter native` for PXE-booted systems.
**Changed:**
- `k3s.service.j2` Template Update - Modified the `ExecStart` line to include a
conditional check for `is_pxe_booted` fact, dynamically setting the
`--snapshotter` option for NFS-mounted systems.
- `main.yml` Task Modification - Added tasks to set `is_pxe_booted` fact based
on the presence of `root=/dev/nfs` in the system's boot command line.
This update allows k3s agents on PXE-booted systems to use the native snapshotter
when running on NFS, addressing compatibility issues with OverlayFS.
* Introduce idiomatic practices for affected areas from previous commits
**Added:**
- Structured HTTP Proxy Configuration Block - Added a structured block in
`http_proxy.yml` for managing HTTP proxy settings, aligning with Ansible's
recommended practices. This includes creating directories and deploying
configuration files in a clear, modular fashion.
- Conditional Execution for Proxy Setup - Implemented conditional execution
for the proxy setup in `http_proxy.yml`, utilizing `proxy_env` to adhere
to Ansible's best practices for conditional tasks.
- Improved PXE-Boot System Check Block - Introduced a more structured approach
in `main.yml` for checking PXE-booted systems, enhancing readability and
maintainability.
**Changed:**
- Adopted Ansible Builtin Modules - Transitioned existing tasks to use
`ansible.builtin` modules, ensuring compatibility and future-proofing the
role.
- Refined Task Grouping - Reorganized tasks into logical blocks, improving
the overall structure and readability, and showcasing Ansible's capabilities
for efficient task management.
- Updated K3s Service Configuration - Modified the K3s service configuration
task in `main.yml` for a more streamlined approach using Ansible's template
module, reflecting community-driven best practices.
**Removed:**
- Streamlined Task Definitions - Optimized task definitions to reduce
redundancy, focusing on clarity and adherence to the evolving Ansible
community standards.
* Added missing checks causing failures for agents
* Add Tigera Operator/Calico CNI option
Small tweak to reduce delta from head
Set calico option to be disabled by default
Add rescue blocks in case updating existing
Refactor items and update comments
Refactor and consolidate calico.yml into block
Refactor to use template for Calico CRs
Revert use_calico to false
Template blockSize
Align default cidr in template with all.yml sample
Apply upstream version tags
Revert to current ver tags. Upstream's don't work.
Update template address detection
Add Tigera Operator/Calico CNI option
* Add calico-apiserver check
* Add eBPF dataplane option
* Add kube svc endpoint configmap when ebpf enabled
* Add /etc/cni/net.d to reset task
* Refactor based on comments
* Add molecule scenario
* Fix lint
---------
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
* feat(k3s): added support for latest raspberrypi os (debian 12 bookworm)
* Update test.yml
* Revert test workflow
---------
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
* Fix http_proxy service dir in k3s_agent role
* Fix http_proxy reset: rm conf files before dirs
* Fix http_proxy reset rm order
---------
Co-authored-by: Techno Tim <timothystewart6@gmail.com>
* Add option to disable MetalLB, for use w/ ext LBs
* Add option to disable MetalLB, for use w/ ext LBs - add defaults
* Skip MetalLB with tags instead of flag
* fixes: ERROR! The requested handler <'Reboot containers' / 'Reboot server' / 'Reboot>' was not found in either the main handlers list nor in the listening handlers list
* Update main.yml
For improved flexibility and maintainability.
* Update tasks in node role to use 'group_name_master' variable instead
of hardcoded 'master' group name
* Update tasks in master role to use 'group_name_master' variable instead
of hardcoded 'master' group name
* Update tasks in post role to use 'group_name_master' variable instead of
hardcoded 'master' group name
Signed-off-by: Christian Berendt <berendt@23technologies.cloud>
* Update pre-commit actions
This was done by running "pre-commit autoupdate --freeze".
* Remove pre-commit only dependencies from requirements.in
Including them in the file would create the illusion that those were the
versions actually used in CI, but they are not. The exact versions are
determined by the pre-commit hooks which are pinned in
.pre-commit-config.yaml.
* Ansible Lint: Fix role-name[path]
* Ansible Lint: Fix name[play]
* Ansible Lint: Fix key-order[task]
* Ansible Lint: Fix jinja[spacing]
* Ansible Lint: Fix no-free-form
* Ansible Lint: Fix var-naming[no-reserved]
* Ansible Lint: Fix yaml[comments]
* Ansible Lint: Fix yaml[line-length]
* Ansible Lint: Fix name[casing]
* Ansible Lint: Fix no-changed-when
* Ansible Lint: Fix fqcn[action]
* Ansible Lint: Fix args[module]
* Improve task naming
Techno Tim
4t1l4
Christian Berendt
Vincent Link
Richard Holmboe
Dov Benyomin Sohacheski
Rina Y
Phil Bolduc
Daniel SP
Jayson Grace
Devin Buhl
sholdee
Gereon Vey
egandro
João Gonçalves
Balázs Hasprai
balazshasprai
johnnyrun
Marek Pilch
Christian Berendt
Simon Leiner