chore(deps): apply dependency updates in one combined change (#694)

* chore(deps): apply dependency updates in one combined change

- Bump ansible-core to 2.19.11 and jmespath to 1.1.0 in requirements.in
- Regenerate the Python 3.11 pip-compile lock in requirements.txt
- Bump molecule-plugins to 23.6.0 while keeping molecule on the stable 6.x
  series (avoids the molecule-plugins 26 major jump that broke vagrant module
  resolution in CI)
- Bump ruamel-yaml-clib to 0.2.15
- Bump the zgosalvez/github-actions-ensure-sha-pinned-actions action to 5.0.6
  (SHA-pinned) in lint.yml

* fix(server): make log_destination conditional boolean for ansible-core 2.19

- The always block's 'Save logs of k3s-init.service' task used when: log_destination
  where log_destination is a path string derived from an env var
- ansible-core 2.19 rejects string-derived conditionals; evaluate the path as a
  real boolean (non-empty) check so the conditional is a true boolean
- Required to keep the k3s_server role working with ansible-core 2.19.11 (the
  dependency bump in this change)

* fix(verify): coerce regex_search assertions to bool for ansible-core 2.19

- ansible-core 2.19 requires assert conditionals to be boolean; regex_search
  returns a string, which is now rejected
- Wrap all regex_search results used in assert.that with | bool so the calico,
  cilium, metallb, and kube-vip image-tag checks produce boolean results

* fix(verify): use boolean is regex_search test instead of | bool

- | bool on a regex_search result coerces a tag string like v0.16.0 to False
  in ansible-core 2.19, failing the image-tag assertions
- Use the is regex_search test which returns a real boolean without string
  coercion for the calico, cilium, metallb, and kube-vip image assertions

* fix(verify): use is not none for regex_search assertions

- ansible-core 2.19 has no "is regex_search" test and rejects bool string
  coercion, so use the regex_search filter with an "is not none" comparison,
  which yields a real boolean for the image-tag assertions
- Applies to calico, cilium, metallb, and kube-vip image checks

* fix(metallb): retry transient apiserver resets in config tests

- The Layer 2 and BGP final configuration checks ran a kubectl get per
  resource with no retry, so a transient connection refused from the kube
  API could abort converge
- Mirror the download_retries/download_delay retry pattern used by the
  'Wait for MetalLB resources' task so these checks survive api server
  resets on slow runners

.github/ISSUE_TEMPLATE.md

@@ -1,85 +0,0 @@
-
-<!-- It's a good idea to check this post first for general troubleshooting https://github.com/timothystewart6/k3s-ansible/discussions/19   -->
-
-<!--- Provide a general summary of the issue in the Title above -->
-
-## Expected Behavior
-
-<!--- Tell us what should happen -->
-
-## Current Behavior
-<!--- Tell us what happens instead of the expected behavior -->
-
-## Steps to Reproduce
-
-<!--- reproduce this bug. Include code to reproduce, if relevant -->
-
-1.
-2.
-3.
-4.
-
-## Context (variables)
-<!--- please include which OS, along with the variables used when running the playbook -->
-
-Operating system:
-
-Hardware:
-
-### Variables Used
-
-`all.yml`
-
-```yml
-k3s_version: ""
-ansible_user: NA
-systemd_dir: ""
-
-flannel_iface: ""
-
-#calico_iface: ""
-calico_ebpf: ""
-calico_cidr: ""
-calico_tag: ""
-
-apiserver_endpoint: ""
-
-k3s_token: "NA"
-
-extra_server_args: ""
-extra_agent_args: ""
-
-kube_vip_tag_version: ""
-
-kube_vip_cloud_provider_tag_version: ""
-kube_vip_lb_ip_range: ""
-
-metal_lb_speaker_tag_version: ""
-metal_lb_controller_tag_version: ""
-
-metal_lb_ip_range: ""
-```
-
-### Hosts
-
-`host.ini`
-
-```ini
-[master]
-IP.ADDRESS.ONE
-IP.ADDRESS.TWO
-IP.ADDRESS.THREE
-
-[node]
-IP.ADDRESS.FOUR
-IP.ADDRESS.FIVE
-
-[k3s_cluster:children]
-master
-node
-```
-
-## Possible Solution
-<!--- Not obligatory, but suggest a fix/reason for the bug, -->
-
-- [ ] I've checked the [General Troubleshooting Guide](https://github.com/timothystewart6/k3s-ansible/discussions/20)

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,102 @@
+---
+name: Bug report
+description: Report a reproducible problem with the playbooks, roles, or generated resources.
+title: "[Bug]: "
+labels:
+  - bug
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for reporting a problem. Search existing issues and review the troubleshooting link first.
+        Remove credentials, tokens, public IP addresses, and private hostnames from all fields and logs.
+
+  - type: checkboxes
+    id: prerequisites
+    attributes:
+      label: Prerequisites
+      options:
+        - label: I searched existing issues and discussions for this problem.
+          required: true
+        - label: I reviewed the troubleshooting guidance linked from the issue chooser.
+          required: true
+        - label: I removed secrets and identifying infrastructure details from this report.
+          required: true
+
+  - type: textarea
+    id: summary
+    attributes:
+      label: Problem summary
+      description: Describe what failed and its impact.
+      placeholder: A concise description of the problem and affected nodes or components.
+    validations:
+      required: true
+
+  - type: textarea
+    id: expected
+    attributes:
+      label: Expected behavior
+      description: What should have happened?
+    validations:
+      required: true
+
+  - type: textarea
+    id: reproduction
+    attributes:
+      label: Steps to reproduce
+      description: Provide the smallest reliable sequence that reproduces the issue.
+      placeholder: |
+        1. Configure ...
+        2. Run ...
+        3. Observe ...
+    validations:
+      required: true
+
+  - type: input
+    id: revision
+    attributes:
+      label: Repository revision
+      description: Release, tag, branch, or commit SHA used.
+      placeholder: v1.36.2+k3s1+tt1 or a commit SHA
+    validations:
+      required: true
+
+  - type: input
+    id: ansible-version
+    attributes:
+      label: Ansible version
+      description: Output of `ansible --version`, shortened to version and Python details.
+      placeholder: ansible-core 2.18.0, Python 3.12
+    validations:
+      required: true
+
+  - type: textarea
+    id: environment
+    attributes:
+      label: Environment
+      description: Include target OS and version, architecture, node counts, platform, and network provider.
+      placeholder: Debian 13, amd64, 3 control nodes and 2 agents, bare metal, Cilium
+    validations:
+      required: true
+
+  - type: textarea
+    id: configuration
+    attributes:
+      label: Relevant sanitized configuration
+      description: Include only variables and inventory groups needed to reproduce the problem.
+      render: yaml
+    validations:
+      required: true
+
+  - type: textarea
+    id: logs
+    attributes:
+      label: Relevant logs or task output
+      description: Include the failing task and surrounding output. Redact sensitive or identifying values.
+      render: shell
+
+  - type: textarea
+    id: context
+    attributes:
+      label: Additional context
+      description: Add attempted fixes, suspected causes, regressions, or other useful context.

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,9 @@
+---
+blank_issues_enabled: false
+contact_links:
+  - name: Troubleshooting and support
+    url: https://github.com/timothystewart6/k3s-ansible/discussions/20
+    about: Review common troubleshooting guidance and ask configuration or usage questions.
+  - name: General discussions
+    url: https://github.com/timothystewart6/k3s-ansible/discussions
+    about: Discuss ideas and questions that are not confirmed bugs or concrete feature requests.

.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,55 @@
+---
+name: Feature request
+description: Propose a focused improvement to supported repository behavior.
+title: "[Feature]: "
+labels:
+  - enhancement
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Describe the use case before proposing an implementation. Search existing issues and discussions first.
+
+  - type: checkboxes
+    id: prerequisites
+    attributes:
+      label: Prerequisites
+      options:
+        - label: I searched existing issues and discussions for this request.
+          required: true
+        - label: This request is about reusable project behavior, not support for one private environment.
+          required: true
+
+  - type: textarea
+    id: problem
+    attributes:
+      label: Problem or use case
+      description: What limitation exists, who encounters it, and why does it matter?
+    validations:
+      required: true
+
+  - type: textarea
+    id: proposal
+    attributes:
+      label: Proposed behavior
+      description: Describe the desired user-visible result. Include example variables or commands when useful.
+    validations:
+      required: true
+
+  - type: textarea
+    id: alternatives
+    attributes:
+      label: Alternatives considered
+      description: Describe workarounds or other designs and their tradeoffs.
+
+  - type: textarea
+    id: compatibility
+    attributes:
+      label: Compatibility and operational impact
+      description: Note affected operating systems, architectures, CNIs, existing clusters, or reset behavior.
+
+  - type: textarea
+    id: context
+    attributes:
+      label: Additional context
+      description: Add relevant upstream documentation, examples, or prior discussion.

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,15 +1,46 @@
-# Proposed Changes
-<!--- Provide a general summary of your changes -->
+## Summary
+
+<!-- Explain the problem and the resulting behavior. Keep implementation details in the sections below. -->
+
+## Changes
 
--
--
 -
 
-## Checklist
+## Related issues
 
-- [ ] Tested locally
-- [ ] Ran `site.yml` playbook
-- [ ] Ran `reset.yml` playbook
-- [ ] Did not add any unnecessary changes
-- [ ] Ran pre-commit install at least once before committing
-- [ ] 🚀
+<!-- Use "Fixes #123" when this pull request should close an issue. Write "None" when not applicable. -->
+
+## Testing
+
+<!-- List exact commands, scenarios, and relevant manual checks. Do not check a box for a test that was not run. -->
+
+- [ ] `pre-commit run --all-files`
+- [ ] Relevant Ansible syntax checks
+- [ ] Relevant focused regression tests
+- [ ] Relevant Molecule scenario
+- [ ] Provisioning tested against a non-production cluster
+- [ ] Reset behavior tested against a non-production cluster
+
+Not run, with reason:
+
+## Risk and compatibility
+
+<!-- Cover existing clusters, upgrades, networking, supported platforms, security, and rollback. Write "None" when a
+category is not affected. -->
+
+- Existing cluster or upgrade impact:
+- Networking or CNI impact:
+- Security impact:
+- Rollback plan:
+
+## Documentation
+
+<!-- Identify updated docs and sample configuration, or explain why no documentation change is needed. -->
+
+## Final checklist
+
+- [ ] The change is focused and contains no unrelated edits.
+- [ ] Tests cover new behavior or a regression, where applicable.
+- [ ] User-facing variables are documented in role defaults, sample inventory, and the README.
+- [ ] Logs, examples, and configuration contain no secrets or identifying infrastructure details.
+- [ ] Generated files and local environment files are not included.

.github/copilot-instructions.md

@@ -0,0 +1,6 @@
+# GitHub Copilot instructions
+
+Read and follow the repository's root-level `AGENTS.md` before proposing or making changes. It is the canonical guide
+for architecture, safety, implementation, validation, and documentation expectations.
+
+Do not duplicate repository guidance here. If instructions need to change, update `AGENTS.md`.

.github/scripts/test-default-interface.py

@@ -0,0 +1,90 @@
+#!/usr/bin/env python3
+"""Assert the sample inventory resolves the flannel interface per host.
+
+flannel_iface defaults to the host's default IPv4 interface rather than a
+hardcoded eth0. This test extracts the flannel_iface expression from the
+sample inventory and proves that a host whose primary interface is not named
+eth0 (e.g. enp1s0, ens3) resolves the interface from ansible facts.
+"""
+
+from __future__ import print_function
+
+import os
+import re
+import subprocess
+
+from jinja2 import Environment, StrictUndefined
+
+
+def repo_root():
+    return subprocess.check_output(
+        ["git", "rev-parse", "--show-toplevel"], text=True
+    ).strip()
+
+
+def fail(message):
+    raise SystemExit("default-interface test failed: " + message)
+
+
+class FakeAnsibleFacts(object):
+    """Stand-in for the per-host ``ansible_facts`` dict."""
+
+    def __init__(self, default_iface, iface_ip):
+        self._default = {"interface": default_iface, "address": iface_ip}
+        self._ifaces = {
+            default_iface: {"ipv4": {"address": iface_ip}},
+        }
+
+    @property
+    def default_ipv4(self):
+        return self._default
+
+    def __getitem__(self, key):
+        return self._ifaces[key]
+
+
+def read_all_yml(root):
+    path = os.path.join(root, "inventory", "sample", "group_vars", "all.yml")
+    with open(path, "r") as handle:
+        return handle.read()
+
+
+def extract_value(content, key):
+    # Match a quoted value assigned to the key, e.g. flannel_iface: "...".
+    match = re.search(r"^%s:\s*\"(.+)\"\s*$" % re.escape(key), content, re.M)
+    if not match:
+        fail("could not find %s in the sample inventory" % key)
+    return match.group(1)
+
+
+def resolve(env, expression, facts):
+    template = env.from_string(expression)
+    return template.render(ansible_facts=facts)
+
+
+def main():
+    root = repo_root()
+    content = read_all_yml(root)
+    env = Environment(undefined=StrictUndefined)
+
+    flannel_expr = extract_value(content, "flannel_iface")
+    if "default_ipv4.interface" not in flannel_expr:
+        fail("flannel_iface no longer defaults from ansible facts")
+
+    # A host whose primary interface is enp1s0 (the core #621 scenario).
+    facts = FakeAnsibleFacts("enp1s0", "192.168.30.11")
+    resolved = resolve(env, flannel_expr, facts)
+    if resolved != "enp1s0":
+        fail("flannel_iface resolved to %r, expected enp1s0" % resolved)
+
+    # A different host with a different interface must resolve independently.
+    facts2 = FakeAnsibleFacts("ens3", "192.168.30.12")
+    resolved2 = resolve(env, flannel_expr, facts2)
+    if resolved2 != "ens3":
+        fail("flannel_iface resolved to %r, expected ens3" % resolved2)
+
+    print("default-interface regression test passed")
+
+
+if __name__ == "__main__":
+    main()

.github/scripts/test-metallb-deploy-condition.py

@@ -0,0 +1,170 @@
+#!/usr/bin/env python3
+"""Regression test for the MetalLB deploy conditions.
+
+The MetalLB manifest (roles/k3s_server/tasks/main.yml) and the MetalLB pool
+(roles/k3s_server_post/tasks/main.yml) are included under a `when` condition
+that decides whether MetalLB provides load balancing. A previous change (#683)
+guarded `cilium_bgp` but accidentally skipped MetalLB whenever a non-BGP
+Cilium CNI was in use (`cilium_iface` defined), breaking the cilium + MetalLB
+scenario.
+
+This test loads the real `when` expressions from both task files and evaluates
+them against representative variable sets, asserting MetalLB is deployed in
+every topology except when kube-vip owns the VIP range or Cilium BGP is enabled.
+"""
+
+from __future__ import print_function
+
+import os
+import re
+import subprocess
+
+import yaml
+from jinja2 import Environment
+
+METALLB_WHEN = (
+    "kube_vip_lb_ip_range is not defined and "
+    "not (cilium_bgp | default(false) | bool)"
+)
+
+
+def repo_root():
+    return subprocess.check_output(
+        ["git", "rev-parse", "--show-toplevel"], text=True
+    ).strip()
+
+
+def fail(message):
+    raise SystemExit("MetalLB deploy condition test failed: " + message)
+
+
+def extract_when(path, task_name):
+    """Return the `when:` expression string for the named task."""
+    with open(path, encoding="utf-8") as handle:
+        doc = yaml.safe_load(handle)
+    for task in doc:
+        if task.get("name") == task_name:
+            when = task.get("when")
+            return (when or "").strip()
+    return None
+
+
+def evaluate(when, variables):
+    """Evaluate a `when` expression against variables using Jinja2."""
+    env = Environment()
+
+    def fake_bool(value):
+        # Minimal stand-in for Ansible's truthiness filter used by `| bool`.
+        if isinstance(value, bool):
+            return value
+        if value is None:
+            return False
+        return str(value).lower() in ("1", "true", "yes", "on")
+
+    env.filters["bool"] = fake_bool
+    template = env.from_string("{{ " + when + " }}")
+    rendered = template.render(**variables)
+    # The expression renders to the literal strings "True"/"False".
+    if rendered == "True":
+        return True
+    if rendered == "False":
+        return False
+    fail("condition did not render to a boolean: {0!r}".format(rendered))
+
+
+def assert_deployment(when, variables, expected, label):
+    result = evaluate(when, variables)
+    verdict = "deploy" if result else "skip"
+    expected_verdict = "deploy" if expected else "skip"
+    if result != expected:
+        fail(
+            "{0}: expected to {1} MetalLB but the condition chose to {2} "
+            "(vars: {3})".format(label, expected_verdict, verdict, variables)
+        )
+
+
+def scenarios():
+    """Yield (variables, expected_deploy, label) pairs."""
+    yield (
+        # Default Flannel inventory (all.yml sets cilium_bgp: false).
+        {
+            "cilium_bgp": False,
+            "cilium_iface": None,
+        },
+        True,
+        "flannel default (cilium_bgp: false)",
+    )
+    yield (
+        # Calico CNI with no Cilium variable in scope (issue #644): cilium_bgp
+        # is genuinely undefined, so `default(false)` must keep MetalLB on.
+        {
+            "calico_iface": "eth1",
+        },
+        True,
+        "calico, cilium_bgp undefined (#644)",
+    )
+    yield (
+        # Cilium CNI with BGP disabled: MetalLB must still be deployed.
+        {
+            "cilium_bgp": False,
+            "cilium_iface": "eth1",
+        },
+        True,
+        "cilium non-BGP (regression catch)",
+    )
+    yield (
+        # Cilium CNI with BGP enabled: Cilium provides the LB, skip MetalLB.
+        {
+            "cilium_bgp": True,
+            "cilium_iface": "eth1",
+        },
+        False,
+        "cilium BGP enabled",
+    )
+    yield (
+        # kube-vip is the load balancer provider: skip MetalLB.
+        {
+            "kube_vip_lb_ip_range": "192.168.30.80-192.168.30.90",
+            "cilium_bgp": False,
+        },
+        False,
+        "kube-vip owns the VIP range",
+    )
+
+
+def main():
+    root = repo_root()
+    server_tasks = os.path.join(root, "roles", "k3s_server", "tasks", "main.yml")
+    server_post_tasks = os.path.join(
+        root, "roles", "k3s_server_post", "tasks", "main.yml"
+    )
+
+    server_when = extract_when(server_tasks, "Deploy metallb manifest")
+    server_post_when = extract_when(server_post_tasks, "Deploy metallb pool")
+
+    if server_when is None:
+        fail("could not find 'Deploy metallb manifest' when condition")
+    if server_post_when is None:
+        fail("could not find 'Deploy metallb pool' when condition")
+
+    for when, source in (
+        (server_when, "k3s_server/tasks/main.yml"),
+        (server_post_when, "k3s_server_post/tasks/main.yml"),
+    ):
+        if when != METALLB_WHEN:
+            fail(
+                "{0} when condition changed unexpectedly:\n"
+                "  expected: {1}\n  got:      {2}".format(source, METALLB_WHEN, when)
+            )
+
+    for variables, expected, label in scenarios():
+        assert_deployment(server_when, variables, expected, "server " + label)
+        assert_deployment(
+            server_post_when, variables, expected, "server_post " + label
+        )
+
+    print("MetalLB deploy condition regression test passed for all scenarios")
+
+
+if __name__ == "__main__":
+    main()

.github/workflows/ci.yml

@@ -14,6 +14,7 @@ on:
       - '**/FUNDING.yml'
       - '**/host.ini'
       - '**/*.md'
+      - '.github/ISSUE_TEMPLATE/**'
       - '**/.editorconfig'
       - '**/ansible.example.cfg'
       - '**/deploy.sh'
@@ -31,10 +32,13 @@ concurrency:
 
 jobs:
   pre:
+    if: github.actor != 'dependabot[bot]'
     uses: ./.github/workflows/cache.yml
     needs: [lint]
   lint:
+    if: github.actor != 'dependabot[bot]'
     uses: ./.github/workflows/lint.yml
   test:
+    if: github.actor != 'dependabot[bot]'
     uses: ./.github/workflows/test.yml
     needs: [pre, lint]

.github/workflows/lint.yml

@@ -68,7 +68,7 @@ jobs:
       - name: Checkout code
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # 7.0.1
       - name: Ensure SHA pinned actions
-        uses: zgosalvez/github-actions-ensure-sha-pinned-actions@3db98c0363e2fa5df3e1c4c471777a7c10b24cc9 # 5.0.5
+        uses: zgosalvez/github-actions-ensure-sha-pinned-actions@46cfe808a5f1588656ef299eedd0ce2fd7ec0dcc # 5.0.6
         with:
           allowlist: |
             aws-actions/

.pre-commit-config.yaml

@@ -84,3 +84,20 @@ repos:
         language: system
         pass_filenames: false
         files: ^roles/k3s_server/tasks/metallb\.yml$|^\.github/scripts/test-metallb-remote-read\.sh$
+      - id: metallb-deploy-condition-test
+        name: MetalLB deploy condition test
+        entry: python3 .github/scripts/test-metallb-deploy-condition.py
+        language: python
+        additional_dependencies:
+          - Jinja2>=3.1
+          - PyYAML
+        pass_filenames: false
+        files: ^roles/k3s_server/tasks/main\.yml$|^roles/k3s_server_post/tasks/main\.yml$|^\.github/scripts/test-metallb-deploy-condition\.py$ # noqa yaml[line-length]
+      - id: default-interface-test
+        name: default interface test
+        entry: python3 .github/scripts/test-default-interface.py
+        language: python
+        additional_dependencies:
+          - Jinja2>=3.1
+        pass_filenames: false
+        files: ^inventory/sample/group_vars/all\.yml$|^\.github/scripts/test-default-interface\.py$

AGENTS.md

@@ -0,0 +1,130 @@
+# Agent Guide
+
+This file is the canonical repository guide for coding agents and automated contributors. Read it before making
+changes. Human contributors should also review [CONTRIBUTING.md](CONTRIBUTING.md).
+
+## Project overview
+
+This repository is an Ansible collection that provisions and resets highly available k3s clusters. It supports
+multiple networking choices, including Flannel, Calico, Cilium, kube-vip, and MetalLB.
+
+The main entry points are:
+
+- `site.yml`: provision or update a cluster.
+- `reset.yml`: remove k3s from a cluster.
+- `reboot.yml`: reboot cluster nodes.
+- `inventory/sample/`: example inventory and variables.
+- `roles/`: reusable Ansible roles used by the playbooks.
+- `molecule/`: integration scenarios run by CI.
+- `.github/scripts/`: CI support scripts and focused regression tests.
+
+## Source of truth
+
+- Role defaults belong in `roles/<role>/defaults/main.yml`.
+- Tasks belong in `roles/<role>/tasks/` and handlers in `roles/<role>/handlers/`.
+- Example user configuration belongs in `inventory/sample/`.
+- User-facing setup and variable documentation belongs in `README.md`.
+- Contributor workflows and review expectations belong in `CONTRIBUTING.md`.
+- Agent-specific repository instructions belong in this file.
+
+Keep `CLAUDE.md` and `.github/copilot-instructions.md` as small pointers to this file. Do not duplicate these
+instructions in tool-specific files.
+
+## Development setup
+
+Use a Python virtual environment. Do not commit the environment, generated logs, inventories, kubeconfigs, or
+credentials.
+
+```bash
+python3 -m venv .env
+source .env/bin/activate
+python3 -m pip install -r requirements.txt
+ansible-galaxy collection install -r collections/requirements.yml
+pre-commit install
+```
+
+`ansible.cfg` is intentionally ignored. Copy `ansible.example.cfg` when local configuration is needed.
+
+## Working rules
+
+1. Inspect the current branch and worktree before editing. Preserve unrelated user changes.
+2. Keep changes focused. Avoid drive-by formatting or dependency updates.
+3. Never add real IP addresses, hostnames, tokens, private keys, kubeconfigs, or inventory secrets.
+4. Use placeholders in examples and redact sensitive values from logs and issue reports.
+5. Preserve idempotence. An already-converged host should not report changes without a real state transition.
+6. Prefer Ansible modules over `ansible.builtin.command` or `ansible.builtin.shell`. When a command is required,
+   define accurate `changed_when` and `failed_when` behavior.
+7. Use fully qualified collection names, such as `ansible.builtin.copy`.
+8. Put configurable values in role defaults or inventory variables. Avoid embedding environment-specific values in
+   tasks and templates.
+9. Maintain compatibility with the operating systems and architectures listed in `README.md`.
+10. Do not weaken lint rules, tests, or CI checks to make a change pass.
+
+## Change guidance
+
+### Ansible tasks and roles
+
+- Use descriptive task names in sentence case.
+- Use YAML booleans (`true` and `false`) rather than aliases.
+- Quote file modes, for example `mode: "0644"`.
+- Notify handlers only when the managed resource changes.
+- Use `become: true` only where privilege escalation is needed.
+- Update role defaults, sample inventory, and the README together when adding or renaming user-facing variables.
+- Check reset behavior when provisioning introduces persistent services, files, mounts, or network state.
+
+### Templates and manifests
+
+- Keep Jinja logic small and readable. Move complicated decisions into task variables where practical.
+- Render valid YAML after Jinja evaluation.
+- Preserve explicit handling for optional and undefined variables.
+- Add or update a focused test under `.github/scripts/` when changing generated Kubernetes manifests or bootstrap
+  behavior.
+
+### Molecule scenarios
+
+- Reuse `molecule/resources/` for shared behavior.
+- Put scenario-specific inputs in `molecule/<scenario>/overrides.yml` and `verify-vars.yml`.
+- Update `molecule/README.md` when adding, removing, or materially changing a scenario.
+- Clean up resources created by tests, including failure paths.
+
+## Validation
+
+Run the smallest relevant checks while iterating, then run the complete local validation before considering a change
+ready:
+
+```bash
+pre-commit run --all-files
+```
+
+For playbook or role changes, also run syntax checks with a non-sensitive inventory:
+
+```bash
+ansible-playbook site.yml --syntax-check -i inventory/sample/hosts.ini
+ansible-playbook reset.yml --syntax-check -i inventory/sample/hosts.ini
+```
+
+Run focused regression scripts when their related files change. The mapping is defined in
+`.pre-commit-config.yaml`.
+
+Molecule tests require Vagrant, VirtualBox, host-only networking, and substantial local resources. Run the most
+relevant scenario when that environment is available:
+
+```bash
+molecule test --scenario-name <scenario>
+```
+
+If a required test can't be run locally, state exactly which check was skipped and why. Never claim a check passed
+unless it was executed.
+
+## Documentation and review
+
+- Keep commands copyable and examples free of secrets.
+- Update documentation in the same change as user-visible behavior.
+- Explain behavior changes, compatibility concerns, operational risks, and rollback steps in the pull request.
+- Use conventional commit messages with a scope, for example `fix(k3s-server): handle an existing token safely`.
+- Do not commit, push, open a pull request, or modify remote resources unless the user explicitly requests it.
+
+## Definition of done
+
+A change is ready for review when it is focused, documented, linted, tested in proportion to its risk, and shown in a
+clean diff with no secrets or generated artifacts.

CLAUDE.md

@@ -0,0 +1,3 @@
+@AGENTS.md
+
+`AGENTS.md` is the canonical repository guide. Follow it for all work in this repository.

CONTRIBUTING.md

@@ -0,0 +1,94 @@
+# Contributing to k3s-ansible
+
+Thank you for improving k3s-ansible. Contributions should be focused, safe to apply to existing clusters, and tested
+in proportion to their operational impact.
+
+## Before opening an issue
+
+- Search existing issues and discussions for the same behavior.
+- Review the [troubleshooting discussion](https://github.com/timothystewart6/k3s-ansible/discussions/20).
+- Remove tokens, credentials, public IP addresses, private hostnames, and other sensitive values from logs and
+  configuration.
+- For support requests, include the k3s-ansible revision, Ansible version, target operating system, architecture,
+  network provider, relevant sanitized variables, and a minimal reproduction.
+
+Use the bug report template for reproducible defects and the feature request template for proposed behavior.
+
+## Development environment
+
+Fork and clone the repository, then create a branch from the latest `master`:
+
+```bash
+git switch master
+git pull --ff-only
+git switch -c <type>/<short-description>
+```
+
+Create a Python environment and install the pinned development dependencies:
+
+```bash
+python3 -m venv .env
+source .env/bin/activate
+python3 -m pip install -r requirements.txt
+ansible-galaxy collection install -r collections/requirements.yml
+pre-commit install
+```
+
+Copy `ansible.example.cfg` to the ignored `ansible.cfg` file if local Ansible configuration is needed. Start custom
+inventories from `inventory/sample/`, keep them out of Git, and never use production credentials in tests.
+
+## Making changes
+
+- Keep each pull request focused on one problem or feature.
+- Follow [AGENTS.md](AGENTS.md) for repository structure, implementation conventions, and safety requirements.
+- Preserve idempotence and existing-cluster compatibility.
+- Add or update tests for behavior changes and regressions.
+- Update role defaults, sample inventory, and documentation when user-facing variables change.
+- Consider both provisioning and reset behavior for persistent resources.
+- Avoid unrelated reformatting and generated files.
+
+## Validation
+
+Run all pre-commit checks before submitting a pull request:
+
+```bash
+pre-commit run --all-files
+```
+
+For playbook or role changes, run syntax checks:
+
+```bash
+ansible-playbook site.yml --syntax-check -i inventory/sample/hosts.ini
+ansible-playbook reset.yml --syntax-check -i inventory/sample/hosts.ini
+```
+
+Run the most relevant Molecule scenario when Vagrant, VirtualBox, and the required host networking are available:
+
+```bash
+molecule test --scenario-name <scenario>
+```
+
+See [molecule/README.md](molecule/README.md) for scenario details and local requirements. Pull requests should list
+every check that was run and clearly identify checks that could not be run locally.
+
+## Commits and pull requests
+
+Use a conventional commit subject with a scope:
+
+```text
+type(scope): short description
+```
+
+Common types are `feat`, `fix`, `refactor`, `test`, `docs`, and `chore`. Write subjects in the imperative mood and
+keep commits logically focused.
+
+Pull requests should:
+
+- Describe the problem and the resulting behavior.
+- Identify compatibility, security, networking, and upgrade risks.
+- Include testing evidence without sensitive data.
+- Call out documentation and sample configuration changes.
+- Link related issues with `Fixes #<issue>` when applicable.
+- Avoid checking boxes for tests that were not run.
+
+Maintainers may ask for a change to be split when unrelated work makes it difficult to review or roll back.

README.md

@@ -2,51 +2,66 @@
 
 ![Fully Automated K3S etcd High Availability Install](https://img.youtube.com/vi/CbkEWcUZ7zM/0.jpg)
 
-This playbook will build an HA Kubernetes cluster with `k3s`, `kube-vip` and MetalLB via `ansible`.
+This Ansible collection builds a highly available Kubernetes cluster with k3s. It supports kube-vip for the control
+plane virtual IP, multiple CNI options, and either MetalLB or kube-vip for service load balancing.
 
 This is based on the work from [this fork](https://github.com/212850a/k3s-ansible) which is based on the work from [k3s-io/k3s-ansible](https://github.com/k3s-io/k3s-ansible). It uses [kube-vip](https://kube-vip.io/) to create a load balancer for control plane, and [metal-lb](https://metallb.universe.tf/installation/) for its service `LoadBalancer`.
 
-If you want more context on how this works, see:
+For more context on how it works, see:
 
 📄 [Documentation](https://technotim.com/posts/k3s-etcd-ansible/) (including example commands)
 
 📺 [Watch the Video](https://www.youtube.com/watch?v=CbkEWcUZ7zM)
 
+## Project guides
+
+- [Getting started](#-getting-started)
+- [Configuration variables](#variables)
+- [Upgrading an existing cluster](#-upgrading-an-existing-cluster)
+- [Local Molecule testing](molecule/README.md)
+- [Contributing guidelines](CONTRIBUTING.md)
+- [Repository guide for coding agents](AGENTS.md)
+
 ## 📖 k3s Ansible Playbook
 
-Build a Kubernetes cluster using Ansible with k3s. The goal is easily install a HA Kubernetes cluster on machines running:
+Build a Kubernetes cluster using Ansible and k3s. The goal is to make a highly available cluster straightforward to
+install on machines running:
 
 - [x] Debian (tested on version 13)
 - [x] Ubuntu (tested on version 26.04 LTS)
 - [x] Rocky (tested on version 10)
 
-on processor architecture:
+Supported processor architectures are:
 
-- [X] x64
-- [X] arm64
-- [X] armhf
+- [x] x64
+- [x] arm64
+- [x] armhf
 
 ## ✅ System requirements
 
-- Control Node (the machine you are running `ansible` commands) must have Ansible 2.11+ If you need a quick primer on Ansible [you can check out my docs and setting up Ansible](https://technotim.com/posts/ansible-automation/).
+- The control node, which runs the Ansible commands, must have Ansible 2.11 or newer. For a quick primer, see
+  [setting up Ansible](https://technotim.com/posts/ansible-automation/).
 
-- You will also need to install collections that this playbook uses by running `ansible-galaxy collection install -r ./collections/requirements.yml` (important❗)
+- Install the required collections with
+  `ansible-galaxy collection install -r ./collections/requirements.yml`.
 
 - [`netaddr` package](https://pypi.org/project/netaddr/) must be available to Ansible. If you have installed Ansible via apt, this is already taken care of. If you have installed Ansible via `pip`, make sure to install `netaddr` into the respective virtual environment.
 
-- `server` and `agent` nodes should have passwordless SSH access, if not you can supply arguments to provide credentials `--ask-pass --ask-become-pass` to each command.
+- Server and agent nodes should support passwordless SSH access. Otherwise, pass `--ask-pass --ask-become-pass` to
+  each playbook command.
 
 ## 🚀 Getting Started
 
 ### 🍴 Preparation
 
-First create a new directory based on the `sample` directory within the `inventory` directory:
+Create a cluster-specific inventory from the sample. The `inventory/` directory ignores custom inventory content so
+credentials and environment details aren't committed accidentally.
 
 ```bash
 cp -R inventory/sample inventory/my-cluster
 ```
 
-Second, edit `inventory/my-cluster/hosts.ini` to match the system information gathered above
+Edit `inventory/my-cluster/hosts.ini` to match the target hosts.
 
 For example:
 
@@ -67,9 +82,10 @@ node
 
 If multiple hosts are in the master group, the playbook will automatically set up k3s in [HA mode with etcd](https://rancher.com/docs/k3s/latest/en/installation/ha-embedded/).
 
-Finally, copy `ansible.example.cfg` to `ansible.cfg` and adapt the inventory path to match the files that you just created.
+Copy `ansible.example.cfg` to `ansible.cfg`, then update its inventory path. The local `ansible.cfg` file is ignored by
+Git.
 
-This requires at least k3s version `1.19.1` however the version is configurable by using the `k3s_version` variable.
+The minimum k3s version is `1.19.1`. Select the desired version with the `k3s_version` variable.
 
 If needed, you can also edit `inventory/my-cluster/group_vars/all.yml` to match your environment.
 
@@ -81,7 +97,8 @@ Start provisioning of the cluster using the following command:
 ansible-playbook site.yml -i inventory/my-cluster/hosts.ini
 ```
 
-After deployment control plane will be accessible via virtual ip-address which is defined in inventory/group_vars/all.yml as `apiserver_endpoint`
+After deployment, the control plane is accessible through the virtual IP defined by `apiserver_endpoint` in the
+inventory variables.
 
 ### 🔥 Remove k3s cluster
 
@@ -89,7 +106,25 @@ After deployment control plane will be accessible via virtual ip-address which i
 ansible-playbook reset.yml -i inventory/my-cluster/hosts.ini
 ```
 
->You should also reboot these nodes due to the VIP not being destroyed
+> Reboot the nodes after reset because the virtual IP may remain configured.
+
+### ⏻️ Reboot Cluster Nodes
+
+Reboot all cluster nodes at once or stage the reboot across the cluster.
+
+```bash
+ansible-playbook reboot.yml -i inventory/my-cluster/hosts.ini
+```
+
+To reboot the nodes in batches, set `concurrent_reboots` to the number of nodes
+to reboot at a time (or a percentage). Optionally set `wait_seconds_after_reboot`
+to pause after each batch so pods in the freshly rebooted batch can settle
+before the next batch reboots.
+
+```bash
+ansible-playbook reboot.yml -i inventory/my-cluster/hosts.ini \
+  --extra-vars 'concurrent_reboots=2 wait_seconds_after_reboot=30'
+```
 
 ## 🔁 Upgrading an existing cluster
 
@@ -124,13 +159,21 @@ To copy your `kube config` locally so that you can access your **Kubernetes** cl
 ```bash
 scp debian@master_ip:/etc/rancher/k3s/k3s.yaml ~/.kube/config
 ```
-If you get file Permission denied, go into the node and temporarly run:
+If the copy fails with a permission error, grant the SSH user temporary read access using the least permissive method
+available for the target system. Restore the original ownership and permissions immediately after copying. Avoid
+world-writable permissions on the kubeconfig because it contains cluster credentials.
+
+For example, copy the file to a temporary user-readable path from the control node:
+
 ```bash
-sudo chmod 777 /etc/rancher/k3s/k3s.yaml
+ssh debian@master_ip 'sudo install -o "$(id -un)" -m 0600 /etc/rancher/k3s/k3s.yaml /tmp/k3s.yaml'
 ```
-Then copy with the scp command and reset the permissions back to:
+
+Copy `/tmp/k3s.yaml`, then remove the temporary remote copy:
+
 ```bash
-sudo chmod 600 /etc/rancher/k3s/k3s.yaml
+scp debian@master_ip:/tmp/k3s.yaml ~/.kube/config
+ssh debian@master_ip rm -f /tmp/k3s.yaml
 ```
 
 You'll then want to modify the config to point to master IP by running:
@@ -150,7 +193,7 @@ See the commands [here](https://technotim.com/posts/k3s-etcd-ansible/#testing-yo
 | `download` | `k3s_version` | string | ❌ | Required | K3s binaries version |
 | `k3s_agent`, `k3s_server`, `k3s_server_post` | `apiserver_endpoint` | string | ❌ | Required | Virtual ip-address configured on each master |
 | `k3s_agent` | `extra_agent_args` | string | `null` | Not required | Extra arguments for agents nodes |
-| `k3s_agent`, `k3s_server` | `group_name_master` | string | `null` | Not required | Name othe master group |
+| `k3s_agent`, `k3s_server` | `group_name_master` | string | `null` | Not required | Name of the master group |
 | `k3s_agent` | `k3s_token` | string | `null` | Not required | Token used to communicate between masters |
 | `k3s_agent`, `k3s_server` | `proxy_env` | dict | `null` | Not required | Internet proxy configurations |
 | `k3s_agent`, `k3s_server` | `proxy_env.HTTP_PROXY` | string | ❌ | Required | HTTP internet proxy |
@@ -211,6 +254,8 @@ See the commands [here](https://technotim.com/posts/k3s-etcd-ansible/#testing-yo
 | `k3s_server_post` | `metal_lb_bgp_peer_asn` | string | `~` | Not required | BGP peer ASN configurations |
 | `k3s_server_post` | `metal_lb_bgp_peer_address` | string | `~` | Not required | BGP peer address |
 | `lxc` | `custom_reboot_command` | string | `~` | Not required | Command to run on reboot |
+| `reboot` (playbook) | `concurrent_reboots` | int/string | `100%` | Not required | Number (or percentage) of nodes to reboot at a time for a staggered reboot |
+| `reboot` (playbook) | `wait_seconds_after_reboot` | int | `0` | Not required | Pause in seconds between staggered reboot batches |
 | `prereq` | `system_timezone` | string | `null` | Not required | Timezone to be set on all nodes |
 | `proxmox_lxc`, `reset_proxmox_lxc` | `proxmox_lxc_ct_ids` | list | ❌ | Required | Proxmox container ID list |
 | `raspberrypi` | `state` | string | `present` | Not required | Indicates whether the k3s prerequisites for Raspberry Pi should be set up (possible values are `present` and `absent`) |
@@ -227,9 +272,11 @@ It is run automatically in CI, but you can also run the tests locally.
 This might be helpful for quick feedback in a few cases.
 You can find more information about it [here](molecule/README.md).
 
-### Pre-commit Hooks
+### Pre-commit hooks
 
-This repo uses `pre-commit` and `pre-commit-hooks` to lint and fix common style and syntax errors.  Be sure to install python packages and then run `pre-commit install`.  For more information, see [pre-commit](https://pre-commit.com/)
+This repository uses `pre-commit` to check style, syntax, Ansible content, and shell scripts. Install the Python
+dependencies, run `pre-commit install` once, and run `pre-commit run --all-files` before submitting a change. See
+[CONTRIBUTING.md](CONTRIBUTING.md) for the complete development workflow.
 
 ## 🌌 Ansible Galaxy
 

inventory/sample/group_vars/all.yml

@@ -8,16 +8,18 @@ systemd_dir: /etc/systemd/system
 system_timezone: Your/Timezone
 
 # interface which will be used for flannel
-flannel_iface: eth0
+# Defaults to each host's default IPv4 interface (e.g. eth0, enp1s0, ens3)
+# so KVM/cloud hosts without eth0 work out of the box. Override per-host if needed.
+flannel_iface: "{{ ansible_facts.default_ipv4.interface }}"
 
 # uncomment calico_iface to use tigera operator/calico cni instead of flannel https://docs.tigera.io/calico/latest/about
-# calico_iface: "eth0"
+# calico_iface: "{{ ansible_facts.default_ipv4.interface }}"
 calico_ebpf: false # use eBPF dataplane instead of iptables
 calico_tag: v3.32.1 # calico version tag
 
 # uncomment cilium_iface to use cilium cni instead of flannel or calico
 # ensure v4.19.57, v5.1.16, v5.2.0 or more recent kernel
-# cilium_iface: "eth0"
+# cilium_iface: "{{ ansible_facts.default_ipv4.interface }}"
 cilium_mode: native # native when nodes are on the same subnet or use BGP, otherwise set tunnel
 cilium_tag: v1.20.0 # cilium version tag
 cilium_cli_tag: v0.19.7 # cilium cli version tag
@@ -47,7 +49,9 @@ kube_vip_bgp_as: "64513"  # Defines the AS for the BGP server
 kube_vip_bgp_peeraddress: "192.168.30.1"  # Defines the address for the BGP peer
 kube_vip_bgp_peeras: "64512"  # Defines the AS for the BGP peer
 
-# apiserver_endpoint is virtual ip-address which will be configured on each master
+# apiserver_endpoint is virtual ip-address which will be configured on each master.
+# This must be a free, routable IP on your network (not already assigned to a host
+# or service), and is used by kube-vip / MetalLB to expose the Kubernetes API.
 apiserver_endpoint: 192.168.30.222
 
 # k3s_token is required  masters can talk together securely

molecule/resources/verify_from_outside/tasks/test/verify-components.yml

@@ -38,6 +38,30 @@
           loop_control:
             label: "{{ item.metadata.name }} ready"
 
+        - name: Assert every node registered a node IP from its interface
+          # Each k3s node is launched with --node-ip derived from flannel_iface.
+          # Confirm every node carries a real InternalIP (not a loopback), which
+          # proves k3s bound to the cluster interface rather than defaulting to 127.0.0.1.
+          ansible.builtin.assert:
+            that: >-
+              (node_internal_ips | length) >= 1 and
+              (node_internal_ips | reject('eq', '127.0.0.1') | list | length) == node_internal_ips | length
+            success_msg: "{{ item.metadata.name }} is bound to {{ node_internal_ips | join(', ') }}"
+            fail_msg: >-
+              {{ item.metadata.name }} has no non-loopback InternalIP
+              (got: {{ node_internal_ips | join(', ') }})
+          vars:
+            node_internal_ips: >-
+              {{
+                (item.status.addresses | default([]))
+                | selectattr('type', 'equalto', 'InternalIP')
+                | map(attribute='address')
+                | list
+              }}
+          loop: "{{ verify_nodes.resources }}"
+          loop_control:
+            label: "{{ item.metadata.name }} InternalIP"
+
         - name: Get any Calico namespaces with Flannel enabled
           kubernetes.core.k8s_info:
             kind: Namespace
@@ -79,7 +103,7 @@
           ansible.builtin.assert:
             that:
               - calico_node_ds.resources | length == 1
-              - calico_node_image | regex_search(':' ~ calico_tag)
+              - calico_node_image | regex_search(':' ~ calico_tag) is not none
             success_msg: "Calico node image uses tag {{ calico_tag }}"
             fail_msg: >-
               Calico node image {{ calico_node_image }},
@@ -145,8 +169,8 @@
         - name: Assert Cilium agent and operator use the expected image tag
           ansible.builtin.assert:
             that:
-              - cilium_agent_image | regex_search(':' ~ cilium_tag)
-              - cilium_operator_image | regex_search(':' ~ cilium_tag)
+              - cilium_agent_image | regex_search(':' ~ cilium_tag) is not none
+              - cilium_operator_image | regex_search(':' ~ cilium_tag) is not none
             success_msg: "Cilium agent and operator use {{ cilium_tag }}"
             fail_msg: >-
               Cilium agent {{ cilium_agent_image }},
@@ -219,17 +243,34 @@
             namespace: metallb-system
             kubeconfig: "{{ kubecfg_path }}"
           register: metallb_info
+          until: metallb_info.resources | length > 0
+          retries: 15
+          delay: 10
           loop:
             - { kind: Deployment, name: controller }
             - { kind: DaemonSet, name: speaker }
           loop_control:
             label: "{{ item.kind }}/{{ item.name }}"
 
+        - name: Fail with a clear message if MetalLB resources are missing
+          ansible.builtin.fail:
+            msg: >-
+              Did not find {{ item.kind | lower }} {{ item.name }} in
+              metallb-system. Expected MetalLB to be deployed in this
+              scenario (verify_lb: {{ verify_lb }}).
+          when: item.resources | length == 0
+          loop: "{{ metallb_info.results }}"
+          loop_control:
+            label: "{{ item.item.kind }}/{{ item.item.name }}"
+
         - name: Assert MetalLB controller and speaker use the expected image tags
           ansible.builtin.assert:
+            # regex_search returns a string or none; check for a match with `is not
+            # none` so the assertion is a real boolean (ansible-core 2.19 rejects
+            # string conditionals and `| bool` deprecates string coercion).
             that:
-              - controller_image | regex_search(metal_lb_controller_tag_version)
-              - speaker_image | regex_search(metal_lb_speaker_tag_version)
+              - controller_image | regex_search(metal_lb_controller_tag_version) is not none
+              - speaker_image | regex_search(metal_lb_speaker_tag_version) is not none
             success_msg: >-
               MetalLB controller {{ metal_lb_controller_tag_version }},
               speaker {{ metal_lb_speaker_tag_version }}
@@ -273,8 +314,8 @@
         - name: Assert the kube-vip and cloud provider image tags
           ansible.builtin.assert:
             that:
-              - kubevip_image | regex_search(':' ~ kube_vip_tag_version)
-              - cloud_provider_image | regex_search(verify_kube_vip_cloud_provider_tag)
+              - kubevip_image | regex_search(':' ~ kube_vip_tag_version) is not none
+              - cloud_provider_image | regex_search(verify_kube_vip_cloud_provider_tag) is not none
             success_msg: >-
               kube-vip {{ kube_vip_tag_version }},
               cloud provider {{ verify_kube_vip_cloud_provider_tag }}

reboot.yml

@@ -2,9 +2,29 @@
 - name: Reboot k3s_cluster
   hosts: k3s_cluster
   gather_facts: true
+
+  # Stagger the reboot across the cluster when concurrent_reboots is set.
+  # Defaults to '100%' so the whole cluster reboots at once (backward compatible).
+  serial: "{{ concurrent_reboots | default('100%') }}"
+
   tasks:
-    - name: Reboot the nodes (and Wait upto 5 mins max)
+    - name: >-
+        {{
+          'Reboot all nodes at once'
+          if (concurrent_reboots is not defined)
+          else 'Reboot nodes with concurrency of ' ~ concurrent_reboots
+        }}
       become: true
       ansible.builtin.reboot:
         reboot_command: "{{ custom_reboot_command | default(omit) }}"
         reboot_timeout: 300
+        test_command: >-
+          {{ 'kubectl get nodes' if 'master' in group_names else 'whoami' }}
+
+    - name: Optional wait before rebooting the next batch of nodes
+      ansible.builtin.pause:
+        seconds: "{{ wait_seconds_after_reboot | int }}"
+      when: >-
+        concurrent_reboots is defined and
+        wait_seconds_after_reboot is defined and
+        wait_seconds_after_reboot | int > 0

requirements.in

@@ -1,5 +1,5 @@
-ansible-core>=2.16.2
-jmespath>=1.0.1
+ansible-core>=2.19.11
+jmespath>=1.1.0
 jsonpatch>=1.33
 kubernetes>=29.0.0
 molecule-plugins[vagrant]

requirements.txt

@@ -2,11 +2,11 @@
 # This file is autogenerated by pip-compile with Python 3.11
 # by the following command:
 #
-#    pip-compile requirements.in
+#    pip-compile --output-file=requirements.txt requirements.in
 #
 ansible-compat==4.1.11
     # via molecule
-ansible-core==2.18.0
+ansible-core==2.19.11
     # via
     #   -r requirements.in
     #   ansible-compat
@@ -53,7 +53,7 @@ jinja2==3.1.3
     # via
     #   ansible-core
     #   molecule
-jmespath==1.0.1
+jmespath==1.1.0
     # via -r requirements.in
 jsonpatch==1.33
     # via -r requirements.in
@@ -77,7 +77,7 @@ molecule==6.0.3
     # via
     #   -r requirements.in
     #   molecule-plugins
-molecule-plugins[vagrant]==23.5.3
+molecule-plugins[vagrant]==23.6.0
     # via -r requirements.in
 netaddr==0.10.1
     # via -r requirements.in
@@ -146,7 +146,7 @@ rsa==4.9
     # via google-auth
 ruamel-yaml==0.18.5
     # via pre-commit-hooks
-ruamel-yaml-clib==0.2.8
+ruamel-yaml-clib==0.2.15
     # via ruamel-yaml
 six==1.16.0
     # via

roles/k3s_agent/templates/k3s.service.j2

@@ -11,7 +11,7 @@ ExecStartPre=-/sbin/modprobe overlay
 ExecStart=/usr/local/bin/k3s agent \
   --server https://{{ apiserver_endpoint | ansible.utils.ipwrap }}:6443 \
   {% if is_pxe_booted | default(false) %}--snapshotter native \
-  {% endif %}--token {{ hostvars[groups[group_name_master | default('master')][0]]['token'] | default(k3s_token) }} \
+  {% endif %}--token {{ hostvars[groups[group_name_master | default('master')][0]]['token'] | default(k3s_token) | replace( '%', '%%' ) }} \
   {{ extra_agent_args }}
 KillMode=process
 Delegate=yes

roles/k3s_server/defaults/main.yml

@@ -23,6 +23,12 @@ metal_lb_controller_tag_version: v0.16.0
 metal_lb_speaker_tag_version: v0.16.0
 metal_lb_type: native
 
+# Shared retry/delay for remote manifest and asset downloads. The CI runner's
+# resolver intermittently times out on GitHub-hosted domains (helm.cilium.io,
+# raw.githubusercontent.com, github.com), so retry transient DNS/network failures.
+download_retries: 5
+download_delay: 10
+
 retry_count: 20
 
 # yamllint disable rule:line-length
@@ -33,7 +39,7 @@ server_init_args: >-
     {% else %}
       --server https://{{ hostvars[groups[group_name_master | default('master')][0]].k3s_node_ip | split(",") | first | ansible.utils.ipwrap }}:6443
     {% endif %}
-    --token {{ k3s_token }}
+    --token {{ k3s_token | replace('%', '%%') }}
   {% endif %}
   {{ extra_server_args }}
 

roles/k3s_server/tasks/kube-vip.yml

@@ -15,6 +15,10 @@
     owner: root
     group: root
     mode: "0644"
+  register: kube_vip_manifest_download
+  retries: "{{ download_retries }}"
+  delay: "{{ download_delay }}"
+  until: kube_vip_manifest_download is succeeded
   when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
 
 - name: Copy kubevip configMap manifest to first master

roles/k3s_server/tasks/main.yml

@@ -27,7 +27,11 @@
 - name: Deploy metallb manifest
   ansible.builtin.include_tasks: metallb.yml
   tags: metallb
-  when: kube_vip_lb_ip_range is not defined and (not cilium_bgp or cilium_iface is not defined)
+  # Deploy MetalLB unless kube-vip owns the load balancer IP range, or Cilium
+  # BGP is enabled (Cilium then provides its own load balancing). The cilium_bgp
+  # default keeps this safe when Cilium variables are not in scope at all (#644)
+  # while still deploying MetalLB when a non-BGP Cilium CNI is in use.
+  when: kube_vip_lb_ip_range is not defined and not (cilium_bgp | default(false) | bool)
 
 - name: Deploy kube-vip manifest
   ansible.builtin.include_tasks: kube-vip.yml
@@ -118,7 +122,10 @@
   always:
     - name: Save logs of k3s-init.service
       ansible.builtin.include_tasks: fetch_k3s_init_logs.yml
-      when: log_destination
+      # ANSIBLE_K3S_LOG_DIR is a path string when set; evaluate it as a boolean
+      # so the conditional is a real boolean (ansible-core 2.19 rejects string
+      # conditionals derived from env vars).
+      when: log_destination | default('') != ''
       vars:
         log_destination: >-
           {{ lookup('ansible.builtin.env', 'ANSIBLE_K3S_LOG_DIR', default=False) }}

roles/k3s_server/tasks/metallb.yml

@@ -15,6 +15,10 @@
     owner: root
     group: root
     mode: "0644"
+  register: metallb_manifest_download
+  retries: "{{ download_retries }}"
+  delay: "{{ download_delay }}"
+  until: metallb_manifest_download is succeeded
   when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
 
 - name: Set image versions in manifest for metallb-{{ metal_lb_type }}

roles/k3s_server_post/defaults/main.yml

@@ -23,6 +23,13 @@ cilium_mode: native
 cilium_tag: v1.20.0
 cilium_cli_tag: v0.19.7
 
+# Shared retry/delay for remote manifest, asset downloads, and waiting on
+# Kubernetes resources. The CI runner's resolver intermittently times out on
+# GitHub-hosted domains and the kube API can transiently return
+# ServiceUnavailable, so retry transient DNS/network/API failures.
+download_retries: 5
+download_delay: 10
+
 cluster_cidr: 10.52.0.0/16
 enable_bpf_masquerade: true
 kube_proxy_replacement: true

roles/k3s_server_post/tasks/calico.yml

@@ -18,6 +18,10 @@
         owner: root
         group: root
         mode: "0755"
+      register: calico_crd_download
+      retries: "{{ download_retries }}"
+      delay: "{{ download_delay }}"
+      until: calico_crd_download is succeeded
 
     - name: "Download to first master: manifest for Tigera Operator and Calico CRDs"
       ansible.builtin.get_url:
@@ -26,6 +30,10 @@
         owner: root
         group: root
         mode: "0755"
+      register: tigera_operator_download
+      retries: "{{ download_retries }}"
+      delay: "{{ download_delay }}"
+      until: tigera_operator_download is succeeded
 
     - name: Apply Calico CRD bundle with server-side apply
       ansible.builtin.command: >-

roles/k3s_server_post/tasks/cilium.yml

@@ -66,6 +66,10 @@
             - .tar.gz.sha256sum
           vars:
             cilium_base_url: https://github.com/cilium/cilium-cli/releases/download/{{ cilium_cli_tag }}
+          register: cilium_cli_download
+          retries: "{{ download_retries }}"
+          delay: "{{ download_delay }}"
+          until: cilium_cli_download is succeeded
 
         - name: Verify the downloaded tarball
           ansible.builtin.shell: |
@@ -182,6 +186,13 @@
         KUBECONFIG: "{{ ansible_user_dir }}/.kube/config"
       register: cilium_install_result
       changed_when: cilium_install_result.rc == 0
+      # cilium install/upgrade fetches the Helm chart from helm.cilium.io, which is
+      # fronted by GitHub Pages and can transiently fail DNS resolution through the
+      # host resolver (intermittent "lookup helm.cilium.io ... i/o timeout"). Retry
+      # so a transient name/network failure does not abort the whole converge play.
+      until: cilium_install_result.rc == 0
+      retries: "{{ download_retries }}"
+      delay: "{{ download_delay }}"
       when: cilium_installed.rc != 0 or cilium_needs_update
 
     - name: Wait for Cilium resources

roles/k3s_server_post/tasks/main.yml

@@ -12,7 +12,11 @@
 - name: Deploy metallb pool
   ansible.builtin.include_tasks: metallb.yml
   tags: metallb
-  when: kube_vip_lb_ip_range is not defined and (not cilium_bgp or cilium_iface is not defined)
+  # Deploy MetalLB unless kube-vip owns the load balancer IP range, or Cilium
+  # BGP is enabled (Cilium then provides its own load balancing). The cilium_bgp
+  # default keeps this safe when Cilium variables are not in scope at all (#644)
+  # while still deploying MetalLB when a non-BGP Cilium CNI is in use.
+  when: kube_vip_lb_ip_range is not defined and not (cilium_bgp | default(false) | bool)
 
 - name: Remove tmp directory used for manifests
   ansible.builtin.file:

roles/k3s_server_post/tasks/metallb.yml

@@ -54,6 +54,12 @@
     {% if item.condition | default(False) -%}{{ item.condition }}{%- endif %}
     --timeout='{{ metal_lb_available_timeout }}'
   changed_when: false
+  # The kube API can briefly return ServiceUnavailable while MetalLB converges,
+  # which would otherwise abort the whole converge play on a transient error.
+  register: metallb_wait_result
+  until: metallb_wait_result.rc == 0
+  retries: "{{ download_retries }}"
+  delay: "{{ download_delay }}"
   run_once: true
   with_items:
     - description: controller
@@ -112,6 +118,10 @@
   changed_when: false
   run_once: true
   when: metal_lb_mode == "layer2"
+  register: metallb_l2_test_result
+  until: metallb_l2_test_result.rc == 0
+  retries: "{{ download_retries }}"
+  delay: "{{ download_delay }}"
   with_items:
     - IPAddressPool
     - L2Advertisement
@@ -122,6 +132,10 @@
   changed_when: false
   run_once: true
   when: metal_lb_mode == "bgp"
+  register: metallb_bgp_test_result
+  until: metallb_bgp_test_result.rc == 0
+  retries: "{{ download_retries }}"
+  delay: "{{ download_delay }}"
   with_items:
     - IPAddressPool
     - BGPPeer

roles/prereq/tasks/main.yml

@@ -33,6 +33,27 @@
     reload: true
   tags: sysctl
 
+- name: Check if br_netfilter module exists
+  ansible.builtin.shell: |
+    set -o pipefail
+    find /lib/modules/$(uname -r) -name "br_netfilter.ko*" | wc -l
+  args:
+    executable: /bin/bash
+  register: br_netfilter_exists
+  changed_when: false
+  when: ansible_os_family == "RedHat"
+  tags: sysctl
+
+- name: Install kernel-modules-extra if br_netfilter missing
+  ansible.builtin.yum:
+    name: kernel-modules-extra
+    state: present
+    update_cache: true
+  register: kernel_modules_installed
+  when:
+    - ansible_os_family == "RedHat"
+    - br_netfilter_exists.stdout | int == 0
+
 - name: Add br_netfilter to /etc/modules-load.d/
   ansible.builtin.copy:
     content: br_netfilter