fix(metallb): retry the webhook endpoint check on transient kube API errors

- retry the webhook-service endpoint get until it succeeds
- extend the regression test to cover the webhook task too

.github/ISSUE_TEMPLATE.md

@@ -1,85 +0,0 @@
-
-<!-- It's a good idea to check this post first for general troubleshooting https://github.com/timothystewart6/k3s-ansible/discussions/19   -->
-
-<!--- Provide a general summary of the issue in the Title above -->
-
-## Expected Behavior
-
-<!--- Tell us what should happen -->
-
-## Current Behavior
-<!--- Tell us what happens instead of the expected behavior -->
-
-## Steps to Reproduce
-
-<!--- reproduce this bug. Include code to reproduce, if relevant -->
-
-1.
-2.
-3.
-4.
-
-## Context (variables)
-<!--- please include which OS, along with the variables used when running the playbook -->
-
-Operating system:
-
-Hardware:
-
-### Variables Used
-
-`all.yml`
-
-```yml
-k3s_version: ""
-ansible_user: NA
-systemd_dir: ""
-
-flannel_iface: ""
-
-#calico_iface: ""
-calico_ebpf: ""
-calico_cidr: ""
-calico_tag: ""
-
-apiserver_endpoint: ""
-
-k3s_token: "NA"
-
-extra_server_args: ""
-extra_agent_args: ""
-
-kube_vip_tag_version: ""
-
-kube_vip_cloud_provider_tag_version: ""
-kube_vip_lb_ip_range: ""
-
-metal_lb_speaker_tag_version: ""
-metal_lb_controller_tag_version: ""
-
-metal_lb_ip_range: ""
-```
-
-### Hosts
-
-`host.ini`
-
-```ini
-[master]
-IP.ADDRESS.ONE
-IP.ADDRESS.TWO
-IP.ADDRESS.THREE
-
-[node]
-IP.ADDRESS.FOUR
-IP.ADDRESS.FIVE
-
-[k3s_cluster:children]
-master
-node
-```
-
-## Possible Solution
-<!--- Not obligatory, but suggest a fix/reason for the bug, -->
-
-- [ ] I've checked the [General Troubleshooting Guide](https://github.com/timothystewart6/k3s-ansible/discussions/20)

.github/ISSUE_TEMPLATE/bug_report.yml

@@ -0,0 +1,102 @@
+---
+name: Bug report
+description: Report a reproducible problem with the playbooks, roles, or generated resources.
+title: "[Bug]: "
+labels:
+  - bug
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Thanks for reporting a problem. Search existing issues and review the troubleshooting link first.
+        Remove credentials, tokens, public IP addresses, and private hostnames from all fields and logs.
+
+  - type: checkboxes
+    id: prerequisites
+    attributes:
+      label: Prerequisites
+      options:
+        - label: I searched existing issues and discussions for this problem.
+          required: true
+        - label: I reviewed the troubleshooting guidance linked from the issue chooser.
+          required: true
+        - label: I removed secrets and identifying infrastructure details from this report.
+          required: true
+
+  - type: textarea
+    id: summary
+    attributes:
+      label: Problem summary
+      description: Describe what failed and its impact.
+      placeholder: A concise description of the problem and affected nodes or components.
+    validations:
+      required: true
+
+  - type: textarea
+    id: expected
+    attributes:
+      label: Expected behavior
+      description: What should have happened?
+    validations:
+      required: true
+
+  - type: textarea
+    id: reproduction
+    attributes:
+      label: Steps to reproduce
+      description: Provide the smallest reliable sequence that reproduces the issue.
+      placeholder: |
+        1. Configure ...
+        2. Run ...
+        3. Observe ...
+    validations:
+      required: true
+
+  - type: input
+    id: revision
+    attributes:
+      label: Repository revision
+      description: Release, tag, branch, or commit SHA used.
+      placeholder: v1.36.2+k3s1+tt1 or a commit SHA
+    validations:
+      required: true
+
+  - type: input
+    id: ansible-version
+    attributes:
+      label: Ansible version
+      description: Output of `ansible --version`, shortened to version and Python details.
+      placeholder: ansible-core 2.18.0, Python 3.12
+    validations:
+      required: true
+
+  - type: textarea
+    id: environment
+    attributes:
+      label: Environment
+      description: Include target OS and version, architecture, node counts, platform, and network provider.
+      placeholder: Debian 13, amd64, 3 control nodes and 2 agents, bare metal, Cilium
+    validations:
+      required: true
+
+  - type: textarea
+    id: configuration
+    attributes:
+      label: Relevant sanitized configuration
+      description: Include only variables and inventory groups needed to reproduce the problem.
+      render: yaml
+    validations:
+      required: true
+
+  - type: textarea
+    id: logs
+    attributes:
+      label: Relevant logs or task output
+      description: Include the failing task and surrounding output. Redact sensitive or identifying values.
+      render: shell
+
+  - type: textarea
+    id: context
+    attributes:
+      label: Additional context
+      description: Add attempted fixes, suspected causes, regressions, or other useful context.

.github/ISSUE_TEMPLATE/config.yml

@@ -0,0 +1,9 @@
+---
+blank_issues_enabled: false
+contact_links:
+  - name: Troubleshooting and support
+    url: https://github.com/timothystewart6/k3s-ansible/discussions/20
+    about: Review common troubleshooting guidance and ask configuration or usage questions.
+  - name: General discussions
+    url: https://github.com/timothystewart6/k3s-ansible/discussions
+    about: Discuss ideas and questions that are not confirmed bugs or concrete feature requests.

.github/ISSUE_TEMPLATE/feature_request.yml

@@ -0,0 +1,55 @@
+---
+name: Feature request
+description: Propose a focused improvement to supported repository behavior.
+title: "[Feature]: "
+labels:
+  - enhancement
+body:
+  - type: markdown
+    attributes:
+      value: |
+        Describe the use case before proposing an implementation. Search existing issues and discussions first.
+
+  - type: checkboxes
+    id: prerequisites
+    attributes:
+      label: Prerequisites
+      options:
+        - label: I searched existing issues and discussions for this request.
+          required: true
+        - label: This request is about reusable project behavior, not support for one private environment.
+          required: true
+
+  - type: textarea
+    id: problem
+    attributes:
+      label: Problem or use case
+      description: What limitation exists, who encounters it, and why does it matter?
+    validations:
+      required: true
+
+  - type: textarea
+    id: proposal
+    attributes:
+      label: Proposed behavior
+      description: Describe the desired user-visible result. Include example variables or commands when useful.
+    validations:
+      required: true
+
+  - type: textarea
+    id: alternatives
+    attributes:
+      label: Alternatives considered
+      description: Describe workarounds or other designs and their tradeoffs.
+
+  - type: textarea
+    id: compatibility
+    attributes:
+      label: Compatibility and operational impact
+      description: Note affected operating systems, architectures, CNIs, existing clusters, or reset behavior.
+
+  - type: textarea
+    id: context
+    attributes:
+      label: Additional context
+      description: Add relevant upstream documentation, examples, or prior discussion.

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,15 +1,46 @@
-# Proposed Changes
-<!--- Provide a general summary of your changes -->
+## Summary
+
+<!-- Explain the problem and the resulting behavior. Keep implementation details in the sections below. -->
+
+## Changes
 
--
--
 -
 
-## Checklist
+## Related issues
 
-- [ ] Tested locally
-- [ ] Ran `site.yml` playbook
-- [ ] Ran `reset.yml` playbook
-- [ ] Did not add any unnecessary changes
-- [ ] Ran pre-commit install at least once before committing
-- [ ] 🚀
+<!-- Use "Fixes #123" when this pull request should close an issue. Write "None" when not applicable. -->
+
+## Testing
+
+<!-- List exact commands, scenarios, and relevant manual checks. Do not check a box for a test that was not run. -->
+
+- [ ] `pre-commit run --all-files`
+- [ ] Relevant Ansible syntax checks
+- [ ] Relevant focused regression tests
+- [ ] Relevant Molecule scenario
+- [ ] Provisioning tested against a non-production cluster
+- [ ] Reset behavior tested against a non-production cluster
+
+Not run, with reason:
+
+## Risk and compatibility
+
+<!-- Cover existing clusters, upgrades, networking, supported platforms, security, and rollback. Write "None" when a
+category is not affected. -->
+
+- Existing cluster or upgrade impact:
+- Networking or CNI impact:
+- Security impact:
+- Rollback plan:
+
+## Documentation
+
+<!-- Identify updated docs and sample configuration, or explain why no documentation change is needed. -->
+
+## Final checklist
+
+- [ ] The change is focused and contains no unrelated edits.
+- [ ] Tests cover new behavior or a regression, where applicable.
+- [ ] User-facing variables are documented in role defaults, sample inventory, and the README.
+- [ ] Logs, examples, and configuration contain no secrets or identifying infrastructure details.
+- [ ] Generated files and local environment files are not included.

.github/copilot-instructions.md

@@ -0,0 +1,6 @@
+# GitHub Copilot instructions
+
+Read and follow the repository's root-level `AGENTS.md` before proposing or making changes. It is the canonical guide
+for architecture, safety, implementation, validation, and documentation expectations.
+
+Do not duplicate repository guidance here. If instructions need to change, update `AGENTS.md`.

.github/scripts/test-cilium-envoy-toggle.py

@@ -0,0 +1,120 @@
+#!/usr/bin/env python3
+"""Regression test for the Cilium Envoy toggle.
+
+The `cilium_envoy` variable lets users enable or disable the Cilium Envoy
+proxy. The Install/upgrade Cilium task in
+roles/k3s_server_post/tasks/cilium.yml passes the value through to Helm as
+`envoy.enabled`. This test:
+
+  - loads the real "Install Cilium" task and confirms the install/upgrade
+    command actually contains the `envoy.enabled` Helm value,
+  - renders the conditional that computes the Helm value and confirms it
+    produces `true` when cilium_envoy is enabled and `false` when disabled,
+  - confirms the task stays forward/backward compatible (no raw `true` /
+    `false` hardcoded in place of the conditional).
+"""
+
+from __future__ import print_function
+
+import os
+import re
+import subprocess
+
+import yaml
+from jinja2 import Environment
+
+ENVOY_EXPRESSION = '{{ "true" if cilium_envoy else "false" }}'
+
+
+def repo_root():
+    return subprocess.check_output(
+        ["git", "rev-parse", "--show-toplevel"], text=True
+    ).strip()
+
+
+def fail(message):
+    raise SystemExit("Cilium Envoy toggle test failed: " + message)
+
+
+def extract_install_command(path):
+    """Return the command string for the 'Install Cilium' task.
+
+    Walks both top-level tasks and tasks nested inside a `block`/`always`/
+    `rescue` list, since the Cilium deploy steps are grouped under the
+    'Prepare Cilium CLI on first master and deploy CNI' block.
+    """
+    with open(path, encoding="utf-8") as handle:
+        doc = yaml.safe_load(handle)
+
+    def find_command(tasks):
+        for task in tasks:
+            if not isinstance(task, dict):
+                continue
+            if task.get("name") == "Install Cilium":
+                command = task.get("ansible.builtin.command")
+                if command is None:
+                    raise SystemExit(
+                        "Cilium Envoy toggle test failed: "
+                        "'Install Cilium' task has no ansible.builtin.command"
+                    )
+                return command
+            # Recurse into block/always/rescue sub-lists.
+            for key in ("block", "always", "rescue"):
+                nested = task.get(key)
+                if isinstance(nested, list):
+                    found = find_command(nested)
+                    if found is not None:
+                        return found
+        return None
+
+    command = find_command(doc)
+    if command is None:
+        raise SystemExit(
+            "Cilium Envoy toggle test failed: could not find 'Install Cilium' task"
+        )
+    return command
+
+
+def assert_envoy_in_command(command):
+    if "envoy.enabled" not in command:
+        fail("install command is missing --helm-set envoy.enabled")
+    if ENVOY_EXPRESSION not in command:
+        fail(
+            "install command does not use the cilium_envoy conditional: "
+            "expected {0!r}".format(ENVOY_EXPRESSION)
+        )
+    # The conditional must be a WYSIWYG helm-set value, not a pre-rendered
+    # true/false literal (which would ignore the cilium_envoy variable).
+    if re.search(r"--helm-set envoy\.enabled=true(?:$|\s)", command):
+        fail("install command hardcodes envoy.enabled=true")
+    if re.search(r"--helm-set envoy\.enabled=false(?:$|\s)", command):
+        fail("install command hardcodes envoy.enabled=false")
+
+
+def assert_render():
+    env = Environment()
+
+    def render_for(value):
+        template = env.from_string(ENVOY_EXPRESSION)
+        return template.render(cilium_envoy=value)
+
+    if render_for(True) != "true":
+        fail("envoy conditional did not render 'true' when enabled")
+    if render_for(False) != "false":
+        fail("envoy conditional did not render 'false' when disabled")
+
+
+def main():
+    root = repo_root()
+    cilium_tasks = os.path.join(
+        root, "roles", "k3s_server_post", "tasks", "cilium.yml"
+    )
+    command = extract_install_command(cilium_tasks)
+    assert_envoy_in_command(command)
+    assert_render()
+
+    print("Cilium Envoy toggle regression test passed")
+
+
+if __name__ == "__main__":
+    main()

.github/scripts/test-default-interface.py

@@ -0,0 +1,90 @@
+#!/usr/bin/env python3
+"""Assert the sample inventory resolves the flannel interface per host.
+
+flannel_iface defaults to the host's default IPv4 interface rather than a
+hardcoded eth0. This test extracts the flannel_iface expression from the
+sample inventory and proves that a host whose primary interface is not named
+eth0 (e.g. enp1s0, ens3) resolves the interface from ansible facts.
+"""
+
+from __future__ import print_function
+
+import os
+import re
+import subprocess
+
+from jinja2 import Environment, StrictUndefined
+
+
+def repo_root():
+    return subprocess.check_output(
+        ["git", "rev-parse", "--show-toplevel"], text=True
+    ).strip()
+
+
+def fail(message):
+    raise SystemExit("default-interface test failed: " + message)
+
+
+class FakeAnsibleFacts(object):
+    """Stand-in for the per-host ``ansible_facts`` dict."""
+
+    def __init__(self, default_iface, iface_ip):
+        self._default = {"interface": default_iface, "address": iface_ip}
+        self._ifaces = {
+            default_iface: {"ipv4": {"address": iface_ip}},
+        }
+
+    @property
+    def default_ipv4(self):
+        return self._default
+
+    def __getitem__(self, key):
+        return self._ifaces[key]
+
+
+def read_all_yml(root):
+    path = os.path.join(root, "inventory", "sample", "group_vars", "all.yml")
+    with open(path, "r") as handle:
+        return handle.read()
+
+
+def extract_value(content, key):
+    # Match a quoted value assigned to the key, e.g. flannel_iface: "...".
+    match = re.search(r"^%s:\s*\"(.+)\"\s*$" % re.escape(key), content, re.M)
+    if not match:
+        fail("could not find %s in the sample inventory" % key)
+    return match.group(1)
+
+
+def resolve(env, expression, facts):
+    template = env.from_string(expression)
+    return template.render(ansible_facts=facts)
+
+
+def main():
+    root = repo_root()
+    content = read_all_yml(root)
+    env = Environment(undefined=StrictUndefined)
+
+    flannel_expr = extract_value(content, "flannel_iface")
+    if "default_ipv4.interface" not in flannel_expr:
+        fail("flannel_iface no longer defaults from ansible facts")
+
+    # A host whose primary interface is enp1s0 (the core #621 scenario).
+    facts = FakeAnsibleFacts("enp1s0", "192.168.30.11")
+    resolved = resolve(env, flannel_expr, facts)
+    if resolved != "enp1s0":
+        fail("flannel_iface resolved to %r, expected enp1s0" % resolved)
+
+    # A different host with a different interface must resolve independently.
+    facts2 = FakeAnsibleFacts("ens3", "192.168.30.12")
+    resolved2 = resolve(env, flannel_expr, facts2)
+    if resolved2 != "ens3":
+        fail("flannel_iface resolved to %r, expected ens3" % resolved2)
+
+    print("default-interface regression test passed")
+
+
+if __name__ == "__main__":
+    main()

.github/scripts/test-disable-swap.sh

@@ -0,0 +1,37 @@
+#!/usr/bin/env bash
+
+set -Eeuo pipefail
+
+repo_root="$(git rev-parse --show-toplevel)"
+prereq_defaults="$repo_root/roles/prereq/defaults/main.yml"
+prereq_tasks="$repo_root/roles/prereq/tasks/main.yml"
+
+# #670: k3s recommends swap be disabled on all nodes. The prereq role must expose
+# a disable_swap toggle (defaulting to true) that turns swap off now and comments
+# out the /etc/fstab swap entries so swap stays off across reboots.
+grep -Eq -- '^disable_swap: true' "$prereq_defaults" || {
+  printf 'prereq defaults are missing disable_swap: true\n' >&2
+  exit 1
+}
+
+grep -Fq -- 'Disable swap on all cluster nodes' "$prereq_tasks" || {
+  printf 'prereq tasks are missing the swap-disable block\n' >&2
+  exit 1
+}
+
+grep -Fq -- 'swapoff -a' "$prereq_tasks" || {
+  printf 'swap-disable block does not run swapoff -a\n' >&2
+  exit 1
+}
+
+grep -Fq -- '/etc/fstab' "$prereq_tasks" || {
+  printf 'swap-disable block does not comment out /etc/fstab swap entries\n' >&2
+  exit 1
+}
+
+if ! grep -Eq -- 'when: disable_swap' "$prereq_tasks"; then
+  printf 'swap-disable block is not gated on the disable_swap toggle\n' >&2
+  exit 1
+fi
+
+printf 'Swap disable regression test passed\n'

.github/scripts/test-kube-vip-manifest.py

@@ -123,6 +123,38 @@ def main():
     if "name: bgp_peers" in output:
         fail("bgp_peers present even though the peer list is empty")
 
+    # kube_vip_endpoint defaults to null (defined in role defaults): the
+    # address and subnet must fall back to the apiserver endpoint. default()
+    # without a truthy flag does NOT fall back on null, only on undefined, so
+    # this case pins the null runtime condition to prevent that regression.
+    output = render(
+        env,
+        {
+            "_kube_vip_bgp_peers": [],
+            "kube_vip_endpoint": None,
+            "kube_vip_arp": True,
+            "kube_vip_bgp": False,
+        },
+    )
+    if "value: 192.168.30.222" not in output:
+        fail("null kube_vip_endpoint does not fall back to apiserver_endpoint")
+
+    # kube_vip_endpoint set: overrides the internal listening address AND the
+    # subnet derivation while the advertised apiserver_endpoint stays separate.
+    output = render(
+        env,
+        {
+            "_kube_vip_bgp_peers": [],
+            "kube_vip_endpoint": "10.66.1.5",
+            "kube_vip_arp": True,
+            "kube_vip_bgp": False,
+        },
+    )
+    if "value: 10.66.1.5" not in output:
+        fail("kube_vip_endpoint did not override the address")
+    if "value: 192.168.30.222" in output:
+        fail("apiserver_endpoint leaked into address when kube_vip_endpoint set")
+
     print("kube-vip manifest regression test passed")
 
 

.github/scripts/test-metallb-deploy-condition.py

@@ -0,0 +1,170 @@
+#!/usr/bin/env python3
+"""Regression test for the MetalLB deploy conditions.
+
+The MetalLB manifest (roles/k3s_server/tasks/main.yml) and the MetalLB pool
+(roles/k3s_server_post/tasks/main.yml) are included under a `when` condition
+that decides whether MetalLB provides load balancing. A previous change (#683)
+guarded `cilium_bgp` but accidentally skipped MetalLB whenever a non-BGP
+Cilium CNI was in use (`cilium_iface` defined), breaking the cilium + MetalLB
+scenario.
+
+This test loads the real `when` expressions from both task files and evaluates
+them against representative variable sets, asserting MetalLB is deployed in
+every topology except when kube-vip owns the VIP range or Cilium BGP is enabled.
+"""
+
+from __future__ import print_function
+
+import os
+import re
+import subprocess
+
+import yaml
+from jinja2 import Environment
+
+METALLB_WHEN = (
+    "kube_vip_lb_ip_range is not defined and "
+    "not (cilium_bgp | default(false) | bool)"
+)
+
+
+def repo_root():
+    return subprocess.check_output(
+        ["git", "rev-parse", "--show-toplevel"], text=True
+    ).strip()
+
+
+def fail(message):
+    raise SystemExit("MetalLB deploy condition test failed: " + message)
+
+
+def extract_when(path, task_name):
+    """Return the `when:` expression string for the named task."""
+    with open(path, encoding="utf-8") as handle:
+        doc = yaml.safe_load(handle)
+    for task in doc:
+        if task.get("name") == task_name:
+            when = task.get("when")
+            return (when or "").strip()
+    return None
+
+
+def evaluate(when, variables):
+    """Evaluate a `when` expression against variables using Jinja2."""
+    env = Environment()
+
+    def fake_bool(value):
+        # Minimal stand-in for Ansible's truthiness filter used by `| bool`.
+        if isinstance(value, bool):
+            return value
+        if value is None:
+            return False
+        return str(value).lower() in ("1", "true", "yes", "on")
+
+    env.filters["bool"] = fake_bool
+    template = env.from_string("{{ " + when + " }}")
+    rendered = template.render(**variables)
+    # The expression renders to the literal strings "True"/"False".
+    if rendered == "True":
+        return True
+    if rendered == "False":
+        return False
+    fail("condition did not render to a boolean: {0!r}".format(rendered))
+
+
+def assert_deployment(when, variables, expected, label):
+    result = evaluate(when, variables)
+    verdict = "deploy" if result else "skip"
+    expected_verdict = "deploy" if expected else "skip"
+    if result != expected:
+        fail(
+            "{0}: expected to {1} MetalLB but the condition chose to {2} "
+            "(vars: {3})".format(label, expected_verdict, verdict, variables)
+        )
+
+
+def scenarios():
+    """Yield (variables, expected_deploy, label) pairs."""
+    yield (
+        # Default Flannel inventory (all.yml sets cilium_bgp: false).
+        {
+            "cilium_bgp": False,
+            "cilium_iface": None,
+        },
+        True,
+        "flannel default (cilium_bgp: false)",
+    )
+    yield (
+        # Calico CNI with no Cilium variable in scope (issue #644): cilium_bgp
+        # is genuinely undefined, so `default(false)` must keep MetalLB on.
+        {
+            "calico_iface": "eth1",
+        },
+        True,
+        "calico, cilium_bgp undefined (#644)",
+    )
+    yield (
+        # Cilium CNI with BGP disabled: MetalLB must still be deployed.
+        {
+            "cilium_bgp": False,
+            "cilium_iface": "eth1",
+        },
+        True,
+        "cilium non-BGP (regression catch)",
+    )
+    yield (
+        # Cilium CNI with BGP enabled: Cilium provides the LB, skip MetalLB.
+        {
+            "cilium_bgp": True,
+            "cilium_iface": "eth1",
+        },
+        False,
+        "cilium BGP enabled",
+    )
+    yield (
+        # kube-vip is the load balancer provider: skip MetalLB.
+        {
+            "kube_vip_lb_ip_range": "192.168.30.80-192.168.30.90",
+            "cilium_bgp": False,
+        },
+        False,
+        "kube-vip owns the VIP range",
+    )
+
+
+def main():
+    root = repo_root()
+    server_tasks = os.path.join(root, "roles", "k3s_server", "tasks", "main.yml")
+    server_post_tasks = os.path.join(
+        root, "roles", "k3s_server_post", "tasks", "main.yml"
+    )
+
+    server_when = extract_when(server_tasks, "Deploy metallb manifest")
+    server_post_when = extract_when(server_post_tasks, "Deploy metallb pool")
+
+    if server_when is None:
+        fail("could not find 'Deploy metallb manifest' when condition")
+    if server_post_when is None:
+        fail("could not find 'Deploy metallb pool' when condition")
+
+    for when, source in (
+        (server_when, "k3s_server/tasks/main.yml"),
+        (server_post_when, "k3s_server_post/tasks/main.yml"),
+    ):
+        if when != METALLB_WHEN:
+            fail(
+                "{0} when condition changed unexpectedly:\n"
+                "  expected: {1}\n  got:      {2}".format(source, METALLB_WHEN, when)
+            )
+
+    for variables, expected, label in scenarios():
+        assert_deployment(server_when, variables, expected, "server " + label)
+        assert_deployment(
+            server_post_when, variables, expected, "server_post " + label
+        )
+
+    print("MetalLB deploy condition regression test passed for all scenarios")
+
+
+if __name__ == "__main__":
+    main()

.github/scripts/test-metallb-interfaces.py

@@ -0,0 +1,88 @@
+#!/usr/bin/env python3
+"""Regression test for the MetalLB L2Advertisement interfaces.
+
+`metal_lb_interfaces` restricts which network interfaces MetalLB announces
+load balancer IPs on in layer2 mode. When the list is non-empty, the
+L2Advertisement in roles/k3s_server_post/templates/metallb.crs.j2 must render
+a `spec.interfaces` block; when it is empty (the default), no spec is rendered
+so MetalLB announces on all interfaces.
+
+This renders the template and asserts both cases plus the BGP path (which must
+not be affected by the L2 interfaces variable).
+"""
+
+from __future__ import print_function
+
+import os
+import subprocess
+
+from jinja2 import Environment, FileSystemLoader, StrictUndefined
+
+
+def repo_root():
+    return subprocess.check_output(
+        ["git", "rev-parse", "--show-toplevel"], text=True
+    ).strip()
+
+
+def fail(message):
+    raise SystemExit("MetalLB interfaces test failed: " + message)
+
+
+def render(env, extra_vars):
+    base_vars = {
+        "metal_lb_mode": "layer2",
+        "metal_lb_ip_range": "192.168.30.80-192.168.30.90",
+    }
+    base_vars.update(extra_vars)
+    template = env.get_template("metallb.crs.j2")
+    return template.render(**base_vars)
+
+
+def main():
+    root = repo_root()
+    template_dir = os.path.join(
+        root, "roles", "k3s_server_post", "templates"
+    )
+    env = Environment(
+        loader=FileSystemLoader(template_dir), undefined=StrictUndefined
+    )
+
+    # Empty list (default): no spec.interfaces in the L2Advertisement.
+    output = render(env, {"metal_lb_interfaces": []})
+    if "spec:\n  interfaces:" in output:
+        fail("spec.interfaces rendered with an empty metal_lb_interfaces")
+    if "kind: L2Advertisement" not in output:
+        fail("L2Advertisement missing in layer2 mode")
+
+    # Single interface.
+    output = render(env, {"metal_lb_interfaces": ["eth1"]})
+    if "spec:\n  interfaces:\n    - eth1" not in output:
+        fail("single interface was not rendered in spec.interfaces")
+
+    # Multiple interfaces.
+    output = render(env, {"metal_lb_interfaces": ["eth1", "eth2"]})
+    if "spec:\n  interfaces:\n    - eth1\n    - eth2" not in output:
+        fail("multiple interfaces were not rendered in spec.interfaces")
+
+    # BGP mode must not emit an L2Advertisement spec at all.
+    output = render(
+        env,
+        {
+            "metal_lb_mode": "bgp",
+            "metal_lb_interfaces": ["eth1"],
+            "metal_lb_bgp_my_asn": "64513",
+            "metal_lb_bgp_peer_asn": "64512",
+            "metal_lb_bgp_peer_address": "192.168.30.1",
+        },
+    )
+    if "kind: L2Advertisement" in output:
+        fail("L2Advertisement rendered in bgp mode")
+    if "interfaces:" in output:
+        fail("interfaces rendered in bgp mode")
+
+    print("MetalLB interfaces regression test passed")
+
+
+if __name__ == "__main__":
+    main()

.github/scripts/test-metallb-namespace.py

@@ -0,0 +1,108 @@
+#!/usr/bin/env python3
+"""Regression test for the MetalLB converge checks.
+
+The MetalLB tasks in roles/k3s_server_post/tasks/metallb.yml must actually
+verify resources through an explicit kubectl get, and must retry on a
+transient kube API error while MetalLB converges.
+
+The "Test metallb-system namespace" task previously ran `k3s kubectl -n
+metallb-system` with no subcommand, which only printed a usage page and always
+exited 0, so it always succeeded even when the namespace did not exist (issue
+#350). It must instead run an explicit `get namespace metallb-system`, which
+returns non-zero when the namespace is absent.
+
+An explicit get actually contacts the API server, so these tasks need the same
+retry wiring as their siblings (register, until rc == 0, retries, delay). A
+bare get with no retry would otherwise abort the converge play on a transient
+kube API error while MetalLB converges.
+"""
+
+from __future__ import print_function
+
+import os
+import subprocess
+
+import yaml
+
+
+def repo_root():
+    return subprocess.check_output(
+        ["git", "rev-parse", "--show-toplevel"], text=True
+    ).strip()
+
+
+def fail(message):
+    raise SystemExit("MetalLB namespace test failed: " + message)
+
+
+def find_task(tasks, name):
+    for entry in tasks:
+        if entry.get("name") == name:
+            return entry
+    fail("could not find the '{0}' task".format(name))
+    return None
+
+
+def command_text(task):
+    cmd = task.get("ansible.builtin.command")
+    if not cmd:
+        cmd = task.get("command")
+    if not cmd:
+        fail("task does not use ansible.builtin.command")
+    return cmd if isinstance(cmd, str) else " ".join(cmd)
+
+
+def check_explicit_get(task, name, needle):
+    text = command_text(task)
+    if needle not in text:
+        fail(
+            "command does not run '{0}'; the task would only print usage and "
+            "never verify the resource (got: {1!r})".format(needle, text)
+        )
+
+
+def check_retry_wiring(task, name):
+    # The sibling k3s_server_post metallb tasks retry kubectl because the kube
+    # API can briefly be unavailable while MetalLB converges. Without the same
+    # retry, a transient API error aborts the whole converge play.
+    if not task.get("register"):
+        fail(
+            "{0} does not register a result; without retry wiring a transient "
+            "kube API error aborts the converge play".format(name)
+        )
+    if not isinstance(task.get("until"), str) or "rc == 0" not in task["until"]:
+        fail(
+            "{0} does not retry on rc == 0; the kube API can transiently fail "
+            "while MetalLB converges and abort the play".format(name)
+        )
+    if task.get("retries") is None:
+        fail("{0} is missing retries".format(name))
+    if task.get("delay") is None:
+        fail("{0} is missing delay".format(name))
+
+
+def main():
+    task_file = os.path.join(
+        repo_root(), "roles", "k3s_server_post", "tasks", "metallb.yml"
+    )
+    with open(task_file, encoding="utf-8") as handle:
+        tasks = yaml.safe_load(handle)
+
+    namespace_task = find_task(tasks, "Test metallb-system namespace")
+    # A bare `-n metallb-system` with no subcommand prints kubectl usage and
+    # always exits 0, so it never proves the namespace exists. The fix must
+    # use an explicit get.
+    check_explicit_get(namespace_task, "Test metallb-system namespace",
+                       "get namespace metallb-system")
+    check_retry_wiring(namespace_task, "Test metallb-system namespace")
+
+    webhook_task = find_task(tasks, "Test metallb-system webhook-service endpoint")
+    check_explicit_get(webhook_task, "Test metallb-system webhook-service endpoint",
+                       "get endpoints")
+    check_retry_wiring(webhook_task, "Test metallb-system webhook-service endpoint")
+
+    print("MetalLB namespace check regression test passed")
+
+
+if __name__ == "__main__":
+    main()

.github/scripts/test-unique-hostname-precheck.sh

@@ -0,0 +1,28 @@
+#!/usr/bin/env bash
+
+set -Eeuo pipefail
+
+repo_root="$(git rev-parse --show-toplevel)"
+site_play="$repo_root/site.yml"
+
+# #636: verify the "Pre tasks" play asserts that all k3s_cluster hosts have
+# unique hostnames, so a duplicate-hostname inventory fails fast instead of
+# silently breaking node registration/joining.
+grep -Fq -- 'Verify all cluster nodes have unique hostnames' "$site_play" || {
+  printf 'site.yml is missing the unique-hostname preflight check\n' >&2
+  exit 1
+}
+
+# The check must deduplicate the cluster hostname list via the `unique` filter
+# and compare lengths, i.e. groups['k3s_cluster'] must be referenced.
+grep -Fq -- "groups['k3s_cluster']" "$site_play" || {
+  printf 'unique-hostname check does not iterate the k3s_cluster group\n' >&2
+  exit 1
+}
+
+if ! grep -Eq -- 'cluster_hostnames.*\|.*unique|\| unique' "$site_play"; then
+  printf 'unique-hostname check does not deduplicate the hostname list\n' >&2
+  exit 1
+fi
+
+printf 'Unique hostname preflight regression test passed\n'

.github/workflows/ci.yml

@@ -14,6 +14,7 @@ on:
       - '**/FUNDING.yml'
       - '**/host.ini'
       - '**/*.md'
+      - '.github/ISSUE_TEMPLATE/**'
       - '**/.editorconfig'
       - '**/ansible.example.cfg'
       - '**/deploy.sh'
@@ -31,10 +32,13 @@ concurrency:
 
 jobs:
   pre:
+    if: github.actor != 'dependabot[bot]'
     uses: ./.github/workflows/cache.yml
     needs: [lint]
   lint:
+    if: github.actor != 'dependabot[bot]'
     uses: ./.github/workflows/lint.yml
   test:
+    if: github.actor != 'dependabot[bot]'
     uses: ./.github/workflows/test.yml
     needs: [pre, lint]

.github/workflows/lint.yml

@@ -68,7 +68,7 @@ jobs:
       - name: Checkout code
         uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # 7.0.1
       - name: Ensure SHA pinned actions
-        uses: zgosalvez/github-actions-ensure-sha-pinned-actions@3db98c0363e2fa5df3e1c4c471777a7c10b24cc9 # 5.0.5
+        uses: zgosalvez/github-actions-ensure-sha-pinned-actions@46cfe808a5f1588656ef299eedd0ce2fd7ec0dcc # 5.0.6
         with:
           allowlist: |
             aws-actions/

.github/workflows/test.yml

@@ -88,7 +88,7 @@ jobs:
           trap stop_monitor EXIT
           /usr/bin/time -v -o "$timing_file" \
             molecule test --scenario-name ${{ matrix.scenario }}
-        timeout-minutes: 150
+        timeout-minutes: 180
         env:
           ANSIBLE_K3S_LOG_DIR: ${{ runner.temp }}/logs/k3s-ansible/${{ matrix.scenario }}
           ANSIBLE_SSH_RETRIES: 4

.pre-commit-config.yaml

@@ -62,6 +62,18 @@ repos:
         language: system
         pass_filenames: false
         files: ^roles/k3s_server/tasks/(main|join_master)\.yml$|^\.github/scripts/test-k3s-server-bootstrap\.sh$
+      - id: unique-hostname-precheck-test
+        name: Unique hostname precheck test
+        entry: .github/scripts/test-unique-hostname-precheck.sh
+        language: system
+        pass_filenames: false
+        files: ^site\.yml$|^\.github/scripts/test-unique-hostname-precheck\.sh$
+      - id: disable-swap-test
+        name: Disable swap test
+        entry: .github/scripts/test-disable-swap.sh
+        language: system
+        pass_filenames: false
+        files: ^roles/prereq/(tasks/main|defaults/main)\.yml$|^\.github/scripts/test-disable-swap\.sh$
       - id: cilium-bgp-manifest-test
         name: Cilium BGP manifest test
         entry: python3 .github/scripts/test-cilium-bgp-manifest.py
@@ -70,6 +82,15 @@ repos:
           - Jinja2>=3.1
         pass_filenames: false
         files: ^roles/k3s_server_post/templates/cilium\.crs\.j2$|^\.github/scripts/test-cilium-bgp-manifest\.py$
+      - id: cilium-envoy-toggle-test
+        name: Cilium Envoy toggle test
+        entry: python3 .github/scripts/test-cilium-envoy-toggle.py
+        language: python
+        additional_dependencies:
+          - Jinja2>=3.1
+          - PyYAML
+        pass_filenames: false
+        files: ^roles/k3s_server_post/tasks/cilium\.yml$|^\.github/scripts/test-cilium-envoy-toggle\.py$
       - id: kube-vip-manifest-test
         name: kube-vip manifest test
         entry: python3 .github/scripts/test-kube-vip-manifest.py
@@ -84,3 +105,36 @@ repos:
         language: system
         pass_filenames: false
         files: ^roles/k3s_server/tasks/metallb\.yml$|^\.github/scripts/test-metallb-remote-read\.sh$
+      - id: metallb-interfaces-test
+        name: MetalLB interfaces test
+        entry: python3 .github/scripts/test-metallb-interfaces.py
+        language: python
+        additional_dependencies:
+          - Jinja2>=3.1
+        pass_filenames: false
+        files: ^roles/k3s_server_post/templates/metallb\.crs\.j2$|^\.github/scripts/test-metallb-interfaces\.py$
+      - id: metallb-namespace-test
+        name: MetalLB namespace test
+        entry: python3 .github/scripts/test-metallb-namespace.py
+        language: python
+        additional_dependencies:
+          - PyYAML
+        pass_filenames: false
+        files: ^roles/k3s_server_post/tasks/metallb\.yml$|^\.github/scripts/test-metallb-namespace\.py$
+      - id: metallb-deploy-condition-test
+        name: MetalLB deploy condition test
+        entry: python3 .github/scripts/test-metallb-deploy-condition.py
+        language: python
+        additional_dependencies:
+          - Jinja2>=3.1
+          - PyYAML
+        pass_filenames: false
+        files: ^roles/k3s_server/tasks/main\.yml$|^roles/k3s_server_post/tasks/main\.yml$|^\.github/scripts/test-metallb-deploy-condition\.py$ # noqa yaml[line-length]
+      - id: default-interface-test
+        name: default interface test
+        entry: python3 .github/scripts/test-default-interface.py
+        language: python
+        additional_dependencies:
+          - Jinja2>=3.1
+        pass_filenames: false
+        files: ^inventory/sample/group_vars/all\.yml$|^\.github/scripts/test-default-interface\.py$

AGENTS.md

@@ -0,0 +1,130 @@
+# Agent Guide
+
+This file is the canonical repository guide for coding agents and automated contributors. Read it before making
+changes. Human contributors should also review [CONTRIBUTING.md](CONTRIBUTING.md).
+
+## Project overview
+
+This repository is an Ansible collection that provisions and resets highly available k3s clusters. It supports
+multiple networking choices, including Flannel, Calico, Cilium, kube-vip, and MetalLB.
+
+The main entry points are:
+
+- `site.yml`: provision or update a cluster.
+- `reset.yml`: remove k3s from a cluster.
+- `reboot.yml`: reboot cluster nodes.
+- `inventory/sample/`: example inventory and variables.
+- `roles/`: reusable Ansible roles used by the playbooks.
+- `molecule/`: integration scenarios run by CI.
+- `.github/scripts/`: CI support scripts and focused regression tests.
+
+## Source of truth
+
+- Role defaults belong in `roles/<role>/defaults/main.yml`.
+- Tasks belong in `roles/<role>/tasks/` and handlers in `roles/<role>/handlers/`.
+- Example user configuration belongs in `inventory/sample/`.
+- User-facing setup and variable documentation belongs in `README.md`.
+- Contributor workflows and review expectations belong in `CONTRIBUTING.md`.
+- Agent-specific repository instructions belong in this file.
+
+Keep `CLAUDE.md` and `.github/copilot-instructions.md` as small pointers to this file. Do not duplicate these
+instructions in tool-specific files.
+
+## Development setup
+
+Use a Python virtual environment. Do not commit the environment, generated logs, inventories, kubeconfigs, or
+credentials.
+
+```bash
+python3 -m venv .env
+source .env/bin/activate
+python3 -m pip install -r requirements.txt
+ansible-galaxy collection install -r collections/requirements.yml
+pre-commit install
+```
+
+`ansible.cfg` is intentionally ignored. Copy `ansible.example.cfg` when local configuration is needed.
+
+## Working rules
+
+1. Inspect the current branch and worktree before editing. Preserve unrelated user changes.
+2. Keep changes focused. Avoid drive-by formatting or dependency updates.
+3. Never add real IP addresses, hostnames, tokens, private keys, kubeconfigs, or inventory secrets.
+4. Use placeholders in examples and redact sensitive values from logs and issue reports.
+5. Preserve idempotence. An already-converged host should not report changes without a real state transition.
+6. Prefer Ansible modules over `ansible.builtin.command` or `ansible.builtin.shell`. When a command is required,
+   define accurate `changed_when` and `failed_when` behavior.
+7. Use fully qualified collection names, such as `ansible.builtin.copy`.
+8. Put configurable values in role defaults or inventory variables. Avoid embedding environment-specific values in
+   tasks and templates.
+9. Maintain compatibility with the operating systems and architectures listed in `README.md`.
+10. Do not weaken lint rules, tests, or CI checks to make a change pass.
+
+## Change guidance
+
+### Ansible tasks and roles
+
+- Use descriptive task names in sentence case.
+- Use YAML booleans (`true` and `false`) rather than aliases.
+- Quote file modes, for example `mode: "0644"`.
+- Notify handlers only when the managed resource changes.
+- Use `become: true` only where privilege escalation is needed.
+- Update role defaults, sample inventory, and the README together when adding or renaming user-facing variables.
+- Check reset behavior when provisioning introduces persistent services, files, mounts, or network state.
+
+### Templates and manifests
+
+- Keep Jinja logic small and readable. Move complicated decisions into task variables where practical.
+- Render valid YAML after Jinja evaluation.
+- Preserve explicit handling for optional and undefined variables.
+- Add or update a focused test under `.github/scripts/` when changing generated Kubernetes manifests or bootstrap
+  behavior.
+
+### Molecule scenarios
+
+- Reuse `molecule/resources/` for shared behavior.
+- Put scenario-specific inputs in `molecule/<scenario>/overrides.yml` and `verify-vars.yml`.
+- Update `molecule/README.md` when adding, removing, or materially changing a scenario.
+- Clean up resources created by tests, including failure paths.
+
+## Validation
+
+Run the smallest relevant checks while iterating, then run the complete local validation before considering a change
+ready:
+
+```bash
+pre-commit run --all-files
+```
+
+For playbook or role changes, also run syntax checks with a non-sensitive inventory:
+
+```bash
+ansible-playbook site.yml --syntax-check -i inventory/sample/hosts.ini
+ansible-playbook reset.yml --syntax-check -i inventory/sample/hosts.ini
+```
+
+Run focused regression scripts when their related files change. The mapping is defined in
+`.pre-commit-config.yaml`.
+
+Molecule tests require Vagrant, VirtualBox, host-only networking, and substantial local resources. Run the most
+relevant scenario when that environment is available:
+
+```bash
+molecule test --scenario-name <scenario>
+```
+
+If a required test can't be run locally, state exactly which check was skipped and why. Never claim a check passed
+unless it was executed.
+
+## Documentation and review
+
+- Keep commands copyable and examples free of secrets.
+- Update documentation in the same change as user-visible behavior.
+- Explain behavior changes, compatibility concerns, operational risks, and rollback steps in the pull request.
+- Use conventional commit messages with a scope, for example `fix(k3s-server): handle an existing token safely`.
+- Do not commit, push, open a pull request, or modify remote resources unless the user explicitly requests it.
+
+## Definition of done
+
+A change is ready for review when it is focused, documented, linted, tested in proportion to its risk, and shown in a
+clean diff with no secrets or generated artifacts.

CLAUDE.md

@@ -0,0 +1,3 @@
+@AGENTS.md
+
+`AGENTS.md` is the canonical repository guide. Follow it for all work in this repository.

CONTRIBUTING.md

@@ -0,0 +1,94 @@
+# Contributing to k3s-ansible
+
+Thank you for improving k3s-ansible. Contributions should be focused, safe to apply to existing clusters, and tested
+in proportion to their operational impact.
+
+## Before opening an issue
+
+- Search existing issues and discussions for the same behavior.
+- Review the [troubleshooting discussion](https://github.com/timothystewart6/k3s-ansible/discussions/20).
+- Remove tokens, credentials, public IP addresses, private hostnames, and other sensitive values from logs and
+  configuration.
+- For support requests, include the k3s-ansible revision, Ansible version, target operating system, architecture,
+  network provider, relevant sanitized variables, and a minimal reproduction.
+
+Use the bug report template for reproducible defects and the feature request template for proposed behavior.
+
+## Development environment
+
+Fork and clone the repository, then create a branch from the latest `master`:
+
+```bash
+git switch master
+git pull --ff-only
+git switch -c <type>/<short-description>
+```
+
+Create a Python environment and install the pinned development dependencies:
+
+```bash
+python3 -m venv .env
+source .env/bin/activate
+python3 -m pip install -r requirements.txt
+ansible-galaxy collection install -r collections/requirements.yml
+pre-commit install
+```
+
+Copy `ansible.example.cfg` to the ignored `ansible.cfg` file if local Ansible configuration is needed. Start custom
+inventories from `inventory/sample/`, keep them out of Git, and never use production credentials in tests.
+
+## Making changes
+
+- Keep each pull request focused on one problem or feature.
+- Follow [AGENTS.md](AGENTS.md) for repository structure, implementation conventions, and safety requirements.
+- Preserve idempotence and existing-cluster compatibility.
+- Add or update tests for behavior changes and regressions.
+- Update role defaults, sample inventory, and documentation when user-facing variables change.
+- Consider both provisioning and reset behavior for persistent resources.
+- Avoid unrelated reformatting and generated files.
+
+## Validation
+
+Run all pre-commit checks before submitting a pull request:
+
+```bash
+pre-commit run --all-files
+```
+
+For playbook or role changes, run syntax checks:
+
+```bash
+ansible-playbook site.yml --syntax-check -i inventory/sample/hosts.ini
+ansible-playbook reset.yml --syntax-check -i inventory/sample/hosts.ini
+```
+
+Run the most relevant Molecule scenario when Vagrant, VirtualBox, and the required host networking are available:
+
+```bash
+molecule test --scenario-name <scenario>
+```
+
+See [molecule/README.md](molecule/README.md) for scenario details and local requirements. Pull requests should list
+every check that was run and clearly identify checks that could not be run locally.
+
+## Commits and pull requests
+
+Use a conventional commit subject with a scope:
+
+```text
+type(scope): short description
+```
+
+Common types are `feat`, `fix`, `refactor`, `test`, `docs`, and `chore`. Write subjects in the imperative mood and
+keep commits logically focused.
+
+Pull requests should:
+
+- Describe the problem and the resulting behavior.
+- Identify compatibility, security, networking, and upgrade risks.
+- Include testing evidence without sensitive data.
+- Call out documentation and sample configuration changes.
+- Link related issues with `Fixes #<issue>` when applicable.
+- Avoid checking boxes for tests that were not run.
+
+Maintainers may ask for a change to be split when unrelated work makes it difficult to review or roll back.

README.md

@@ -2,51 +2,70 @@
 
 ![Fully Automated K3S etcd High Availability Install](https://img.youtube.com/vi/CbkEWcUZ7zM/0.jpg)
 
-This playbook will build an HA Kubernetes cluster with `k3s`, `kube-vip` and MetalLB via `ansible`.
+This Ansible collection builds a highly available Kubernetes cluster with k3s. It supports kube-vip for the control
+plane virtual IP, multiple CNI options, and either MetalLB or kube-vip for service load balancing.
 
 This is based on the work from [this fork](https://github.com/212850a/k3s-ansible) which is based on the work from [k3s-io/k3s-ansible](https://github.com/k3s-io/k3s-ansible). It uses [kube-vip](https://kube-vip.io/) to create a load balancer for control plane, and [metal-lb](https://metallb.universe.tf/installation/) for its service `LoadBalancer`.
 
-If you want more context on how this works, see:
+For more context on how it works, see:
 
 📄 [Documentation](https://technotim.com/posts/k3s-etcd-ansible/) (including example commands)
 
 📺 [Watch the Video](https://www.youtube.com/watch?v=CbkEWcUZ7zM)
 
+## Project guides
+
+- [Getting started](#-getting-started)
+- [Configuration variables](#variables)
+- [Upgrading an existing cluster](#-upgrading-an-existing-cluster)
+- [Local Molecule testing](molecule/README.md)
+- [Contributing guidelines](CONTRIBUTING.md)
+- [Repository guide for coding agents](AGENTS.md)
+
 ## 📖 k3s Ansible Playbook
 
-Build a Kubernetes cluster using Ansible with k3s. The goal is easily install a HA Kubernetes cluster on machines running:
+Build a Kubernetes cluster using Ansible and k3s. The goal is to make a highly available cluster straightforward to
+install on machines running:
 
 - [x] Debian (tested on version 13)
 - [x] Ubuntu (tested on version 26.04 LTS)
 - [x] Rocky (tested on version 10)
 
-on processor architecture:
+Supported processor architectures are:
 
-- [X] x64
-- [X] arm64
-- [X] armhf
+- [x] x64
+- [x] arm64
+- [x] armhf
 
 ## ✅ System requirements
 
-- Control Node (the machine you are running `ansible` commands) must have Ansible 2.11+ If you need a quick primer on Ansible [you can check out my docs and setting up Ansible](https://technotim.com/posts/ansible-automation/).
+- The control node, which runs the Ansible commands, must have Ansible 2.11 or newer. For a quick primer, see
+  [setting up Ansible](https://technotim.com/posts/ansible-automation/).
 
-- You will also need to install collections that this playbook uses by running `ansible-galaxy collection install -r ./collections/requirements.yml` (important❗)
+- Install the required collections with
+  `ansible-galaxy collection install -r ./collections/requirements.yml`.
 
 - [`netaddr` package](https://pypi.org/project/netaddr/) must be available to Ansible. If you have installed Ansible via apt, this is already taken care of. If you have installed Ansible via `pip`, make sure to install `netaddr` into the respective virtual environment.
 
-- `server` and `agent` nodes should have passwordless SSH access, if not you can supply arguments to provide credentials `--ask-pass --ask-become-pass` to each command.
+- Server and agent nodes should support passwordless SSH access. Otherwise, pass `--ask-pass --ask-become-pass` to
+  each playbook command.
+
+- Every node in the cluster must have a **unique hostname**. k3s registers each node keyed by its hostname, so
+  two nodes with the same hostname cannot join the cluster. `site.yml` asserts this up front and fails fast if
+  any duplicate is found.
 
 ## 🚀 Getting Started
 
 ### 🍴 Preparation
 
-First create a new directory based on the `sample` directory within the `inventory` directory:
+Create a cluster-specific inventory from the sample. The `inventory/` directory ignores custom inventory content so
+credentials and environment details aren't committed accidentally.
 
 ```bash
 cp -R inventory/sample inventory/my-cluster
 ```
 
-Second, edit `inventory/my-cluster/hosts.ini` to match the system information gathered above
+Edit `inventory/my-cluster/hosts.ini` to match the target hosts.
 
 For example:
 
@@ -67,9 +86,10 @@ node
 
 If multiple hosts are in the master group, the playbook will automatically set up k3s in [HA mode with etcd](https://rancher.com/docs/k3s/latest/en/installation/ha-embedded/).
 
-Finally, copy `ansible.example.cfg` to `ansible.cfg` and adapt the inventory path to match the files that you just created.
+Copy `ansible.example.cfg` to `ansible.cfg`, then update its inventory path. The local `ansible.cfg` file is ignored by
+Git.
 
-This requires at least k3s version `1.19.1` however the version is configurable by using the `k3s_version` variable.
+The minimum k3s version is `1.19.1`. Select the desired version with the `k3s_version` variable.
 
 If needed, you can also edit `inventory/my-cluster/group_vars/all.yml` to match your environment.
 
@@ -81,7 +101,8 @@ Start provisioning of the cluster using the following command:
 ansible-playbook site.yml -i inventory/my-cluster/hosts.ini
 ```
 
-After deployment control plane will be accessible via virtual ip-address which is defined in inventory/group_vars/all.yml as `apiserver_endpoint`
+After deployment, the control plane is accessible through the virtual IP defined by `apiserver_endpoint` in the
+inventory variables.
 
 ### 🔥 Remove k3s cluster
 
@@ -89,7 +110,25 @@ After deployment control plane will be accessible via virtual ip-address which i
 ansible-playbook reset.yml -i inventory/my-cluster/hosts.ini
 ```
 
->You should also reboot these nodes due to the VIP not being destroyed
+> Reboot the nodes after reset because the virtual IP may remain configured.
+
+### ⏻️ Reboot Cluster Nodes
+
+Reboot all cluster nodes at once or stage the reboot across the cluster.
+
+```bash
+ansible-playbook reboot.yml -i inventory/my-cluster/hosts.ini
+```
+
+To reboot the nodes in batches, set `concurrent_reboots` to the number of nodes
+to reboot at a time (or a percentage). Optionally set `wait_seconds_after_reboot`
+to pause after each batch so pods in the freshly rebooted batch can settle
+before the next batch reboots.
+
+```bash
+ansible-playbook reboot.yml -i inventory/my-cluster/hosts.ini \
+  --extra-vars 'concurrent_reboots=2 wait_seconds_after_reboot=30'
+```
 
 ## 🔁 Upgrading an existing cluster
 
@@ -124,13 +163,21 @@ To copy your `kube config` locally so that you can access your **Kubernetes** cl
 ```bash
 scp debian@master_ip:/etc/rancher/k3s/k3s.yaml ~/.kube/config
 ```
-If you get file Permission denied, go into the node and temporarly run:
+If the copy fails with a permission error, grant the SSH user temporary read access using the least permissive method
+available for the target system. Restore the original ownership and permissions immediately after copying. Avoid
+world-writable permissions on the kubeconfig because it contains cluster credentials.
+
+For example, copy the file to a temporary user-readable path from the control node:
+
 ```bash
-sudo chmod 777 /etc/rancher/k3s/k3s.yaml
+ssh debian@master_ip 'sudo install -o "$(id -un)" -m 0600 /etc/rancher/k3s/k3s.yaml /tmp/k3s.yaml'
 ```
-Then copy with the scp command and reset the permissions back to:
+
+Copy `/tmp/k3s.yaml`, then remove the temporary remote copy:
+
 ```bash
-sudo chmod 600 /etc/rancher/k3s/k3s.yaml
+scp debian@master_ip:/tmp/k3s.yaml ~/.kube/config
+ssh debian@master_ip rm -f /tmp/k3s.yaml
 ```
 
 You'll then want to modify the config to point to master IP by running:
@@ -150,7 +197,7 @@ See the commands [here](https://technotim.com/posts/k3s-etcd-ansible/#testing-yo
 | `download` | `k3s_version` | string | ❌ | Required | K3s binaries version |
 | `k3s_agent`, `k3s_server`, `k3s_server_post` | `apiserver_endpoint` | string | ❌ | Required | Virtual ip-address configured on each master |
 | `k3s_agent` | `extra_agent_args` | string | `null` | Not required | Extra arguments for agents nodes |
-| `k3s_agent`, `k3s_server` | `group_name_master` | string | `null` | Not required | Name othe master group |
+| `k3s_agent`, `k3s_server` | `group_name_master` | string | `null` | Not required | Name of the master group |
 | `k3s_agent` | `k3s_token` | string | `null` | Not required | Token used to communicate between masters |
 | `k3s_agent`, `k3s_server` | `proxy_env` | dict | `null` | Not required | Internet proxy configurations |
 | `k3s_agent`, `k3s_server` | `proxy_env.HTTP_PROXY` | string | ❌ | Required | HTTP internet proxy |
@@ -172,6 +219,7 @@ See the commands [here](https://technotim.com/posts/k3s-etcd-ansible/#testing-yo
 | `k3s_server` | `kube_vip_bgp_peers` | list | `[]` | Not required | List of BGP peer ASN & address pairs |
 | `k3s_server` | `kube_vip_bgp_peers_groups` | list | `['k3s_master']` | Not required | Inventory group in which to search for additional `kube_vip_bgp_peers` parameters to merge. |
 | `k3s_server` | `kube_vip_iface` | string | `~` | Not required | Explicitly define an interface that ALL control nodes should use to propagate the VIP, define it here. Otherwise, kube-vip will determine the right interface automatically at runtime. |
+| `k3s_server` | `kube_vip_endpoint` | string | `~` | Not required | Overrides the internal address kube-vip binds/listens on, which can differ from the announced apiserver_endpoint for complex routing/tunnels. Defaults to apiserver_endpoint. |
 | `k3s_server` | `kube_vip_tag_version` | string | `v1.2.2` | Not required | Image tag for kube-vip |
 | `k3s_server` | `kube_vip_cloud_provider_tag_version` | string | `v0.0.12` | Not required | Tag for kube-vip-cloud-provider manifest when enable |
 | `k3s_server`, `k3_server_post` | `kube_vip_lb_ip_range` | string | `~` | Not required | IP range for kube-vip load balancer |
@@ -211,7 +259,10 @@ See the commands [here](https://technotim.com/posts/k3s-etcd-ansible/#testing-yo
 | `k3s_server_post` | `metal_lb_bgp_peer_asn` | string | `~` | Not required | BGP peer ASN configurations |
 | `k3s_server_post` | `metal_lb_bgp_peer_address` | string | `~` | Not required | BGP peer address |
 | `lxc` | `custom_reboot_command` | string | `~` | Not required | Command to run on reboot |
+| `reboot` (playbook) | `concurrent_reboots` | int/string | `100%` | Not required | Number (or percentage) of nodes to reboot at a time for a staggered reboot |
+| `reboot` (playbook) | `wait_seconds_after_reboot` | int | `0` | Not required | Pause in seconds between staggered reboot batches |
 | `prereq` | `system_timezone` | string | `null` | Not required | Timezone to be set on all nodes |
+| `prereq` | `disable_swap` | bool | `true` | Not required | Disable swap on all cluster nodes (swapoff + comment out /etc/fstab swap entries), all-or-nothing |
 | `proxmox_lxc`, `reset_proxmox_lxc` | `proxmox_lxc_ct_ids` | list | ❌ | Required | Proxmox container ID list |
 | `raspberrypi` | `state` | string | `present` | Not required | Indicates whether the k3s prerequisites for Raspberry Pi should be set up (possible values are `present` and `absent`) |
 
@@ -227,9 +278,11 @@ It is run automatically in CI, but you can also run the tests locally.
 This might be helpful for quick feedback in a few cases.
 You can find more information about it [here](molecule/README.md).
 
-### Pre-commit Hooks
+### Pre-commit hooks
 
-This repo uses `pre-commit` and `pre-commit-hooks` to lint and fix common style and syntax errors.  Be sure to install python packages and then run `pre-commit install`.  For more information, see [pre-commit](https://pre-commit.com/)
+This repository uses `pre-commit` to check style, syntax, Ansible content, and shell scripts. Install the Python
+dependencies, run `pre-commit install` once, and run `pre-commit run --all-files` before submitting a change. See
+[CONTRIBUTING.md](CONTRIBUTING.md) for the complete development workflow.
 
 ## 🌌 Ansible Galaxy
 

inventory/sample/group_vars/all.yml

@@ -7,21 +7,31 @@ systemd_dir: /etc/systemd/system
 # Set your timezone
 system_timezone: Your/Timezone
 
+# k3s recommends swap be disabled on every cluster node. Applied uniformly to all
+# nodes (all-or-nothing) in the prereq role. Set to false to leave swap enabled.
+disable_swap: true
+
 # interface which will be used for flannel
-flannel_iface: eth0
+# Defaults to each host's default IPv4 interface (e.g. eth0, enp1s0, ens3)
+# so KVM/cloud hosts without eth0 work out of the box. Override per-host if needed.
+flannel_iface: "{{ ansible_facts.default_ipv4.interface }}"
 
 # uncomment calico_iface to use tigera operator/calico cni instead of flannel https://docs.tigera.io/calico/latest/about
-# calico_iface: "eth0"
+# calico_iface: "{{ ansible_facts.default_ipv4.interface }}"
 calico_ebpf: false # use eBPF dataplane instead of iptables
 calico_tag: v3.32.1 # calico version tag
 
 # uncomment cilium_iface to use cilium cni instead of flannel or calico
 # ensure v4.19.57, v5.1.16, v5.2.0 or more recent kernel
-# cilium_iface: "eth0"
+# cilium_iface: "{{ ansible_facts.default_ipv4.interface }}"
 cilium_mode: native # native when nodes are on the same subnet or use BGP, otherwise set tunnel
 cilium_tag: v1.20.0 # cilium version tag
 cilium_cli_tag: v0.19.7 # cilium cli version tag
 cilium_hubble: true # enable hubble observability relay and ui
+cilium_envoy: true # enable the Envoy proxy for Cilium L7 policies
+
+# disable cilium_envoy to skip the Envoy proxy entirely (e.g. no L7 policies)
+# cilium_envoy: false
 
 # if using calico or cilium, you may specify the cluster pod cidr pool
 cluster_cidr: 10.52.0.0/16
@@ -38,6 +48,11 @@ cilium_bgp_lb_cidr: 192.168.31.0/24 # cidr for cilium loadbalancer ipam
 # enable kube-vip ARP broadcasts
 kube_vip_arp: true
 
+# (optional) overrides the address kube-vip binds/listens on internally, which
+# can differ from the announced apiserver_endpoint for complex routing/tunnels.
+# Defaults to apiserver_endpoint. Also used to derive the kube-vip subnet.
+# kube_vip_endpoint: 10.66.1.5
+
 # enable kube-vip BGP peering
 kube_vip_bgp: false
 
@@ -47,7 +62,9 @@ kube_vip_bgp_as: "64513"  # Defines the AS for the BGP server
 kube_vip_bgp_peeraddress: "192.168.30.1"  # Defines the address for the BGP peer
 kube_vip_bgp_peeras: "64512"  # Defines the AS for the BGP peer
 
-# apiserver_endpoint is virtual ip-address which will be configured on each master
+# apiserver_endpoint is virtual ip-address which will be configured on each master.
+# This must be a free, routable IP on your network (not already assigned to a host
+# or service), and is used by kube-vip / MetalLB to expose the Kubernetes API.
 apiserver_endpoint: 192.168.30.222
 
 # k3s_token is required  masters can talk together securely
@@ -112,6 +129,12 @@ metal_lb_controller_tag_version: v0.16.0
 # metallb ip range for load balancer
 metal_lb_ip_range: 192.168.30.80-192.168.30.90
 
+# (optional) limit MetalLB layer2 announcements to specific network interfaces.
+# Leave empty (default) to announce on all interfaces.
+# metal_lb_interfaces:
+#   - eth1
+#   - eth2
+
 # Only enable if your nodes are proxmox LXC nodes, make sure to configure your proxmox nodes
 # in your hosts.ini file.
 # Please read https://gist.github.com/triangletodd/02f595cd4c0dc9aac5f7763ca2264185 before using this.

molecule/resources/verify_from_outside/tasks/test/verify-components.yml

@@ -38,6 +38,30 @@
           loop_control:
             label: "{{ item.metadata.name }} ready"
 
+        - name: Assert every node registered a node IP from its interface
+          # Each k3s node is launched with --node-ip derived from flannel_iface.
+          # Confirm every node carries a real InternalIP (not a loopback), which
+          # proves k3s bound to the cluster interface rather than defaulting to 127.0.0.1.
+          ansible.builtin.assert:
+            that: >-
+              (node_internal_ips | length) >= 1 and
+              (node_internal_ips | reject('eq', '127.0.0.1') | list | length) == node_internal_ips | length
+            success_msg: "{{ item.metadata.name }} is bound to {{ node_internal_ips | join(', ') }}"
+            fail_msg: >-
+              {{ item.metadata.name }} has no non-loopback InternalIP
+              (got: {{ node_internal_ips | join(', ') }})
+          vars:
+            node_internal_ips: >-
+              {{
+                (item.status.addresses | default([]))
+                | selectattr('type', 'equalto', 'InternalIP')
+                | map(attribute='address')
+                | list
+              }}
+          loop: "{{ verify_nodes.resources }}"
+          loop_control:
+            label: "{{ item.metadata.name }} InternalIP"
+
         - name: Get any Calico namespaces with Flannel enabled
           kubernetes.core.k8s_info:
             kind: Namespace
@@ -79,7 +103,7 @@
           ansible.builtin.assert:
             that:
               - calico_node_ds.resources | length == 1
-              - calico_node_image | regex_search(':' ~ calico_tag)
+              - calico_node_image | regex_search(':' ~ calico_tag) is not none
             success_msg: "Calico node image uses tag {{ calico_tag }}"
             fail_msg: >-
               Calico node image {{ calico_node_image }},
@@ -145,8 +169,8 @@
         - name: Assert Cilium agent and operator use the expected image tag
           ansible.builtin.assert:
             that:
-              - cilium_agent_image | regex_search(':' ~ cilium_tag)
-              - cilium_operator_image | regex_search(':' ~ cilium_tag)
+              - cilium_agent_image | regex_search(':' ~ cilium_tag) is not none
+              - cilium_operator_image | regex_search(':' ~ cilium_tag) is not none
             success_msg: "Cilium agent and operator use {{ cilium_tag }}"
             fail_msg: >-
               Cilium agent {{ cilium_agent_image }},
@@ -219,17 +243,34 @@
             namespace: metallb-system
             kubeconfig: "{{ kubecfg_path }}"
           register: metallb_info
+          until: metallb_info.resources | length > 0
+          retries: 15
+          delay: 10
           loop:
             - { kind: Deployment, name: controller }
             - { kind: DaemonSet, name: speaker }
           loop_control:
             label: "{{ item.kind }}/{{ item.name }}"
 
+        - name: Fail with a clear message if MetalLB resources are missing
+          ansible.builtin.fail:
+            msg: >-
+              Did not find {{ item.kind | lower }} {{ item.name }} in
+              metallb-system. Expected MetalLB to be deployed in this
+              scenario (verify_lb: {{ verify_lb }}).
+          when: item.resources | length == 0
+          loop: "{{ metallb_info.results }}"
+          loop_control:
+            label: "{{ item.item.kind }}/{{ item.item.name }}"
+
         - name: Assert MetalLB controller and speaker use the expected image tags
           ansible.builtin.assert:
+            # regex_search returns a string or none; check for a match with `is not
+            # none` so the assertion is a real boolean (ansible-core 2.19 rejects
+            # string conditionals and `| bool` deprecates string coercion).
             that:
-              - controller_image | regex_search(metal_lb_controller_tag_version)
-              - speaker_image | regex_search(metal_lb_speaker_tag_version)
+              - controller_image | regex_search(metal_lb_controller_tag_version) is not none
+              - speaker_image | regex_search(metal_lb_speaker_tag_version) is not none
             success_msg: >-
               MetalLB controller {{ metal_lb_controller_tag_version }},
               speaker {{ metal_lb_speaker_tag_version }}
@@ -273,8 +314,8 @@
         - name: Assert the kube-vip and cloud provider image tags
           ansible.builtin.assert:
             that:
-              - kubevip_image | regex_search(':' ~ kube_vip_tag_version)
-              - cloud_provider_image | regex_search(verify_kube_vip_cloud_provider_tag)
+              - kubevip_image | regex_search(':' ~ kube_vip_tag_version) is not none
+              - cloud_provider_image | regex_search(verify_kube_vip_cloud_provider_tag) is not none
             success_msg: >-
               kube-vip {{ kube_vip_tag_version }},
               cloud provider {{ verify_kube_vip_cloud_provider_tag }}

reboot.yml

@@ -2,9 +2,29 @@
 - name: Reboot k3s_cluster
   hosts: k3s_cluster
   gather_facts: true
+
+  # Stagger the reboot across the cluster when concurrent_reboots is set.
+  # Defaults to '100%' so the whole cluster reboots at once (backward compatible).
+  serial: "{{ concurrent_reboots | default('100%') }}"
+
   tasks:
-    - name: Reboot the nodes (and Wait upto 5 mins max)
+    - name: >-
+        {{
+          'Reboot all nodes at once'
+          if (concurrent_reboots is not defined)
+          else 'Reboot nodes with concurrency of ' ~ concurrent_reboots
+        }}
       become: true
       ansible.builtin.reboot:
         reboot_command: "{{ custom_reboot_command | default(omit) }}"
         reboot_timeout: 300
+        test_command: >-
+          {{ 'kubectl get nodes' if 'master' in group_names else 'whoami' }}
+
+    - name: Optional wait before rebooting the next batch of nodes
+      ansible.builtin.pause:
+        seconds: "{{ wait_seconds_after_reboot | int }}"
+      when: >-
+        concurrent_reboots is defined and
+        wait_seconds_after_reboot is defined and
+        wait_seconds_after_reboot | int > 0

requirements.in

@@ -1,5 +1,5 @@
-ansible-core>=2.16.2
-jmespath>=1.0.1
+ansible-core>=2.19.11
+jmespath>=1.1.0
 jsonpatch>=1.33
 kubernetes>=29.0.0
 molecule-plugins[vagrant]

requirements.txt

@@ -2,11 +2,11 @@
 # This file is autogenerated by pip-compile with Python 3.11
 # by the following command:
 #
-#    pip-compile requirements.in
+#    pip-compile --output-file=requirements.txt requirements.in
 #
 ansible-compat==4.1.11
     # via molecule
-ansible-core==2.18.0
+ansible-core==2.19.11
     # via
     #   -r requirements.in
     #   ansible-compat
@@ -53,7 +53,7 @@ jinja2==3.1.3
     # via
     #   ansible-core
     #   molecule
-jmespath==1.0.1
+jmespath==1.1.0
     # via -r requirements.in
 jsonpatch==1.33
     # via -r requirements.in
@@ -77,7 +77,7 @@ molecule==6.0.3
     # via
     #   -r requirements.in
     #   molecule-plugins
-molecule-plugins[vagrant]==23.5.3
+molecule-plugins[vagrant]==23.6.0
     # via -r requirements.in
 netaddr==0.10.1
     # via -r requirements.in
@@ -146,7 +146,7 @@ rsa==4.9
     # via google-auth
 ruamel-yaml==0.18.5
     # via pre-commit-hooks
-ruamel-yaml-clib==0.2.8
+ruamel-yaml-clib==0.2.15
     # via ruamel-yaml
 six==1.16.0
     # via

roles/k3s_agent/templates/k3s.service.j2

@@ -11,7 +11,7 @@ ExecStartPre=-/sbin/modprobe overlay
 ExecStart=/usr/local/bin/k3s agent \
   --server https://{{ apiserver_endpoint | ansible.utils.ipwrap }}:6443 \
   {% if is_pxe_booted | default(false) %}--snapshotter native \
-  {% endif %}--token {{ hostvars[groups[group_name_master | default('master')][0]]['token'] | default(k3s_token) }} \
+  {% endif %}--token {{ hostvars[groups[group_name_master | default('master')][0]]['token'] | default(k3s_token) | replace( '%', '%%' ) }} \
   {{ extra_agent_args }}
 KillMode=process
 Delegate=yes

roles/k3s_server/defaults/main.yml

@@ -7,6 +7,7 @@ group_name_master: master
 
 kube_vip_arp: true
 kube_vip_iface:
+kube_vip_endpoint:
 kube_vip_cloud_provider_tag_version: v0.0.12
 kube_vip_tag_version: v1.2.2
 
@@ -23,6 +24,12 @@ metal_lb_controller_tag_version: v0.16.0
 metal_lb_speaker_tag_version: v0.16.0
 metal_lb_type: native
 
+# Shared retry/delay for remote manifest and asset downloads. The CI runner's
+# resolver intermittently times out on GitHub-hosted domains (helm.cilium.io,
+# raw.githubusercontent.com, github.com), so retry transient DNS/network failures.
+download_retries: 5
+download_delay: 10
+
 retry_count: 20
 
 # yamllint disable rule:line-length
@@ -33,7 +40,7 @@ server_init_args: >-
     {% else %}
       --server https://{{ hostvars[groups[group_name_master | default('master')][0]].k3s_node_ip | split(",") | first | ansible.utils.ipwrap }}:6443
     {% endif %}
-    --token {{ k3s_token }}
+    --token {{ k3s_token | replace('%', '%%') }}
   {% endif %}
   {{ extra_server_args }}
 

roles/k3s_server/meta/main.yml

@@ -78,6 +78,15 @@ argument_specs:
           - automatically at runtime.
         default: ~
 
+      kube_vip_endpoint:
+        description:
+          - Overrides the address kube-vip binds/listens on internally, which
+          - can differ from the announced apiserver_endpoint for complex
+          - routing and site-to-site tunnels.
+          - Defaults to apiserver_endpoint and is used to derive the kube-vip
+          - subnet.
+        default: ~
+
       kube_vip_tag_version:
         description: Image tag for kube-vip
         default: v1.2.2

roles/k3s_server/tasks/kube-vip.yml

@@ -15,6 +15,10 @@
     owner: root
     group: root
     mode: "0644"
+  register: kube_vip_manifest_download
+  retries: "{{ download_retries }}"
+  delay: "{{ download_delay }}"
+  until: kube_vip_manifest_download is succeeded
   when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
 
 - name: Copy kubevip configMap manifest to first master

roles/k3s_server/tasks/main.yml

@@ -27,7 +27,11 @@
 - name: Deploy metallb manifest
   ansible.builtin.include_tasks: metallb.yml
   tags: metallb
-  when: kube_vip_lb_ip_range is not defined and (not cilium_bgp or cilium_iface is not defined)
+  # Deploy MetalLB unless kube-vip owns the load balancer IP range, or Cilium
+  # BGP is enabled (Cilium then provides its own load balancing). The cilium_bgp
+  # default keeps this safe when Cilium variables are not in scope at all (#644)
+  # while still deploying MetalLB when a non-BGP Cilium CNI is in use.
+  when: kube_vip_lb_ip_range is not defined and not (cilium_bgp | default(false) | bool)
 
 - name: Deploy kube-vip manifest
   ansible.builtin.include_tasks: kube-vip.yml
@@ -118,7 +122,10 @@
   always:
     - name: Save logs of k3s-init.service
       ansible.builtin.include_tasks: fetch_k3s_init_logs.yml
-      when: log_destination
+      # ANSIBLE_K3S_LOG_DIR is a path string when set; evaluate it as a boolean
+      # so the conditional is a real boolean (ansible-core 2.19 rejects string
+      # conditionals derived from env vars).
+      when: log_destination | default('') != ''
       vars:
         log_destination: >-
           {{ lookup('ansible.builtin.env', 'ANSIBLE_K3S_LOG_DIR', default=False) }}

roles/k3s_server/tasks/metallb.yml

@@ -15,6 +15,10 @@
     owner: root
     group: root
     mode: "0644"
+  register: metallb_manifest_download
+  retries: "{{ download_retries }}"
+  delay: "{{ download_delay }}"
+  until: metallb_manifest_download is succeeded
   when: ansible_hostname == hostvars[groups[group_name_master | default('master')][0]]['ansible_hostname']
 
 - name: Set image versions in manifest for metallb-{{ metal_lb_type }}

roles/k3s_server/templates/vip.yaml.j2

@@ -37,7 +37,7 @@ spec:
           value: {{ kube_vip_iface }}
 {% endif %}
         - name: vip_subnet
-          value: "{{ apiserver_endpoint | ansible.utils.ipsubnet | ansible.utils.ipaddr('prefix') }}"
+          value: "{{ (kube_vip_endpoint | default(apiserver_endpoint, true)) | ansible.utils.ipsubnet | ansible.utils.ipaddr('prefix') }}"
         - name: cp_enable
           value: "true"
         - name: cp_namespace
@@ -55,7 +55,7 @@ spec:
         - name: vip_retryperiod
           value: "2"
         - name: address
-          value: {{ apiserver_endpoint }}
+          value: {{ kube_vip_endpoint | default(apiserver_endpoint, true) }}
 {% if kube_vip_bgp | default(false) | bool %}
 {% if kube_vip_bgp_routerid is defined %}
         - name: bgp_routerid

roles/k3s_server_post/defaults/main.yml

@@ -18,11 +18,19 @@ cilium_bgp_peer_asn: 64512
 cilium_bgp_neighbors: []
 cilium_bgp_neighbors_groups: ['k3s_all']
 cilium_bgp_lb_cidr: 192.168.31.0/24
+cilium_envoy: true
 cilium_hubble: true
 cilium_mode: native
 cilium_tag: v1.20.0
 cilium_cli_tag: v0.19.7
 
+# Shared retry/delay for remote manifest, asset downloads, and waiting on
+# Kubernetes resources. The CI runner's resolver intermittently times out on
+# GitHub-hosted domains and the kube API can transiently return
+# ServiceUnavailable, so retry transient DNS/network/API failures.
+download_retries: 5
+download_delay: 10
+
 cluster_cidr: 10.52.0.0/16
 enable_bpf_masquerade: true
 kube_proxy_replacement: true
@@ -31,4 +39,5 @@ group_name_master: master
 metal_lb_mode: layer2
 metal_lb_available_timeout: 240s
 metal_lb_controller_tag_version: v0.16.0
+metal_lb_interfaces: []
 metal_lb_ip_range: 192.168.30.80-192.168.30.90

roles/k3s_server_post/meta/main.yml

@@ -141,6 +141,14 @@ argument_specs:
         description: MetalLB ip range for load balancer
         default: 192.168.30.80-192.168.30.90
 
+      metal_lb_interfaces:
+        description: >-
+          List of network interfaces on which MetalLB should announce the
+          load balancer IPs in layer2 mode. When empty (default), MetalLB
+          announces on all interfaces.
+        type: list
+        default: []
+
       metal_lb_controller_tag_version:
         description: Image tag for MetalLB
         default: v0.16.0

roles/k3s_server_post/tasks/calico.yml

@@ -18,6 +18,10 @@
         owner: root
         group: root
         mode: "0755"
+      register: calico_crd_download
+      retries: "{{ download_retries }}"
+      delay: "{{ download_delay }}"
+      until: calico_crd_download is succeeded
 
     - name: "Download to first master: manifest for Tigera Operator and Calico CRDs"
       ansible.builtin.get_url:
@@ -26,6 +30,10 @@
         owner: root
         group: root
         mode: "0755"
+      register: tigera_operator_download
+      retries: "{{ download_retries }}"
+      delay: "{{ download_delay }}"
+      until: tigera_operator_download is succeeded
 
     - name: Apply Calico CRD bundle with server-side apply
       ansible.builtin.command: >-

roles/k3s_server_post/tasks/cilium.yml

@@ -66,6 +66,10 @@
             - .tar.gz.sha256sum
           vars:
             cilium_base_url: https://github.com/cilium/cilium-cli/releases/download/{{ cilium_cli_tag }}
+          register: cilium_cli_download
+          retries: "{{ download_retries }}"
+          delay: "{{ download_delay }}"
+          until: cilium_cli_download is succeeded
 
         - name: Verify the downloaded tarball
           ansible.builtin.shell: |
@@ -174,6 +178,7 @@
         --helm-set hubble.enabled={{ "true" if cilium_hubble else "false" }}
         --helm-set hubble.relay.enabled={{ "true" if cilium_hubble else "false" }}
         --helm-set hubble.ui.enabled={{ "true" if cilium_hubble else "false" }}
+        --helm-set envoy.enabled={{ "true" if cilium_envoy else "false" }}
         {% if kube_proxy_replacement is not false %}
         --helm-set loadBalancer.algorithm={{ bpf_lb_algorithm }}
         --helm-set loadBalancer.mode={{ bpf_lb_mode }}
@@ -182,6 +187,13 @@
         KUBECONFIG: "{{ ansible_user_dir }}/.kube/config"
       register: cilium_install_result
       changed_when: cilium_install_result.rc == 0
+      # cilium install/upgrade fetches the Helm chart from helm.cilium.io, which is
+      # fronted by GitHub Pages and can transiently fail DNS resolution through the
+      # host resolver (intermittent "lookup helm.cilium.io ... i/o timeout"). Retry
+      # so a transient name/network failure does not abort the whole converge play.
+      until: cilium_install_result.rc == 0
+      retries: "{{ download_retries }}"
+      delay: "{{ download_delay }}"
       when: cilium_installed.rc != 0 or cilium_needs_update
 
     - name: Wait for Cilium resources

roles/k3s_server_post/tasks/main.yml

@@ -12,7 +12,11 @@
 - name: Deploy metallb pool
   ansible.builtin.include_tasks: metallb.yml
   tags: metallb
-  when: kube_vip_lb_ip_range is not defined and (not cilium_bgp or cilium_iface is not defined)
+  # Deploy MetalLB unless kube-vip owns the load balancer IP range, or Cilium
+  # BGP is enabled (Cilium then provides its own load balancing). The cilium_bgp
+  # default keeps this safe when Cilium variables are not in scope at all (#644)
+  # while still deploying MetalLB when a non-BGP Cilium CNI is in use.
+  when: kube_vip_lb_ip_range is not defined and not (cilium_bgp | default(false) | bool)
 
 - name: Remove tmp directory used for manifests
   ansible.builtin.file:

roles/k3s_server_post/tasks/metallb.yml

@@ -40,8 +40,14 @@
 
 - name: Test metallb-system namespace
   ansible.builtin.command: >-
-    {{ k3s_kubectl_binary | default('k3s kubectl') }} -n metallb-system
+    {{ k3s_kubectl_binary | default('k3s kubectl') }} get namespace metallb-system
   changed_when: false
+  # The kube API can briefly return ServiceUnavailable while MetalLB converges,
+  # which would otherwise abort the whole converge play on a transient error.
+  register: metallb_namespace_result
+  until: metallb_namespace_result.rc == 0
+  retries: "{{ download_retries }}"
+  delay: "{{ download_delay }}"
   with_items: "{{ groups[group_name_master | default('master')] }}"
   run_once: true
 
@@ -54,6 +60,12 @@
     {% if item.condition | default(False) -%}{{ item.condition }}{%- endif %}
     --timeout='{{ metal_lb_available_timeout }}'
   changed_when: false
+  # The kube API can briefly return ServiceUnavailable while MetalLB converges,
+  # which would otherwise abort the whole converge play on a transient error.
+  register: metallb_wait_result
+  until: metallb_wait_result.rc == 0
+  retries: "{{ download_retries }}"
+  delay: "{{ download_delay }}"
   run_once: true
   with_items:
     - description: controller
@@ -93,6 +105,12 @@
   ansible.builtin.command: >-
     {{ k3s_kubectl_binary | default('k3s kubectl') }} -n metallb-system get endpoints {{ metallb_webhook_service_name }}
   changed_when: false
+  # The kube API can briefly return ServiceUnavailable while MetalLB converges,
+  # which would otherwise abort the whole converge play on a transient error.
+  register: metallb_webhook_result
+  until: metallb_webhook_result.rc == 0
+  retries: "{{ download_retries }}"
+  delay: "{{ download_delay }}"
   with_items: "{{ groups[group_name_master | default('master')] }}"
   run_once: true
 
@@ -112,6 +130,10 @@
   changed_when: false
   run_once: true
   when: metal_lb_mode == "layer2"
+  register: metallb_l2_test_result
+  until: metallb_l2_test_result.rc == 0
+  retries: "{{ download_retries }}"
+  delay: "{{ download_delay }}"
   with_items:
     - IPAddressPool
     - L2Advertisement
@@ -122,6 +144,10 @@
   changed_when: false
   run_once: true
   when: metal_lb_mode == "bgp"
+  register: metallb_bgp_test_result
+  until: metallb_bgp_test_result.rc == 0
+  retries: "{{ download_retries }}"
+  delay: "{{ download_delay }}"
   with_items:
     - IPAddressPool
     - BGPPeer

roles/k3s_server_post/templates/metallb.crs.j2

@@ -21,6 +21,11 @@ kind: L2Advertisement
 metadata:
   name: default
   namespace: metallb-system
+{% if metal_lb_interfaces | default([]) | length > 0 %}
+spec:
+  interfaces:{% for iface in metal_lb_interfaces %}
+    - {{ iface }}{% endfor %}
+{% endif %}
 {% endif %}
 {% if metal_lb_mode == "bgp" %}
 ---

roles/prereq/defaults/main.yml

@@ -1,4 +1,6 @@
 ---
+disable_swap: true
+
 secure_path:
   RedHat: /sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin
   Suse: /usr/sbin:/usr/bin:/sbin:/bin:/usr/local/bin

roles/prereq/tasks/main.yml

@@ -4,6 +4,28 @@
     name: "{{ system_timezone }}"
   when: (system_timezone is defined) and (system_timezone != "Your/Timezone")
 
+# k3s recommends swap be disabled on all nodes. Disabling swap is all-or-nothing
+# across the cluster: leaving it enabled on some nodes but not others creates
+# uneven scheduling/latency behavior. This block turns swap off and comments out
+# the swap entries in /etc/fstab so it stays off across reboots. It is idempotent
+# and a no-op when swap is already disabled or swapoff is unavailable.
+- name: Disable swap on all cluster nodes
+  when: disable_swap
+  block:
+    - name: Turn off swap now
+      ansible.builtin.command: swapoff -a
+      register: swapoff_result
+      changed_when: false
+      failed_when: false
+
+    - name: Comment out swap entries in fstab
+      ansible.builtin.replace:
+        path: /etc/fstab
+        regexp: '^([^#][^\n]*\s+swap\s+)'
+        replace: '# \\1'
+      register: fstab_swap
+
+
 - name: Set SELinux to disabled state
   ansible.posix.selinux:
     state: disabled
@@ -33,6 +55,27 @@
     reload: true
   tags: sysctl
 
+- name: Check if br_netfilter module exists
+  ansible.builtin.shell: |
+    set -o pipefail
+    find /lib/modules/$(uname -r) -name "br_netfilter.ko*" | wc -l
+  args:
+    executable: /bin/bash
+  register: br_netfilter_exists
+  changed_when: false
+  when: ansible_os_family == "RedHat"
+  tags: sysctl
+
+- name: Install kernel-modules-extra if br_netfilter missing
+  ansible.builtin.yum:
+    name: kernel-modules-extra
+    state: present
+    update_cache: true
+  register: kernel_modules_installed
+  when:
+    - ansible_os_family == "RedHat"
+    - br_netfilter_exists.stdout | int == 0
+
 - name: Add br_netfilter to /etc/modules-load.d/
   ansible.builtin.copy:
     content: br_netfilter

site.yml

@@ -8,6 +8,23 @@
         msg: >
           "Ansible is out of date. See here for more info: https://docs.technotim.com/posts/ansible-automation/"
 
+    - name: Verify all cluster nodes have unique hostnames
+      ansible.builtin.assert:
+        that: (cluster_hostnames | unique | length) == (cluster_hostnames | length)
+        msg: >-
+          k3s nodes must have unique hostnames. Found a duplicate in:
+          {{ cluster_hostnames | unique }}. Each node registers in the cluster
+          keyed by its hostname, so matching hostnames prevent nodes from joining.
+      vars:
+        cluster_hostnames: >-
+          {{
+            groups['k3s_cluster']
+            | map('extract', hostvars, 'ansible_hostname')
+            | list
+          }}
+      run_once: true
+      when: "'k3s_cluster' in groups"
+
 - name: Prepare Proxmox cluster
   hosts: proxmox
   gather_facts: true