edge-host was a single role doing eight unrelated jobs: flash-wear
mitigation, log shipping, VLAN networking, an MQTT bridge, a CAN interface,
the heating CAN services, a Z-Wave gateway and a power-meter reader.
That is not just untidy. It is what blocked a second thin client from
joining as an NTP peer: the host-generic work (journald to RAM, Alloy, the
VLAN sub-interface) could not be applied without also handing the box a
Z-Wave radio and a CAN bus it does not have. The role bound its services to
specific USB device paths, so a second host would have tried to start
containers for hardware that is not there.
Now:
edge-flash-wear journald -> RAM, noatime/lazytime, TRIM, eMMC health
edge-log-shipper Alloy: journal -> Loki
edge-vlan tagged VLAN sub-interface
mqtt-bridge loopback mosquitto -> cluster broker over TLS
socketcan can0 only
heating-can open3e (Vitocal/VX3) + e3oncan (E380 meter)
zwave-gateway zwave-js-ui, own compose project
power-meter-sml sml2mqtt, own compose project
The E380 reader lives with open3e rather than in its own role: it is part
of the same heating installation -- it feeds the heat pump's own energy
management -- not a separate system that happens to share a bus. can0 does
stay separate, because an interface is not the same concern as the services
consuming it.
Each role installs its own packages, so it stands alone. An edge host with
no peripherals takes the first five and none of the rest.
docker-daemon.json moves to docker-install with its own docker_log_driver
variable and Restart docker handler, since configuring the docker daemon was
never an edge-host concern. Handlers split ten ways; defaults split per
role; all sixteen templates and files moved with git mv.
Two deliberate behaviour changes on the node, both accepted:
1. Compose split. zwave-js and sml2mqtt shared a compose file, so touching
either recreated both. They are now separate projects in their own
directories. container_name is fixed, so the old project must release the
names first -- a guarded one-time task downs it and removes the file.
First apply costs a brief down/up on both. Data paths are unchanged; the
zwave store is not migrated.
2. Header repointing. Templates and scripts carried "Managed by
roles/edge-host" headers, which render onto the host -- they would have
sent an operator to a role that no longer exists. Updating them changes
file content, so it restarts the services that consume those files.
Verified by --check --diff against the live host. 24 changed items, and
every one is accounted for:
5 pre-existing, unchanged from the baseline before this work:
common timesyncd; open3e venv; E3onCAN venv; and the open3e /
e3oncan restart handlers. The venv tasks install from a git URL,
which pip cannot verify, so they report changed on every run --
a separate defect, not introduced here.
5 the compose split: zwave env + compose and sml compose to their
new paths, plus the two legacy-project retirement tasks.
14 the header repointing: journald conf, eMMC health script, alloy
config, mosquitto bridge config, 80-can0.network, open3e overlay /
env / unit, e3oncan unit, zwave backup script + env, sml2mqtt
config -- and the resulting restarts of journald, alloy, mosquitto,
open3e and e3oncan.
Nothing changed outside those three groups.
One consequence check mode cannot show: 80-can0.network.j2 changed, so the
"Reload networkd" handler fires on a real run and reconfigures can0,
briefly dropping the bus under open3e and e3oncan. It is absent from the
check-mode output only because shell handlers are skipped there. The VLAN
templates did NOT change, so the riskier reconfigure of the parent uplink
on this headless host is not triggered.
Also fixed in passing: the keepalived-ntp guard told operators to recover
with `--limit <host> --tags vlan40`, and referred to the edge-host role.
Both corrected. The role README becomes ansible/docs/edge-host.md -- it
documents the host, not any single role now.
Co-Authored-By: Claude Bot <claude-bot@renner.uno>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
191 lines
7.6 KiB
YAML
191 lines
7.6 KiB
YAML
---
|
|
# Keepalived NTP VIP Role
|
|
#
|
|
# Supports Debian (PVE cluster nodes) and Arch (edge-1). Unlike chrony, the
|
|
# keepalived package uses the same config path (/etc/keepalived/keepalived.conf)
|
|
# and unit name (keepalived.service) on both, so only the install step branches.
|
|
#
|
|
# The chrony unit name does differ, and the health check script needs it -- see
|
|
# the fact resolution below.
|
|
|
|
- name: Fail on unsupported distribution
|
|
ansible.builtin.fail:
|
|
msg: >-
|
|
keepalived-ntp supports Debian and Archlinux only;
|
|
this host reports os_family={{ ansible_facts['os_family'] }}
|
|
when: ansible_facts['os_family'] not in ['Debian', 'Archlinux']
|
|
|
|
# Resolved here as well as in chrony-ntp-server so this role remains usable
|
|
# standalone. Both compute the same value from the same fact.
|
|
- name: Resolve chrony unit name for the health check
|
|
ansible.builtin.set_fact:
|
|
chrony_service_name: >-
|
|
{{ 'chrony' if ansible_facts['os_family'] == 'Debian' else 'chronyd' }}
|
|
|
|
- name: Install keepalived (Debian)
|
|
ansible.builtin.apt:
|
|
name: keepalived
|
|
state: present
|
|
update_cache: true
|
|
when: ansible_facts['os_family'] == 'Debian'
|
|
|
|
- name: Install keepalived (Arch)
|
|
ansible.builtin.package:
|
|
name: keepalived
|
|
state: present
|
|
when: ansible_facts['os_family'] == 'Archlinux'
|
|
|
|
# --- Verify nothing else is assigning the IPv6 service address ---------------
|
|
#
|
|
# The removal itself lives in withdraw-static.yml, run as a separate fleet-wide
|
|
# play before any host reaches this role (see install-ntp-ha.yml). It has to
|
|
# complete everywhere first: DAD is enabled on the VRRP address, so a static
|
|
# copy on one host and a VRRP-managed copy on another cannot coexist -- the
|
|
# kernel marks the newcomer `dadfailed` and its master serves nothing.
|
|
#
|
|
# What remains here are the guards. They run on every execution, not just the
|
|
# migration, and fail the play if any static assignment survives -- which would
|
|
# both defeat the health gating (a static copy does not disappear when
|
|
# keepalived decides the host is unfit) and break the VRRP copy via DAD.
|
|
#
|
|
# The config file is what gets checked, not the running address, and that is
|
|
# what makes it safe to run every time: keepalived never writes to these files,
|
|
# so anything found is by definition static, whereas the live address is
|
|
# legitimately present on whichever host is master.
|
|
#
|
|
# Prefix-agnostic and \b-anchored on purpose. Matching the full
|
|
# address/prefixlen would fail open against a hand-written assignment using any
|
|
# other mask.
|
|
- name: Detect any remaining static assignment of the service address (ifupdown)
|
|
ansible.builtin.command:
|
|
argv:
|
|
- grep
|
|
- -rlE
|
|
- "{{ keepalived_ntp_vip6.split('/')[0] }}\\b"
|
|
- /etc/network/interfaces
|
|
- /etc/network/interfaces.d/
|
|
register: keepalived_ntp_static6_ifupdown
|
|
changed_when: false
|
|
failed_when: false
|
|
when:
|
|
- keepalived_ntp_vip6 | length > 0
|
|
- ansible_facts['os_family'] == 'Debian'
|
|
|
|
- name: Assert ifupdown is not also assigning the service address
|
|
ansible.builtin.assert:
|
|
that:
|
|
- keepalived_ntp_static6_ifupdown.stdout | default('') | length == 0
|
|
fail_msg: >-
|
|
{{ keepalived_ntp_vip6 }} is still assigned statically by ifupdown on
|
|
this host
|
|
({{ keepalived_ntp_static6_ifupdown.stdout_lines | default([]) | join(', ') }}).
|
|
The automatic removal only matches the exact form
|
|
`up ip -6 addr replace {{ keepalived_ntp_vip6 }} dev <if> nodad`, so a
|
|
different prefix length or layout is left in place. Handing the address
|
|
to keepalived while ifupdown also assigns it means it would NOT be
|
|
withdrawn when check_ntp fails -- the host would keep serving time on an
|
|
address VRRP believes it has released. Remove the assignment from the
|
|
file(s) above, then re-run.
|
|
success_msg: >-
|
|
No static ifupdown assignment of {{ keepalived_ntp_vip6 }}; safe for
|
|
keepalived to own it.
|
|
when:
|
|
- keepalived_ntp_vip6 | length > 0
|
|
- ansible_facts['os_family'] == 'Debian'
|
|
|
|
# On Arch (edge-1) the static assignment lives in a systemd-networkd file owned
|
|
# by the edge-vlan role, so this role must not rewrite it -- but it must not
|
|
# quietly proceed either. install-edge-host.yml is a separate play with no
|
|
# ordering relationship to this one, so nothing guarantees it has been re-run
|
|
# to drop the [Address] block. If it has not, edge-1 would end up holding
|
|
# fd40::123 statically AND running a VRRP instance for the same address:
|
|
# keepalived would believe it had withdrawn the address while networkd kept it
|
|
# assigned, and the host would go on answering NTP with a clock the health
|
|
# check had already rejected.
|
|
#
|
|
# That is the precise failure of 2026-08-04, on the host it happened to, so it
|
|
# fails fast rather than being left to inspection.
|
|
# Matched on the bare address, not the full address/prefixlen, and anchored
|
|
# with \b so it cannot also hit a longer address sharing the prefix (fd40::1234).
|
|
#
|
|
# Searching for the literal "fd40::123/128" would fail open: a hand-written
|
|
# [Address] using any other mask -- /64, say -- would not match, the assert
|
|
# below would pass, and keepalived would be handed an address networkd is
|
|
# still assigning. Since this check is detect-only, with no delete afterwards
|
|
# to catch the mistake, a miss here is silent. The Debian path does not share
|
|
# the problem: `ip -6 addr show ... to <prefix>` matches by range regardless of
|
|
# the assigned prefix length, and its delete is re-verified by assert.
|
|
- name: Detect a static assignment of the service address (systemd-networkd)
|
|
ansible.builtin.command:
|
|
argv:
|
|
- grep
|
|
- -rlE
|
|
- "{{ keepalived_ntp_vip6.split('/')[0] }}\\b"
|
|
- /etc/systemd/network/
|
|
register: keepalived_ntp_static6_networkd
|
|
changed_when: false
|
|
failed_when: false
|
|
when:
|
|
- keepalived_ntp_vip6 | length > 0
|
|
- ansible_facts['os_family'] == 'Archlinux'
|
|
|
|
- name: Assert networkd is not also assigning the service address
|
|
ansible.builtin.assert:
|
|
that:
|
|
- keepalived_ntp_static6_networkd.stdout | default('') | length == 0
|
|
fail_msg: >-
|
|
{{ keepalived_ntp_vip6 }} is still assigned statically by
|
|
systemd-networkd on this host
|
|
({{ keepalived_ntp_static6_networkd.stdout_lines | default([]) | join(', ') }}).
|
|
Handing the address to keepalived while networkd also assigns it means
|
|
it would NOT be withdrawn when check_ntp fails -- the host would keep
|
|
serving time on an address VRRP believes it has released.
|
|
Re-apply the edge-vlan role first, whose template no longer carries the
|
|
[Address] block:
|
|
ansible-playbook -i inventory/pve_hosts.yml plays/install-edge-host.yml
|
|
success_msg: >-
|
|
No static networkd assignment of {{ keepalived_ntp_vip6 }}; safe for
|
|
keepalived to own it.
|
|
when:
|
|
- keepalived_ntp_vip6 | length > 0
|
|
- ansible_facts['os_family'] == 'Archlinux'
|
|
|
|
- name: Create keepalived configuration
|
|
ansible.builtin.template:
|
|
src: keepalived.conf.j2
|
|
dest: /etc/keepalived/keepalived.conf
|
|
owner: root
|
|
group: root
|
|
mode: '0644'
|
|
notify: restart keepalived
|
|
|
|
- name: Install health check script
|
|
ansible.builtin.template:
|
|
src: check-ntp-health.sh.j2
|
|
dest: "{{ keepalived_ntp_check_script }}"
|
|
owner: root
|
|
group: root
|
|
mode: '0755'
|
|
|
|
- name: Install notification script
|
|
ansible.builtin.copy:
|
|
src: keepalived-notify.sh
|
|
dest: /usr/local/bin/keepalived-notify.sh
|
|
owner: root
|
|
group: root
|
|
mode: '0755'
|
|
|
|
- name: Configure sysctl for VRRP
|
|
ansible.posix.sysctl:
|
|
name: "{{ item.name }}"
|
|
value: "{{ item.value }}"
|
|
state: present
|
|
reload: true
|
|
loop: "{{ keepalived_ntp_sysctl }}"
|
|
|
|
- name: Enable and start keepalived
|
|
ansible.builtin.systemd:
|
|
name: keepalived
|
|
enabled: true
|
|
state: started
|