hermes-bot
eed77ecb9a
www.multi-circuit-boards.eu (third-party Apache) occasionally resets a single TLS handshake with ECONNRESET. Kuma 2.5.6 opens a fresh connection for every check, so two such resets in a row were enough for DOWN with retryInterval 60 / maxRetries 1: 14 false episodes since 2026-10-20 21:51 UTC. A 30-min probe from the cluster (559 requests at 3s) saw 557x 200 and 2 handshake resets, and got 200 during Kuma's DOWN window. With retryInterval 20 (CRD minimum) and maxRetries 3, DOWN now takes 4 failures in a row over at least 60s, so a real outage longer than about 1 min still alerts. The other third-party monitors are left unchanged. Refs #1407
homelab
GitOps source for the Renner homelab cluster. Flux applies main to
production, so every merge is a production change. Changes go in by pull
request only; see CONTRIBUTING.md for YAML conventions.
Change rules that protect Home Assistant
Owner decision (REN-240, REN-243). Home Assistant runs the house, so it gets two rules on top of normal review.
1. Disruptive work runs only 02:00–05:00 Europe/Berlin
Planned work that can restart Home Assistant or disrupt the cluster is merged and applied only between 02:00 and 05:00 Europe/Berlin. That covers:
- node drain, node reboot, Talos upgrade
- CNI (Cilium, Multus) and cluster-wide network policy
- storage (Rook/Ceph, CSI, snapshot controller, CNPG operator)
- Flux core (
kubernetes/clusters/*/flux-system/, Flux controllers) - Authentik
- OpenBao, including secret rotation that restarts consumers
- anything in
kubernetes/apps/home-assistant/
A PR in this list states in its description that it waits for the window. Flux applies on merge, so "wait for the window" means merge in the window, not just apply. If unsure whether a change is disruptive, treat it as disruptive.
Not covered: Home Assistant image bumps by Flux image automation. They stay fully automatic by owner decision. If one breaks Home Assistant, roll it back with RUNBOOK-home-assistant-image-rollback.md. An incident rollback is not planned work; the owner decides its timing.
Also not covered, by owner decision: the Hermes ha-updates job
(kubernetes/apps/hermes/skills/ha-updates/) installs HACS and device-firmware
updates that Home Assistant offers, and restarts Home Assistant for them, on
its own inside the 02:00–05:00 window (UniFi and SMLIGHT firmware excluded;
reported only). The window is enforced by its ha.py, not only by the prompt.
2. HA-impact review before merge
Every PR that touches kubernetes/apps/home-assistant/, or makes a
cluster-wide disruptive change from the list above, needs an HA-impact
review before merge: a named reviewer states in the PR what Home
Assistant will notice (restart, lost integration, lost history, none) and
approves.
- Reviewer: Hestia, the Home Assistant maintainer.
- Until the owner hands this review to Hestia (the
hestiaHermes profile, which today only states the expected impact in its PRs): the owner does the HA-impact review.
This is in addition to the normal review and merge by Argus. Argus does not merge such a PR without the HA-impact approval.