feat(netbird): relay + STUN cluster members on the netcup proxy hosts #412
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: konairius/homelab#412
Reference in New Issue
Block a user
Closes mitigation option 2 from the SPOF assessment: relay diversity off the Hetzner controller VPS.
Changes
netbird-proxyrole: newrelaycompose service on proxy-1/proxy-2 —netbirdio/relay:{{ netbird_version }}, host network, WebSocket listener:33080, embedded STUN:3478/udp, sharedNB_AUTH_SECRETfrom the existing vault var (verified it resolves for thenetbird_proxygroup; upstream docs: mismatched secrets fail silently).relay.envdeployed 0600 with its own force-recreate handler, mirroring the proxy.env pattern.netbird-serverrole:management.jsonnow lists three relays (rels://netbird.renner.uno:443,rel://proxy-2.renner.uno:33080,rel://proxy-1.renner.uno:33080) and three STUN servers. Template change triggers the existing management-only force-recreate handler.Design notes
rel://: Let's Encrypt is impossible on these hosts (:443= reverse-proxy tls-alpn-01,:80= redirect), and shipping a zone-wide DNS-01 token to internet-facing VPSes is a worse risk than plaintext transport. Peer payload remains WireGuard-encrypted end to end; exposure is relay framing metadata plus credential sniff-and-reuse bounded by the 24 h TTL. Upgrading torels://via centrally-issued DNS-01 certs is a possible follow-up.ip_unprivileged_port_startlesson from #409 doesn't apply here.Rollout & verification
Merge triggers both
deploy-netbird-proxy(serial: 1) anddeploy-netbird-serverpipelines. Brief window where management may advertise a relay that isn't up yet — harmless, clients retry/fail over.Afterwards I'll verify:
ssshows :33080/tcp + :3478/udp listeners on both hosts,netbird status -don an ingress pod lists the new relays as Available, and (destructive-test optional) stopping the VPS relay leaves relayed peers connected via a netcup relay.🤖 Generated with Claude Code
🤖 Automated review — model
CLI default(complexity: complex), commit7eff0940daafe01a28585623552570d556131017.Summary
Adds a NetBird relay + embedded STUN service to the
netbird-proxyrole (deployed onproxy-1/proxy-2), and registers those relays/STUN servers in the management server'smanagement.json, so mesh peers keep a working relay/STUN path during a Hetzner controller-VPS outage. The change reuses the existingvault_netbird_relay_secretvar (already consumed by thenetbird-serverrole's own relay container) rather than minting a new secret, follows the establishedproxy.env/force-recreate-handler pattern for the newrelay.env, and its handler wiring is correct:management.jsonchanges only notify the existing management-only restart handler (ansible/roles/netbird-server/handlers/main.yml:12), so the relay container on the controller host — which doesn't read that file — isn't needlessly bounced. I traced the CNP referenced in the PR body (kubernetes/infrastructure/network-policies/ingress-isolation.yaml:62-65) and confirmed thetoCIDRSetegress rule to the proxy hosts has no port restriction, so it already covers the new:33080/tcpand:3478/udpports without a policy change, as claimed. No blocking issues found.🟡 Suggestions
ansible/roles/netbird-proxy/templates/relay.env.j2:102-103—NB_ENABLE_STUN/NB_STUN_PORTSare new env var names not otherwise used anywhere in this repo (the existing relay container innetbird-server/templates/docker-compose.yml.j2:126-129only setsNB_LOG_LEVEL/NB_LISTEN_ADDRESS/NB_EXPOSED_ADDRESS/NB_AUTH_SECRET). I could not reach the internet from this sandbox to confirm these are the correct flag names/format fornetbirdio/relay:0.73.2's embedded-STUN support, so I can't verify this beyond the PR author's citation of the upstream docs. Worth double-checking against the actualnetbirdio/relayrelease notes/--helpfor 0.73.2 before merge, since a wrong env var name would silently no-op (relay comes up fine, STUN just never binds) rather than fail loudly.ansible/roles/netbird-proxy-watchdog, both the URL-probe and log-flood variants) only targets{{ netbird_watchdog_container }}(thenetbird-proxycontainer) and isn't extended to the newrelaycontainer. That's consistent with this being a proxy-health watchdog and out of scope for a first cut, but worth a follow-up if a wedged/crash-looping relay becomes an operational issue in practice — right now a hung relay would only be caught byrestart: unless-stopped+ external monitoring, if any.Tests
This is an Ansible/infra-config repo with no automated test suite for role changes (consistent with the rest of the codebase — verification here is inherently operational). The PR body lays out a concrete post-deploy verification plan (
sslistener check,netbird status -don an ingress pod, optional destructive relay-stop test), which is the appropriate level of "testing" for this kind of change. I confirmed both.woodpecker/deploy-netbird-proxy.ymland.woodpecker/deploy-netbird-server.ymlpath-filter on the changed role directories, so merging will actually trigger both deploys as the PR claims.Recommendation
Looks reasonable — no blocking issues; comments above are optional (the NB_ENABLE_STUN/NB_STUN_PORTS env var names are worth a manual double-check against upstream before merge, but I can't confirm or refute them myself).
Review follow-up — both 🟡 points addressed:
1.
NB_ENABLE_STUN/NB_STUN_PORTSon relay 0.73.2 — verified empirically, not just from docs.docker run --rm netbirdio/relay:0.73.2 --helplists--enable-stunand--stun-ports(default[3478]), and a local run with exactly the env vars fromrelay.env.j2(NB_ENABLE_STUN=true,NB_STUN_PORTS=13478) logged:So the env binding is honored at our pinned version and the plaintext
rel://mode works. (Expected side note:WARN Not starting QUIC listener: valid TLS config is required— QUIC transport is TLS-only, so the netcup relays serve WebSocket only; the VPS relay behind Traefik is ws-only today as well.)2. Watchdog not extended to the relay — deliberate, now documented in the compose template (
c6a578b4). The relay is a fallback component with client-side failover to the other relays, so a wedged instance degrades redundancy, not availability;restart: unless-stoppedcatches crashes. The binary serveshttp://localhost:9000/health(verified on 0.73.2) — recorded in the template as the hook for future active monitoring, which is already an open item from the SPOF assessment (external probes).🤖 Automated review — model
CLI default(complexity: complex), commitc6a578b4f25cd71dd719146563980c7cce9a69dc.Summary
Adds a
relaycompose service (+ embedded STUN) to thenetbird-proxyrole so proxy-1/proxy-2 host a second and third NetBird relay outside Hetzner, and updatesnetbird-server'smanagement.jsonto register the two new relays and STUN servers. The change follows the existing patterns in this repo closely (host networking, vault-sourced secrets,env_file+ force-recreate handler mirroringproxy.env), and I could confirm several of the PR's specific claims against the checkout (see below). No blocking issues found.Checks I performed against the repo, not just the diff:
vault_netbird_relay_secretis already defined (it's consumed bynetbird-server's existingRelay.Secret) and lives ininventory/group_vars/all/vault.yml, so it resolves for thenetbird_proxygroup too — the PR's claim checks out (ansible/roles/netbird-proxy/defaults/main.yml:13,ansible/roles/netbird-server/defaults/main.yml:25).inventory_hostnamefor the two hosts is literallyproxy-1.renner.uno/proxy-2.renner.uno(ansible/inventory/pve_hosts.yml:22-23), soNB_EXPOSED_ADDRESS=rel://{{ inventory_hostname }}:33080(relay.env.j2:102) renders to exactly the URIs hardcoded inmanagement.json.j2:137-138.kubernetes/infrastructure/network-policies/ingress-isolation.yaml:55-65), confirming "no policy change needed."netbird-proxy-watchdogrestarts a hardcodednetbird_watchdog_container: netbird-proxy(ansible/roles/netbird-proxy-watchdog/defaults/main.yml:28), so the newrelaycontainer is indeed outside its blast radius, as claimed.relay.envis templated (with its own notify) beforedocker compose pull/docker compose up -druns, so the container starts with the right env from the very first apply; the plaindocker compose up -dalso picks up the new service even without the handler firing, consistent with howproxy.env/proxyalready work.I could not verify one load-bearing claim: whether
NB_ENABLE_STUN/NB_STUN_PORTS(relay.env.j2:109-110) are real, correctly-named env vars fornetbirdio/relay:0.73.2's embedded STUN feature — I don't have network access in this environment to check the cited docs page or the relay's source/help output. If either name is wrong, the relay would most likely just silently ignore it (no crash), so the STUN half of the SPOF mitigation would quietly do nothing until an actual controller outage exposed it. This isn't a report of a bug — I couldn't confirm or refute it — but it's worth an explicit check before or right after rollout, e.g.docker exec netbird-relay <binary> --help | grep -i stun, or simply confirming withss -lunthat something is actually bound to:3478/udppost-deploy (the PR's own rollout plan already includes this check).🟡 Suggestions
ansible/roles/netbird-server/templates/management.json.j2:9,13,37-38— the new STUN/relay entries hardcodeproxy-1.renner.uno/proxy-2.renner.unoin a template that lives in a different role than the one that owns those hosts (netbird-proxy, driven by the[netbird_proxy]inventory group). Adding, removing, or renaming a proxy cluster member now requires editing two roles by hand and keeping them in sync manually, with no coupling enforced by Ansible. Not worth a refactor for a 2-3 host homelab, but if the proxy cluster grows, consider deriving this list fromgroups['netbird_proxy']instead.Tests
No automated test/lint step exists for Ansible role changes in this repo (no molecule/ansible-lint pipeline was found for any role, so this isn't a gap introduced by this PR). The
deploy-netbird-proxyanddeploy-netbird-serverWoodpecker pipelines apply the change directly to production hosts on merge. The PR body's stated manual verification plan (ssshowing:33080/tcp+:3478/udplisteners,netbird status -dlisting the new relays as Available, optional destructive failover test) is a reasonable acceptance check for this kind of infra change and should be treated as required, not optional, given the unverified STUN env-var names noted above.Recommendation
Looks reasonable — no blocking issues; comments above are optional (though I'd treat the STUN env-var verification as a should-do before considering the rollout complete).