Files
homelab/kubernetes/apps/monitoring-rules/prometheusrule-blackbox.yaml
T
atlas-botandClaude Opus 5.5 21a7583523
ci/woodpecker/pr/ci-manifests-test Pipeline was successful
ci/woodpecker/pr/ci-scripts-test Pipeline was successful
ci/woodpecker/pr/pr-review Pipeline was successful
ci: cite YAML by anchor instead of line number, and guard it (REN-183)
Comments that pointed at other YAML as `file.yaml:N` went stale on any
edit above the cited line; three drift fixes had been merged by hand
(REN-181). The cited block now carries `# anchor: <name>` and the
comment cites `path.yaml#<name>`.

- Sweep: all 32 line-number citations in YAML comments converted.
  Several were already stale at head and now point at the right block
  (see PR). Vendored files (Flux gotk-components, IMC CRD) get no
  anchor; those two citations name the object/schema path instead.
- Guard: scripts/ci/yaml-citations.sh in ci-manifests-test.yml; fails
  on a line-number citation or a path#name whose file/anchor is gone.
  Self-test in ci-scripts-test.yml.
- CONTRIBUTING.md: the convention in three lines.

Comment-only under kubernetes/: every file parses to identical objects
before and after, so Flux applies nothing and no ConfigMap data changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-26 08:44:54 +00:00

62 lines
3.2 KiB
YAML

apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: blackbox
namespace: monitoring
labels:
# kube-prometheus-stack's Prometheus resource selects on this label;
# without it the rule is created but never loaded into Prometheus.
release: kube-prometheus-stack
spec:
groups:
- name: blackbox.rules
rules:
# Catch-all for every blackbox-http target that has no rule of its own.
# Before this existed, a target could be probed, graphed and completely
# unalerted — which is how file.renner.uno served 502 for months.
#
# THE EXCLUSION LIST AND THE DEDICATED RULES ARE ONE INVARIANT: an
# instance belongs here only while a rule elsewhere covers it, and
# adding a dedicated rule means adding its instance here. Get it wrong
# in one direction and the alert fires twice; wrong in the other and it
# never fires at all. Currently excluded:
#
# fsn1.your-objectstorage.com HetznerBackupEndpointDown (critical)
# netbird.renner.uno prometheusrule-netbird-availability
# proxy-1/2:9000/health same file, relay redundancy
# ci.renner.uno WoodpeckerDown
# openbao .../sys/health OpenBaoUnavailable
#
# Warning, not critical, on purpose: this rule cannot know what a given
# target means. A target whose loss deserves a critical page gets its
# own rule — and lands in the list above. That is what keeps the
# exclusion list from being arbitrary.
# anchor: blackbox-probe-failing
- alert: BlackboxProbeFailing
expr: probe_success{job="blackbox-http", instance!~"https://fsn1\\.your-objectstorage\\.com|https://netbird\\.renner\\.uno|http://proxy-[12]\\.renner\\.uno:9000/health|https://ci\\.renner\\.uno|https://openbao\\.renner\\.uno/v1/sys/health.*"} == 0
for: 10m
annotations:
summary: "Blackbox probe of {{ $labels.instance }} has been failing for 10 minutes"
description: >-
The endpoint is not answering as expected. For the Authentik-gated
hostnames this is a reachability check covering DNS, TLS, Traefik
and the gate outpost — it says nothing about the app behind the
gate, which the internal /-/healthy probes cover separately.
labels:
severity: warning
# The check that the checks are running. Every rule above reads
# probe_success; if the series stop being produced at all, none of them
# can fire and the silence looks exactly like health.
# anchor: blackbox-probes-absent
- alert: BlackboxProbesAbsent
expr: absent(probe_success{job="blackbox-http"})
for: 15m
annotations:
summary: "No blackbox-http probe results are reaching Prometheus"
description: >-
Either the blackbox exporter is down or the Probe objects stopped
being discovered. Every endpoint alert depends on this job, so
they are all blind until it returns.
labels:
severity: critical