chore: remove the Proxmox observability and access surface #620

2026-08-26T04:47:18Z

claude-bot commented

2026-08-26 04:47:18 +00:00

No Proxmox remains. All three nodes run Talos on bare metal, so everything that scraped, proxied, alerted on or rotated credentials for the PVE API now points at hosts answering nothing on :8006.

Two of these are not merely dead but actively failing on a schedule:


`gatus`	an Infrastructure check on `https://pve.renner.uno` every 60s, which can now only ever fail
`openbao` rotation	a Class-B CronJob rolling the `pve-exporter@pve` API token against `pve.apps.svc.cluster.local:8006`

The Gatus one would have sat there red indefinitely; the rotation CronJob would fail on its schedule and, by design, a failed rotation Job is itself an alarm signal.

Removed

apps/pve/                         Service, EndpointSlice (10.2.40.201-203),
                                  IngressRoute for the PVE web UIs
monitoring/pve-exporter-*         Deployment + config ExternalSecret
prometheus scrape configs         job_name pve and pve-node
dashboards/proxmox-cluster.json   and its ConfigMap generator entry
image-automation/pve-exporter     image policy for an image nothing runs
openbao/rotation/pve-exporter     the rotation CronJob
openbao/policies/rotator-proxmox.hcl
crowdsec pve-ui-polling-whitelist suppressed false bans on PVE UI API
                                  polling that no longer happens

709 deletions, 3 insertions.

Kept

The insecure-backend ServersTransport stays — apps/file still uses it for the NAS web UI on a self-signed cert. Its comment loses the apps/pve line.

One judgement call

The Gatus removal leaves a numbering gap at "Infrastructure 2" rather than renumbering 3 and 4 up. The labels are deliberately opaque (they are not served publicly, there is a maintainer legend in the file), and renumbering would relabel the surviving checks and discard their history in Gatus for a cosmetic gain. The legend now reads Infrastructure 2 = retired (was Proxmox VE).

Verified

Every affected kustomization still builds:

kubectl kustomize apps/ | apps/monitoring/ | apps/monitoring/dashboards/
                        | apps/gatus/ | infrastructure/crowdsec/
                        | infrastructure/image-automation/
                        | infrastructure/openbao/rotation/
                        | infrastructure/network-policies/    → all rc=0

Deliberately not in scope

The OpenTofu Proxmox provider, the proxmox_vm/proxmox_lxc modules, vm_152_k8s_3.tf, sync_state_from_pve.py, and ansible/inventory/api.proxmox.yaml. Those carry state implications — removing a .tf whose resource is still in state produces a destroy plan for a VM that no longer exists — and want a deliberate pass with someone able to run tofu, not a manifest cleanup at 3am.

Note the ExternalSecret removal does not delete the underlying Proxmox credentials from OpenBao; that is a separate secret-store cleanup whenever you want it.

No Proxmox remains. All three nodes run Talos on bare metal, so everything that scraped, proxied, alerted on or rotated credentials for the PVE API now points at hosts answering nothing on `:8006`. **Two of these are not merely dead but actively failing on a schedule:** | | | | --- | --- | | `gatus` | an Infrastructure check on `https://pve.renner.uno` every 60s, which can now only ever fail | | `openbao` rotation | a Class-B CronJob rolling the `pve-exporter@pve` API token against `pve.apps.svc.cluster.local:8006` | The Gatus one would have sat there red indefinitely; the rotation CronJob would fail on its schedule and, by design, a failed rotation Job is itself an alarm signal. ## Removed ``` apps/pve/ Service, EndpointSlice (10.2.40.201-203), IngressRoute for the PVE web UIs monitoring/pve-exporter-* Deployment + config ExternalSecret prometheus scrape configs job_name pve and pve-node dashboards/proxmox-cluster.json and its ConfigMap generator entry image-automation/pve-exporter image policy for an image nothing runs openbao/rotation/pve-exporter the rotation CronJob openbao/policies/rotator-proxmox.hcl crowdsec pve-ui-polling-whitelist suppressed false bans on PVE UI API polling that no longer happens ``` 709 deletions, 3 insertions. ## Kept The `insecure-backend` ServersTransport stays — `apps/file` still uses it for the NAS web UI on a self-signed cert. Its comment loses the `apps/pve` line. ## One judgement call The Gatus removal leaves a numbering gap at "Infrastructure 2" rather than renumbering 3 and 4 up. The labels are deliberately opaque (they are not served publicly, there is a maintainer legend in the file), and renumbering would relabel the surviving checks and discard their history in Gatus for a cosmetic gain. The legend now reads `Infrastructure 2 = retired (was Proxmox VE)`. ## Verified Every affected kustomization still builds: ``` kubectl kustomize apps/ | apps/monitoring/ | apps/monitoring/dashboards/ | apps/gatus/ | infrastructure/crowdsec/ | infrastructure/image-automation/ | infrastructure/openbao/rotation/ | infrastructure/network-policies/ → all rc=0 ``` ## Deliberately not in scope The OpenTofu Proxmox provider, the `proxmox_vm`/`proxmox_lxc` modules, `vm_152_k8s_3.tf`, `sync_state_from_pve.py`, and `ansible/inventory/api.proxmox.yaml`. Those carry state implications — removing a `.tf` whose resource is still in state produces a destroy plan for a VM that no longer exists — and want a deliberate pass with someone able to run tofu, not a manifest cleanup at 3am. Note the ExternalSecret removal does not delete the underlying Proxmox credentials from OpenBao; that is a separate secret-store cleanup whenever you want it.

claude-bot added 1 commit 2026-08-26 04:47:18 +00:00

chore: remove the Proxmox observability and access surface

ci/woodpecker/pr/ci-manifests-test Pipeline was successful

Details

ci/woodpecker/pr/ci-scripts-test Pipeline was successful

Details

ci/woodpecker/pr/pr-review Pipeline was successful

Details

d50be81ed1

No Proxmox remains. jupiter, saturn and neptune all run Talos on bare metal,
so everything that scraped, proxied, alerted on or rotated credentials for the
PVE API now points at hosts that answer nothing on :8006.

Two of these were not merely dead but actively failing on a schedule:

  gatus            an Infrastructure check on https://pve.renner.uno, every
                   60s, which can now only ever fail
  openbao rotation a Class-B CronJob rolling the pve-exporter@pve API token
                   against pve.apps.svc.cluster.local:8006

Removed:

  apps/pve/                       Service, EndpointSlice (10.2.40.201-203),
                                  IngressRoute for the PVE web UIs
  monitoring/pve-exporter-*       Deployment and its config ExternalSecret
  prometheus scrape configs       job_name pve and pve-node
  dashboards/proxmox-cluster.json and its ConfigMap generator entry
  image-automation/pve-exporter   image policy for an image nothing runs
  openbao/rotation/pve-exporter   the rotation CronJob
  openbao/policies/rotator-proxmox.hcl
  crowdsec pve-ui-polling-whitelist, which suppressed false bans on PVE UI
                                  API polling that no longer happens

Kept, with comments corrected: the insecure-backend ServersTransport, still
used by apps/file for the NAS web UI.

The gatus check leaves a numbering gap at "Infrastructure 2" rather than
renumbering 3 and 4. The labels are deliberately opaque and renumbering would
relabel the surviving checks, discarding their history for cosmetics.

Not touched: the opentofu Proxmox provider, VM and LXC modules,
vm_152_k8s_3.tf, sync_state_from_pve.py, and the ansible PVE inventory. Those
carry state implications and want a deliberate pass rather than folding into a
manifest cleanup.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

k.renner merged commit 8173cf5517 into master

2026-08-26 04:49:17 +00:00

k.renner referenced this issue from a commit

2026-08-26 04:49:18 +00:00

Merge pull request 'chore: remove the Proxmox observability and access surface' (#620) from drop-pve-monitoring into master

review-bot reviewed

2026-08-26 04:50:48 +00:00

review-bot left a comment

🤖 Automated review — model CLI default (complexity: complex), commit d50be81ed1026fc8d8768741583564e2312cd291.

Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.

Summary

Pure-deletion PR that removes the entire Proxmox VE monitoring/access surface (Gatus check, pve-exporter Deployment + ExternalSecret + Prometheus scrape configs, the Grafana dashboard, the apps/pve Service/EndpointSlice/IngressRoute, the OpenBao rotation CronJob + policy, the image-automation policy, and the CrowdSec whitelist parser) now that all nodes are bare-metal Talos. I traced every removed resource back to its consumers and found no dangling references — all affected kustomization.yaml resource lists were updated consistently, no ServiceMonitor/PrometheusRule/dashboard still points at pve_* metrics or the pve-exporter/pve Services, and the two YAML files with in-place edits (gatus/configmap.yaml, crowdsec/helmrelease.yaml) remain structurally valid after the removal. No blocking issues found.

🟡 Suggestions

kubernetes/infrastructure/openbao/rotation/kustomization.yaml:23-25 and kubernetes/infrastructure/network-policies/openbao-ns.yaml:89 still describe "proxmox-token" / "Proxmox" as an active rotator example, and secret-rotator/src/main.rs:37,208-228 + secret-rotator/src/proxmox.rs still implement the proxmox-token mode end-to-end. With rotator-proxmox.hcl and the CronJob gone, this mode is now unreachable dead code with no bootstrap credential left to exercise it. Not this PR's stated scope (which is manifests, not the rotator binary), but worth a follow-up issue so the binary doesn't quietly carry unusable Proxmox logic indefinitely.
kubernetes/infrastructure/network-policies/monitoring-alloy-syslog.yaml:3 still lists "Proxmox" as an example external syslog sender. Given the PR body states "No Proxmox remains. All three nodes run Talos on bare metal," this comment is now inaccurate (there's no Proxmox host left to send syslog) — minor doc drift outside this diff's touched files.

🟢 Nits

kubernetes/apps/monitoring/ci-diagnostics-rbac.yaml:11 references "the pve-exporter 401" as a historical example of a verified diagnosis; harmless since it's phrased as past history, but could confuse a future reader into thinking pve-exporter still exists.

Tests

No automated tests apply to this repo's manifest changes; the PR body documents manual verification via kubectl kustomize against every affected Kustomization, which I could not independently re-run (no kubectl/kustomize binary in this environment) but whose logic I verified by hand: every resource removed from a kustomization.yaml resources list corresponds to a file actually deleted in the same diff, and no surviving manifest references the removed Services, Secrets, dashboards, scrape jobs, or CronJob.

Recommendation

Looks reasonable — no blocking issues; comments above are optional.

🤖 **Automated review** — model `CLI default` (complexity: complex), commit `d50be81ed1026fc8d8768741583564e2312cd291`. > Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review. --- ## Summary Pure-deletion PR that removes the entire Proxmox VE monitoring/access surface (Gatus check, pve-exporter Deployment + ExternalSecret + Prometheus scrape configs, the Grafana dashboard, the `apps/pve` Service/EndpointSlice/IngressRoute, the OpenBao rotation CronJob + policy, the image-automation policy, and the CrowdSec whitelist parser) now that all nodes are bare-metal Talos. I traced every removed resource back to its consumers and found no dangling references — all affected `kustomization.yaml` resource lists were updated consistently, no ServiceMonitor/PrometheusRule/dashboard still points at `pve_*` metrics or the `pve-exporter`/`pve` Services, and the two YAML files with in-place edits (`gatus/configmap.yaml`, `crowdsec/helmrelease.yaml`) remain structurally valid after the removal. No blocking issues found. ## 🟡 Suggestions - `kubernetes/infrastructure/openbao/rotation/kustomization.yaml:23-25` and `kubernetes/infrastructure/network-policies/openbao-ns.yaml:89` still describe "proxmox-token" / "Proxmox" as an active rotator example, and `secret-rotator/src/main.rs:37,208-228` + `secret-rotator/src/proxmox.rs` still implement the `proxmox-token` mode end-to-end. With `rotator-proxmox.hcl` and the CronJob gone, this mode is now unreachable dead code with no bootstrap credential left to exercise it. Not this PR's stated scope (which is manifests, not the rotator binary), but worth a follow-up issue so the binary doesn't quietly carry unusable Proxmox logic indefinitely. - `kubernetes/infrastructure/network-policies/monitoring-alloy-syslog.yaml:3` still lists "Proxmox" as an example external syslog sender. Given the PR body states "No Proxmox remains. All three nodes run Talos on bare metal," this comment is now inaccurate (there's no Proxmox host left to send syslog) — minor doc drift outside this diff's touched files. ## 🟢 Nits - `kubernetes/apps/monitoring/ci-diagnostics-rbac.yaml:11` references "the pve-exporter 401" as a historical example of a verified diagnosis; harmless since it's phrased as past history, but could confuse a future reader into thinking pve-exporter still exists. ## Tests No automated tests apply to this repo's manifest changes; the PR body documents manual verification via `kubectl kustomize` against every affected Kustomization, which I could not independently re-run (no `kubectl`/`kustomize` binary in this environment) but whose logic I verified by hand: every resource removed from a `kustomization.yaml` resources list corresponds to a file actually deleted in the same diff, and no surviving manifest references the removed Services, Secrets, dashboards, scrape jobs, or CronJob. ## Recommendation Looks reasonable — no blocking issues; comments above are optional.

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: konairius/homelab#620