docs(chrony): correct the root cause recorded for the makestep fix #466

2026-08-05T15:36:59Z

claude-bot commented

2026-08-05 15:36:59 +00:00

Follow-up to #465. Comments only — no behaviour change.

The problem

#465 landed the makestep directive with the wrong root cause baked into its comment, claiming edge-1 "rebooted with an RTC ~16 days in the past". That is false, and I only established the real cause after #465 had already merged.

What actually happened

edge-1's CMOS battery is dead. After a power loss the RTC came up reading 2018-08-20 — roughly eight years stale, not 16 days:

PM: RTC time: 09:18:56, date: 2018-08-20
rtc_cmos 00:01: setting system clock to 2018-08-20T09:18:58 UTC (1534756738)

The clock was then reconstructed entirely in software. systemd floored it forward to its own build date:

systemd[1]: System time advanced to built-in epoch: Fri 2026-06-26 11:27:23 UTC

and it settled at ~2026-07-19, about 16 days behind real time.

So the 16 days is incidental — just where the flooring happened to land. There is no 16-day mechanism and nothing "drifted". The two real faults are a dead RTC battery and a chrony config that never steps. #465 fixed the second; the first is hardware.

This matters for what to do next: without the battery replacement, edge-1 will still come up with a wrong clock after every power loss. It will now self-correct rather than sit broken, but the underlying fault is unaddressed.

Also in this PR

Collapses the duplicated rationale in chrony.conf.j2 to a pointer at defaults/main.yml, as suggested in review on #465, so the explanation lives in exactly one place.

Note on the commit message

7fcaa4b0 on master still carries the wrong explanation. Merged history is left alone rather than rewritten; this PR corrects the tree, and this description is the record of why the merged message is wrong.

Evidence caveat

The clock was wrong for the whole window, so journald and Loki timestamps from that period are unreliable and cannot be used to order events. The conclusions above rest on message content and on /proc/uptime (monotonic, immune to clock changes) — the real boot time was 2026-08-04 19:47:48, which is not what the journal's own timestamps suggest.

Still open, not addressed here

Replace the CR2032 in edge-1.
Why the host lost power at ~19:47 on 2026-08-04.
/etc/machine-id was empty at that boot (Initializing machine ID from random generator).
local stratum lets these hosts serve NTP with a known-wrong clock.

🤖 Generated with Claude Code

Follow-up to #465. Comments only — no behaviour change. ## The problem #465 landed the `makestep` directive with the **wrong root cause** baked into its comment, claiming edge-1 "rebooted with an RTC ~16 days in the past". That is false, and I only established the real cause after #465 had already merged. ## What actually happened **edge-1's CMOS battery is dead.** After a power loss the RTC came up reading `2018-08-20` — roughly eight years stale, not 16 days: ``` PM: RTC time: 09:18:56, date: 2018-08-20 rtc_cmos 00:01: setting system clock to 2018-08-20T09:18:58 UTC (1534756738) ``` The clock was then reconstructed entirely in software. systemd floored it forward to its own build date: ``` systemd[1]: System time advanced to built-in epoch: Fri 2026-06-26 11:27:23 UTC ``` and it settled at ~2026-07-19, about 16 days behind real time. So **the 16 days is incidental** — just where the flooring happened to land. There is no 16-day mechanism and nothing "drifted". The two real faults are a dead RTC battery and a chrony config that never steps. #465 fixed the second; the first is hardware. This matters for what to do next: without the battery replacement, edge-1 will still come up with a wrong clock after every power loss. It will now self-correct rather than sit broken, but the underlying fault is unaddressed. ## Also in this PR Collapses the duplicated rationale in `chrony.conf.j2` to a pointer at `defaults/main.yml`, as suggested in review on #465, so the explanation lives in exactly one place. ## Note on the commit message `7fcaa4b0` on master still carries the wrong explanation. Merged history is left alone rather than rewritten; this PR corrects the tree, and this description is the record of why the merged message is wrong. ## Evidence caveat The clock was wrong for the whole window, so journald and Loki timestamps from that period are unreliable and cannot be used to order events. The conclusions above rest on message *content* and on `/proc/uptime` (monotonic, immune to clock changes) — the real boot time was 2026-08-04 19:47:48, which is not what the journal's own timestamps suggest. ## Still open, not addressed here - Replace the CR2032 in edge-1. - Why the host lost power at ~19:47 on 2026-08-04. - `/etc/machine-id` was empty at that boot (`Initializing machine ID from random generator`). - `local stratum` lets these hosts serve NTP with a known-wrong clock. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

claude-bot added 1 commit 2026-08-05 15:37:00 +00:00

docs(chrony): correct the root cause recorded for the makestep fix

ci/woodpecker/pr/pr-review Pipeline was successful

Details

ci/woodpecker/pr/ci-scripts-test Pipeline was successful

Details

32d12f93d8

#465 landed the makestep directive with the wrong root cause in its
comment. It claimed edge-1 "rebooted with an RTC ~16 days in the past".
That is false, and the real cause matters for what to do next.

edge-1's CMOS battery is dead. After a power loss the RTC came up reading
2018-08-20 -- roughly eight years stale, not 16 days:

  PM: RTC time: 09:18:56, date: 2018-08-20
  rtc_cmos 00:01: setting system clock to 2018-08-20T09:18:58 UTC

The clock was then reconstructed entirely in software. systemd floored it
forward to its own build date:

  systemd[1]: System time advanced to built-in epoch: Fri 2026-06-26 11:27:23 UTC

and it settled at ~2026-07-19, about 16 days behind. So the 16 days is
incidental -- it is just where the flooring landed. There is no 16-day
mechanism, and nothing "drifted". The two real faults are a dead RTC
battery and a chrony config that never steps. #465 fixed the second.

Also collapses the duplicated rationale in chrony.conf.j2 to a pointer at
defaults/main.yml, as suggested in review on #465, so the explanation
lives in exactly one place.

No behaviour change: the makestep directive and its defaults are
untouched. Comments only.

The commit message on 7fcaa4b0 still carries the wrong explanation and is
left alone rather than rewriting merged history.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

review-bot reviewed

2026-08-05 15:38:25 +00:00

review-bot left a comment

🤖 Automated review — model claude-haiku-4-5-20251001 (complexity: trivial), commit 32d12f93d80500fabdaddfd5d851bba54e29b0dc.

Advisory review by the review-bot pipeline. It never approves; a human merges. Push new commits to get a fresh review.

Summary

This PR corrects the documented root cause of the makestep configuration added in #465. The original comment incorrectly claimed edge-1's RTC was "~16 days in the past," when the actual cause is a dead CMOS battery that read 2018 after a power loss; the 16 days refers to where software flooring landed, not the fault mechanism. The PR also consolidates duplicated rationale between the template and defaults, following DRY principles. No behavior changes—comments only.

🟢 Nits

Line 19 in chrony.conf.j2: The pointer references chrony_makestep_* which is a pattern matching both threshold and limit variables in defaults/main.yml. Clear enough, but could say chrony_makestep_threshold and chrony_makestep_limit for explicitness. Very minor—the current phrasing is acceptable.

Tests

No tests needed—documentation-only change.

Recommendation

Looks reasonable — no blocking issues; comment above is optional.

🤖 **Automated review** — model `claude-haiku-4-5-20251001` (complexity: trivial), commit `32d12f93d80500fabdaddfd5d851bba54e29b0dc`. > Advisory review by the `review-bot` pipeline. It never approves; a human merges. Push new commits to get a fresh review. --- ## Summary This PR corrects the documented root cause of the `makestep` configuration added in #465. The original comment incorrectly claimed edge-1's RTC was "~16 days in the past," when the actual cause is a dead CMOS battery that read 2018 after a power loss; the 16 days refers to where software flooring landed, not the fault mechanism. The PR also consolidates duplicated rationale between the template and defaults, following DRY principles. No behavior changes—comments only. ## 🟢 Nits **Line 19 in `chrony.conf.j2`**: The pointer references `chrony_makestep_*` which is a pattern matching both `threshold` and `limit` variables in `defaults/main.yml`. Clear enough, but could say `chrony_makestep_threshold` and `chrony_makestep_limit` for explicitness. Very minor—the current phrasing is acceptable. ## Tests No tests needed—documentation-only change. ## Recommendation Looks reasonable — no blocking issues; comment above is optional.

claude-bot referenced this pull request

2026-08-05 17:28:45 +00:00

fix(ntp): make the keepalived health check verify the clock, not liveness #467

k.renner merged commit d39115a55d into master

2026-08-05 17:49:08 +00:00

k.renner referenced this issue from a commit

2026-08-05 17:49:10 +00:00

Merge pull request 'docs(chrony): correct the root cause recorded for the makestep fix' (#466) from chrony-comment-correction into master

Sign in to join this conversation.

2 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: konairius/homelab#466