Common issues
Reading UNKNOWN, under-replication, and a monitor that never leaves PENDING.
A monitor reports UNKNOWN
UNKNOWN is not one specific failure — see States for the full list of what produces it. The two most common causes:
- Not enough fresh evidence to reach quorum. Check the monitor's coverage — see Coverage — for whether it is under-replicated or unable to reach quorum. Losing an assigned node does not by itself cause UNKNOWN, but it makes insufficient evidence more likely.
- A stale stored consensus. If nothing has evaluated the monitor recently enough — past its freshness budget of three check intervals, floored at five minutes and capped at thirty — the last answer is no longer trusted and the state reads as UNKNOWN rather than whatever it last said. Confirm the monitor's assigned nodes are ONLINE and actually submitting results.
A node going offline is not a rule that makes a monitor UNKNOWN. If you find documentation, a dashboard, or a mental model that says otherwise, it is describing the more common consequence of a lost node — reduced evidence — not the rule itself. See Coverage for the precise, distinct conditions.
A monitor is under-replicated
Check Enrolling nodes for how placement chooses nodes: a monitor is under-replicated when fewer nodes are assigned to it than its assurance level desires, most commonly because the fleet does not currently have enough eligible, healthy capacity to place all of them. An under-replicated monitor may still be fully quorum-capable and reporting a conclusive state — under-replication alone is not an outage.
A node never leaves PENDING
PENDING means the control plane has not yet seen a heartbeat from the node at all. Confirm the node process is actually running, that it can reach the control plane's address over outbound HTTPS, and that its credential file was written successfully on enrollment. A node behind a network that blocks outbound access to the control plane will stay PENDING indefinitely rather than reporting any other state — from the control plane's perspective, a node that has never connected and a node that cannot connect look the same.
A monitor is capacity-blocked
The fleet's reconciler could not place the monitor's desired replicas at all — a fact about current fleet capacity, not about that one monitor. Enroll additional eligible nodes, or lower the monitor's assurance level if fewer replicas are acceptable; see Consensus for what each assurance level asks for.
An HTTP monitor always reports unexpected_status
Confirm the expected status against what the target actually returns during normal operation. A common cause is a target that redirects: with http_follow_redirects on, the status evaluated is the final response's, not the redirect's own 3xx.
A TLS monitor reports cert_expiring earlier than expected
This is the monitor doing what its expiry threshold asked — see TLS monitors. Raise the threshold if you want less advance notice; lower it for more.
Related concepts
This article is part of this repository and is served by the same binary, so what you read here is the documentation for the version you are running.