Replication and quorum

Replicated volumes are 2-way synchronous (DRBD "protocol C"): a write completes only once both legs have it on disk, so the copies are identical at every instant, not eventually.

The hard problem in any replicated system is not a dead node; it is a partition: both nodes are alive but can't see each other. If both keep accepting writes, the copies diverge (split-brain) and one side's writes must eventually be thrown away. The quorum policy is the per-class answer to that situation, a trade between never diverging and staying writable. It is set per StorageClass via the miroir.home-operations.com/quorum parameter on classes whose replicas is above 1.

freeze (default)

DRBD majority quorum with on-no-quorum io-error: a replica that cannot reach a majority of the volume's peers refuses writes (the workload sees I/O errors) instead of carrying on alone. Two isolated sides can never both write, so there is nothing to un-diverge later; when connectivity returns, the stale side simply resyncs. Expect the filesystem on top to have gone read-only in the meantime, so the pod usually needs a restart once the volume recovers.

Two data replicas on their own are only 2 votes, and majority of 2 is 2: any disconnect would halt writes on both sides. That is what the tie-breaker fixes.

The diskless tie-breaker

When a 2-replica freeze volume is created and a third storage node in the topology holds neither leg, the controller adds that node as a diskless tie-breaker: a DRBD peer configured with disk none that joins quorum voting but stores nothing; no backing device, no capacity used, no snapshot leg, no part in CSI topology (when a pod stages the volume through it, it does coordinate the snapshot write barrier — see remote consumers). With 3 votes, losing any single node (a data leg or the tie-breaker) leaves a majority of 2 and the volume keeps serving.

Behavior and knobs:

• Placement is zone-aware. A spare node in a zone neither data leg occupies is preferred (nodes.<node>.spec.zone); ties break by node name.
• Existing volumes are retrofitted. Adding a third node to the topology (applying its MiroirNode) appends a tie-breaker to every 2-replica freeze volume that lacks one; the controller follows MiroirNode changes live, no restart involved. Editing a volume's spec.quorumPolicy from last-man-standing to freeze triggers the same reconciler.
• No spare node, no tie-breaker. On a 2-node cluster the volume is created with majority quorum on 2 votes: never diverges, but a single node loss stops I/O until the node returns. If availability matters more, use last-man-standing, or add a third node and let the retrofit pick it up.
• Opt out with autoTieBreaker: false in Helm values. This disables both placement at create time and the retrofit; tie-breakers can still be added manually by appending {node: <name>, diskless: true} to a volume's spec.replicas.
• Remove or move one by deleting its entry from spec.replicas; the node's agent detaches and cleans up (removal waits until both data legs are connected and UpToDate). A diskful replica can never become diskless in place; remove and re-add the entry instead. The other direction is allowed: flipping diskless: false on a tie-breaker makes its agent attach a fresh backing device to the live leg and full-sync it (this is how auto-diskful converts a tie-breaker).

last-man-standing

quorum off: the surviving replica keeps accepting writes even with no peers in sight. Maximum availability, but if both sides run while partitioned they diverge. DRBD detects the split-brain on reconnect and deliberately stays disconnected (after-sb-* disconnect, never auto-resolve); an operator inspects both legs and picks the loser. Volumes with this policy never get a tie-breaker.

At a glance

Failure	freeze + tie-breaker	freeze, 2 nodes	last-man-standing
One node down	keeps serving (2/3 votes)	I/O errors until it returns	survivor keeps writing
Replication link partitioned	the majority side serves; the minority side refuses writes	both sides refuse writes	both sides may write → split-brain, manual resolve
Two nodes down	remaining node refuses writes	I/O errors	survivor keeps writing

The default was last-man-standing before v0.3. Volumes keep whatever quorumPolicy is stored in their spec; nothing is rewritten on upgrade, and the new default only applies to volumes created after it.

Volumes are also consumable from nodes that hold no replica at all; see Remote consumers and auto-diskful. For what happens when a disk (rather than a node) fails, and how legs are verified against each other, see Disk failures, rebuilds, and verification.

Changing the replica count on restore

A live volume can never cross the replication boundary: an unreplicated (replicas: 1) volume cannot gain a DRBD layer in place, and its PersistentVolume is pinned to its one node forever. What is supported is requesting a different replica count when restoring a snapshot (or cloning a PVC): the restored volume is born with the class's shape, whatever the source had.

This is the growth story for a cluster going from one storage node to two. For each PVC:

1. Snapshot the PVC (any VolumeSnapshotClass backed by miroir).
2. Create a new PVC from that snapshot using the replicated StorageClass.
3. Repoint the workload at the new PVC and delete the old one.

The leg on the node that holds the snapshot restores as an instant copy-on-write clone; every additional leg is placed on another node carrying the class's pool and receives its content over the wire as a DRBD full sync (the restore reports Ready once the sync completes). The new PersistentVolume carries the replicated topology from the start, so pods can immediately follow either node.

Shrinking works the same way in reverse: restoring a replicated volume's snapshot with a replicas: 1 class keeps one copy-on-write leg and drops the replication layer. When the counts differ, the restore keeps the legs that hold the snapshot in preference to ones that would need a full sync.

One implementation detail worth knowing: a restore that crosses from unreplicated to replicated grows every backing slightly past the requested size (spec.source.padForMetadata), because the source filesystem spans its full nominal size and DRBD's internal metadata must fit behind it. The padding is virtual on every miroir backend and invisible to the workload.