From 1061804d09565363aba73e369faf310a7d2c4d86 Mon Sep 17 00:00:00 2001 From: Jan Friesse Date: Mon, 15 Jul 2019 14:08:39 +0200 Subject: [PATCH] totem: Increase ring_id seq after load This patch handles the situation where the leader node (the node with lowest node_id) crashes and is started again before token timeout of the rest of the cluster. The newly restarted node restores the ringid of the old ring from stable storage, so it has the same ringid as rest of the nodes, but ARU is zero. If the node is able to create a singleton membership before receiving the joinlist from rest of the cluster, everything works as expected, because the ring id gets increased correctly. But if the node receives a joinlist from another cluster node before its own joinlist, then it continues as it would had it never left the cluster. This is not correct, because the new node should always create a singleton configuration first. During the recovery phase, ARUs are compared and because they differ (the ARU of the old leader node is 0), the other nodes try to sent all of their previous messages. This is impossible (even if it was correct), because other nodes have already freed most of those messages. The implementation uses an assert to limit maximum number of messages sent during recovery (we could fix this, but it's not really the point). The solution here is to increase the ring_id sequence number by 1 after loading it from storage. During creation of the commit token it is always increased by 4, so it will not collide with an existing sequence. Thanks Christine Caulfield for clarify commit message. Signed-off-by: Jan Friesse Reviewed-by: Christine Caulfield --- exec/totemsrp.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/exec/totemsrp.c b/exec/totemsrp.c index 0410ed9..c07bd43 100644 --- a/exec/totemsrp.c +++ b/exec/totemsrp.c @@ -5094,6 +5094,14 @@ void main_iface_change_fn ( if (instance->iface_changes++ == 0) { instance->memb_ring_id_create_or_load (&instance->my_ring_id, &instance->my_id.addr[0]); + /* + * Increase the ring_id sequence number. This doesn't follow specification. + * Solves problem with restarted leader node (node with lowest nodeid) before + * rest of the cluster forms new membership and guarantees unique ring_id for + * new singleton configuration. + */ + instance->my_ring_id.seq++; + instance->token_ring_id_seq = instance->my_ring_id.seq; log_printf ( instance->totemsrp_log_level_debug, -- 1.8.3.1