Why Mesh assigns sequence numbers on the server

Every real-time chat system has to answer one question before it answers any other: when two messages are sent at the same moment, which one came first?

The tempting answer is to let the client decide. Stamp each message with Date.now() on send, sort by that timestamp on receive, and move on. It works in development. It works in the demo. It falls apart the first time two people on two machines with two slightly different clocks type at the same time.

The failure mode

Client clocks drift. Not by much — NTP keeps most machines within tens of milliseconds — but "tens of milliseconds" is exactly the window in which a conversation happens.

Consider two clients whose clocks differ by 40ms:

// Client A, clock is 40ms fast
msg := Message{Body: "ship it", SentAt: 1719230400040}
 
// Client B, clock is accurate. Types 20ms *later* in wall-clock time.
msg := Message{Body: "wait, no", SentAt: 1719230400020}

B typed second. B's timestamp is lower. Sort by SentAt and the transcript now reads "wait, no" followed by "ship it" — the exact inversion of what happened, and a meaningfully different conversation.

What we do instead

Mesh does not trust client timestamps for ordering. Every message passes through a sequencer — a small, Raft-backed service that owns one job: hand out a monotonically increasing sequence number per channel.

func (s *Sequencer) Assign(ctx context.Context, channelID ChannelID) (uint64, error) {
    // Raft ensures a single leader per channel range, so this increment
    // is the only one happening for this channel anywhere in the cluster.
    seq, err := s.raft.Apply(ctx, IncrementCommand{Channel: channelID})
    if err != nil {
        return 0, fmt.Errorf("assign sequence for %s: %w", channelID, err)
    }
    return seq, nil
}

A message is not durable and not visible until it has a sequence number. Clients sort by that number and nothing else. The client's own clock is retained for display — "sent at 3:42pm" — but it has no say in ordering.

The cost, stated honestly

This is a real trade-off, and it is worth being explicit about which side of CAP we landed on.

Because the sequencer needs a Raft quorum to assign a number, a network partition means the minority side cannot accept writes. Clients on that side see their messages stay in a pending state until the partition heals.

That is a genuine availability cost. We took it deliberately. The alternative — accepting messages optimistically on both sides and reconciling afterwards — means that some conversations get silently reordered after the fact. A user who watched a transcript appear in one order would later find it in another.

A chat system that occasionally rewrites history is worse than one that occasionally makes you wait.

What it buys

With server-assigned ordering, a set of properties fall out for free:

• Resume is trivial. A reconnecting client sends its last-seen sequence number and gets exactly what it missed. No timestamp windows, no overlap, no dedupe.
• Pagination is exact. WHERE seq > $1 ORDER BY seq LIMIT 50 — no cursor encoding, no ties to break.
• Idempotency is structural. A message with a sequence number has already been accepted. Retrying delivery cannot duplicate it.

None of these needed to be designed. They are consequences of having a total order, which is the general shape of a good architectural decision: you make one guarantee real, and a category of problems stops existing.