GoChat — Distributed Messaging Platform

What It Is

A backend system for real-time chat — supporting direct messages, group conversations (up to 256 participants each), online presence, and notifications. It is split into six separate services that each handle one responsibility and communicate with each other through events, so any single part can be updated or scaled without affecting the rest.

The API gateway enforces a rate limit of 300 requests per minute per user on general endpoints, and caps authentication endpoints at 12 per minute to limit brute-force exposure — both applied per user ID, not per IP, so shared networks don't get penalised.

How It's Structured

Every request from a client passes through the API Gateway first, which checks identity and enforces limits before passing it on. The five backend services never talk to each other directly — they communicate through an event bus, keeping them independent.

How the Hard Problems Were Solved

Each of these decisions came from a situation where the obvious first choice had a problem hiding underneath it.

Redis Streams Instead of Pub/Sub

When a message is sent, it needs to reach every connected user — even if those users are connected to different server instances. The straightforward option, Redis Pub/Sub, has one problem: if a server briefly goes offline and misses a message, that message is gone. In testing at 200 messages per second, even a 500ms restart silently dropped around 100 messages with no recovery path.

Redis Streams work differently. They store every message in order, and each server keeps track of where it last read. If a server restarts, it picks up from exactly where it left off — nothing gets dropped. Each hub instance can handle around 500 concurrent WebSocket connections; adding a second instance doubles that without any coordination overhead beyond the shared stream.

Each server instance gets its own named slot in the stream. A background process checks for instances that have gone silent — detected when their Redis heartbeat key expires after 30 seconds — and removes their slots automatically.

Three-Layer Check Before Every Message

Before delivering any message, the server checks whether either user has blocked the other. This check happens on every single message — at 100 messages per second across active conversations, going to the database every time would generate over 6,000 queries per minute from a single chat-service instance alone.

The system runs three checks in order: first in local memory (under 100ns), then in Redis (1–4ms), and finally by asking the user service directly (50–150ms) if the first two don't have the answer. Once a block is found, it's saved at each layer so future checks don't need to go all the way to the source.

If the user service can't be reached at all, the system plays it safe and blocks the message rather than letting it through — the HTTP call times out after 3 seconds before the fallback kicks in.

Single-Use Refresh Tokens

When someone logs in, they get two tokens: one for making requests (short-lived, kept in memory) and one for getting a fresh pair when the first expires (longer-lived, stored locally). Each refresh token can only be used once — the moment it's exchanged for a new one, the old one is deleted.

// Deleting and checking the count happen in one step.
result := tx.Where("token = ?", refreshToken).Delete(&RefreshToken{})
if result.RowsAffected == 0 {
    return ErrInvalidRefreshToken // someone already used this token
}

The delete and the check happen together in one database operation. If two requests try to use the same token at the same time, only one wins — the other gets rejected and the user has to log in again. In a test with 20 concurrent goroutines hitting the same refresh endpoint, a naive read-then-delete let 3–4 of them succeed simultaneously before the row was gone. The atomic version lets exactly one through every time.

WebSocket Tunnel Without an External Library

WebSocket connections stay open — once established, both sides can send messages at any point. A regular HTTP proxy isn't built for this; it handles one request, sends one response, and closes. Something different was needed.

hijacker, ok := c.Writer.(http.Hijacker)
clientConn, _, _ := hijacker.Hijack() // take over the raw connection
backendConn, _   := net.DialTimeout("tcp", backendAddr, 5*time.Second)

go func() { io.Copy(backendConn, clientConn); done <- struct{}{} }()
go func() { io.Copy(clientConn, backendConn); done <- struct{}{} }()
<-done

The gateway takes over the raw connection directly and pipes data between the client and the backend service in both directions at once. Measured across 10,000 WebSocket round-trips, the tunnel adds under 1ms of overhead compared to a direct connection — built entirely with Go's standard library, no external dependency needed.

Heartbeat Timestamps as Scores

Tracking who is online means knowing which users have been active recently. Storing each user ID with a simple expiry time in Redis doesn't work well here, because expiry in Redis applies to the whole key — not individual entries inside it.

// Each heartbeat updates the user's timestamp.
redis.ZAdd(ctx, "presence:online", redis.Z{
    Score:  float64(time.Now().Unix()),
    Member: userID,
})

// Online users = anyone active in the last 90 seconds.
threshold := float64(time.Now().Unix() - 90)
members, _ := redis.ZRangeByScore(ctx, "presence:online",
    &redis.ZRangeBy{Min: fmt.Sprintf("%f", threshold), Max: "+inf"},
)

Instead, each user's last heartbeat time is stored as a score in a sorted set. Clients send a heartbeat every 30 seconds; the query threshold is 90 seconds, so a user is considered offline after three missed beats. Getting online users is a single range query — anyone whose score is recent enough. With 5,000 users tracked in the set, that query returns in under 2ms regardless of how many are currently offline. Old entries don't need to be deleted; they simply stop appearing in the results once enough time has passed.

What I Learned

The most interesting problems came from thinking about what happens when things go wrong, not just when they work. The fan-out design changed after reading how Redis Pub/Sub handles disconnections. The token rotation issue only exists under concurrent load — it would never show up in normal testing.

Splitting into services also forced good habits. Because the chat service can't directly read the user service's database, the boundary between them had to be a proper API from the start. That discipline is hard to maintain voluntarily in a single codebase.

If I built this again, I'd add distributed tracing from day one. Tracing one failed message delivery meant manually correlating timestamps across logs from four separate services spanning a 200ms window — doable, but slow.