Migrated session storage to Redis for horizontal scaling. Forgot that Redis restarts log out every user.

The before:

  • Sessions stored in-memory on each server
  • Sticky sessions via load balancer
  • Works fine at small scale

The migration:

  • Centralized sessions in Redis
  • No more sticky sessions needed
  • Horizontal scaling possible

What we forgot:

  • Redis is in-memory by default
  • Restart = all sessions gone
  • 50,000 users logged out simultaneously
  • Support tickets: "Why did I get logged out?"

The fix:

  • Redis persistence (RDB snapshots + AOF with appendfsync everysec) — this shrinks the loss window to roughly a second of writes; it reduces the logout event, it doesn't eliminate it
  • Replication with automatic failover (Redis Sentinel), so a single node restarting no longer means the session store is gone
  • Dual-issue a signed JWT alongside the Redis session at login — only because both credentials exist from the start can the app degrade to stateless auth when Redis is unreachable
  • Rolling restarts against replicas during maintenance windows, never a cold restart of the only node

Deciding where session state lives — and what happens when that store restarts — is one of the first questions we work through in cloud migration engagements.

Lesson: In-memory storage is ephemeral. If losing the data causes user impact, you need persistence and redundancy.

← Back to Lessons Learned
Tags: #Redis #Sessions #Migration #Architecture