Cache Invalidation Nightmare
"There are only two hard things in computer science: cache invalidation and naming things."
After this incident, I believe the person who said this.
Our cache invalidation strategy:
- User updates profile → invalidate user cache
- User updates profile → invalidate user's posts cache
- User updates profile → invalidate followers' feeds
- User updates profile → invalidate search index
- User updates profile → invalidate recommendation cache
What we forgot:
- User's comments on other people's posts (still showed old name)
- Notification history (old avatar)
- Email preview cache (old bio)
The realization:
As your system grows, the number of places you cache data grows faster than your ability to track them.
Our new approach:
- Document every cache in a central registry
- Cache keys include version numbers
- Global cache busting for user data changes
- Accept some staleness where it doesn't matter
Lesson: If you can't list every place a piece of data is cached, you can't reliably invalidate it.