Open a social app and you see a feed: recent posts from the accounts you follow, newest or "best" first, loading instantly and scrolling forever. Behind it is one central question: when do you assemble each person's feed — when a post is written, or when the feed is read?
Requirements and numbers
Functional: post text and images; follow accounts; see a feed of followed accounts' posts, newest first (ranking later); infinite scroll.
Non-functional: the feed loads in under ~200 ms; a new post appears in followers' feeds within seconds; reads hugely outnumber writes.
Scale: 200 million daily users, each opening the feed 10 times a day (2 billion feed reads a day, ~23,000/s) and posting about once every few days (~50 million posts a day, ~600/s). Average 200 followers per account, with a long tail up to tens of millions.
Option 1: fan-out on read (pull)
Build the feed when it's requested:
SELECT * FROM posts
WHERE author_id IN (SELECT followee FROM follows WHERE follower = :me)
ORDER BY created_at DESC LIMIT 20;
Writes are trivial — one row. Reads are expensive: for someone who follows 500 accounts, every feed load is a merge across 500 authors' recent posts. At 23,000 reads a second, that's a lot of repeated work, and latency grows with the number of accounts you follow.
Option 2: fan-out on write (push)
Do the work once per post instead. When someone posts, push the post's ID onto a precomputed feed list for each follower. Reading a feed is then one lookup of a list that already exists.
Asha publishes a post. It is stored once, in the posts table, by ID.
The numbers favour push: 600 posts a second × 200 followers = 120,000 small list writes a second, done asynchronously by workers. That buys 23,000 cheap reads a second with predictable latency. Feed lists hold only post IDs (8 bytes each), capped at a few hundred, so 200 million feeds × 800 IDs × 8 bytes ≈ 1.3 TB — a large but ordinary Redis cluster.
The celebrity problem
Push breaks on the long tail. An account with 40 million followers turns one post into 40 million writes: the fan-out queue backs up, and followers see the post minutes late.
The hybrid most large feeds use:
- Accounts below a follower threshold (say 10,000): fan-out on write.
- Accounts above it: not fanned out. At read time, the feed service fetches recent posts from the handful of big accounts the reader follows and merges them with the precomputed list.
Most readers follow only a few huge accounts, so the read-time merge is small. Inactive users are another cheap win: don't maintain feeds for accounts that haven't logged in for months, and rebuild one with a pull query when they come back.
Storing posts and media
- Posts: a table keyed by post ID. Use time-sortable IDs so the IDs in feed lists also encode order.
- Media: images and video go to object storage and are served through a CDN. The post stores only URLs.
- Hydration: a feed read gets 20 IDs, then fetches those posts in one batch from a post cache. Counts that change constantly — likes, replies — are fetched separately, so updating them doesn't invalidate the post.
Pagination
Offset pagination breaks here, because new posts shift every offset while the user scrolls. Use a cursor — "posts older than ID X" — which stays correct however many posts arrive. See cursor vs. offset pagination.
Ranking
A chronological feed is the baseline. Ranked feeds add a second stage: fetch a few hundred candidates (the precomputed list plus celebrity posts, plus perhaps recommended posts), score them with a model using features such as author affinity, recency and predicted engagement, and return the top 20. The candidate stage is the architecture above; ranking is a service that runs on top of it.
What breaks first
- Fan-out lag during spikes: queue depth grows. Workers scale horizontally; prioritise fan-out for active followers.
- Unfollows and deletes: a deleted post may still be in a million feed lists. Filter at read time (hydration finds no post) rather than chasing every copy.
- Hot posts: one viral post read millions of times. It's in every cache, so replicate hot keys across cache nodes.
Practise it
"News feed" is a prompt in the system design practice tool — draw the fan-out workers and feed cache, and have Claude check whether your notes justify every box.